Nov 04 00:59:44.549007 ip-10-0-132-67 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Nov 04 00:59:44.549018 ip-10-0-132-67 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Nov 04 00:59:44.549027 ip-10-0-132-67 systemd[1]: kubelet.service: Failed with result 'resources'.
Nov 04 00:59:44.549368 ip-10-0-132-67 systemd[1]: Failed to start Kubernetes Kubelet.
Nov 04 00:59:54.576541 ip-10-0-132-67 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Nov 04 00:59:54.576559 ip-10-0-132-67 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 4583952d5bf24dfdbe58c8de4cec02ad --
Nov 04 01:01:56.646926 ip-10-0-132-67 systemd[1]: Starting Kubernetes Kubelet...
Nov 04 01:01:57.097021 ip-10-0-132-67 kubenswrapper[2582]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 01:01:57.097021 ip-10-0-132-67 kubenswrapper[2582]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 04 01:01:57.097021 ip-10-0-132-67 kubenswrapper[2582]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 01:01:57.097021 ip-10-0-132-67 kubenswrapper[2582]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 01:01:57.101199 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101122    2582 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:01:57.101199 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101198    2582 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:01:57.101199 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101202    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:01:57.101332 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101205    2582 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:01:57.101332 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101209    2582 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:01:57.101332 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101211    2582 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:01:57.101332 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101214    2582 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:01:57.101332 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101217    2582 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:01:57.101332 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101220    2582 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:01:57.101332 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101222    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:01:57.101332 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101225    2582 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:01:57.101332 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101228    2582 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:01:57.101332 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101230    2582 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:01:57.101332 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101233    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:01:57.101332 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101236    2582 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:01:57.101332 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101238    2582 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:01:57.101332 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101241    2582 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:01:57.101332 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101243    2582 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:01:57.101332 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101246    2582 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:01:57.101332 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101249    2582 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:01:57.101332 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101252    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:01:57.101332 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101255    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:01:57.101784 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101257    2582 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:01:57.101784 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101260    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:01:57.101784 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101262    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:01:57.101784 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101270    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:01:57.101784 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101273    2582 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:01:57.101784 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101276    2582 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:01:57.101784 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101278    2582 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:01:57.101784 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101281    2582 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:01:57.101784 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101284    2582 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:01:57.101784 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101286    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:01:57.101784 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101289    2582 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:01:57.101784 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101292    2582 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:01:57.101784 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101295    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:01:57.101784 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101297    2582 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:01:57.101784 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101301    2582 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:01:57.101784 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101304    2582 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:01:57.101784 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101307    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:01:57.101784 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101311    2582 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:01:57.101784 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101316    2582 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:01:57.102226 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101319    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:01:57.102226 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101322    2582 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:01:57.102226 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101325    2582 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:01:57.102226 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101329    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:01:57.102226 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101333    2582 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:01:57.102226 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101337    2582 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:01:57.102226 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101340    2582 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:01:57.102226 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101344    2582 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:01:57.102226 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101347    2582 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:01:57.102226 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101350    2582 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:01:57.102226 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101353    2582 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:01:57.102226 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101355    2582 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:01:57.102226 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101358    2582 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:01:57.102226 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101361    2582 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:01:57.102226 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101363    2582 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:01:57.102226 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101366    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:01:57.102226 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101369    2582 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:01:57.102226 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101371    2582 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:01:57.102226 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101374    2582 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:01:57.102721 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101377    2582 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:01:57.102721 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101379    2582 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:01:57.102721 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101382    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:01:57.102721 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101385    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:01:57.102721 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101387    2582 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:01:57.102721 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101390    2582 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:01:57.102721 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101392    2582 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:01:57.102721 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101395    2582 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:01:57.102721 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101399    2582 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:01:57.102721 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101401    2582 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:01:57.102721 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101404    2582 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:01:57.102721 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101407    2582 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:01:57.102721 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101410    2582 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:01:57.102721 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101412    2582 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:01:57.102721 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101415    2582 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:01:57.102721 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101418    2582 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:01:57.102721 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101420    2582 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:01:57.102721 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101423    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:01:57.102721 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101425    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:01:57.102721 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101428    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:01:57.103179 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101431    2582 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:01:57.103179 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101434    2582 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:01:57.103179 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101436    2582 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:01:57.103179 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101439    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:01:57.103179 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101441    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:01:57.103179 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101444    2582 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:01:57.103179 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101446    2582 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:01:57.103179 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101449    2582 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:01:57.103179 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101451    2582 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:01:57.103179 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101454    2582 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:01:57.103179 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101456    2582 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:01:57.103179 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101459    2582 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:01:57.103179 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101461    2582 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:01:57.103179 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101464    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:01:57.103179 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101466    2582 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:01:57.103179 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101469    2582 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:01:57.103179 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101471    2582 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:01:57.103179 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101474    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:01:57.103179 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101476    2582 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:01:57.103637 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101479    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:01:57.103637 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101483    2582 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:01:57.103637 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101486    2582 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:01:57.103637 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101489    2582 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:01:57.103637 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101491    2582 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:01:57.103637 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101494    2582 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:01:57.103637 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.101496    2582 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:01:57.103637 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102314    2582 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:01:57.103637 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102323    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:01:57.103637 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102326    2582 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:01:57.103637 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102329    2582 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:01:57.103637 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102332    2582 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:01:57.103637 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102334    2582 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:01:57.103637 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102337    2582 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:01:57.103637 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102340    2582 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:01:57.103637 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102344    2582 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:01:57.103637 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102346    2582 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:01:57.103637 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102349    2582 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:01:57.103637 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102352    2582 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:01:57.103637 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102355    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:01:57.104121 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102357    2582 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:01:57.104121 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102360    2582 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:01:57.104121 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102362    2582 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:01:57.104121 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102365    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:01:57.104121 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102368    2582 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:01:57.104121 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102371    2582 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:01:57.104121 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102373    2582 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:01:57.104121 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102376    2582 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:01:57.104121 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102379    2582 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:01:57.104121 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102381    2582 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:01:57.104121 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102384    2582 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:01:57.104121 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102386    2582 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:01:57.104121 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102389    2582 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:01:57.104121 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102393    2582 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:01:57.104121 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102396    2582 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:01:57.104121 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102398    2582 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:01:57.104121 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102401    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:01:57.104121 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102403    2582 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:01:57.104121 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102406    2582 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:01:57.104121 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102409    2582 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:01:57.104589 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102412    2582 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:01:57.104589 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102414    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:01:57.104589 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102417    2582 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:01:57.104589 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102419    2582 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:01:57.104589 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102422    2582 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:01:57.104589 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102424    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:01:57.104589 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102427    2582 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:01:57.104589 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102429    2582 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:01:57.104589 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102432    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:01:57.104589 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102435    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:01:57.104589 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102437    2582 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:01:57.104589 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102440    2582 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:01:57.104589 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102442    2582 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:01:57.104589 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102445    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:01:57.104589 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102447    2582 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:01:57.104589 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102449    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:01:57.104589 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102452    2582 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:01:57.104589 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102454    2582 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:01:57.104589 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102457    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:01:57.104589 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102459    2582 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:01:57.105114 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102462    2582 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:01:57.105114 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102464    2582 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:01:57.105114 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102467    2582 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:01:57.105114 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102469    2582 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:01:57.105114 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102477    2582 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:01:57.105114 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102483    2582 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:01:57.105114 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102487    2582 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:01:57.105114 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102491    2582 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:01:57.105114 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102495    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:01:57.105114 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102498    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:01:57.105114 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102501    2582 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:01:57.105114 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102504    2582 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:01:57.105114 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102506    2582 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:01:57.105114 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102509    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:01:57.105114 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102512    2582 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:01:57.105114 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102514    2582 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:01:57.105114 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102517    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:01:57.105114 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102519    2582 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:01:57.105604 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102522    2582 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:01:57.105604 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102524    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:01:57.105604 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102527    2582 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:01:57.105604 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102529    2582 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:01:57.105604 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102532    2582 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:01:57.105604 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102535    2582 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:01:57.105604 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102537    2582 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:01:57.105604 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102540    2582 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:01:57.105604 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102543    2582 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:01:57.105604 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102545    2582 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:01:57.105604 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102548    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:01:57.105604 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102550    2582 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:01:57.105604 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102553    2582 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:01:57.105604 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102556    2582 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:01:57.105604 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102558    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:01:57.105604 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102560    2582 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:01:57.105604 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102563    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:01:57.105604 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102566    2582 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:01:57.105604 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102569    2582 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:01:57.105604 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102574    2582 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:01:57.106093 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102576    2582 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:01:57.106093 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102579    2582 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:01:57.106093 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102582    2582 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:01:57.106093 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102584    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:01:57.106093 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102587    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:01:57.106093 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102589    2582 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:01:57.106093 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102592    2582 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:01:57.106093 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102595    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:01:57.106093 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102599    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:01:57.106093 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102601    2582 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:01:57.106093 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102603    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:01:57.106093 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102606    2582 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:01:57.106093 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102623    2582 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:01:57.106093 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102626    2582 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:01:57.106093 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.102628    2582 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:01:57.106093 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103121    2582 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:01:57.106093 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103127    2582 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:01:57.106093 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103130    2582 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:01:57.106093 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103132    2582 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:01:57.106093 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103135    2582 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:01:57.106567 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103139    2582 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:01:57.106567 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103142    2582 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:01:57.106567 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103144    2582 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:01:57.106567 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103147    2582 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:01:57.106567 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103150    2582 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:01:57.106567 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103153    2582 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:01:57.106567 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103155    2582 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:01:57.106567 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103158    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:01:57.106567 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103160    2582 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:01:57.106567 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103163    2582 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:01:57.106567 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103166    2582 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:01:57.106567 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103171    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:01:57.106567 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103174    2582 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:01:57.106567 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103176    2582 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:01:57.106567 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103179    2582 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:01:57.106567 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103182    2582 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:01:57.106567 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103184    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:01:57.106567 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103187    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:01:57.106567 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103190    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:01:57.107035 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103192    2582 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:01:57.107035 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103195    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:01:57.107035 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103198    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:01:57.107035 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103200    2582 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:01:57.107035 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103203    2582 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:01:57.107035 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103205    2582 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:01:57.107035 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103208    2582 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:01:57.107035 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103210    2582 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:01:57.107035 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103213    2582 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:01:57.107035 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103216    2582 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:01:57.107035 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103219    2582 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:01:57.107035 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103221    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:01:57.107035 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103225    2582 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:01:57.107035 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103228    2582 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:01:57.107035 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103231    2582 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:01:57.107035 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103233    2582 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:01:57.107035 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103236    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:01:57.107035 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103239    2582 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:01:57.107035 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103241    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:01:57.107483 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103244    2582 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:01:57.107483 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103246    2582 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:01:57.107483 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103249    2582 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:01:57.107483 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103252    2582 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:01:57.107483 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103254    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:01:57.107483 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103258    2582 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:01:57.107483 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103261    2582 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:01:57.107483 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103263    2582 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:01:57.107483 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103266    2582 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:01:57.107483 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103268    2582 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:01:57.107483 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103271    2582 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:01:57.107483 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103273    2582 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:01:57.107483 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103276    2582 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:01:57.107483 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103278    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:01:57.107483 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103281    2582 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:01:57.107483 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103284    2582 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:01:57.107483 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103286    2582 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:01:57.107483 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103289    2582 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:01:57.107483 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103291    2582 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:01:57.107483 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103294    2582 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:01:57.107968 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103297    2582 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:01:57.107968 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103300    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:01:57.107968 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103302    2582 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:01:57.107968 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103305    2582 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:01:57.107968 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103308    2582 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:01:57.107968 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103311    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:01:57.107968 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103313    2582 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:01:57.107968 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103316    2582 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:01:57.107968 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103318    2582 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:01:57.107968 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103321    2582 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:01:57.107968 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103323    2582 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:01:57.107968 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103326    2582 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:01:57.107968 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103328    2582 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:01:57.107968 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103331    2582 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:01:57.107968 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103334    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:01:57.107968 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103336    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:01:57.107968 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103339    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:01:57.107968 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103343    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:01:57.107968 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103345    2582 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:01:57.107968 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103348    2582 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:01:57.108436 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103351    2582 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:01:57.108436 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103355    2582 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:01:57.108436 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103358    2582 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:01:57.108436 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103362    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:01:57.108436 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103364    2582 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:01:57.108436 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103368    2582 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:01:57.108436 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103370    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:01:57.108436 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103373    2582 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:01:57.108436 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103375    2582 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:01:57.108436 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103378    2582 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:01:57.108436 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103380    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:01:57.108436 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103383    2582 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:01:57.108436 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103385    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:01:57.108436 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103388    2582 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:01:57.108436 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103391    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:01:57.108436 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103395    2582 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:01:57.108436 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103399    2582 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:01:57.108436 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103401    2582 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:01:57.108436 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103404    2582 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:01:57.108436 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103406    2582 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:01:57.108933 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103409    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:01:57.108933 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103411    2582 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:01:57.108933 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.103414    2582 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:01:57.108933 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103501    2582 flags.go:64] FLAG: --address="0.0.0.0"
Nov 04 01:01:57.108933 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103508    2582 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 04 01:01:57.108933 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103514    2582 flags.go:64] FLAG: --anonymous-auth="true"
Nov 04 01:01:57.108933 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103519    2582 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 04 01:01:57.108933 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103523    2582 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 04 01:01:57.108933 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103526    2582 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 04 01:01:57.108933 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103531    2582 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 04 01:01:57.108933 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103536    2582 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 04 01:01:57.108933 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103539    2582 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 04 01:01:57.108933 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103542    2582 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 04 01:01:57.108933 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103546    2582 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 04 01:01:57.108933 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103549    2582 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 04 01:01:57.108933 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103552    2582 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 04 01:01:57.108933 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103555    2582 flags.go:64] FLAG: --cgroup-root=""
Nov 04 01:01:57.108933 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103558    2582 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 04 01:01:57.108933 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103561    2582 flags.go:64] FLAG: --client-ca-file=""
Nov 04 01:01:57.108933 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103564    2582 flags.go:64] FLAG: --cloud-provider="external"
Nov 04 01:01:57.108933 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103567    2582 flags.go:64] FLAG: --cluster-dns="[]"
Nov 04 01:01:57.108933 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103571    2582 flags.go:64] FLAG: --cluster-domain=""
Nov 04 01:01:57.108933 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103574    2582 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 04 01:01:57.108933 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103577    2582 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 04 01:01:57.109502 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103581    2582 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 04 01:01:57.109502 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103584    2582 flags.go:64] FLAG: --container-log-max-files="5"
Nov 04 01:01:57.109502 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103591    2582 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 04 01:01:57.109502 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103596    2582 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 04 01:01:57.109502 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103599    2582 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 04 01:01:57.109502 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103603    2582 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 04 01:01:57.109502 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103620    2582 flags.go:64] FLAG: --contention-profiling="false"
Nov 04 01:01:57.109502 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103624    2582 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 04 01:01:57.109502 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103627    2582 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 04 01:01:57.109502 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103630    2582 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 04 01:01:57.109502 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103633    2582 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 04 01:01:57.109502 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103638    2582 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 04 01:01:57.109502 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103641    2582 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 04 01:01:57.109502 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103645    2582 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 04 01:01:57.109502 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103648    2582 flags.go:64] FLAG: --enable-load-reader="false"
Nov 04 01:01:57.109502 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103651    2582 flags.go:64] FLAG: --enable-server="true"
Nov 04 01:01:57.109502 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103653    2582 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 04 01:01:57.109502 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103658    2582 flags.go:64] FLAG: --event-burst="100"
Nov 04 01:01:57.109502 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103662    2582 flags.go:64] FLAG: --event-qps="50"
Nov 04 01:01:57.109502 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103664    2582 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 04 01:01:57.109502 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103668    2582 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 04 01:01:57.109502 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103670    2582 flags.go:64] FLAG: --eviction-hard=""
Nov 04 01:01:57.109502 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103674    2582 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 04 01:01:57.109502 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103677    2582 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 04 01:01:57.110142 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103681    2582 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 04 01:01:57.110142 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103684    2582 flags.go:64] FLAG: --eviction-soft=""
Nov 04 01:01:57.110142 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103687    2582 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 04 01:01:57.110142 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103690    2582 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 04 01:01:57.110142 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103692    2582 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 04 01:01:57.110142 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103695    2582 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 04 01:01:57.110142 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103698    2582 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 04 01:01:57.110142 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103701    2582 flags.go:64] FLAG: --fail-swap-on="true"
Nov 04 01:01:57.110142 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103704    2582 flags.go:64] FLAG: --feature-gates=""
Nov 04 01:01:57.110142 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103708    2582 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 04 01:01:57.110142 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103711    2582 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 04 01:01:57.110142 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103716    2582 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 04 01:01:57.110142 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103719    2582 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 04 01:01:57.110142 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103722    2582 flags.go:64] FLAG: --healthz-port="10248"
Nov 04 01:01:57.110142 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103725    2582 flags.go:64] FLAG: --help="false"
Nov 04 01:01:57.110142 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103728    2582 flags.go:64] FLAG: --hostname-override="ip-10-0-132-67.us-east-2.compute.internal"
Nov 04 01:01:57.110142 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103731    2582 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 04 01:01:57.110142 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103734    2582 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 04 01:01:57.110142 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103737    2582 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 04 01:01:57.110142 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103740    2582 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 04 01:01:57.110142 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103744    2582 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 04 01:01:57.110142 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103747    2582 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 04 01:01:57.110142 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103750    2582 flags.go:64] FLAG: --image-service-endpoint=""
Nov 04 01:01:57.111056 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103753    2582 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 04 01:01:57.111056 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103755    2582 flags.go:64] FLAG: --kube-api-burst="100"
Nov 04 01:01:57.111056 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103758    2582 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 04 01:01:57.111056 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103762    2582 flags.go:64] FLAG: --kube-api-qps="50"
Nov 04 01:01:57.111056 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103765    2582 flags.go:64] FLAG: --kube-reserved=""
Nov 04 01:01:57.111056 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103767    2582 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 04 01:01:57.111056 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103771    2582 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 04 01:01:57.111056 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103774    2582 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 04 01:01:57.111056 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103777    2582 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 04 01:01:57.111056 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103780    2582 flags.go:64] FLAG: --lock-file=""
Nov 04 01:01:57.111056 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103782    2582 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 04 01:01:57.111056 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103785    2582 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 04 01:01:57.111056 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103788    2582 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 04 01:01:57.111056 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103794    2582 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 04 01:01:57.111056 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103797    2582 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 04 01:01:57.111056 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103800    2582 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 04 01:01:57.111056 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103803    2582 flags.go:64] FLAG: --logging-format="text"
Nov 04 01:01:57.111056 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103806    2582 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 04 01:01:57.111056 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103809    2582 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 04 01:01:57.111056 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103812    2582 flags.go:64] FLAG: --manifest-url=""
Nov 04 01:01:57.111056 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103816    2582 flags.go:64] FLAG: --manifest-url-header=""
Nov 04 01:01:57.111056 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103821    2582 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 04 01:01:57.111056 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103824    2582 flags.go:64] FLAG: --max-open-files="1000000"
Nov 04 01:01:57.111056 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103828    2582 flags.go:64] FLAG: --max-pods="110"
Nov 04 01:01:57.111056 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103831    2582 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 04 01:01:57.112106 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103834    2582 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 04 01:01:57.112106 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103837    2582 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 04 01:01:57.112106 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103840    2582 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 04 01:01:57.112106 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103843    2582 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 04 01:01:57.112106 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103846    2582 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 04 01:01:57.112106 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103849    2582 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 04 01:01:57.112106 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103856    2582 flags.go:64] FLAG: --node-status-max-images="50"
Nov 04 01:01:57.112106 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103859    2582 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 04 01:01:57.112106 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103862    2582 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 04 01:01:57.112106 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103866    2582 flags.go:64] FLAG: --pod-cidr=""
Nov 04 01:01:57.112106 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103868    2582 flags.go:64] FLAG: --pod-manifest-path=""
Nov 04 01:01:57.112106 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103871    2582 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 04 01:01:57.112106 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103874    2582 flags.go:64] FLAG: --pods-per-core="0"
Nov 04 01:01:57.112106 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103877    2582 flags.go:64] FLAG: --port="10250"
Nov 04 01:01:57.112106 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103880    2582 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 04 01:01:57.112106 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103883    2582 flags.go:64] FLAG: --provider-id="aws:///us-east-2a/i-06148214c3363a096"
Nov 04 01:01:57.112106 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103886    2582 flags.go:64] FLAG: --qos-reserved=""
Nov 04 01:01:57.112106 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103889    2582 flags.go:64] FLAG: --read-only-port="10255"
Nov 04 01:01:57.112106 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103892    2582 flags.go:64] FLAG: --register-node="true"
Nov 04 01:01:57.112106 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103895    2582 flags.go:64] FLAG: --register-with-taints=""
Nov 04 01:01:57.112106 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103898    2582 flags.go:64] FLAG: --registry-burst="10"
Nov 04 01:01:57.112106 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103901    2582 flags.go:64] FLAG: --registry-qps="5"
Nov 04 01:01:57.112106 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103904    2582 flags.go:64] FLAG: --reserved-cpus=""
Nov 04 01:01:57.112106 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103907    2582 flags.go:64] FLAG: --reserved-memory=""
Nov 04 01:01:57.112106 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103911    2582 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 04 01:01:57.112106 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103914    2582 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 04 01:01:57.113217 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103917    2582 flags.go:64] FLAG: --rotate-certificates="false"
Nov 04 01:01:57.113217 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103921    2582 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 04 01:01:57.113217 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103924    2582 flags.go:64] FLAG: --runonce="false"
Nov 04 01:01:57.113217 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103927    2582 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 04 01:01:57.113217 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103930    2582 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 04 01:01:57.113217 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103933    2582 flags.go:64] FLAG: --seccomp-default="false"
Nov 04 01:01:57.113217 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103937    2582 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 04 01:01:57.113217 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103940    2582 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 04 01:01:57.113217 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103943    2582 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 04 01:01:57.113217 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103946    2582 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 04 01:01:57.113217 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103949    2582 flags.go:64] FLAG: --storage-driver-password="root"
Nov 04 01:01:57.113217 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103952    2582 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 04 01:01:57.113217 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103955    2582 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 04 01:01:57.113217 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103958    2582 flags.go:64] FLAG: --storage-driver-user="root"
Nov 04 01:01:57.113217 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103961    2582 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 04 01:01:57.113217 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103964    2582 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 04 01:01:57.113217 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103967    2582 flags.go:64] FLAG: --system-cgroups=""
Nov 04 01:01:57.113217 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103969    2582 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 04 01:01:57.113217 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103974    2582 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 04 01:01:57.113217 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103977    2582 flags.go:64] FLAG: --tls-cert-file=""
Nov 04 01:01:57.113217 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103980    2582 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 04 01:01:57.113217 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103984    2582 flags.go:64] FLAG: --tls-min-version=""
Nov 04 01:01:57.113217 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103987    2582 flags.go:64] FLAG: --tls-private-key-file=""
Nov 04 01:01:57.113217 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103990    2582 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 04 01:01:57.113217 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103993    2582 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 04 01:01:57.114301 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103996    2582 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 04 01:01:57.114301 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.103999    2582 flags.go:64] FLAG: --v="2"
Nov 04 01:01:57.114301 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.104003    2582 flags.go:64] FLAG: --version="false"
Nov 04 01:01:57.114301 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.104007    2582 flags.go:64] FLAG: --vmodule=""
Nov 04 01:01:57.114301 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.104010    2582 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 04 01:01:57.114301 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.104014    2582 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 04 01:01:57.114301 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104149    2582 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:01:57.114301 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104154    2582 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:01:57.114301 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104159    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:01:57.114301 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104162    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:01:57.114301 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104165    2582 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:01:57.114301 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104168    2582 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:01:57.114301 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104170    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:01:57.114301 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104173    2582 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:01:57.114301 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104176    2582 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:01:57.114301 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104179    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:01:57.114301 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104181    2582 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:01:57.114301 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104184    2582 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:01:57.114301 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104186    2582 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:01:57.114301 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104189    2582 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:01:57.114301 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104191    2582 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:01:57.115320 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104194    2582 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:01:57.115320 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104198    2582 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:01:57.115320 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104201    2582 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:01:57.115320 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104204    2582 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:01:57.115320 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104206    2582 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:01:57.115320 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104209    2582 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:01:57.115320 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104212    2582 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:01:57.115320 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104214    2582 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:01:57.115320 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104217    2582 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:01:57.115320 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104219    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:01:57.115320 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104222    2582 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:01:57.115320 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104224    2582 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:01:57.115320 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104227    2582 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:01:57.115320 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104230    2582 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:01:57.115320 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104232    2582 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:01:57.115320 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104235    2582 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:01:57.115320 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104237    2582 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:01:57.115320 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104240    2582 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:01:57.115320 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104242    2582 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:01:57.115859 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104246    2582 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:01:57.115859 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104248    2582 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:01:57.115859 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104251    2582 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:01:57.115859 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104254    2582 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:01:57.115859 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104256    2582 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:01:57.115859 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104259    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:01:57.115859 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104261    2582 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:01:57.115859 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104264    2582 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:01:57.115859 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104267    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:01:57.115859 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104269    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:01:57.115859 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104272    2582 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:01:57.115859 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104274    2582 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:01:57.115859 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104277    2582 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:01:57.115859 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104280    2582 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:01:57.115859 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104284    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:01:57.115859 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104287    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:01:57.115859 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104289    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:01:57.115859 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104292    2582 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:01:57.115859 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104295    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:01:57.116564 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104297    2582 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:01:57.116564 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104299    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:01:57.116564 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104302    2582 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:01:57.116564 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104304    2582 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:01:57.116564 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104307    2582 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:01:57.116564 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104310    2582 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:01:57.116564 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104312    2582 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:01:57.116564 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104315    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:01:57.116564 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104318    2582 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:01:57.116564 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104320    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:01:57.116564 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104323    2582 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:01:57.116564 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104325    2582 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:01:57.116564 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104328    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:01:57.116564 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104332    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:01:57.116564 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104335    2582 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:01:57.116564 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104338    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:01:57.116564 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104340    2582 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:01:57.116564 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104343    2582 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:01:57.116564 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104345    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:01:57.117382 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104348    2582 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:01:57.117382 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104350    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:01:57.117382 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104353    2582 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:01:57.117382 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104356    2582 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:01:57.117382 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104358    2582 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:01:57.117382 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104361    2582 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:01:57.117382 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104364    2582 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:01:57.117382 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104366    2582 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:01:57.117382 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104370    2582 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:01:57.117382 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104373    2582 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:01:57.117382 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104376    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:01:57.117382 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104378    2582 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:01:57.117382 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104381    2582 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:01:57.117382 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104384    2582 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:01:57.117382 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104386    2582 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:01:57.117382 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104389    2582 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:01:57.117382 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104391    2582 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:01:57.117382 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104394    2582 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:01:57.117382 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104396    2582 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:01:57.117382 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104399    2582 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:01:57.117990 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104402    2582 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:01:57.117990 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104404    2582 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:01:57.117990 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104407    2582 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:01:57.117990 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104409    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:01:57.117990 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104412    2582 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:01:57.117990 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104414    2582 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:01:57.117990 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104418    2582 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:01:57.117990 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104421    2582 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:01:57.117990 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104423    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:01:57.117990 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104426    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:01:57.117990 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104428    2582 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:01:57.117990 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104432    2582 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:01:57.117990 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104435    2582 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:01:57.117990 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104438    2582 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:01:57.117990 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.104463    2582 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 01:01:57.117990 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104507    2582 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:01:57.118414 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104512    2582 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:01:57.118414 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104515    2582 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:01:57.118414 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104517    2582 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:01:57.118414 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104521    2582 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:01:57.118414 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104524    2582 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:01:57.118414 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104528    2582 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:01:57.118414 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104531    2582 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:01:57.118414 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104533    2582 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:01:57.118414 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104536    2582 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:01:57.118414 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104539    2582 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:01:57.118414 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104541    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:01:57.118414 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104544    2582 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:01:57.118414 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104547    2582 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:01:57.118414 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104549    2582 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:01:57.118414 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104552    2582 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:01:57.118414 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104554    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:01:57.118414 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104557    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:01:57.118414 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104560    2582 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:01:57.118414 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104564    2582 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:01:57.118414 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104567    2582 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:01:57.119072 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104570    2582 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:01:57.119072 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104573    2582 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:01:57.119072 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104579    2582 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:01:57.119072 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104583    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:01:57.119072 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104587    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:01:57.119072 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104589    2582 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:01:57.119072 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104592    2582 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:01:57.119072 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104595    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:01:57.119072 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104598    2582 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:01:57.119072 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104601    2582 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:01:57.119072 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104603    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:01:57.119072 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104617    2582 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:01:57.119072 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104621    2582 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:01:57.119072 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104623    2582 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:01:57.119072 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104626    2582 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:01:57.119072 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104629    2582 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:01:57.119072 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104631    2582 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:01:57.119072 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104635    2582 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:01:57.119072 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104637    2582 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:01:57.119521 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104640    2582 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:01:57.119521 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104643    2582 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:01:57.119521 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104645    2582 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:01:57.119521 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104648    2582 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:01:57.119521 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104651    2582 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:01:57.119521 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104653    2582 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:01:57.119521 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104656    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:01:57.119521 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104658    2582 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:01:57.119521 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104661    2582 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:01:57.119521 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104664    2582 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:01:57.119521 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104667    2582 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:01:57.119521 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104669    2582 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:01:57.119521 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104671    2582 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:01:57.119521 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104674    2582 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:01:57.119521 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104676    2582 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:01:57.119521 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104681    2582 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:01:57.119521 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104684    2582 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:01:57.119521 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104686    2582 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:01:57.119521 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104689    2582 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:01:57.119993 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104691    2582 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:01:57.119993 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104694    2582 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:01:57.119993 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104697    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:01:57.119993 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104699    2582 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:01:57.119993 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104702    2582 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:01:57.119993 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104704    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:01:57.119993 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104707    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:01:57.119993 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104710    2582 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:01:57.119993 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104712    2582 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:01:57.119993 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104715    2582 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:01:57.119993 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104718    2582 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:01:57.119993 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104722    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:01:57.119993 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104724    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:01:57.119993 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104727    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:01:57.119993 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104729    2582 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:01:57.119993 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104732    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:01:57.119993 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104734    2582 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:01:57.119993 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104737    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:01:57.119993 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104739    2582 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:01:57.120455 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104742    2582 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:01:57.120455 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104744    2582 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:01:57.120455 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104747    2582 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:01:57.120455 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104750    2582 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:01:57.120455 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104752    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:01:57.120455 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104755    2582 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:01:57.120455 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104757    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:01:57.120455 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104760    2582 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:01:57.120455 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104763    2582 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:01:57.120455 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104767    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:01:57.120455 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104769    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:01:57.120455 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104772    2582 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:01:57.120455 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104774    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:01:57.120455 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104777    2582 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:01:57.120455 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104780    2582 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:01:57.120455 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104782    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:01:57.120455 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104785    2582 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:01:57.120455 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104788    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:01:57.120455 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104791    2582 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:01:57.120951 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104793    2582 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:01:57.120951 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104796    2582 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:01:57.120951 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104799    2582 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:01:57.120951 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104801    2582 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:01:57.120951 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104804    2582 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:01:57.120951 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104808    2582 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:01:57.120951 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104811    2582 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:01:57.120951 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104814    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:01:57.120951 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.104817    2582 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:01:57.120951 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.113150    2582 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 04 01:01:57.120951 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.113171    2582 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 04 01:01:57.120951 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.113190    2582 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 04 01:01:57.120951 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.113199    2582 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 04 01:01:57.120951 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113283    2582 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:01:57.120951 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113292    2582 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:01:57.120951 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113297    2582 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:01:57.120951 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113303    2582 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:01:57.120951 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113307    2582 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:01:57.120951 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113311    2582 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:01:57.120951 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113316    2582 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:01:57.121422 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113320    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:01:57.121422 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113325    2582 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:01:57.121422 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113329    2582 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:01:57.121422 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113333    2582 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:01:57.121422 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113338    2582 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:01:57.121422 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113345    2582 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:01:57.121422 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113351    2582 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:01:57.121422 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113357    2582 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:01:57.121422 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113362    2582 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:01:57.121422 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113366    2582 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:01:57.121422 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113371    2582 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:01:57.121422 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113376    2582 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:01:57.121422 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113380    2582 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:01:57.121422 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113384    2582 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:01:57.121422 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113389    2582 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:01:57.121422 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113394    2582 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:01:57.121422 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113398    2582 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:01:57.121422 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113402    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:01:57.121422 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113407    2582 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:01:57.121925 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113411    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:01:57.121925 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113415    2582 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:01:57.121925 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113420    2582 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:01:57.121925 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113433    2582 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:01:57.121925 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113438    2582 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:01:57.121925 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113442    2582 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:01:57.121925 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113447    2582 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:01:57.121925 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113451    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:01:57.121925 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113455    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:01:57.121925 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113459    2582 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:01:57.121925 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113464    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:01:57.121925 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113468    2582 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:01:57.121925 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113473    2582 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:01:57.121925 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113477    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:01:57.121925 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113481    2582 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:01:57.121925 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113486    2582 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:01:57.121925 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113490    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:01:57.121925 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113494    2582 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:01:57.121925 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113498    2582 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:01:57.122368 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113503    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:01:57.122368 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113507    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:01:57.122368 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113511    2582 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:01:57.122368 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113515    2582 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:01:57.122368 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113520    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:01:57.122368 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113524    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:01:57.122368 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113529    2582 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:01:57.122368 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113533    2582 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:01:57.122368 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113538    2582 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:01:57.122368 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113542    2582 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:01:57.122368 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113547    2582 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:01:57.122368 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113551    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:01:57.122368 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113556    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:01:57.122368 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113560    2582 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:01:57.122368 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113564    2582 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:01:57.122368 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113569    2582 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:01:57.122368 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113573    2582 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:01:57.122368 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113588    2582 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:01:57.122368 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113593    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:01:57.122934 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113597    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:01:57.122934 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113602    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:01:57.122934 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113606    2582 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:01:57.122934 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113625    2582 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:01:57.122934 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113629    2582 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:01:57.122934 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113634    2582 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:01:57.122934 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113638    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:01:57.122934 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113642    2582 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:01:57.122934 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113647    2582 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:01:57.122934 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113650    2582 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:01:57.122934 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113655    2582 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:01:57.122934 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113659    2582 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:01:57.122934 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113663    2582 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:01:57.122934 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113667    2582 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:01:57.122934 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113672    2582 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:01:57.122934 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113676    2582 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:01:57.122934 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113680    2582 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:01:57.122934 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113684    2582 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:01:57.122934 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113688    2582 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:01:57.122934 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113713    2582 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:01:57.123386 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113720    2582 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:01:57.123386 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113725    2582 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:01:57.123386 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113730    2582 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:01:57.123386 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113735    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:01:57.123386 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113739    2582 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:01:57.123386 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113743    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:01:57.123386 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113749    2582 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:01:57.123386 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113755    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:01:57.123386 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113760    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:01:57.123386 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113764    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:01:57.123386 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113769    2582 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:01:57.123386 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113782    2582 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:01:57.123386 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113787    2582 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:01:57.123386 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113792    2582 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:01:57.123386 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113796    2582 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:01:57.123386 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113800    2582 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:01:57.123386 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113804    2582 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:01:57.123386 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113808    2582 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:01:57.123386 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113812    2582 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:01:57.124086 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113817    2582 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:01:57.124086 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113821    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:01:57.124086 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.113825    2582 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:01:57.124086 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.113867    2582 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 01:01:57.124086 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114146    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:01:57.124086 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114154    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:01:57.124086 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114160    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:01:57.124086 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114165    2582 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:01:57.124086 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114169    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:01:57.124086 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114175    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:01:57.124086 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114181    2582 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:01:57.124086 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114186    2582 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:01:57.124086 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114190    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:01:57.124086 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114194    2582 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:01:57.124086 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114199    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:01:57.124686 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114203    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:01:57.124686 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114208    2582 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:01:57.124686 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114212    2582 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:01:57.124686 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114216    2582 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:01:57.124686 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114221    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:01:57.124686 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114225    2582 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:01:57.124686 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114230    2582 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:01:57.124686 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114234    2582 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:01:57.124686 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114238    2582 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:01:57.124686 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114243    2582 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:01:57.124686 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114254    2582 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:01:57.124686 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114259    2582 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:01:57.124686 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114263    2582 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:01:57.124686 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114267    2582 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:01:57.124686 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114272    2582 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:01:57.124686 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114276    2582 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:01:57.124686 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114280    2582 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:01:57.124686 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114285    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:01:57.124686 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114289    2582 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:01:57.125401 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114293    2582 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:01:57.125401 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114297    2582 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:01:57.125401 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114301    2582 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:01:57.125401 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114305    2582 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:01:57.125401 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114309    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:01:57.125401 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114313    2582 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:01:57.125401 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114316    2582 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:01:57.125401 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114320    2582 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:01:57.125401 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114324    2582 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:01:57.125401 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114328    2582 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:01:57.125401 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114332    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:01:57.125401 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114337    2582 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:01:57.125401 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114341    2582 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:01:57.125401 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114345    2582 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:01:57.125401 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114349    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:01:57.125401 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114353    2582 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:01:57.125401 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114358    2582 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:01:57.125401 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114362    2582 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:01:57.125401 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114367    2582 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:01:57.125401 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114371    2582 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:01:57.125947 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114376    2582 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:01:57.125947 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114380    2582 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:01:57.125947 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114384    2582 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:01:57.125947 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114388    2582 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:01:57.125947 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114402    2582 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:01:57.125947 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114409    2582 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:01:57.125947 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114415    2582 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:01:57.125947 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114419    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:01:57.125947 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114424    2582 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:01:57.125947 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114429    2582 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:01:57.125947 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114433    2582 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:01:57.125947 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114437    2582 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:01:57.125947 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114441    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:01:57.125947 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114446    2582 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:01:57.125947 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114450    2582 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:01:57.125947 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114454    2582 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:01:57.125947 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114458    2582 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:01:57.125947 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114462    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:01:57.125947 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114467    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:01:57.126392 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114471    2582 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:01:57.126392 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114475    2582 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:01:57.126392 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114479    2582 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:01:57.126392 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114483    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:01:57.126392 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114487    2582 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:01:57.126392 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114491    2582 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:01:57.126392 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114505    2582 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:01:57.126392 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114510    2582 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:01:57.126392 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114514    2582 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:01:57.126392 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114519    2582 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:01:57.126392 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114523    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:01:57.126392 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114527    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:01:57.126392 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114532    2582 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:01:57.126392 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114536    2582 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:01:57.126392 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114540    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:01:57.126392 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114545    2582 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:01:57.126392 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114549    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:01:57.126392 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114553    2582 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:01:57.126392 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114564    2582 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:01:57.126392 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114569    2582 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:01:57.126918 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114573    2582 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:01:57.126918 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114577    2582 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:01:57.126918 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114581    2582 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:01:57.126918 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114585    2582 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:01:57.126918 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114589    2582 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:01:57.126918 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114594    2582 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:01:57.126918 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114598    2582 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:01:57.126918 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114604    2582 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:01:57.126918 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114625    2582 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:01:57.126918 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114631    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:01:57.126918 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114636    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:01:57.126918 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114640    2582 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:01:57.126918 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114644    2582 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:01:57.126918 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114648    2582 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:01:57.126918 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114652    2582 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:01:57.126918 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114656    2582 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:01:57.126918 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114659    2582 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:01:57.127303 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.114697    2582 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 01:01:57.127303 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114762    2582 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:01:57.127303 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114769    2582 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:01:57.127303 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114773    2582 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:01:57.127303 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114777    2582 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:01:57.127303 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114782    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:01:57.127303 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114786    2582 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:01:57.127303 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114791    2582 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:01:57.127303 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114799    2582 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:01:57.127303 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114803    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:01:57.127303 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114807    2582 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:01:57.127303 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114811    2582 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:01:57.127303 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114816    2582 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:01:57.127303 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114820    2582 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:01:57.127303 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114832    2582 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:01:57.127303 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114837    2582 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:01:57.127700 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114842    2582 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:01:57.127700 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114846    2582 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:01:57.127700 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114850    2582 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:01:57.127700 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114857    2582 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:01:57.127700 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114864    2582 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:01:57.127700 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114869    2582 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:01:57.127700 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114874    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:01:57.127700 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114879    2582 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:01:57.127700 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114884    2582 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:01:57.127700 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114888    2582 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:01:57.127700 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114893    2582 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:01:57.127700 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114897    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:01:57.127700 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114901    2582 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:01:57.127700 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114906    2582 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:01:57.127700 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114910    2582 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:01:57.127700 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114914    2582 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:01:57.127700 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114919    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:01:57.127700 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114923    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:01:57.127700 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114927    2582 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:01:57.128154 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114932    2582 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:01:57.128154 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114936    2582 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:01:57.128154 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114940    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:01:57.128154 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114944    2582 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:01:57.128154 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114949    2582 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:01:57.128154 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114955    2582 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:01:57.128154 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114959    2582 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:01:57.128154 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114964    2582 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:01:57.128154 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114968    2582 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:01:57.128154 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114972    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:01:57.128154 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114976    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:01:57.128154 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114980    2582 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:01:57.128154 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114993    2582 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:01:57.128154 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.114998    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:01:57.128154 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115002    2582 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:01:57.128154 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115007    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:01:57.128154 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115011    2582 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:01:57.128154 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115015    2582 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:01:57.128154 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115019    2582 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:01:57.128154 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115023    2582 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:01:57.128621 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115027    2582 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:01:57.128621 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115032    2582 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:01:57.128621 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115038    2582 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:01:57.128621 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115043    2582 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:01:57.128621 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115048    2582 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:01:57.128621 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115053    2582 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:01:57.128621 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115059    2582 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:01:57.128621 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115064    2582 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:01:57.128621 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115069    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:01:57.128621 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115073    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:01:57.128621 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115078    2582 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:01:57.128621 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115082    2582 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:01:57.128621 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115087    2582 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:01:57.128621 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115091    2582 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:01:57.128621 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115095    2582 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:01:57.128621 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115099    2582 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:01:57.128621 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115103    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:01:57.128621 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115109    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:01:57.128621 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115114    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:01:57.129060 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115118    2582 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:01:57.129060 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115123    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:01:57.129060 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115127    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:01:57.129060 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115131    2582 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:01:57.129060 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115135    2582 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:01:57.129060 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115139    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:01:57.129060 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115151    2582 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:01:57.129060 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115155    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:01:57.129060 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115160    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:01:57.129060 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115164    2582 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:01:57.129060 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115168    2582 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:01:57.129060 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115173    2582 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:01:57.129060 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115177    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:01:57.129060 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115181    2582 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:01:57.129060 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115186    2582 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:01:57.129060 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115191    2582 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:01:57.129060 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115196    2582 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:01:57.129060 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115200    2582 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:01:57.129060 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115204    2582 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:01:57.129543 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115209    2582 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:01:57.129543 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115213    2582 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:01:57.129543 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115217    2582 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:01:57.129543 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115221    2582 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:01:57.129543 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115226    2582 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:01:57.129543 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115230    2582 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:01:57.129543 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115235    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:01:57.129543 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115239    2582 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:01:57.129543 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115243    2582 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:01:57.129543 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115247    2582 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:01:57.129543 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115252    2582 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:01:57.129543 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115257    2582 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:01:57.129543 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115262    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:01:57.129543 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:57.115266    2582 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:01:57.129543 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.115344    2582 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 04 01:01:57.129543 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.118424    2582 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Nov 04 01:01:57.129543 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.120425    2582 server.go:1014] "Starting client certificate rotation"
Nov 04 01:01:57.129543 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.120542    2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 04 01:01:57.129543 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.120586    2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 04 01:01:57.130031 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.126239    2582 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 04 01:01:57.130031 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.129264    2582 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 04 01:01:57.139675 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.139653    2582 log.go:25] "Validated CRI v1 runtime API"
Nov 04 01:01:57.145863 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.145847    2582 log.go:25] "Validated CRI v1 image API"
Nov 04 01:01:57.146700 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.146670    2582 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 04 01:01:57.150828 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.150809    2582 fs.go:135] Filesystem UUIDs: map[2f245827-4bad-472e-b665-4f970b464ebf:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 917c0278-c1e3-466a-b9cc-4bd2d67be92e:/dev/nvme0n1p3]
Nov 04 01:01:57.150925 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.150830    2582 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Nov 04 01:01:57.156397 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.156285    2582 manager.go:217] Machine: {Timestamp:2026-11-04 01:01:57.155048722 +0000 UTC m=+0.380896898 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3097650 MemoryCapacity:33164341248 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e12c6543deb065ef9c9649fccaefe SystemUUID:ec2e12c6-543d-eb06-5ef9-c9649fccaefe BootID:4583952d-5bf2-4dfd-be58-c8de4cec02ad Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582172672 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:c6:b1:41:ef Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:c6:b1:41:ef Speed:0 Mtu:9001} {Name:ovs-system MacAddress:6e:29:90:5d:8c:22 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164341248 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 04 01:01:57.156959 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.156946    2582 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 04 01:01:57.157066 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.157050    2582 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 04 01:01:57.158904 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.158888    2582 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 04 01:01:57.160698 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.160669    2582 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 04 01:01:57.160828 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.160701    2582 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-67.us-east-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 04 01:01:57.160875 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.160830    2582 topology_manager.go:143] "Creating topology manager with none policy"
Nov 04 01:01:57.160875 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.160838    2582 container_manager_linux.go:311] "Creating device plugin manager"
Nov 04 01:01:57.160875 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.160848    2582 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 04 01:01:57.161762 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.161751    2582 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 04 01:01:57.162868 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.162858    2582 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 04 01:01:57.167459 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.167447    2582 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 04 01:01:57.167575 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.167566    2582 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 04 01:01:57.167632 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.167595    2582 kubelet.go:493] "Attempting to sync node with API server"
Nov 04 01:01:57.167632 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.167604    2582 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 04 01:01:57.167632 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.167629    2582 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 04 01:01:57.167740 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.167640    2582 kubelet.go:405] "Adding apiserver pod source"
Nov 04 01:01:57.167740 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.167652    2582 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 04 01:01:57.168074 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.168058    2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 04 01:01:57.172088 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.172070    2582 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 04 01:01:57.178533 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.178518    2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 04 01:01:57.179402 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.179390    2582 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 04 01:01:57.179441 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.179420    2582 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 04 01:01:57.179570 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.179560    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 04 01:01:57.179603 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.179575    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 04 01:01:57.179603 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.179581    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 04 01:01:57.179603 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.179587    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 04 01:01:57.179603 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.179593    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 04 01:01:57.179603 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.179599    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 04 01:01:57.180335 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.180326    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 04 01:01:57.180362 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.180336    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 04 01:01:57.180362 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.180343    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 04 01:01:57.180362 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.180350    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 04 01:01:57.180362 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.180358    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 04 01:01:57.180550 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.180366    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 04 01:01:57.182289 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.182279    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 04 01:01:57.182322 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.182291    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 04 01:01:57.186272 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.186247    2582 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-67.us-east-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Nov 04 01:01:57.186374 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.186290    2582 server.go:1263] "Started kubelet"
Nov 04 01:01:57.186551 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.186384    2582 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 04 01:01:57.186753 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.186694    2582 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 04 01:01:57.186845 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.186775    2582 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 04 01:01:57.187757 ip-10-0-132-67 systemd[1]: Started Kubernetes Kubelet.
Nov 04 01:01:57.188193 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.188152    2582 server.go:317] "Adding debug handlers to kubelet server"
Nov 04 01:01:57.188368 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.188352    2582 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 04 01:01:57.192888 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:01:57.192870    2582 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Nov 04 01:01:57.195255 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.195237    2582 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 04 01:01:57.195346 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.195247    2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 04 01:01:57.195422 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:01:57.195398    2582 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-67.us-east-2.compute.internal\" not found"
Nov 04 01:01:57.196400 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.196382    2582 factory.go:55] Registering systemd factory
Nov 04 01:01:57.196400 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.196402    2582 factory.go:223] Registration of the systemd container factory successfully
Nov 04 01:01:57.196656 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:01:57.196590    2582 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-67.us-east-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Nov 04 01:01:57.196740 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.196662    2582 factory.go:153] Registering CRI-O factory
Nov 04 01:01:57.196740 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.196675    2582 factory.go:223] Registration of the crio container factory successfully
Nov 04 01:01:57.196740 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.196716    2582 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 04 01:01:57.196740 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.196733    2582 factory.go:103] Registering Raw factory
Nov 04 01:01:57.196905 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.196746    2582 manager.go:1199] Started watching for new ooms in manager
Nov 04 01:01:57.196905 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.196814    2582 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 04 01:01:57.196905 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.196820    2582 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 04 01:01:57.196905 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.196837    2582 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 04 01:01:57.196905 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:01:57.196817    2582 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Nov 04 01:01:57.197051 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.196939    2582 reconstruct.go:97] "Volume reconstruction finished"
Nov 04 01:01:57.197051 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.196947    2582 reconciler.go:29] "Reconciler: start to sync state"
Nov 04 01:01:57.197460 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:01:57.196405    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-67.us-east-2.compute.internal.18e4b1dda94817f6  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-67.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-67.us-east-2.compute.internal,},FirstTimestamp:2026-11-04 01:01:57.18626303 +0000 UTC m=+0.412111208,LastTimestamp:2026-11-04 01:01:57.18626303 +0000 UTC m=+0.412111208,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-67.us-east-2.compute.internal,}"
Nov 04 01:01:57.197853 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.197839    2582 manager.go:319] Starting recovery of all containers
Nov 04 01:01:57.201704 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:01:57.201670    2582 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-67.us-east-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Nov 04 01:01:57.206171 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:01:57.206142    2582 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver"
Nov 04 01:01:57.210477 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.210452    2582 manager.go:324] Recovery completed
Nov 04 01:01:57.214815 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.214423    2582 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:01:57.216395 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.216378    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-67.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:01:57.216483 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.216406    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-67.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:01:57.216483 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.216417    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-67.us-east-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:01:57.217002 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.216985    2582 cpu_manager.go:226] "Starting" policy="none"
Nov 04 01:01:57.217002 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.216993    2582 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:01:57.217002 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.216998    2582 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 04 01:01:57.217176 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.217016    2582 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 04 01:01:57.217297 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.217278    2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-v4g2h"
Nov 04 01:01:57.217728 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.217712    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-67.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:01:57.217803 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.217744    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-67.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:01:57.217803 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.217763    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-67.us-east-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:01:57.218935 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:01:57.218848    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-67.us-east-2.compute.internal.18e4b1ddab13d7b6  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-67.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-67.us-east-2.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-67.us-east-2.compute.internal,},FirstTimestamp:2026-11-04 01:01:57.216393142 +0000 UTC m=+0.442241319,LastTimestamp:2026-11-04 01:01:57.216393142 +0000 UTC m=+0.442241319,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-67.us-east-2.compute.internal,}"
Nov 04 01:01:57.219213 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.219200    2582 policy_none.go:50] "Start"
Nov 04 01:01:57.219272 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.219217    2582 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 04 01:01:57.219272 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.219230    2582 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 04 01:01:57.221243 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.221230    2582 policy_none.go:44] "Start"
Nov 04 01:01:57.229312 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.229296    2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-v4g2h"
Nov 04 01:01:57.234116 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:01:57.234036    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-67.us-east-2.compute.internal.18e4b1ddab141c1e  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-67.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-132-67.us-east-2.compute.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-132-67.us-east-2.compute.internal,},FirstTimestamp:2026-11-04 01:01:57.216410654 +0000 UTC m=+0.442258831,LastTimestamp:2026-11-04 01:01:57.216410654 +0000 UTC m=+0.442258831,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-67.us-east-2.compute.internal,}"
Nov 04 01:01:57.252177 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.252144    2582 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 04 01:01:57.254533 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.254521    2582 manager.go:341] "Starting Device Plugin manager"
Nov 04 01:01:57.254630 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:01:57.254554    2582 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 04 01:01:57.254630 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.254564    2582 server.go:87] "Starting device plugin registration server"
Nov 04 01:01:57.254812 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.254801    2582 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 04 01:01:57.254852 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.254815    2582 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 04 01:01:57.254940 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.254922    2582 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 04 01:01:57.255012 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.255004    2582 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 04 01:01:57.255079 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.255012    2582 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 04 01:01:57.255405 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:01:57.255383    2582 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 04 01:01:57.255482 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:01:57.255438    2582 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-67.us-east-2.compute.internal\" not found"
Nov 04 01:01:57.355291 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.355232    2582 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:01:57.356136 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.356121    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-67.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:01:57.356211 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.356151    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-67.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:01:57.356211 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.356166    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-67.us-east-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:01:57.356211 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.356191    2582 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-132-67.us-east-2.compute.internal"
Nov 04 01:01:57.366259 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.366241    2582 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-132-67.us-east-2.compute.internal"
Nov 04 01:01:57.366259 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:01:57.366259    2582 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-67.us-east-2.compute.internal\": node \"ip-10-0-132-67.us-east-2.compute.internal\" not found"
Nov 04 01:01:57.394758 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:01:57.394738    2582 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-67.us-east-2.compute.internal\" not found"
Nov 04 01:01:57.495317 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:01:57.495160    2582 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-67.us-east-2.compute.internal\" not found"
Nov 04 01:01:57.588837 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.588804    2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 04 01:01:57.590024 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.589996    2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 04 01:01:57.590024 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.590020    2582 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 04 01:01:57.590158 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.590045    2582 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 04 01:01:57.590721 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:01:57.590079    2582 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Nov 04 01:01:57.595268 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.595251    2582 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 01:01:57.596274 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:01:57.596262    2582 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-67.us-east-2.compute.internal\" not found"
Nov 04 01:01:57.690822 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.690761    2582 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-67.us-east-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-67.us-east-2.compute.internal"]
Nov 04 01:01:57.690914 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.690834    2582 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:01:57.692014 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.691998    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-67.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:01:57.692084 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.692034    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-67.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:01:57.692084 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.692050    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-67.us-east-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:01:57.694199 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.694188    2582 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:01:57.694328 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.694314    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-67.us-east-2.compute.internal"
Nov 04 01:01:57.694373 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.694342    2582 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:01:57.695045 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.695026    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-67.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:01:57.695147 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.695057    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-67.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:01:57.695147 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.695071    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-67.us-east-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:01:57.695147 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.695030    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-67.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:01:57.695147 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.695136    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-67.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:01:57.695147 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.695147    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-67.us-east-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:01:57.697017 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:01:57.697004    2582 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-67.us-east-2.compute.internal\" not found"
Nov 04 01:01:57.697238 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.697222    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-67.us-east-2.compute.internal"
Nov 04 01:01:57.697309 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.697253    2582 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:01:57.699425 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.699410    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/210e3b81ce84c7df763e2a095c51a6d5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-67.us-east-2.compute.internal\" (UID: \"210e3b81ce84c7df763e2a095c51a6d5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-67.us-east-2.compute.internal"
Nov 04 01:01:57.699501 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.699438    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9589190378fb255196269e8a3bdf442c-config\") pod \"kube-apiserver-proxy-ip-10-0-132-67.us-east-2.compute.internal\" (UID: \"9589190378fb255196269e8a3bdf442c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-67.us-east-2.compute.internal"
Nov 04 01:01:57.699501 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.699463    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/210e3b81ce84c7df763e2a095c51a6d5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-67.us-east-2.compute.internal\" (UID: \"210e3b81ce84c7df763e2a095c51a6d5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-67.us-east-2.compute.internal"
Nov 04 01:01:57.700041 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.700023    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-67.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:01:57.700131 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.700049    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-67.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:01:57.700131 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.700059    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-67.us-east-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:01:57.721923 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:01:57.721901    2582 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-67.us-east-2.compute.internal\" not found" node="ip-10-0-132-67.us-east-2.compute.internal"
Nov 04 01:01:57.725151 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:01:57.725133    2582 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-67.us-east-2.compute.internal\" not found" node="ip-10-0-132-67.us-east-2.compute.internal"
Nov 04 01:01:57.798022 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:01:57.797999    2582 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-67.us-east-2.compute.internal\" not found"
Nov 04 01:01:57.800175 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.800156    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/210e3b81ce84c7df763e2a095c51a6d5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-67.us-east-2.compute.internal\" (UID: \"210e3b81ce84c7df763e2a095c51a6d5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-67.us-east-2.compute.internal"
Nov 04 01:01:57.800217 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.800184    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/210e3b81ce84c7df763e2a095c51a6d5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-67.us-east-2.compute.internal\" (UID: \"210e3b81ce84c7df763e2a095c51a6d5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-67.us-east-2.compute.internal"
Nov 04 01:01:57.800217 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.800201    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9589190378fb255196269e8a3bdf442c-config\") pod \"kube-apiserver-proxy-ip-10-0-132-67.us-east-2.compute.internal\" (UID: \"9589190378fb255196269e8a3bdf442c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-67.us-east-2.compute.internal"
Nov 04 01:01:57.800282 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.800245    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9589190378fb255196269e8a3bdf442c-config\") pod \"kube-apiserver-proxy-ip-10-0-132-67.us-east-2.compute.internal\" (UID: \"9589190378fb255196269e8a3bdf442c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-67.us-east-2.compute.internal"
Nov 04 01:01:57.800282 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.800267    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/210e3b81ce84c7df763e2a095c51a6d5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-67.us-east-2.compute.internal\" (UID: \"210e3b81ce84c7df763e2a095c51a6d5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-67.us-east-2.compute.internal"
Nov 04 01:01:57.800381 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:57.800269    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/210e3b81ce84c7df763e2a095c51a6d5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-67.us-east-2.compute.internal\" (UID: \"210e3b81ce84c7df763e2a095c51a6d5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-67.us-east-2.compute.internal"
Nov 04 01:01:57.899449 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:01:57.899411    2582 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-67.us-east-2.compute.internal\" not found"
Nov 04 01:01:57.999542 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:01:57.999479    2582 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-67.us-east-2.compute.internal\" not found"
Nov 04 01:01:58.025656 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:58.025633    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-67.us-east-2.compute.internal"
Nov 04 01:01:58.027889 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:58.027871    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-67.us-east-2.compute.internal"
Nov 04 01:01:58.099878 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:01:58.099851    2582 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-67.us-east-2.compute.internal\" not found"
Nov 04 01:01:58.120311 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:58.120292    2582 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Nov 04 01:01:58.120422 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:58.120405    2582 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Nov 04 01:01:58.200785 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:01:58.200767    2582 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-67.us-east-2.compute.internal\" not found"
Nov 04 01:01:58.215972 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:58.215956    2582 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 01:01:58.235681 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:58.235605    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-03 00:56:57 +0000 UTC" deadline="2028-06-08 06:55:35.772585456 +0000 UTC"
Nov 04 01:01:58.235756 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:58.235705    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13973h53m37.536910047s"
Nov 04 01:01:58.301394 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:01:58.301372    2582 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-67.us-east-2.compute.internal\" not found"
Nov 04 01:01:58.383497 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:58.383475    2582 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:01:58.384609 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:58.384589    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-67.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:01:58.384712 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:58.384636    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-67.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:01:58.384712 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:58.384648    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-67.us-east-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:01:58.401717 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:01:58.401695    2582 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-67.us-east-2.compute.internal\" not found"
Nov 04 01:01:58.502806 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:01:58.502774    2582 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-67.us-east-2.compute.internal\" not found"
Nov 04 01:01:58.603558 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:01:58.603496    2582 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-67.us-east-2.compute.internal\" not found"
Nov 04 01:01:58.620550 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:58.620527    2582 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 01:01:58.704035 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:01:58.704010    2582 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-67.us-east-2.compute.internal\" not found"
Nov 04 01:01:58.790035 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:58.790011    2582 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 01:01:58.796131 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:58.796109    2582 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-67.us-east-2.compute.internal"
Nov 04 01:01:58.811282 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:58.811262    2582 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 01:01:58.812110 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:58.812096    2582 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-67.us-east-2.compute.internal"
Nov 04 01:01:58.821878 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:58.821859    2582 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 01:01:58.881530 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:58.881498    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod210e3b81ce84c7df763e2a095c51a6d5.slice/crio-f0dffe660314752a2e102bfc7eed66425cddd61b4ca0027f4132de7c475c8af2 WatchSource:0}: Error finding container f0dffe660314752a2e102bfc7eed66425cddd61b4ca0027f4132de7c475c8af2: Status 404 returned error can't find the container with id f0dffe660314752a2e102bfc7eed66425cddd61b4ca0027f4132de7c475c8af2
Nov 04 01:01:58.892659 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:58.892641    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 01:01:58.899914 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:58.899891    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9589190378fb255196269e8a3bdf442c.slice/crio-ff43c868abd19cc8cc3756d6c7b1acaeafa2c2dc60a90562bb5df20889f60d6b WatchSource:0}: Error finding container ff43c868abd19cc8cc3756d6c7b1acaeafa2c2dc60a90562bb5df20889f60d6b: Status 404 returned error can't find the container with id ff43c868abd19cc8cc3756d6c7b1acaeafa2c2dc60a90562bb5df20889f60d6b
Nov 04 01:01:59.171926 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.171861    2582 apiserver.go:52] "Watching apiserver"
Nov 04 01:01:59.180912 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.180883    2582 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 04 01:01:59.181245 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.181224    2582 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-fvjq7","openshift-cluster-node-tuning-operator/tuned-pwv64","openshift-network-diagnostics/network-check-target-wxhng","openshift-ovn-kubernetes/ovnkube-node-sz7gg","kube-system/konnectivity-agent-sjcm7","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-67.us-east-2.compute.internal","openshift-multus/network-metrics-daemon-rx7gq","openshift-network-operator/iptables-alerter-l7n65","kube-system/kube-apiserver-proxy-ip-10-0-132-67.us-east-2.compute.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt2x6","openshift-multus/multus-additional-cni-plugins-qlr2s","openshift-multus/multus-jfd2m","kube-system/global-pull-secret-syncer-gnrzw","openshift-dns/node-resolver-8gvq8"]
Nov 04 01:01:59.185020 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.184999    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx7gq"
Nov 04 01:01:59.185103 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:01:59.185065    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rx7gq" podUID="a409c58b-b4eb-428f-9177-a09b8a229ec7"
Nov 04 01:01:59.187494 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.187475    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gnrzw"
Nov 04 01:01:59.187576 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:01:59.187524    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gnrzw" podUID="6be44550-8625-4296-aeb9-da833aaa4b2f"
Nov 04 01:01:59.189741 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.189727    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wxhng"
Nov 04 01:01:59.189834 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:01:59.189767    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wxhng" podUID="588c643c-b16a-4d5c-ad85-ccb8d486dd43"
Nov 04 01:01:59.192234 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.192215    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.194547 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.194524    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-sjcm7"
Nov 04 01:01:59.196178 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.196162    2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Nov 04 01:01:59.196701 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.196666    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 04 01:01:59.196793 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.196722    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 04 01:01:59.196893 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.196873    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 04 01:01:59.196951 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.196920    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8gvq8"
Nov 04 01:01:59.196951 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.196944    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 04 01:01:59.197077 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.197059    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 04 01:01:59.198373 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.198353    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-dh9vh\""
Nov 04 01:01:59.198458 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.198441    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 04 01:01:59.198508 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.198448    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 04 01:01:59.198508 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.198485    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-n5n45\""
Nov 04 01:01:59.198572 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.198530    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 04 01:01:59.199330 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.199311    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-l7n65"
Nov 04 01:01:59.201612 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.201590    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 04 01:01:59.201750 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.201696    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-pvmvv\""
Nov 04 01:01:59.201812 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.201754    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-fvjq7"
Nov 04 01:01:59.206358 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.206340    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:01:59.206768 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.206744    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-run-systemd\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.206875 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.206771    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-host-cni-bin\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.206875 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.206809    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-host-cni-netd\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.206875 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.206848    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-ovn-node-metrics-cert\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.207040 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.206914    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6a4c158-e115-4d2f-af0d-1d9249dc7384-tmp-dir\") pod \"node-resolver-8gvq8\" (UID: \"d6a4c158-e115-4d2f-af0d-1d9249dc7384\") " pod="openshift-dns/node-resolver-8gvq8"
Nov 04 01:01:59.207040 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.206940    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8r85c\" (UniqueName: \"kubernetes.io/projected/a409c58b-b4eb-428f-9177-a09b8a229ec7-kube-api-access-8r85c\") pod \"network-metrics-daemon-rx7gq\" (UID: \"a409c58b-b4eb-428f-9177-a09b8a229ec7\") " pod="openshift-multus/network-metrics-daemon-rx7gq"
Nov 04 01:01:59.207040 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.206961    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6be44550-8625-4296-aeb9-da833aaa4b2f-dbus\") pod \"global-pull-secret-syncer-gnrzw\" (UID: \"6be44550-8625-4296-aeb9-da833aaa4b2f\") " pod="kube-system/global-pull-secret-syncer-gnrzw"
Nov 04 01:01:59.207040 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.206976    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-host-slash\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.207040 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.207026    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-log-socket\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.207311 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.207062    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-env-overrides\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.207311 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.207087    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3a53d2f9-8ce8-4ae5-9974-a8c865065ff3-iptables-alerter-script\") pod \"iptables-alerter-l7n65\" (UID: \"3a53d2f9-8ce8-4ae5-9974-a8c865065ff3\") " pod="openshift-network-operator/iptables-alerter-l7n65"
Nov 04 01:01:59.207311 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.207111    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6be44550-8625-4296-aeb9-da833aaa4b2f-kubelet-config\") pod \"global-pull-secret-syncer-gnrzw\" (UID: \"6be44550-8625-4296-aeb9-da833aaa4b2f\") " pod="kube-system/global-pull-secret-syncer-gnrzw"
Nov 04 01:01:59.207311 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.207136    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-systemd-units\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.207311 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.207169    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-etc-openvswitch\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.207311 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.207206    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-run-ovn\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.207311 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.207228    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wsn9\" (UniqueName: \"kubernetes.io/projected/588c643c-b16a-4d5c-ad85-ccb8d486dd43-kube-api-access-4wsn9\") pod \"network-check-target-wxhng\" (UID: \"588c643c-b16a-4d5c-ad85-ccb8d486dd43\") " pod="openshift-network-diagnostics/network-check-target-wxhng"
Nov 04 01:01:59.207311 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.207260    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3f74f502-9d11-4533-8bdf-cf880a0fbe0c-konnectivity-ca\") pod \"konnectivity-agent-sjcm7\" (UID: \"3f74f502-9d11-4533-8bdf-cf880a0fbe0c\") " pod="kube-system/konnectivity-agent-sjcm7"
Nov 04 01:01:59.207311 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.207286    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-node-log\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.207807 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.207337    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wj8p2\" (UniqueName: \"kubernetes.io/projected/d6a4c158-e115-4d2f-af0d-1d9249dc7384-kube-api-access-wj8p2\") pod \"node-resolver-8gvq8\" (UID: \"d6a4c158-e115-4d2f-af0d-1d9249dc7384\") " pod="openshift-dns/node-resolver-8gvq8"
Nov 04 01:01:59.207807 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.207380    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-host-run-netns\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.207807 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.207431    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-run-openvswitch\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.207807 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.207466    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-host-run-ovn-kubernetes\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.207807 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.207494    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.207807 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.207533    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vgx6\" (UniqueName: \"kubernetes.io/projected/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-kube-api-access-8vgx6\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.207807 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.207566    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-var-lib-openvswitch\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.207807 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.207594    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3a53d2f9-8ce8-4ae5-9974-a8c865065ff3-host-slash\") pod \"iptables-alerter-l7n65\" (UID: \"3a53d2f9-8ce8-4ae5-9974-a8c865065ff3\") " pod="openshift-network-operator/iptables-alerter-l7n65"
Nov 04 01:01:59.207807 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.207634    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-ovnkube-script-lib\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.207807 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.207691    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3f74f502-9d11-4533-8bdf-cf880a0fbe0c-agent-certs\") pod \"konnectivity-agent-sjcm7\" (UID: \"3f74f502-9d11-4533-8bdf-cf880a0fbe0c\") " pod="kube-system/konnectivity-agent-sjcm7"
Nov 04 01:01:59.207807 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.207723    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpjd9\" (UniqueName: \"kubernetes.io/projected/3a53d2f9-8ce8-4ae5-9974-a8c865065ff3-kube-api-access-mpjd9\") pod \"iptables-alerter-l7n65\" (UID: \"3a53d2f9-8ce8-4ae5-9974-a8c865065ff3\") " pod="openshift-network-operator/iptables-alerter-l7n65"
Nov 04 01:01:59.207807 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.207774    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-host-kubelet\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.207807 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.207801    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-ovnkube-config\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.208577 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.207827    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d6a4c158-e115-4d2f-af0d-1d9249dc7384-hosts-file\") pod \"node-resolver-8gvq8\" (UID: \"d6a4c158-e115-4d2f-af0d-1d9249dc7384\") " pod="openshift-dns/node-resolver-8gvq8"
Nov 04 01:01:59.208577 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.207854    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a409c58b-b4eb-428f-9177-a09b8a229ec7-metrics-certs\") pod \"network-metrics-daemon-rx7gq\" (UID: \"a409c58b-b4eb-428f-9177-a09b8a229ec7\") " pod="openshift-multus/network-metrics-daemon-rx7gq"
Nov 04 01:01:59.208577 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.207890    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6be44550-8625-4296-aeb9-da833aaa4b2f-original-pull-secret\") pod \"global-pull-secret-syncer-gnrzw\" (UID: \"6be44550-8625-4296-aeb9-da833aaa4b2f\") " pod="kube-system/global-pull-secret-syncer-gnrzw"
Nov 04 01:01:59.209257 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.209232    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt2x6"
Nov 04 01:01:59.211312 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.211294    2582 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 04 01:01:59.211563 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.211545    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qlr2s"
Nov 04 01:01:59.214009 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.213990    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jfd2m"
Nov 04 01:01:59.234276 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.234256    2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-2z5xm"
Nov 04 01:01:59.246893 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.246876    2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-2z5xm"
Nov 04 01:01:59.297971 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.297952    2582 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 04 01:01:59.306882 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.306828    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 04 01:01:59.307041 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.307021    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 04 01:01:59.307133 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.307044    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 04 01:01:59.307133 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.307029    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 04 01:01:59.307133 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.307020    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 04 01:01:59.307287 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.307154    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 04 01:01:59.307287 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.307160    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 04 01:01:59.307287 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.307154    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-rq94r\""
Nov 04 01:01:59.307287 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.307197    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 04 01:01:59.307287 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.307204    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zw2l8\""
Nov 04 01:01:59.307287 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.307251    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 04 01:01:59.307910 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.307896    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 04 01:01:59.307971 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.307939    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-mhcr5\""
Nov 04 01:01:59.307971 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.307960    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 04 01:01:59.308052 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.307995    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 04 01:01:59.308090 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.308066    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3f74f502-9d11-4533-8bdf-cf880a0fbe0c-konnectivity-ca\") pod \"konnectivity-agent-sjcm7\" (UID: \"3f74f502-9d11-4533-8bdf-cf880a0fbe0c\") " pod="kube-system/konnectivity-agent-sjcm7"
Nov 04 01:01:59.308125 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.308088    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-tmp\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:01:59.308125 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.308109    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4965a588-c16b-488b-82c7-b53bb390f7d0-device-dir\") pod \"aws-ebs-csi-driver-node-tt2x6\" (UID: \"4965a588-c16b-488b-82c7-b53bb390f7d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt2x6"
Nov 04 01:01:59.308219 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.308131    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-host-var-lib-cni-bin\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:01:59.308219 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.308169    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 04 01:01:59.308219 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.308198    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-host-var-lib-kubelet\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:01:59.308423 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.308403    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-node-log\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.308484 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.308442    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-etc-tuned\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:01:59.308484 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.308472    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-cnibin\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:01:59.308578 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.308518    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-host-var-lib-cni-multus\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:01:59.308578 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.308508    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-node-log\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.308578 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.308565    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-multus-conf-dir\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:01:59.308578 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.308574    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 04 01:01:59.308791 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.308574    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-h8pc2\""
Nov 04 01:01:59.308791 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.308657    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-run-openvswitch\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.308791 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.308710    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 04 01:01:59.308791 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.308736    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-host-run-ovn-kubernetes\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.308996 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.308801    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 04 01:01:59.308996 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.308801    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-run-openvswitch\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.308996 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.308862    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-host-run-ovn-kubernetes\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.308996 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.308864    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 04 01:01:59.308996 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.308893    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.308996 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.308869    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3f74f502-9d11-4533-8bdf-cf880a0fbe0c-konnectivity-ca\") pod \"konnectivity-agent-sjcm7\" (UID: \"3f74f502-9d11-4533-8bdf-cf880a0fbe0c\") " pod="kube-system/konnectivity-agent-sjcm7"
Nov 04 01:01:59.309302 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.309052    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 04 01:01:59.309302 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.309065    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.309302 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.309064    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-78n7j\""
Nov 04 01:01:59.309302 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.309069    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-sys\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:01:59.309302 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.309120    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-cfm9p\""
Nov 04 01:01:59.309302 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.309124    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-host\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:01:59.309302 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.309142    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b3bd337e-c495-4349-b56d-c16ebe3a9b11-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qlr2s\" (UID: \"b3bd337e-c495-4349-b56d-c16ebe3a9b11\") " pod="openshift-multus/multus-additional-cni-plugins-qlr2s"
Nov 04 01:01:59.309302 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.309162    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b3bd337e-c495-4349-b56d-c16ebe3a9b11-cni-binary-copy\") pod \"multus-additional-cni-plugins-qlr2s\" (UID: \"b3bd337e-c495-4349-b56d-c16ebe3a9b11\") " pod="openshift-multus/multus-additional-cni-plugins-qlr2s"
Nov 04 01:01:59.309302 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.309214    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b3bd337e-c495-4349-b56d-c16ebe3a9b11-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qlr2s\" (UID: \"b3bd337e-c495-4349-b56d-c16ebe3a9b11\") " pod="openshift-multus/multus-additional-cni-plugins-qlr2s"
Nov 04 01:01:59.309893 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.309318    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4965a588-c16b-488b-82c7-b53bb390f7d0-socket-dir\") pod \"aws-ebs-csi-driver-node-tt2x6\" (UID: \"4965a588-c16b-488b-82c7-b53bb390f7d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt2x6"
Nov 04 01:01:59.310638 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.310572    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-ovnkube-script-lib\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.310746 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.310726    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-ovnkube-script-lib\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.310826 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.310781    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpjd9\" (UniqueName: \"kubernetes.io/projected/3a53d2f9-8ce8-4ae5-9974-a8c865065ff3-kube-api-access-mpjd9\") pod \"iptables-alerter-l7n65\" (UID: \"3a53d2f9-8ce8-4ae5-9974-a8c865065ff3\") " pod="openshift-network-operator/iptables-alerter-l7n65"
Nov 04 01:01:59.310905 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.310836    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b3bd337e-c495-4349-b56d-c16ebe3a9b11-system-cni-dir\") pod \"multus-additional-cni-plugins-qlr2s\" (UID: \"b3bd337e-c495-4349-b56d-c16ebe3a9b11\") " pod="openshift-multus/multus-additional-cni-plugins-qlr2s"
Nov 04 01:01:59.310905 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.310868    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-host-kubelet\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.311029 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.310903    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a409c58b-b4eb-428f-9177-a09b8a229ec7-metrics-certs\") pod \"network-metrics-daemon-rx7gq\" (UID: \"a409c58b-b4eb-428f-9177-a09b8a229ec7\") " pod="openshift-multus/network-metrics-daemon-rx7gq"
Nov 04 01:01:59.311029 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.310981    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-etc-kubernetes\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:01:59.311157 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.311051    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-host-kubelet\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.311157 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.311054    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-host-cni-bin\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.311157 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.311118    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-host-cni-bin\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.311297 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:01:59.311208    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:01:59.311297 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.311264    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6a4c158-e115-4d2f-af0d-1d9249dc7384-tmp-dir\") pod \"node-resolver-8gvq8\" (UID: \"d6a4c158-e115-4d2f-af0d-1d9249dc7384\") " pod="openshift-dns/node-resolver-8gvq8"
Nov 04 01:01:59.311387 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.311344    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8r85c\" (UniqueName: \"kubernetes.io/projected/a409c58b-b4eb-428f-9177-a09b8a229ec7-kube-api-access-8r85c\") pod \"network-metrics-daemon-rx7gq\" (UID: \"a409c58b-b4eb-428f-9177-a09b8a229ec7\") " pod="openshift-multus/network-metrics-daemon-rx7gq"
Nov 04 01:01:59.311513 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.311460    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-etc-modprobe-d\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:01:59.311589 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.311539    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b3bd337e-c495-4349-b56d-c16ebe3a9b11-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qlr2s\" (UID: \"b3bd337e-c495-4349-b56d-c16ebe3a9b11\") " pod="openshift-multus/multus-additional-cni-plugins-qlr2s"
Nov 04 01:01:59.311589 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.311573    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-system-cni-dir\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:01:59.311734 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:01:59.311620    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a409c58b-b4eb-428f-9177-a09b8a229ec7-metrics-certs podName:a409c58b-b4eb-428f-9177-a09b8a229ec7 nodeName:}" failed. No retries permitted until 2026-11-04 01:01:59.811594482 +0000 UTC m=+3.037442671 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a409c58b-b4eb-428f-9177-a09b8a229ec7-metrics-certs") pod "network-metrics-daemon-rx7gq" (UID: "a409c58b-b4eb-428f-9177-a09b8a229ec7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:01:59.311843 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.311811    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-host-run-k8s-cni-cncf-io\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:01:59.312981 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.312124    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-log-socket\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.312981 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.312172    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7b044258-a241-4bd2-a6eb-9ed586a0f2cc-serviceca\") pod \"node-ca-fvjq7\" (UID: \"7b044258-a241-4bd2-a6eb-9ed586a0f2cc\") " pod="openshift-image-registry/node-ca-fvjq7"
Nov 04 01:01:59.312981 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.312228    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b3bd337e-c495-4349-b56d-c16ebe3a9b11-cnibin\") pod \"multus-additional-cni-plugins-qlr2s\" (UID: \"b3bd337e-c495-4349-b56d-c16ebe3a9b11\") " pod="openshift-multus/multus-additional-cni-plugins-qlr2s"
Nov 04 01:01:59.312981 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.312261    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-systemd-units\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.312981 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.312290    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-etc-openvswitch\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.312981 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.312338    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-etc-sysconfig\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:01:59.312981 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.312341    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6a4c158-e115-4d2f-af0d-1d9249dc7384-tmp-dir\") pod \"node-resolver-8gvq8\" (UID: \"d6a4c158-e115-4d2f-af0d-1d9249dc7384\") " pod="openshift-dns/node-resolver-8gvq8"
Nov 04 01:01:59.312981 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.312454    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-etc-kubernetes\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:01:59.312981 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.312497    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-etc-sysctl-conf\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:01:59.312981 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.312539    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/80a681c1-e74e-4887-a56d-6be313103581-cni-binary-copy\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:01:59.312981 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.312565    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wj8p2\" (UniqueName: \"kubernetes.io/projected/d6a4c158-e115-4d2f-af0d-1d9249dc7384-kube-api-access-wj8p2\") pod \"node-resolver-8gvq8\" (UID: \"d6a4c158-e115-4d2f-af0d-1d9249dc7384\") " pod="openshift-dns/node-resolver-8gvq8"
Nov 04 01:01:59.312981 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.312599    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-var-lib-kubelet\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:01:59.312981 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.312665    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-host-run-netns\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.312981 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.312698    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vgx6\" (UniqueName: \"kubernetes.io/projected/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-kube-api-access-8vgx6\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.312981 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.312851    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-etc-sysctl-d\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:01:59.313608 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.312971    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-etc-systemd\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:01:59.313608 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.313025    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-etc-openvswitch\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.313608 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.313053    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-log-socket\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.313608 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.313130    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-systemd-units\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.313608 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.313280    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-run\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:01:59.313608 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.313274    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-host-run-netns\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.313608 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.313387    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqnfn\" (UniqueName: \"kubernetes.io/projected/4965a588-c16b-488b-82c7-b53bb390f7d0-kube-api-access-xqnfn\") pod \"aws-ebs-csi-driver-node-tt2x6\" (UID: \"4965a588-c16b-488b-82c7-b53bb390f7d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt2x6"
Nov 04 01:01:59.313608 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.313452    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-hostroot\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:01:59.313608 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.313521    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-var-lib-openvswitch\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.313608 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.313589    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3a53d2f9-8ce8-4ae5-9974-a8c865065ff3-host-slash\") pod \"iptables-alerter-l7n65\" (UID: \"3a53d2f9-8ce8-4ae5-9974-a8c865065ff3\") " pod="openshift-network-operator/iptables-alerter-l7n65"
Nov 04 01:01:59.314046 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.313650    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3a53d2f9-8ce8-4ae5-9974-a8c865065ff3-host-slash\") pod \"iptables-alerter-l7n65\" (UID: \"3a53d2f9-8ce8-4ae5-9974-a8c865065ff3\") " pod="openshift-network-operator/iptables-alerter-l7n65"
Nov 04 01:01:59.314046 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.313693    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pksw4\" (UniqueName: \"kubernetes.io/projected/7b044258-a241-4bd2-a6eb-9ed586a0f2cc-kube-api-access-pksw4\") pod \"node-ca-fvjq7\" (UID: \"7b044258-a241-4bd2-a6eb-9ed586a0f2cc\") " pod="openshift-image-registry/node-ca-fvjq7"
Nov 04 01:01:59.314046 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.313708    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-var-lib-openvswitch\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.314046 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.313766    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4965a588-c16b-488b-82c7-b53bb390f7d0-sys-fs\") pod \"aws-ebs-csi-driver-node-tt2x6\" (UID: \"4965a588-c16b-488b-82c7-b53bb390f7d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt2x6"
Nov 04 01:01:59.314046 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.313815    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/80a681c1-e74e-4887-a56d-6be313103581-multus-daemon-config\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:01:59.314346 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.314308    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-host-run-multus-certs\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:01:59.314405 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.314368    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-499cg\" (UniqueName: \"kubernetes.io/projected/80a681c1-e74e-4887-a56d-6be313103581-kube-api-access-499cg\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:01:59.314451 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.314428    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3f74f502-9d11-4533-8bdf-cf880a0fbe0c-agent-certs\") pod \"konnectivity-agent-sjcm7\" (UID: \"3f74f502-9d11-4533-8bdf-cf880a0fbe0c\") " pod="kube-system/konnectivity-agent-sjcm7"
Nov 04 01:01:59.314499 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.314475    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-ovnkube-config\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.314556 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.314497    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d6a4c158-e115-4d2f-af0d-1d9249dc7384-hosts-file\") pod \"node-resolver-8gvq8\" (UID: \"d6a4c158-e115-4d2f-af0d-1d9249dc7384\") " pod="openshift-dns/node-resolver-8gvq8"
Nov 04 01:01:59.314556 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.314518    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9n9lx\" (UniqueName: \"kubernetes.io/projected/b3bd337e-c495-4349-b56d-c16ebe3a9b11-kube-api-access-9n9lx\") pod \"multus-additional-cni-plugins-qlr2s\" (UID: \"b3bd337e-c495-4349-b56d-c16ebe3a9b11\") " pod="openshift-multus/multus-additional-cni-plugins-qlr2s"
Nov 04 01:01:59.314556 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.314537    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-multus-socket-dir-parent\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:01:59.314556 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.314554    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6be44550-8625-4296-aeb9-da833aaa4b2f-original-pull-secret\") pod \"global-pull-secret-syncer-gnrzw\" (UID: \"6be44550-8625-4296-aeb9-da833aaa4b2f\") " pod="kube-system/global-pull-secret-syncer-gnrzw"
Nov 04 01:01:59.314774 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.314569    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-run-systemd\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.314774 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:01:59.314650    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 01:01:59.314774 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.314657    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d6a4c158-e115-4d2f-af0d-1d9249dc7384-hosts-file\") pod \"node-resolver-8gvq8\" (UID: \"d6a4c158-e115-4d2f-af0d-1d9249dc7384\") " pod="openshift-dns/node-resolver-8gvq8"
Nov 04 01:01:59.314774 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:01:59.314702    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6be44550-8625-4296-aeb9-da833aaa4b2f-original-pull-secret podName:6be44550-8625-4296-aeb9-da833aaa4b2f nodeName:}" failed. No retries permitted until 2026-11-04 01:01:59.814686475 +0000 UTC m=+3.040534656 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6be44550-8625-4296-aeb9-da833aaa4b2f-original-pull-secret") pod "global-pull-secret-syncer-gnrzw" (UID: "6be44550-8625-4296-aeb9-da833aaa4b2f") : object "kube-system"/"original-pull-secret" not registered
Nov 04 01:01:59.314774 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.314695    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-host-cni-netd\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.314774 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.314734    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-ovn-node-metrics-cert\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.314774 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.314741    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-host-cni-netd\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.314774 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.314761    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7b044258-a241-4bd2-a6eb-9ed586a0f2cc-host\") pod \"node-ca-fvjq7\" (UID: \"7b044258-a241-4bd2-a6eb-9ed586a0f2cc\") " pod="openshift-image-registry/node-ca-fvjq7"
Nov 04 01:01:59.314774 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.314744    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-run-systemd\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.315186 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.314790    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-lib-modules\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:01:59.315186 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.314818    2582 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 04 01:01:59.315186 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.314841    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p559g\" (UniqueName: \"kubernetes.io/projected/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-kube-api-access-p559g\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:01:59.315186 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.314912    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4965a588-c16b-488b-82c7-b53bb390f7d0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tt2x6\" (UID: \"4965a588-c16b-488b-82c7-b53bb390f7d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt2x6"
Nov 04 01:01:59.315186 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.314940    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6be44550-8625-4296-aeb9-da833aaa4b2f-dbus\") pod \"global-pull-secret-syncer-gnrzw\" (UID: \"6be44550-8625-4296-aeb9-da833aaa4b2f\") " pod="kube-system/global-pull-secret-syncer-gnrzw"
Nov 04 01:01:59.315186 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.314964    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-host-slash\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.315186 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.314984    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-env-overrides\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.315186 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.314999    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3a53d2f9-8ce8-4ae5-9974-a8c865065ff3-iptables-alerter-script\") pod \"iptables-alerter-l7n65\" (UID: \"3a53d2f9-8ce8-4ae5-9974-a8c865065ff3\") " pod="openshift-network-operator/iptables-alerter-l7n65"
Nov 04 01:01:59.315186 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.315021    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4965a588-c16b-488b-82c7-b53bb390f7d0-registration-dir\") pod \"aws-ebs-csi-driver-node-tt2x6\" (UID: \"4965a588-c16b-488b-82c7-b53bb390f7d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt2x6"
Nov 04 01:01:59.315186 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.315046    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4965a588-c16b-488b-82c7-b53bb390f7d0-etc-selinux\") pod \"aws-ebs-csi-driver-node-tt2x6\" (UID: \"4965a588-c16b-488b-82c7-b53bb390f7d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt2x6"
Nov 04 01:01:59.315186 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.315073    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-host-slash\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.315186 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.315123    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6be44550-8625-4296-aeb9-da833aaa4b2f-dbus\") pod \"global-pull-secret-syncer-gnrzw\" (UID: \"6be44550-8625-4296-aeb9-da833aaa4b2f\") " pod="kube-system/global-pull-secret-syncer-gnrzw"
Nov 04 01:01:59.315186 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.315127    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-multus-cni-dir\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:01:59.315186 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.315180    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-host-run-netns\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:01:59.315839 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.315216    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6be44550-8625-4296-aeb9-da833aaa4b2f-kubelet-config\") pod \"global-pull-secret-syncer-gnrzw\" (UID: \"6be44550-8625-4296-aeb9-da833aaa4b2f\") " pod="kube-system/global-pull-secret-syncer-gnrzw"
Nov 04 01:01:59.315839 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.315269    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-run-ovn\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.315839 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.315311    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6be44550-8625-4296-aeb9-da833aaa4b2f-kubelet-config\") pod \"global-pull-secret-syncer-gnrzw\" (UID: \"6be44550-8625-4296-aeb9-da833aaa4b2f\") " pod="kube-system/global-pull-secret-syncer-gnrzw"
Nov 04 01:01:59.315839 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.315358    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wsn9\" (UniqueName: \"kubernetes.io/projected/588c643c-b16a-4d5c-ad85-ccb8d486dd43-kube-api-access-4wsn9\") pod \"network-check-target-wxhng\" (UID: \"588c643c-b16a-4d5c-ad85-ccb8d486dd43\") " pod="openshift-network-diagnostics/network-check-target-wxhng"
Nov 04 01:01:59.315839 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.315480    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-run-ovn\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.315839 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.315514    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-env-overrides\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.315839 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.315520    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-ovnkube-config\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.315839 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.315646    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3a53d2f9-8ce8-4ae5-9974-a8c865065ff3-iptables-alerter-script\") pod \"iptables-alerter-l7n65\" (UID: \"3a53d2f9-8ce8-4ae5-9974-a8c865065ff3\") " pod="openshift-network-operator/iptables-alerter-l7n65"
Nov 04 01:01:59.318219 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.318199    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-ovn-node-metrics-cert\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.318303 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.318287    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3f74f502-9d11-4533-8bdf-cf880a0fbe0c-agent-certs\") pod \"konnectivity-agent-sjcm7\" (UID: \"3f74f502-9d11-4533-8bdf-cf880a0fbe0c\") " pod="kube-system/konnectivity-agent-sjcm7"
Nov 04 01:01:59.324405 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.324381    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpjd9\" (UniqueName: \"kubernetes.io/projected/3a53d2f9-8ce8-4ae5-9974-a8c865065ff3-kube-api-access-mpjd9\") pod \"iptables-alerter-l7n65\" (UID: \"3a53d2f9-8ce8-4ae5-9974-a8c865065ff3\") " pod="openshift-network-operator/iptables-alerter-l7n65"
Nov 04 01:01:59.324601 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.324585    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8r85c\" (UniqueName: \"kubernetes.io/projected/a409c58b-b4eb-428f-9177-a09b8a229ec7-kube-api-access-8r85c\") pod \"network-metrics-daemon-rx7gq\" (UID: \"a409c58b-b4eb-428f-9177-a09b8a229ec7\") " pod="openshift-multus/network-metrics-daemon-rx7gq"
Nov 04 01:01:59.325376 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:01:59.325353    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 01:01:59.325376 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:01:59.325372    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 01:01:59.325507 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:01:59.325381    2582 projected.go:196] Error preparing data for projected volume kube-api-access-4wsn9 for pod openshift-network-diagnostics/network-check-target-wxhng: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:01:59.325507 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:01:59.325419    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/588c643c-b16a-4d5c-ad85-ccb8d486dd43-kube-api-access-4wsn9 podName:588c643c-b16a-4d5c-ad85-ccb8d486dd43 nodeName:}" failed. No retries permitted until 2026-11-04 01:01:59.825409046 +0000 UTC m=+3.051257225 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-4wsn9" (UniqueName: "kubernetes.io/projected/588c643c-b16a-4d5c-ad85-ccb8d486dd43-kube-api-access-4wsn9") pod "network-check-target-wxhng" (UID: "588c643c-b16a-4d5c-ad85-ccb8d486dd43") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:01:59.325738 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.325720    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wj8p2\" (UniqueName: \"kubernetes.io/projected/d6a4c158-e115-4d2f-af0d-1d9249dc7384-kube-api-access-wj8p2\") pod \"node-resolver-8gvq8\" (UID: \"d6a4c158-e115-4d2f-af0d-1d9249dc7384\") " pod="openshift-dns/node-resolver-8gvq8"
Nov 04 01:01:59.325941 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.325919    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vgx6\" (UniqueName: \"kubernetes.io/projected/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-kube-api-access-8vgx6\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.369795 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.369776    2582 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 01:01:59.415674 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.415647    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-host-run-k8s-cni-cncf-io\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:01:59.415854 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.415682    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7b044258-a241-4bd2-a6eb-9ed586a0f2cc-serviceca\") pod \"node-ca-fvjq7\" (UID: \"7b044258-a241-4bd2-a6eb-9ed586a0f2cc\") " pod="openshift-image-registry/node-ca-fvjq7"
Nov 04 01:01:59.415854 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.415707    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b3bd337e-c495-4349-b56d-c16ebe3a9b11-cnibin\") pod \"multus-additional-cni-plugins-qlr2s\" (UID: \"b3bd337e-c495-4349-b56d-c16ebe3a9b11\") " pod="openshift-multus/multus-additional-cni-plugins-qlr2s"
Nov 04 01:01:59.415854 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.415732    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-etc-sysconfig\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:01:59.415854 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.415709    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-host-run-k8s-cni-cncf-io\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:01:59.415854 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.415752    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-etc-kubernetes\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:01:59.415854 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.415772    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b3bd337e-c495-4349-b56d-c16ebe3a9b11-cnibin\") pod \"multus-additional-cni-plugins-qlr2s\" (UID: \"b3bd337e-c495-4349-b56d-c16ebe3a9b11\") " pod="openshift-multus/multus-additional-cni-plugins-qlr2s"
Nov 04 01:01:59.415854 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.415773    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-etc-sysctl-conf\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:01:59.415854 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.415850    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-etc-kubernetes\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:01:59.416274 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.415858    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/80a681c1-e74e-4887-a56d-6be313103581-cni-binary-copy\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:01:59.416274 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.415892    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-var-lib-kubelet\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:01:59.416274 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.415912    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-etc-sysctl-d\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:01:59.416274 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.415933    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-etc-systemd\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:01:59.416274 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.415955    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-run\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:01:59.416274 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.415971    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-etc-sysconfig\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:01:59.416274 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.415975    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqnfn\" (UniqueName: \"kubernetes.io/projected/4965a588-c16b-488b-82c7-b53bb390f7d0-kube-api-access-xqnfn\") pod \"aws-ebs-csi-driver-node-tt2x6\" (UID: \"4965a588-c16b-488b-82c7-b53bb390f7d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt2x6"
Nov 04 01:01:59.416274 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.416012    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-hostroot\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:01:59.416274 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.416051    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pksw4\" (UniqueName: \"kubernetes.io/projected/7b044258-a241-4bd2-a6eb-9ed586a0f2cc-kube-api-access-pksw4\") pod \"node-ca-fvjq7\" (UID: \"7b044258-a241-4bd2-a6eb-9ed586a0f2cc\") " pod="openshift-image-registry/node-ca-fvjq7"
Nov 04 01:01:59.416274 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.416062    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-hostroot\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:01:59.416274 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.416066    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-etc-systemd\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:01:59.416274 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.416078    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4965a588-c16b-488b-82c7-b53bb390f7d0-sys-fs\") pod \"aws-ebs-csi-driver-node-tt2x6\" (UID: \"4965a588-c16b-488b-82c7-b53bb390f7d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt2x6"
Nov 04 01:01:59.416274 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.416109    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-var-lib-kubelet\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:01:59.416274 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.416123    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/80a681c1-e74e-4887-a56d-6be313103581-multus-daemon-config\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:01:59.416274 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.416149    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-host-run-multus-certs\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:01:59.416274 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.416170    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-499cg\" (UniqueName: \"kubernetes.io/projected/80a681c1-e74e-4887-a56d-6be313103581-kube-api-access-499cg\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:01:59.416274 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.416191    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9n9lx\" (UniqueName: \"kubernetes.io/projected/b3bd337e-c495-4349-b56d-c16ebe3a9b11-kube-api-access-9n9lx\") pod \"multus-additional-cni-plugins-qlr2s\" (UID: \"b3bd337e-c495-4349-b56d-c16ebe3a9b11\") " pod="openshift-multus/multus-additional-cni-plugins-qlr2s"
Nov 04 01:01:59.416274 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.416207    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-multus-socket-dir-parent\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:01:59.416984 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.416211    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-run\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:01:59.416984 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.416244    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-host-run-multus-certs\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:01:59.416984 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.416257    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-multus-socket-dir-parent\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:01:59.416984 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.415933    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-etc-sysctl-conf\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:01:59.416984 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.416257    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-etc-sysctl-d\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:01:59.416984 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.416287    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7b044258-a241-4bd2-a6eb-9ed586a0f2cc-host\") pod \"node-ca-fvjq7\" (UID: \"7b044258-a241-4bd2-a6eb-9ed586a0f2cc\") " pod="openshift-image-registry/node-ca-fvjq7"
Nov 04 01:01:59.416984 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.416304    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4965a588-c16b-488b-82c7-b53bb390f7d0-sys-fs\") pod \"aws-ebs-csi-driver-node-tt2x6\" (UID: \"4965a588-c16b-488b-82c7-b53bb390f7d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt2x6"
Nov 04 01:01:59.416984 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.416315    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-lib-modules\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:01:59.416984 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.416331    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p559g\" (UniqueName: \"kubernetes.io/projected/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-kube-api-access-p559g\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:01:59.416984 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.416360    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4965a588-c16b-488b-82c7-b53bb390f7d0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tt2x6\" (UID: \"4965a588-c16b-488b-82c7-b53bb390f7d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt2x6"
Nov 04 01:01:59.416984 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.416398    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4965a588-c16b-488b-82c7-b53bb390f7d0-registration-dir\") pod \"aws-ebs-csi-driver-node-tt2x6\" (UID: \"4965a588-c16b-488b-82c7-b53bb390f7d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt2x6"
Nov 04 01:01:59.416984 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.416415    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4965a588-c16b-488b-82c7-b53bb390f7d0-etc-selinux\") pod \"aws-ebs-csi-driver-node-tt2x6\" (UID: \"4965a588-c16b-488b-82c7-b53bb390f7d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt2x6"
Nov 04 01:01:59.416984 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.416430    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-multus-cni-dir\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:01:59.416984 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.416452    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-host-run-netns\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:01:59.416984 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.416475    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-lib-modules\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:01:59.416984 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.416489    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-tmp\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:01:59.416984 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.416503    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7b044258-a241-4bd2-a6eb-9ed586a0f2cc-host\") pod \"node-ca-fvjq7\" (UID: \"7b044258-a241-4bd2-a6eb-9ed586a0f2cc\") " pod="openshift-image-registry/node-ca-fvjq7"
Nov 04 01:01:59.416984 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.416506    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4965a588-c16b-488b-82c7-b53bb390f7d0-registration-dir\") pod \"aws-ebs-csi-driver-node-tt2x6\" (UID: \"4965a588-c16b-488b-82c7-b53bb390f7d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt2x6"
Nov 04 01:01:59.417882 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.416552    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4965a588-c16b-488b-82c7-b53bb390f7d0-device-dir\") pod \"aws-ebs-csi-driver-node-tt2x6\" (UID: \"4965a588-c16b-488b-82c7-b53bb390f7d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt2x6"
Nov 04 01:01:59.417882 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.416554    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-multus-cni-dir\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:01:59.417882 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.416568    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4965a588-c16b-488b-82c7-b53bb390f7d0-etc-selinux\") pod \"aws-ebs-csi-driver-node-tt2x6\" (UID: \"4965a588-c16b-488b-82c7-b53bb390f7d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt2x6"
Nov 04 01:01:59.417882 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.416634    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-host-run-netns\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:01:59.417882 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.416504    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4965a588-c16b-488b-82c7-b53bb390f7d0-device-dir\") pod \"aws-ebs-csi-driver-node-tt2x6\" (UID: \"4965a588-c16b-488b-82c7-b53bb390f7d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt2x6"
Nov 04 01:01:59.417882 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.416746    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7b044258-a241-4bd2-a6eb-9ed586a0f2cc-serviceca\") pod \"node-ca-fvjq7\" (UID: \"7b044258-a241-4bd2-a6eb-9ed586a0f2cc\") " pod="openshift-image-registry/node-ca-fvjq7"
Nov 04 01:01:59.417882 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.416745    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-host-var-lib-cni-bin\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:01:59.417882 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.416785    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-host-var-lib-kubelet\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:01:59.417882 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.416784    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-host-var-lib-cni-bin\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:01:59.417882 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.416806    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-etc-tuned\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:01:59.417882 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.416846    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-cnibin\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:01:59.417882 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.416704    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4965a588-c16b-488b-82c7-b53bb390f7d0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tt2x6\" (UID: \"4965a588-c16b-488b-82c7-b53bb390f7d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt2x6"
Nov 04 01:01:59.417882 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.416895    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-host-var-lib-cni-multus\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:01:59.417882 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.416924    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-multus-conf-dir\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:01:59.417882 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.416937    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-host-var-lib-kubelet\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:01:59.417882 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.416951    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-sys\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:01:59.417882 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.416976    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-host\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:01:59.417882 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.416996    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-host-var-lib-cni-multus\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:01:59.418461 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.417002    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b3bd337e-c495-4349-b56d-c16ebe3a9b11-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qlr2s\" (UID: \"b3bd337e-c495-4349-b56d-c16ebe3a9b11\") " pod="openshift-multus/multus-additional-cni-plugins-qlr2s"
Nov 04 01:01:59.418461 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.417030    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b3bd337e-c495-4349-b56d-c16ebe3a9b11-cni-binary-copy\") pod \"multus-additional-cni-plugins-qlr2s\" (UID: \"b3bd337e-c495-4349-b56d-c16ebe3a9b11\") " pod="openshift-multus/multus-additional-cni-plugins-qlr2s"
Nov 04 01:01:59.418461 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.417043    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/80a681c1-e74e-4887-a56d-6be313103581-cni-binary-copy\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:01:59.418461 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.417034    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-cnibin\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:01:59.418461 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.417058    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b3bd337e-c495-4349-b56d-c16ebe3a9b11-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qlr2s\" (UID: \"b3bd337e-c495-4349-b56d-c16ebe3a9b11\") " pod="openshift-multus/multus-additional-cni-plugins-qlr2s"
Nov 04 01:01:59.418461 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.417046    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-multus-conf-dir\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:01:59.418461 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.417111    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-sys\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:01:59.418461 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.417104    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-host\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:01:59.418461 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.417173    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b3bd337e-c495-4349-b56d-c16ebe3a9b11-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qlr2s\" (UID: \"b3bd337e-c495-4349-b56d-c16ebe3a9b11\") " pod="openshift-multus/multus-additional-cni-plugins-qlr2s"
Nov 04 01:01:59.418461 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.417204    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4965a588-c16b-488b-82c7-b53bb390f7d0-socket-dir\") pod \"aws-ebs-csi-driver-node-tt2x6\" (UID: \"4965a588-c16b-488b-82c7-b53bb390f7d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt2x6"
Nov 04 01:01:59.418461 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.417194    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/80a681c1-e74e-4887-a56d-6be313103581-multus-daemon-config\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:01:59.418461 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.417243    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b3bd337e-c495-4349-b56d-c16ebe3a9b11-system-cni-dir\") pod \"multus-additional-cni-plugins-qlr2s\" (UID: \"b3bd337e-c495-4349-b56d-c16ebe3a9b11\") " pod="openshift-multus/multus-additional-cni-plugins-qlr2s"
Nov 04 01:01:59.418461 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.417285    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-etc-kubernetes\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:01:59.418461 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.417299    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4965a588-c16b-488b-82c7-b53bb390f7d0-socket-dir\") pod \"aws-ebs-csi-driver-node-tt2x6\" (UID: \"4965a588-c16b-488b-82c7-b53bb390f7d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt2x6"
Nov 04 01:01:59.418461 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.417313    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-etc-modprobe-d\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:01:59.418461 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.417339    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b3bd337e-c495-4349-b56d-c16ebe3a9b11-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qlr2s\" (UID: \"b3bd337e-c495-4349-b56d-c16ebe3a9b11\") " pod="openshift-multus/multus-additional-cni-plugins-qlr2s"
Nov 04 01:01:59.418461 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.417349    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b3bd337e-c495-4349-b56d-c16ebe3a9b11-system-cni-dir\") pod \"multus-additional-cni-plugins-qlr2s\" (UID: \"b3bd337e-c495-4349-b56d-c16ebe3a9b11\") " pod="openshift-multus/multus-additional-cni-plugins-qlr2s"
Nov 04 01:01:59.418987 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.417362    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-system-cni-dir\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:01:59.418987 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.417351    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-etc-kubernetes\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:01:59.418987 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.417402    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-etc-modprobe-d\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:01:59.418987 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.417474    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-system-cni-dir\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:01:59.418987 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.417673    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b3bd337e-c495-4349-b56d-c16ebe3a9b11-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qlr2s\" (UID: \"b3bd337e-c495-4349-b56d-c16ebe3a9b11\") " pod="openshift-multus/multus-additional-cni-plugins-qlr2s"
Nov 04 01:01:59.418987 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.417820    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b3bd337e-c495-4349-b56d-c16ebe3a9b11-cni-binary-copy\") pod \"multus-additional-cni-plugins-qlr2s\" (UID: \"b3bd337e-c495-4349-b56d-c16ebe3a9b11\") " pod="openshift-multus/multus-additional-cni-plugins-qlr2s"
Nov 04 01:01:59.418987 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.417924    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b3bd337e-c495-4349-b56d-c16ebe3a9b11-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qlr2s\" (UID: \"b3bd337e-c495-4349-b56d-c16ebe3a9b11\") " pod="openshift-multus/multus-additional-cni-plugins-qlr2s"
Nov 04 01:01:59.419879 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.419855    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-etc-tuned\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:01:59.419879 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.419878    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-tmp\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:01:59.429640 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.429571    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-499cg\" (UniqueName: \"kubernetes.io/projected/80a681c1-e74e-4887-a56d-6be313103581-kube-api-access-499cg\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:01:59.429807 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.429774    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9n9lx\" (UniqueName: \"kubernetes.io/projected/b3bd337e-c495-4349-b56d-c16ebe3a9b11-kube-api-access-9n9lx\") pod \"multus-additional-cni-plugins-qlr2s\" (UID: \"b3bd337e-c495-4349-b56d-c16ebe3a9b11\") " pod="openshift-multus/multus-additional-cni-plugins-qlr2s"
Nov 04 01:01:59.429895 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.429822    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p559g\" (UniqueName: \"kubernetes.io/projected/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-kube-api-access-p559g\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:01:59.429953 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.429936    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pksw4\" (UniqueName: \"kubernetes.io/projected/7b044258-a241-4bd2-a6eb-9ed586a0f2cc-kube-api-access-pksw4\") pod \"node-ca-fvjq7\" (UID: \"7b044258-a241-4bd2-a6eb-9ed586a0f2cc\") " pod="openshift-image-registry/node-ca-fvjq7"
Nov 04 01:01:59.429989 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.429967    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqnfn\" (UniqueName: \"kubernetes.io/projected/4965a588-c16b-488b-82c7-b53bb390f7d0-kube-api-access-xqnfn\") pod \"aws-ebs-csi-driver-node-tt2x6\" (UID: \"4965a588-c16b-488b-82c7-b53bb390f7d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt2x6"
Nov 04 01:01:59.519852 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.519832    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-sjcm7"
Nov 04 01:01:59.522129 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.522112    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8gvq8"
Nov 04 01:01:59.527751 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.527728    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:01:59.528792 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.528775    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-l7n65"
Nov 04 01:01:59.531428 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:59.531405    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f74f502_9d11_4533_8bdf_cf880a0fbe0c.slice/crio-f9d36abdc505c14e0014478293acdb283ec8f81dde22697d6ffe71873e2c62f2 WatchSource:0}: Error finding container f9d36abdc505c14e0014478293acdb283ec8f81dde22697d6ffe71873e2c62f2: Status 404 returned error can't find the container with id f9d36abdc505c14e0014478293acdb283ec8f81dde22697d6ffe71873e2c62f2
Nov 04 01:01:59.532254 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.532233    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-fvjq7"
Nov 04 01:01:59.535152 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:59.535115    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6a4c158_e115_4d2f_af0d_1d9249dc7384.slice/crio-4a430137277d3d191db07d8b5f20cb298c9ff6674f04c7635fd64f98c46fc91c WatchSource:0}: Error finding container 4a430137277d3d191db07d8b5f20cb298c9ff6674f04c7635fd64f98c46fc91c: Status 404 returned error can't find the container with id 4a430137277d3d191db07d8b5f20cb298c9ff6674f04c7635fd64f98c46fc91c
Nov 04 01:01:59.537919 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.537897    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:01:59.539115 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:59.539094    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabaec7c0_0005_4bdd_9d8f_0ef0998629a6.slice/crio-2aa8127eda532fa7ee4c334ca641dc30fb42d41497fb07b2e4a0376a0ea3c416 WatchSource:0}: Error finding container 2aa8127eda532fa7ee4c334ca641dc30fb42d41497fb07b2e4a0376a0ea3c416: Status 404 returned error can't find the container with id 2aa8127eda532fa7ee4c334ca641dc30fb42d41497fb07b2e4a0376a0ea3c416
Nov 04 01:01:59.542395 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:59.542353    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a53d2f9_8ce8_4ae5_9974_a8c865065ff3.slice/crio-7da1e32f59d67ca747c01f1ce91c2973944962cd032beabd2ccd2c13c342366a WatchSource:0}: Error finding container 7da1e32f59d67ca747c01f1ce91c2973944962cd032beabd2ccd2c13c342366a: Status 404 returned error can't find the container with id 7da1e32f59d67ca747c01f1ce91c2973944962cd032beabd2ccd2c13c342366a
Nov 04 01:01:59.546117 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:59.546098    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b044258_a241_4bd2_a6eb_9ed586a0f2cc.slice/crio-f616a7b67b469b0b7393a53e6376c7f3f28b86c54b76a8f0f35df11a947e268b WatchSource:0}: Error finding container f616a7b67b469b0b7393a53e6376c7f3f28b86c54b76a8f0f35df11a947e268b: Status 404 returned error can't find the container with id f616a7b67b469b0b7393a53e6376c7f3f28b86c54b76a8f0f35df11a947e268b
Nov 04 01:01:59.548348 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.548328    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt2x6"
Nov 04 01:01:59.548918 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:59.548898    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f70c6f7_47f0_46fc_b7c6_88a780ae6ddd.slice/crio-906e0ff5d9b80c712a0e219f8dba28aac2d718eb6092bb8b1cb3d7e9021c751d WatchSource:0}: Error finding container 906e0ff5d9b80c712a0e219f8dba28aac2d718eb6092bb8b1cb3d7e9021c751d: Status 404 returned error can't find the container with id 906e0ff5d9b80c712a0e219f8dba28aac2d718eb6092bb8b1cb3d7e9021c751d
Nov 04 01:01:59.557751 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:59.557726    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4965a588_c16b_488b_82c7_b53bb390f7d0.slice/crio-f6645f2c05b3c7556458c7f2bd08199e2f168775c846c5b5cf2aa3593a6916f0 WatchSource:0}: Error finding container f6645f2c05b3c7556458c7f2bd08199e2f168775c846c5b5cf2aa3593a6916f0: Status 404 returned error can't find the container with id f6645f2c05b3c7556458c7f2bd08199e2f168775c846c5b5cf2aa3593a6916f0
Nov 04 01:01:59.559522 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.559508    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jfd2m"
Nov 04 01:01:59.566645 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.566609    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qlr2s"
Nov 04 01:01:59.576087 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:01:59.576066    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3bd337e_c495_4349_b56d_c16ebe3a9b11.slice/crio-08e42593c7cf86de79c059dab595951e8d13b366390903d025ad4b53c8719f8c WatchSource:0}: Error finding container 08e42593c7cf86de79c059dab595951e8d13b366390903d025ad4b53c8719f8c: Status 404 returned error can't find the container with id 08e42593c7cf86de79c059dab595951e8d13b366390903d025ad4b53c8719f8c
Nov 04 01:01:59.597066 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.597028    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jfd2m" event={"ID":"80a681c1-e74e-4887-a56d-6be313103581","Type":"ContainerStarted","Data":"da0ea2a4d569949cd653e044b028fe259960f7770a377c653ae58477e2547f30"}
Nov 04 01:01:59.598192 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.598163    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt2x6" event={"ID":"4965a588-c16b-488b-82c7-b53bb390f7d0","Type":"ContainerStarted","Data":"f6645f2c05b3c7556458c7f2bd08199e2f168775c846c5b5cf2aa3593a6916f0"}
Nov 04 01:01:59.599306 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.599283    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pwv64" event={"ID":"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd","Type":"ContainerStarted","Data":"906e0ff5d9b80c712a0e219f8dba28aac2d718eb6092bb8b1cb3d7e9021c751d"}
Nov 04 01:01:59.600362 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.600334    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-l7n65" event={"ID":"3a53d2f9-8ce8-4ae5-9974-a8c865065ff3","Type":"ContainerStarted","Data":"7da1e32f59d67ca747c01f1ce91c2973944962cd032beabd2ccd2c13c342366a"}
Nov 04 01:01:59.601474 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.601454    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg" event={"ID":"abaec7c0-0005-4bdd-9d8f-0ef0998629a6","Type":"ContainerStarted","Data":"2aa8127eda532fa7ee4c334ca641dc30fb42d41497fb07b2e4a0376a0ea3c416"}
Nov 04 01:01:59.602523 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.602502    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-sjcm7" event={"ID":"3f74f502-9d11-4533-8bdf-cf880a0fbe0c","Type":"ContainerStarted","Data":"f9d36abdc505c14e0014478293acdb283ec8f81dde22697d6ffe71873e2c62f2"}
Nov 04 01:01:59.603537 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.603512    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-67.us-east-2.compute.internal" event={"ID":"9589190378fb255196269e8a3bdf442c","Type":"ContainerStarted","Data":"ff43c868abd19cc8cc3756d6c7b1acaeafa2c2dc60a90562bb5df20889f60d6b"}
Nov 04 01:01:59.604693 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.604661    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qlr2s" event={"ID":"b3bd337e-c495-4349-b56d-c16ebe3a9b11","Type":"ContainerStarted","Data":"08e42593c7cf86de79c059dab595951e8d13b366390903d025ad4b53c8719f8c"}
Nov 04 01:01:59.605689 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.605665    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-fvjq7" event={"ID":"7b044258-a241-4bd2-a6eb-9ed586a0f2cc","Type":"ContainerStarted","Data":"f616a7b67b469b0b7393a53e6376c7f3f28b86c54b76a8f0f35df11a947e268b"}
Nov 04 01:01:59.606632 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.606602    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8gvq8" event={"ID":"d6a4c158-e115-4d2f-af0d-1d9249dc7384","Type":"ContainerStarted","Data":"4a430137277d3d191db07d8b5f20cb298c9ff6674f04c7635fd64f98c46fc91c"}
Nov 04 01:01:59.607671 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.607650    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-67.us-east-2.compute.internal" event={"ID":"210e3b81ce84c7df763e2a095c51a6d5","Type":"ContainerStarted","Data":"f0dffe660314752a2e102bfc7eed66425cddd61b4ca0027f4132de7c475c8af2"}
Nov 04 01:01:59.818777 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.818740    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6be44550-8625-4296-aeb9-da833aaa4b2f-original-pull-secret\") pod \"global-pull-secret-syncer-gnrzw\" (UID: \"6be44550-8625-4296-aeb9-da833aaa4b2f\") " pod="kube-system/global-pull-secret-syncer-gnrzw"
Nov 04 01:01:59.818925 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.818804    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a409c58b-b4eb-428f-9177-a09b8a229ec7-metrics-certs\") pod \"network-metrics-daemon-rx7gq\" (UID: \"a409c58b-b4eb-428f-9177-a09b8a229ec7\") " pod="openshift-multus/network-metrics-daemon-rx7gq"
Nov 04 01:01:59.818969 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:01:59.818952    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:01:59.819016 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:01:59.819006    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a409c58b-b4eb-428f-9177-a09b8a229ec7-metrics-certs podName:a409c58b-b4eb-428f-9177-a09b8a229ec7 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:00.81899256 +0000 UTC m=+4.044840740 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a409c58b-b4eb-428f-9177-a09b8a229ec7-metrics-certs") pod "network-metrics-daemon-rx7gq" (UID: "a409c58b-b4eb-428f-9177-a09b8a229ec7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:01:59.819066 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:01:59.818951    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 01:01:59.819151 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:01:59.819115    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6be44550-8625-4296-aeb9-da833aaa4b2f-original-pull-secret podName:6be44550-8625-4296-aeb9-da833aaa4b2f nodeName:}" failed. No retries permitted until 2026-11-04 01:02:00.819095526 +0000 UTC m=+4.044943707 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6be44550-8625-4296-aeb9-da833aaa4b2f-original-pull-secret") pod "global-pull-secret-syncer-gnrzw" (UID: "6be44550-8625-4296-aeb9-da833aaa4b2f") : object "kube-system"/"original-pull-secret" not registered
Nov 04 01:01:59.919200 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:01:59.919149    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wsn9\" (UniqueName: \"kubernetes.io/projected/588c643c-b16a-4d5c-ad85-ccb8d486dd43-kube-api-access-4wsn9\") pod \"network-check-target-wxhng\" (UID: \"588c643c-b16a-4d5c-ad85-ccb8d486dd43\") " pod="openshift-network-diagnostics/network-check-target-wxhng"
Nov 04 01:01:59.919352 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:01:59.919300    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 01:01:59.919352 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:01:59.919325    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 01:01:59.919352 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:01:59.919340    2582 projected.go:196] Error preparing data for projected volume kube-api-access-4wsn9 for pod openshift-network-diagnostics/network-check-target-wxhng: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:01:59.919481 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:01:59.919402    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/588c643c-b16a-4d5c-ad85-ccb8d486dd43-kube-api-access-4wsn9 podName:588c643c-b16a-4d5c-ad85-ccb8d486dd43 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:00.919386245 +0000 UTC m=+4.145234418 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-4wsn9" (UniqueName: "kubernetes.io/projected/588c643c-b16a-4d5c-ad85-ccb8d486dd43-kube-api-access-4wsn9") pod "network-check-target-wxhng" (UID: "588c643c-b16a-4d5c-ad85-ccb8d486dd43") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:02:00.249858 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:00.249751    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 00:56:59 +0000 UTC" deadline="2028-08-19 15:24:40.716032942 +0000 UTC"
Nov 04 01:02:00.249858 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:00.249813    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15710h22m40.466224178s"
Nov 04 01:02:00.590975 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:00.590933    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gnrzw"
Nov 04 01:02:00.591180 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:00.591088    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gnrzw" podUID="6be44550-8625-4296-aeb9-da833aaa4b2f"
Nov 04 01:02:00.828279 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:00.828237    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6be44550-8625-4296-aeb9-da833aaa4b2f-original-pull-secret\") pod \"global-pull-secret-syncer-gnrzw\" (UID: \"6be44550-8625-4296-aeb9-da833aaa4b2f\") " pod="kube-system/global-pull-secret-syncer-gnrzw"
Nov 04 01:02:00.828583 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:00.828329    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a409c58b-b4eb-428f-9177-a09b8a229ec7-metrics-certs\") pod \"network-metrics-daemon-rx7gq\" (UID: \"a409c58b-b4eb-428f-9177-a09b8a229ec7\") " pod="openshift-multus/network-metrics-daemon-rx7gq"
Nov 04 01:02:00.828583 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:00.828472    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:02:00.828583 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:00.828539    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a409c58b-b4eb-428f-9177-a09b8a229ec7-metrics-certs podName:a409c58b-b4eb-428f-9177-a09b8a229ec7 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:02.828519445 +0000 UTC m=+6.054367617 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a409c58b-b4eb-428f-9177-a09b8a229ec7-metrics-certs") pod "network-metrics-daemon-rx7gq" (UID: "a409c58b-b4eb-428f-9177-a09b8a229ec7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:02:00.828819 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:00.828758    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 01:02:00.828819 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:00.828806    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6be44550-8625-4296-aeb9-da833aaa4b2f-original-pull-secret podName:6be44550-8625-4296-aeb9-da833aaa4b2f nodeName:}" failed. No retries permitted until 2026-11-04 01:02:02.82879339 +0000 UTC m=+6.054641556 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6be44550-8625-4296-aeb9-da833aaa4b2f-original-pull-secret") pod "global-pull-secret-syncer-gnrzw" (UID: "6be44550-8625-4296-aeb9-da833aaa4b2f") : object "kube-system"/"original-pull-secret" not registered
Nov 04 01:02:00.929724 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:00.928948    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wsn9\" (UniqueName: \"kubernetes.io/projected/588c643c-b16a-4d5c-ad85-ccb8d486dd43-kube-api-access-4wsn9\") pod \"network-check-target-wxhng\" (UID: \"588c643c-b16a-4d5c-ad85-ccb8d486dd43\") " pod="openshift-network-diagnostics/network-check-target-wxhng"
Nov 04 01:02:00.929724 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:00.929176    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 01:02:00.929724 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:00.929201    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 01:02:00.929724 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:00.929214    2582 projected.go:196] Error preparing data for projected volume kube-api-access-4wsn9 for pod openshift-network-diagnostics/network-check-target-wxhng: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:02:00.929724 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:00.929277    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/588c643c-b16a-4d5c-ad85-ccb8d486dd43-kube-api-access-4wsn9 podName:588c643c-b16a-4d5c-ad85-ccb8d486dd43 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:02.929258341 +0000 UTC m=+6.155106523 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-4wsn9" (UniqueName: "kubernetes.io/projected/588c643c-b16a-4d5c-ad85-ccb8d486dd43-kube-api-access-4wsn9") pod "network-check-target-wxhng" (UID: "588c643c-b16a-4d5c-ad85-ccb8d486dd43") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:02:01.251063 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:01.250962    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 00:56:59 +0000 UTC" deadline="2028-04-11 11:12:44.22762267 +0000 UTC"
Nov 04 01:02:01.251063 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:01.251007    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12586h10m42.976620765s"
Nov 04 01:02:01.590918 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:01.590887    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx7gq"
Nov 04 01:02:01.591153 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:01.591028    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rx7gq" podUID="a409c58b-b4eb-428f-9177-a09b8a229ec7"
Nov 04 01:02:01.591526 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:01.591502    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wxhng"
Nov 04 01:02:01.591688 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:01.591601    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wxhng" podUID="588c643c-b16a-4d5c-ad85-ccb8d486dd43"
Nov 04 01:02:02.591326 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:02.590857    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gnrzw"
Nov 04 01:02:02.591326 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:02.590973    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gnrzw" podUID="6be44550-8625-4296-aeb9-da833aaa4b2f"
Nov 04 01:02:02.844806 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:02.844743    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a409c58b-b4eb-428f-9177-a09b8a229ec7-metrics-certs\") pod \"network-metrics-daemon-rx7gq\" (UID: \"a409c58b-b4eb-428f-9177-a09b8a229ec7\") " pod="openshift-multus/network-metrics-daemon-rx7gq"
Nov 04 01:02:02.844923 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:02.844815    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6be44550-8625-4296-aeb9-da833aaa4b2f-original-pull-secret\") pod \"global-pull-secret-syncer-gnrzw\" (UID: \"6be44550-8625-4296-aeb9-da833aaa4b2f\") " pod="kube-system/global-pull-secret-syncer-gnrzw"
Nov 04 01:02:02.845032 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:02.844939    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 01:02:02.845032 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:02.844999    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6be44550-8625-4296-aeb9-da833aaa4b2f-original-pull-secret podName:6be44550-8625-4296-aeb9-da833aaa4b2f nodeName:}" failed. No retries permitted until 2026-11-04 01:02:06.844980505 +0000 UTC m=+10.070828689 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6be44550-8625-4296-aeb9-da833aaa4b2f-original-pull-secret") pod "global-pull-secret-syncer-gnrzw" (UID: "6be44550-8625-4296-aeb9-da833aaa4b2f") : object "kube-system"/"original-pull-secret" not registered
Nov 04 01:02:02.845400 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:02.845378    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:02:02.845502 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:02.845435    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a409c58b-b4eb-428f-9177-a09b8a229ec7-metrics-certs podName:a409c58b-b4eb-428f-9177-a09b8a229ec7 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:06.845421264 +0000 UTC m=+10.071269435 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a409c58b-b4eb-428f-9177-a09b8a229ec7-metrics-certs") pod "network-metrics-daemon-rx7gq" (UID: "a409c58b-b4eb-428f-9177-a09b8a229ec7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:02:02.945182 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:02.945141    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wsn9\" (UniqueName: \"kubernetes.io/projected/588c643c-b16a-4d5c-ad85-ccb8d486dd43-kube-api-access-4wsn9\") pod \"network-check-target-wxhng\" (UID: \"588c643c-b16a-4d5c-ad85-ccb8d486dd43\") " pod="openshift-network-diagnostics/network-check-target-wxhng"
Nov 04 01:02:02.945422 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:02.945396    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 01:02:02.945422 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:02.945422    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 01:02:02.945606 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:02.945435    2582 projected.go:196] Error preparing data for projected volume kube-api-access-4wsn9 for pod openshift-network-diagnostics/network-check-target-wxhng: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:02:02.945606 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:02.945498    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/588c643c-b16a-4d5c-ad85-ccb8d486dd43-kube-api-access-4wsn9 podName:588c643c-b16a-4d5c-ad85-ccb8d486dd43 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:06.945475069 +0000 UTC m=+10.171323239 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-4wsn9" (UniqueName: "kubernetes.io/projected/588c643c-b16a-4d5c-ad85-ccb8d486dd43-kube-api-access-4wsn9") pod "network-check-target-wxhng" (UID: "588c643c-b16a-4d5c-ad85-ccb8d486dd43") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:02:03.591674 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:03.591403    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx7gq"
Nov 04 01:02:03.592132 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:03.591823    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rx7gq" podUID="a409c58b-b4eb-428f-9177-a09b8a229ec7"
Nov 04 01:02:03.594229 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:03.594196    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wxhng"
Nov 04 01:02:03.594377 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:03.594353    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wxhng" podUID="588c643c-b16a-4d5c-ad85-ccb8d486dd43"
Nov 04 01:02:03.647784 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:03.647434    2582 generic.go:356] "Generic (PLEG): container finished" podID="210e3b81ce84c7df763e2a095c51a6d5" containerID="acb80777421a025a1efc66bd483cb1690ba6307d1e0d34cab7479a93c1fa8495" exitCode=0
Nov 04 01:02:03.647784 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:03.647534    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-67.us-east-2.compute.internal" event={"ID":"210e3b81ce84c7df763e2a095c51a6d5","Type":"ContainerDied","Data":"acb80777421a025a1efc66bd483cb1690ba6307d1e0d34cab7479a93c1fa8495"}
Nov 04 01:02:03.654710 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:03.653744    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-67.us-east-2.compute.internal" event={"ID":"9589190378fb255196269e8a3bdf442c","Type":"ContainerStarted","Data":"d12e32b345211851ccc3a1f44adc1a1683b0c0d46661274955118ae7a7a4b5cc"}
Nov 04 01:02:03.667720 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:03.667497    2582 generic.go:356] "Generic (PLEG): container finished" podID="b3bd337e-c495-4349-b56d-c16ebe3a9b11" containerID="98c46866a770d6bad4c965fb1cd3a4d69ea6a720a56febeacdde21d7d4b688da" exitCode=0
Nov 04 01:02:03.667720 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:03.667536    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qlr2s" event={"ID":"b3bd337e-c495-4349-b56d-c16ebe3a9b11","Type":"ContainerDied","Data":"98c46866a770d6bad4c965fb1cd3a4d69ea6a720a56febeacdde21d7d4b688da"}
Nov 04 01:02:04.590878 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:04.590844    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gnrzw"
Nov 04 01:02:04.591074 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:04.590979    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gnrzw" podUID="6be44550-8625-4296-aeb9-da833aaa4b2f"
Nov 04 01:02:04.690206 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:04.690135    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-67.us-east-2.compute.internal" event={"ID":"210e3b81ce84c7df763e2a095c51a6d5","Type":"ContainerStarted","Data":"2706c4453c2d26c4260b528d49ecb541481c44394c07099ea1d95ad7c6203c12"}
Nov 04 01:02:04.709786 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:04.708590    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-67.us-east-2.compute.internal" podStartSLOduration=5.708570874 podStartE2EDuration="5.708570874s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:01:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:02:03.723560733 +0000 UTC m=+6.949408924" watchObservedRunningTime="2026-11-04 01:02:04.708570874 +0000 UTC m=+7.934419068"
Nov 04 01:02:05.594640 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:05.593942    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx7gq"
Nov 04 01:02:05.594640 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:05.594098    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rx7gq" podUID="a409c58b-b4eb-428f-9177-a09b8a229ec7"
Nov 04 01:02:05.594640 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:05.594492    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wxhng"
Nov 04 01:02:05.594640 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:05.594584    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wxhng" podUID="588c643c-b16a-4d5c-ad85-ccb8d486dd43"
Nov 04 01:02:05.695034 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:05.694969    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-sjcm7" event={"ID":"3f74f502-9d11-4533-8bdf-cf880a0fbe0c","Type":"ContainerStarted","Data":"d1041c852de28dcb47153aa7ec0b3c0156576b0e2dfa9082e8a97fd9a031ae35"}
Nov 04 01:02:05.713392 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:05.713018    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-sjcm7" podStartSLOduration=2.6340809309999997 podStartE2EDuration="7.71300059s" totalImagesPullingTime="5.078919659s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:01:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:02:05.71220927 +0000 UTC m=+8.938057472" watchObservedRunningTime="2026-11-04 01:02:05.71300059 +0000 UTC m=+8.938848785"
Nov 04 01:02:05.713392 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:05.713134    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-67.us-east-2.compute.internal" podStartSLOduration=6.713124699 podStartE2EDuration="6.713124699s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:01:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:02:04.709760094 +0000 UTC m=+7.935608284" watchObservedRunningTime="2026-11-04 01:02:05.713124699 +0000 UTC m=+8.938972889"
Nov 04 01:02:06.590895 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:06.590856    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gnrzw"
Nov 04 01:02:06.591047 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:06.590991    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gnrzw" podUID="6be44550-8625-4296-aeb9-da833aaa4b2f"
Nov 04 01:02:06.701567 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:06.701093    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-fvjq7" event={"ID":"7b044258-a241-4bd2-a6eb-9ed586a0f2cc","Type":"ContainerStarted","Data":"159cf68b87cb255c55cb9c0d3622353c6b313de3a8ac16de859b142060aba6c2"}
Nov 04 01:02:06.886537 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:06.886499    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a409c58b-b4eb-428f-9177-a09b8a229ec7-metrics-certs\") pod \"network-metrics-daemon-rx7gq\" (UID: \"a409c58b-b4eb-428f-9177-a09b8a229ec7\") " pod="openshift-multus/network-metrics-daemon-rx7gq"
Nov 04 01:02:06.886744 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:06.886585    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6be44550-8625-4296-aeb9-da833aaa4b2f-original-pull-secret\") pod \"global-pull-secret-syncer-gnrzw\" (UID: \"6be44550-8625-4296-aeb9-da833aaa4b2f\") " pod="kube-system/global-pull-secret-syncer-gnrzw"
Nov 04 01:02:06.886841 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:06.886745    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 01:02:06.886841 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:06.886809    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6be44550-8625-4296-aeb9-da833aaa4b2f-original-pull-secret podName:6be44550-8625-4296-aeb9-da833aaa4b2f nodeName:}" failed. No retries permitted until 2026-11-04 01:02:14.886790417 +0000 UTC m=+18.112638589 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6be44550-8625-4296-aeb9-da833aaa4b2f-original-pull-secret") pod "global-pull-secret-syncer-gnrzw" (UID: "6be44550-8625-4296-aeb9-da833aaa4b2f") : object "kube-system"/"original-pull-secret" not registered
Nov 04 01:02:06.887234 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:06.887203    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:02:06.887334 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:06.887253    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a409c58b-b4eb-428f-9177-a09b8a229ec7-metrics-certs podName:a409c58b-b4eb-428f-9177-a09b8a229ec7 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:14.887238192 +0000 UTC m=+18.113086374 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a409c58b-b4eb-428f-9177-a09b8a229ec7-metrics-certs") pod "network-metrics-daemon-rx7gq" (UID: "a409c58b-b4eb-428f-9177-a09b8a229ec7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:02:06.987505 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:06.987460    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wsn9\" (UniqueName: \"kubernetes.io/projected/588c643c-b16a-4d5c-ad85-ccb8d486dd43-kube-api-access-4wsn9\") pod \"network-check-target-wxhng\" (UID: \"588c643c-b16a-4d5c-ad85-ccb8d486dd43\") " pod="openshift-network-diagnostics/network-check-target-wxhng"
Nov 04 01:02:06.987734 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:06.987665    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 01:02:06.987734 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:06.987690    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 01:02:06.987734 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:06.987704    2582 projected.go:196] Error preparing data for projected volume kube-api-access-4wsn9 for pod openshift-network-diagnostics/network-check-target-wxhng: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:02:06.987933 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:06.987768    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/588c643c-b16a-4d5c-ad85-ccb8d486dd43-kube-api-access-4wsn9 podName:588c643c-b16a-4d5c-ad85-ccb8d486dd43 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:14.987746259 +0000 UTC m=+18.213594428 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-4wsn9" (UniqueName: "kubernetes.io/projected/588c643c-b16a-4d5c-ad85-ccb8d486dd43-kube-api-access-4wsn9") pod "network-check-target-wxhng" (UID: "588c643c-b16a-4d5c-ad85-ccb8d486dd43") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:02:07.592493 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:07.592459    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wxhng"
Nov 04 01:02:07.592680 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:07.592537    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wxhng" podUID="588c643c-b16a-4d5c-ad85-ccb8d486dd43"
Nov 04 01:02:07.592680 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:07.592574    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx7gq"
Nov 04 01:02:07.592680 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:07.592666    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rx7gq" podUID="a409c58b-b4eb-428f-9177-a09b8a229ec7"
Nov 04 01:02:07.707762 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:07.707735    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt2x6" event={"ID":"4965a588-c16b-488b-82c7-b53bb390f7d0","Type":"ContainerStarted","Data":"f67caeab7f9894e14e2eb22ddc264609aff59edeb7068a2c76bbbf3b4acb4923"}
Nov 04 01:02:08.591126 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:08.590857    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gnrzw"
Nov 04 01:02:08.591126 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:08.590991    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gnrzw" podUID="6be44550-8625-4296-aeb9-da833aaa4b2f"
Nov 04 01:02:08.712182 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:08.711443    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jfd2m" event={"ID":"80a681c1-e74e-4887-a56d-6be313103581","Type":"ContainerStarted","Data":"584f4f3effc199015dbb73dad5e9daa885adcc35cbfec0095e894aa59c9fdcc8"}
Nov 04 01:02:08.735415 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:08.734836    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-fvjq7" podStartSLOduration=4.608506062 podStartE2EDuration="10.73481814s" totalImagesPullingTime="6.126312078s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:01:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:02:06.722849965 +0000 UTC m=+9.948698155" watchObservedRunningTime="2026-11-04 01:02:08.73481814 +0000 UTC m=+11.960666328"
Nov 04 01:02:08.772756 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:08.772738    2582 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 04 01:02:09.261646 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:09.260935    2582 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-04T01:02:08.772754846Z","UUID":"0c8cba9e-3a84-4503-8757-f5bb669a4ebe","Handler":null,"Name":"","Endpoint":""}
Nov 04 01:02:09.262710 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:09.262683    2582 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 04 01:02:09.262710 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:09.262714    2582 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 04 01:02:09.590455 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:09.590423    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx7gq"
Nov 04 01:02:09.590627 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:09.590569    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rx7gq" podUID="a409c58b-b4eb-428f-9177-a09b8a229ec7"
Nov 04 01:02:09.590707 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:09.590634    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wxhng"
Nov 04 01:02:09.590777 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:09.590758    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wxhng" podUID="588c643c-b16a-4d5c-ad85-ccb8d486dd43"
Nov 04 01:02:09.715320 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:09.715287    2582 generic.go:356] "Generic (PLEG): container finished" podID="b3bd337e-c495-4349-b56d-c16ebe3a9b11" containerID="ac5e28a5c11ea0d56a2e397f89743f1ffa843571a9bb5dc89a57b112d378749f" exitCode=0
Nov 04 01:02:09.715788 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:09.715374    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qlr2s" event={"ID":"b3bd337e-c495-4349-b56d-c16ebe3a9b11","Type":"ContainerDied","Data":"ac5e28a5c11ea0d56a2e397f89743f1ffa843571a9bb5dc89a57b112d378749f"}
Nov 04 01:02:09.717138 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:09.717114    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8gvq8" event={"ID":"d6a4c158-e115-4d2f-af0d-1d9249dc7384","Type":"ContainerStarted","Data":"942f3294fe1fa43e2e8dc73732ca8b986ca261736f79dd03b7e95d2f3998f2bf"}
Nov 04 01:02:09.719819 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:09.719795    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt2x6" event={"ID":"4965a588-c16b-488b-82c7-b53bb390f7d0","Type":"ContainerStarted","Data":"d8294220ee0c4433b2b576e620a0051d1287c29f90ce14e424a2d8eec979e6bf"}
Nov 04 01:02:09.741941 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:09.741890    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-jfd2m" podStartSLOduration=2.961928136 podStartE2EDuration="11.741873566s" totalImagesPullingTime="8.77994543s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:01:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:02:08.735599947 +0000 UTC m=+11.961448136" watchObservedRunningTime="2026-11-04 01:02:09.741873566 +0000 UTC m=+12.967721754"
Nov 04 01:02:09.758986 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:09.758937    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-8gvq8" podStartSLOduration=2.571660529 podStartE2EDuration="11.758921637s" totalImagesPullingTime="9.187261108s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:01:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:02:09.758489166 +0000 UTC m=+12.984337352" watchObservedRunningTime="2026-11-04 01:02:09.758921637 +0000 UTC m=+12.984769824"
Nov 04 01:02:10.590694 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:10.590665    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gnrzw"
Nov 04 01:02:10.590846 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:10.590779    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gnrzw" podUID="6be44550-8625-4296-aeb9-da833aaa4b2f"
Nov 04 01:02:10.648292 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:10.648268    2582 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-sjcm7"
Nov 04 01:02:10.648898 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:10.648878    2582 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-sjcm7"
Nov 04 01:02:10.724857 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:10.724828    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-l7n65" event={"ID":"3a53d2f9-8ce8-4ae5-9974-a8c865065ff3","Type":"ContainerStarted","Data":"e8b31c64e77038cd7001270191326a015759d69ad67a82fbd8ebc6e94483582e"}
Nov 04 01:02:10.741422 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:10.741385    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-l7n65" podStartSLOduration=3.647230965 podStartE2EDuration="12.741371923s" totalImagesPullingTime="9.094140958s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:01:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:02:10.741268528 +0000 UTC m=+13.967116714" watchObservedRunningTime="2026-11-04 01:02:10.741371923 +0000 UTC m=+13.967220109"
Nov 04 01:02:11.591256 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:11.591233    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx7gq"
Nov 04 01:02:11.591430 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:11.591397    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rx7gq" podUID="a409c58b-b4eb-428f-9177-a09b8a229ec7"
Nov 04 01:02:11.591501 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:11.591446    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wxhng"
Nov 04 01:02:11.591576 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:11.591555    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wxhng" podUID="588c643c-b16a-4d5c-ad85-ccb8d486dd43"
Nov 04 01:02:11.728581 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:11.728549    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt2x6" event={"ID":"4965a588-c16b-488b-82c7-b53bb390f7d0","Type":"ContainerStarted","Data":"2fa983217e3bcf62dd714cf870a129a62dff5499d59c510a5b87ad7c6b210adb"}
Nov 04 01:02:11.730183 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:11.730159    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pwv64" event={"ID":"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd","Type":"ContainerStarted","Data":"5e7c8425e9d436f6538fbc6c61da22907dbbbceedfa2ae36e3429d3c27c2251c"}
Nov 04 01:02:11.732411 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:11.732390    2582 generic.go:356] "Generic (PLEG): container finished" podID="b3bd337e-c495-4349-b56d-c16ebe3a9b11" containerID="badfa2f0633d7138f9da22eca143668068d34ea6e7c245cb46e8cdbfcf4659d5" exitCode=0
Nov 04 01:02:11.732489 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:11.732437    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qlr2s" event={"ID":"b3bd337e-c495-4349-b56d-c16ebe3a9b11","Type":"ContainerDied","Data":"badfa2f0633d7138f9da22eca143668068d34ea6e7c245cb46e8cdbfcf4659d5"}
Nov 04 01:02:11.749336 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:11.749297    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt2x6" podStartSLOduration=2.985875551 podStartE2EDuration="13.749286404s" totalImagesPullingTime="10.763410853s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:01:58 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:02:11.74922906 +0000 UTC m=+14.975077246" watchObservedRunningTime="2026-11-04 01:02:11.749286404 +0000 UTC m=+14.975134590"
Nov 04 01:02:11.768939 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:11.768894    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-pwv64" podStartSLOduration=2.346453053 podStartE2EDuration="13.768883464s" totalImagesPullingTime="11.422430411s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:01:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:02:11.7682756 +0000 UTC m=+14.994123787" watchObservedRunningTime="2026-11-04 01:02:11.768883464 +0000 UTC m=+14.994731649"
Nov 04 01:02:12.590429 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:12.590395    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gnrzw"
Nov 04 01:02:12.590587 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:12.590514    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gnrzw" podUID="6be44550-8625-4296-aeb9-da833aaa4b2f"
Nov 04 01:02:13.590847 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:13.590661    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wxhng"
Nov 04 01:02:13.591326 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:13.590703    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx7gq"
Nov 04 01:02:13.591326 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:13.590933    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wxhng" podUID="588c643c-b16a-4d5c-ad85-ccb8d486dd43"
Nov 04 01:02:13.591326 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:13.591019    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rx7gq" podUID="a409c58b-b4eb-428f-9177-a09b8a229ec7"
Nov 04 01:02:13.738517 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:13.738481    2582 generic.go:356] "Generic (PLEG): container finished" podID="b3bd337e-c495-4349-b56d-c16ebe3a9b11" containerID="891c44a2cbd939e78ef6b0c631e1ef7e08373df28106b068356a2bafe8179db3" exitCode=0
Nov 04 01:02:13.738709 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:13.738548    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qlr2s" event={"ID":"b3bd337e-c495-4349-b56d-c16ebe3a9b11","Type":"ContainerDied","Data":"891c44a2cbd939e78ef6b0c631e1ef7e08373df28106b068356a2bafe8179db3"}
Nov 04 01:02:14.591034 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:14.590996    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gnrzw"
Nov 04 01:02:14.591522 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:14.591141    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gnrzw" podUID="6be44550-8625-4296-aeb9-da833aaa4b2f"
Nov 04 01:02:14.951601 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:14.951521    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6be44550-8625-4296-aeb9-da833aaa4b2f-original-pull-secret\") pod \"global-pull-secret-syncer-gnrzw\" (UID: \"6be44550-8625-4296-aeb9-da833aaa4b2f\") " pod="kube-system/global-pull-secret-syncer-gnrzw"
Nov 04 01:02:14.951788 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:14.951604    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a409c58b-b4eb-428f-9177-a09b8a229ec7-metrics-certs\") pod \"network-metrics-daemon-rx7gq\" (UID: \"a409c58b-b4eb-428f-9177-a09b8a229ec7\") " pod="openshift-multus/network-metrics-daemon-rx7gq"
Nov 04 01:02:14.951788 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:14.951711    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 01:02:14.951788 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:14.951780    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6be44550-8625-4296-aeb9-da833aaa4b2f-original-pull-secret podName:6be44550-8625-4296-aeb9-da833aaa4b2f nodeName:}" failed. No retries permitted until 2026-11-04 01:02:30.951764773 +0000 UTC m=+34.177612948 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6be44550-8625-4296-aeb9-da833aaa4b2f-original-pull-secret") pod "global-pull-secret-syncer-gnrzw" (UID: "6be44550-8625-4296-aeb9-da833aaa4b2f") : object "kube-system"/"original-pull-secret" not registered
Nov 04 01:02:14.951950 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:14.951711    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:02:14.951950 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:14.951837    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a409c58b-b4eb-428f-9177-a09b8a229ec7-metrics-certs podName:a409c58b-b4eb-428f-9177-a09b8a229ec7 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:30.951826845 +0000 UTC m=+34.177675031 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a409c58b-b4eb-428f-9177-a09b8a229ec7-metrics-certs") pod "network-metrics-daemon-rx7gq" (UID: "a409c58b-b4eb-428f-9177-a09b8a229ec7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:02:15.052940 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:15.052906    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wsn9\" (UniqueName: \"kubernetes.io/projected/588c643c-b16a-4d5c-ad85-ccb8d486dd43-kube-api-access-4wsn9\") pod \"network-check-target-wxhng\" (UID: \"588c643c-b16a-4d5c-ad85-ccb8d486dd43\") " pod="openshift-network-diagnostics/network-check-target-wxhng"
Nov 04 01:02:15.053100 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:15.053044    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 01:02:15.053100 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:15.053065    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 01:02:15.053100 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:15.053079    2582 projected.go:196] Error preparing data for projected volume kube-api-access-4wsn9 for pod openshift-network-diagnostics/network-check-target-wxhng: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:02:15.053243 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:15.053137    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/588c643c-b16a-4d5c-ad85-ccb8d486dd43-kube-api-access-4wsn9 podName:588c643c-b16a-4d5c-ad85-ccb8d486dd43 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:31.05311618 +0000 UTC m=+34.278964389 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-4wsn9" (UniqueName: "kubernetes.io/projected/588c643c-b16a-4d5c-ad85-ccb8d486dd43-kube-api-access-4wsn9") pod "network-check-target-wxhng" (UID: "588c643c-b16a-4d5c-ad85-ccb8d486dd43") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:02:15.591751 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:15.591718    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx7gq"
Nov 04 01:02:15.592242 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:15.591866    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rx7gq" podUID="a409c58b-b4eb-428f-9177-a09b8a229ec7"
Nov 04 01:02:15.592242 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:15.591898    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wxhng"
Nov 04 01:02:15.592242 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:15.591998    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wxhng" podUID="588c643c-b16a-4d5c-ad85-ccb8d486dd43"
Nov 04 01:02:16.591202 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:16.591027    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gnrzw"
Nov 04 01:02:16.591333 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:16.591288    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gnrzw" podUID="6be44550-8625-4296-aeb9-da833aaa4b2f"
Nov 04 01:02:16.748053 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:16.747977    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg" event={"ID":"abaec7c0-0005-4bdd-9d8f-0ef0998629a6","Type":"ContainerStarted","Data":"eae7d6b26f94d2f4d08f0289cfc21e27582e0af1f6245b729e783e61de2acb54"}
Nov 04 01:02:16.748053 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:16.748040    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg" event={"ID":"abaec7c0-0005-4bdd-9d8f-0ef0998629a6","Type":"ContainerStarted","Data":"44882fb8c87f39b7c50eff4b3eeac5d66ccbac78476aecee298fe25eddef8976"}
Nov 04 01:02:16.748455 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:16.748069    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg" event={"ID":"abaec7c0-0005-4bdd-9d8f-0ef0998629a6","Type":"ContainerStarted","Data":"3558e0cd31cfda33f3ef4398267ad980c1058d955b04855372c35bf4ac38b0f0"}
Nov 04 01:02:16.748455 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:16.748096    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg" event={"ID":"abaec7c0-0005-4bdd-9d8f-0ef0998629a6","Type":"ContainerStarted","Data":"d0b5a7d4c2299f4a1071f6c98b9782dc5729f1604304ab57db412c6199c042f4"}
Nov 04 01:02:16.748455 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:16.748124    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg" event={"ID":"abaec7c0-0005-4bdd-9d8f-0ef0998629a6","Type":"ContainerStarted","Data":"8138a17ad2dc3719d19dec13025d51f1e03faa39f371f480f16f477bfe9cb131"}
Nov 04 01:02:16.748455 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:16.748152    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg" event={"ID":"abaec7c0-0005-4bdd-9d8f-0ef0998629a6","Type":"ContainerStarted","Data":"d0f0965c562d56f981573ea8d76413b6cab18fd77d6f72e690c103b4be57e9c2"}
Nov 04 01:02:16.750746 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:16.750724    2582 generic.go:356] "Generic (PLEG): container finished" podID="b3bd337e-c495-4349-b56d-c16ebe3a9b11" containerID="a86f747a6f2075432e47541b1846c8075ad7f4426ac0596a5d46923a1ca5383f" exitCode=0
Nov 04 01:02:16.750793 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:16.750769    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qlr2s" event={"ID":"b3bd337e-c495-4349-b56d-c16ebe3a9b11","Type":"ContainerDied","Data":"a86f747a6f2075432e47541b1846c8075ad7f4426ac0596a5d46923a1ca5383f"}
Nov 04 01:02:17.591303 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:17.591272    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx7gq"
Nov 04 01:02:17.591496 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:17.591422    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rx7gq" podUID="a409c58b-b4eb-428f-9177-a09b8a229ec7"
Nov 04 01:02:17.591496 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:17.591462    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wxhng"
Nov 04 01:02:17.591646 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:17.591517    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wxhng" podUID="588c643c-b16a-4d5c-ad85-ccb8d486dd43"
Nov 04 01:02:17.758546 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:17.758517    2582 generic.go:356] "Generic (PLEG): container finished" podID="b3bd337e-c495-4349-b56d-c16ebe3a9b11" containerID="64da235aa48ce0025b528cca6f283dc408d2f0d043240515b18a66c35ebcf232" exitCode=0
Nov 04 01:02:17.758911 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:17.758564    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qlr2s" event={"ID":"b3bd337e-c495-4349-b56d-c16ebe3a9b11","Type":"ContainerDied","Data":"64da235aa48ce0025b528cca6f283dc408d2f0d043240515b18a66c35ebcf232"}
Nov 04 01:02:18.545684 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:18.545654    2582 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-sjcm7"
Nov 04 01:02:18.545827 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:18.545806    2582 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 04 01:02:18.546240 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:18.546221    2582 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-sjcm7"
Nov 04 01:02:18.591160 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:18.591128    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gnrzw"
Nov 04 01:02:18.591276 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:18.591220    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gnrzw" podUID="6be44550-8625-4296-aeb9-da833aaa4b2f"
Nov 04 01:02:18.763599 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:18.763567    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg" event={"ID":"abaec7c0-0005-4bdd-9d8f-0ef0998629a6","Type":"ContainerStarted","Data":"5d7d4d33785574320f978ed410baeff16dc96ab33166a29a98cee83993115b66"}
Nov 04 01:02:18.766515 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:18.766492    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qlr2s" event={"ID":"b3bd337e-c495-4349-b56d-c16ebe3a9b11","Type":"ContainerStarted","Data":"16b4693d24af814a415cdaa80d8d5882cc6085836d165233102a605be0d9327b"}
Nov 04 01:02:18.791548 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:18.791511    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-qlr2s" podStartSLOduration=7.171241316 podStartE2EDuration="20.791499886s" totalImagesPullingTime="13.62025857s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:01:58 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:02:18.791058316 +0000 UTC m=+22.016906501" watchObservedRunningTime="2026-11-04 01:02:18.791499886 +0000 UTC m=+22.017348072"
Nov 04 01:02:19.593346 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:19.593320    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wxhng"
Nov 04 01:02:19.593486 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:19.593320    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx7gq"
Nov 04 01:02:19.593486 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:19.593405    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wxhng" podUID="588c643c-b16a-4d5c-ad85-ccb8d486dd43"
Nov 04 01:02:19.593572 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:19.593496    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rx7gq" podUID="a409c58b-b4eb-428f-9177-a09b8a229ec7"
Nov 04 01:02:20.590936 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:20.590916    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gnrzw"
Nov 04 01:02:20.591344 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:20.591003    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gnrzw" podUID="6be44550-8625-4296-aeb9-da833aaa4b2f"
Nov 04 01:02:20.774516 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:20.774432    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg" event={"ID":"abaec7c0-0005-4bdd-9d8f-0ef0998629a6","Type":"ContainerStarted","Data":"a1fe74219b79e2d71f2b03f16695078b2333e9847baa3a8d79525777c724d79c"}
Nov 04 01:02:20.774737 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:20.774717    2582 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:20.774824 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:20.774789    2582 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:20.789768 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:20.789744    2582 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:20.809823 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:20.809783    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg" podStartSLOduration=6.207341399 podStartE2EDuration="22.809772359s" totalImagesPullingTime="16.60243096s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:01:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:02:20.808000633 +0000 UTC m=+24.033848818" watchObservedRunningTime="2026-11-04 01:02:20.809772359 +0000 UTC m=+24.035620577"
Nov 04 01:02:21.590436 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:21.590407    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx7gq"
Nov 04 01:02:21.590687 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:21.590407    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wxhng"
Nov 04 01:02:21.590687 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:21.590504    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rx7gq" podUID="a409c58b-b4eb-428f-9177-a09b8a229ec7"
Nov 04 01:02:21.590687 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:21.590570    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wxhng" podUID="588c643c-b16a-4d5c-ad85-ccb8d486dd43"
Nov 04 01:02:21.776932 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:21.776896    2582 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:21.794901 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:21.794875    2582 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:22.302788 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:22.302592    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-gnrzw"]
Nov 04 01:02:22.302923 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:22.302861    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gnrzw"
Nov 04 01:02:22.302963 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:22.302944    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gnrzw" podUID="6be44550-8625-4296-aeb9-da833aaa4b2f"
Nov 04 01:02:22.305284 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:22.305260    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-wxhng"]
Nov 04 01:02:22.305389 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:22.305360    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wxhng"
Nov 04 01:02:22.305480 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:22.305460    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wxhng" podUID="588c643c-b16a-4d5c-ad85-ccb8d486dd43"
Nov 04 01:02:22.308398 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:22.308377    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rx7gq"]
Nov 04 01:02:22.308511 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:22.308471    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx7gq"
Nov 04 01:02:22.308583 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:22.308563    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rx7gq" podUID="a409c58b-b4eb-428f-9177-a09b8a229ec7"
Nov 04 01:02:23.590833 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:23.590793    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx7gq"
Nov 04 01:02:23.591186 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:23.590934    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rx7gq" podUID="a409c58b-b4eb-428f-9177-a09b8a229ec7"
Nov 04 01:02:24.590366 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:24.590332    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wxhng"
Nov 04 01:02:24.590366 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:24.590371    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gnrzw"
Nov 04 01:02:24.590723 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:24.590455    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wxhng" podUID="588c643c-b16a-4d5c-ad85-ccb8d486dd43"
Nov 04 01:02:24.590723 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:24.590493    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gnrzw" podUID="6be44550-8625-4296-aeb9-da833aaa4b2f"
Nov 04 01:02:25.590983 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:25.590958    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx7gq"
Nov 04 01:02:25.591375 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:25.591057    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rx7gq" podUID="a409c58b-b4eb-428f-9177-a09b8a229ec7"
Nov 04 01:02:26.591233 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:26.591205    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gnrzw"
Nov 04 01:02:26.591658 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:26.591214    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wxhng"
Nov 04 01:02:26.591658 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:26.591303    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gnrzw" podUID="6be44550-8625-4296-aeb9-da833aaa4b2f"
Nov 04 01:02:26.591658 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:26.591409    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wxhng" podUID="588c643c-b16a-4d5c-ad85-ccb8d486dd43"
Nov 04 01:02:27.591655 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:27.591599    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx7gq"
Nov 04 01:02:27.592148 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:27.591713    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rx7gq" podUID="a409c58b-b4eb-428f-9177-a09b8a229ec7"
Nov 04 01:02:28.088290 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.088266    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-67.us-east-2.compute.internal" event="NodeReady"
Nov 04 01:02:28.088443 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.088388    2582 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Nov 04 01:02:28.124829 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.124803    2582 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-hhfmk"]
Nov 04 01:02:28.127933 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.127912    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-hhfmk"
Nov 04 01:02:28.128281 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.128258    2582 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-6lcfv"]
Nov 04 01:02:28.131574 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.131553    2582 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-rff54"]
Nov 04 01:02:28.131761 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.131737    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-6lcfv"
Nov 04 01:02:28.134562 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.134359    2582 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-gdqjb"]
Nov 04 01:02:28.134733 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.134713    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-rff54"
Nov 04 01:02:28.137271 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.137254    2582 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-5569c8658c-v7txl"]
Nov 04 01:02:28.137391 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.137373    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-gdqjb"
Nov 04 01:02:28.138187 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.138169    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\""
Nov 04 01:02:28.140201 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.140174    2582 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-6vxbv"]
Nov 04 01:02:28.140282 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.140242    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5569c8658c-v7txl"
Nov 04 01:02:28.142803 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.142788    2582 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-58wxt"]
Nov 04 01:02:28.142926 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.142908    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6vxbv"
Nov 04 01:02:28.145373 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.145358    2582 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-7c4986d449-sbjxr"]
Nov 04 01:02:28.145457 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.145446    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-58wxt"
Nov 04 01:02:28.149023 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.149002    2582 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jmxvd"]
Nov 04 01:02:28.149427 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.149407    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-sbjxr"
Nov 04 01:02:28.156753 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.152340    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Nov 04 01:02:28.156753 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.152495    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Nov 04 01:02:28.156753 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.152732    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Nov 04 01:02:28.161156 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.161135    2582 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-tr8b2"]
Nov 04 01:02:28.161265 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.161234    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jmxvd"
Nov 04 01:02:28.164099 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.164081    2582 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-8jlt4"]
Nov 04 01:02:28.164283 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.164266    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-tr8b2"
Nov 04 01:02:28.167510 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.167493    2582 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6574d8498f-74vmj"]
Nov 04 01:02:28.168418 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.168402    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-8jlt4"
Nov 04 01:02:28.170349 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.170332    2582 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5c5894ddd-j4phf"]
Nov 04 01:02:28.170723 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.170703    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6574d8498f-74vmj"
Nov 04 01:02:28.175487 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.175449    2582 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-27rmf"]
Nov 04 01:02:28.178679 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.178661    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5c5894ddd-j4phf"
Nov 04 01:02:28.182803 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.182784    2582 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-zc968"]
Nov 04 01:02:28.183006 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.182987    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-27rmf"
Nov 04 01:02:28.187008 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.186987    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-6lcfv"]
Nov 04 01:02:28.187099 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.187021    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-rff54"]
Nov 04 01:02:28.187099 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.187042    2582 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-klq54"]
Nov 04 01:02:28.187204 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.187098    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-zc968"
Nov 04 01:02:28.190385 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.190161    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-58wxt"]
Nov 04 01:02:28.190385 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.190305    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-gdqjb"]
Nov 04 01:02:28.190385 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.190331    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-6vxbv"]
Nov 04 01:02:28.190385 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.190354    2582 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-4827b"]
Nov 04 01:02:28.190385 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.190258    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-klq54"
Nov 04 01:02:28.193443 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.193426    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-hhfmk"]
Nov 04 01:02:28.193568 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.193465    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-zc968"]
Nov 04 01:02:28.193568 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.193484    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5569c8658c-v7txl"]
Nov 04 01:02:28.193568 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.193503    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-8jlt4"]
Nov 04 01:02:28.193568 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.193531    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-sbjxr"]
Nov 04 01:02:28.193568 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.193545    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5c5894ddd-j4phf"]
Nov 04 01:02:28.193568 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.193558    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4827b"]
Nov 04 01:02:28.194415 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.193575    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jmxvd"]
Nov 04 01:02:28.194415 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.193600    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-klq54"]
Nov 04 01:02:28.194415 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.193626    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4827b"
Nov 04 01:02:28.194415 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.193643    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-tr8b2"]
Nov 04 01:02:28.194415 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.193663    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6574d8498f-74vmj"]
Nov 04 01:02:28.194835 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.194813    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-27rmf"]
Nov 04 01:02:28.238987 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.238969    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Nov 04 01:02:28.239061 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.239022    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Nov 04 01:02:28.239061 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.239023    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Nov 04 01:02:28.239061 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.238969    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Nov 04 01:02:28.239156 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.239066    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Nov 04 01:02:28.239156 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.239093    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\""
Nov 04 01:02:28.239156 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.239094    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\""
Nov 04 01:02:28.239156 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.239127    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Nov 04 01:02:28.239365 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.239352    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Nov 04 01:02:28.239408 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.239393    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Nov 04 01:02:28.239454 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.239407    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Nov 04 01:02:28.239503 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.239478    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Nov 04 01:02:28.239503 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.239495    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 04 01:02:28.239677 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.239495    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Nov 04 01:02:28.239677 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.239486    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Nov 04 01:02:28.239677 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.239524    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Nov 04 01:02:28.239677 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.239539    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Nov 04 01:02:28.239677 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.239547    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Nov 04 01:02:28.239677 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.239552    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Nov 04 01:02:28.239677 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.239572    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 04 01:02:28.239913 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.239739    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Nov 04 01:02:28.240432 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.240415    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Nov 04 01:02:28.240432 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.240430    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Nov 04 01:02:28.241099 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.241082    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 04 01:02:28.241099 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.241098    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Nov 04 01:02:28.241253 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.241083    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 04 01:02:28.241253 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.241081    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 04 01:02:28.241253 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.241128    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 04 01:02:28.246240 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.246222    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Nov 04 01:02:28.248034 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.248017    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 04 01:02:28.256204 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.256049    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Nov 04 01:02:28.256204 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.256101    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 04 01:02:28.256204 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.256118    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Nov 04 01:02:28.256204 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.256130    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Nov 04 01:02:28.256204 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.256125    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Nov 04 01:02:28.256204 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.256148    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Nov 04 01:02:28.256204 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.256160    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Nov 04 01:02:28.256204 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.256167    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Nov 04 01:02:28.256204 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.256175    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Nov 04 01:02:28.256602 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.256214    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Nov 04 01:02:28.256602 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.256224    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Nov 04 01:02:28.256602 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.256121    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Nov 04 01:02:28.256602 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.256333    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 04 01:02:28.256602 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.256324    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-w5mjn\""
Nov 04 01:02:28.256602 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.256417    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Nov 04 01:02:28.256602 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.256442    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vklqk\""
Nov 04 01:02:28.256602 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.256487    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-c48q9\""
Nov 04 01:02:28.256861 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.256647    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Nov 04 01:02:28.257289 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.257273    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-6kj2d\""
Nov 04 01:02:28.257783 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.257767    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/21d1d23b-7b6e-4ab5-9593-cf7784fffea2-trusted-ca\") pod \"console-operator-7cd56b8dbc-tr8b2\" (UID: \"21d1d23b-7b6e-4ab5-9593-cf7784fffea2\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tr8b2"
Nov 04 01:02:28.257872 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.257802    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-ca-trust-extracted\") pod \"image-registry-5c5894ddd-j4phf\" (UID: \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\") " pod="openshift-image-registry/image-registry-5c5894ddd-j4phf"
Nov 04 01:02:28.257872 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.257825    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-trusted-ca\") pod \"image-registry-5c5894ddd-j4phf\" (UID: \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\") " pod="openshift-image-registry/image-registry-5c5894ddd-j4phf"
Nov 04 01:02:28.257976 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.257914    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnhzq\" (UniqueName: \"kubernetes.io/projected/23cb8b1a-61f6-4045-9f5a-3ccc1dc346a5-kube-api-access-wnhzq\") pod \"kas-connection-checker-685895c7c9-6lcfv\" (UID: \"23cb8b1a-61f6-4045-9f5a-3ccc1dc346a5\") " pod="kube-system/kas-connection-checker-685895c7c9-6lcfv"
Nov 04 01:02:28.258036 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.258004    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/20f4989c-4fe8-49d2-9430-5030a4274b60-image-registry-private-configuration\") pod \"image-registry-6574d8498f-74vmj\" (UID: \"20f4989c-4fe8-49d2-9430-5030a4274b60\") " pod="openshift-image-registry/image-registry-6574d8498f-74vmj"
Nov 04 01:02:28.258156 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.258037    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/20f4989c-4fe8-49d2-9430-5030a4274b60-trusted-ca\") pod \"image-registry-6574d8498f-74vmj\" (UID: \"20f4989c-4fe8-49d2-9430-5030a4274b60\") " pod="openshift-image-registry/image-registry-6574d8498f-74vmj"
Nov 04 01:02:28.258156 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.258132    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pt8q7\" (UniqueName: \"kubernetes.io/projected/28a8304b-9226-422d-9044-b50678d52971-kube-api-access-pt8q7\") pod \"cluster-monitoring-operator-884576c57-6vxbv\" (UID: \"28a8304b-9226-422d-9044-b50678d52971\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6vxbv"
Nov 04 01:02:28.258245 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.258157    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/de97ada4-9cae-4775-a8f8-014004e17d25-snapshots\") pod \"insights-operator-7c4986d449-sbjxr\" (UID: \"de97ada4-9cae-4775-a8f8-014004e17d25\") " pod="openshift-insights/insights-operator-7c4986d449-sbjxr"
Nov 04 01:02:28.258245 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.258176    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-image-registry-private-configuration\") pod \"image-registry-5c5894ddd-j4phf\" (UID: \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\") " pod="openshift-image-registry/image-registry-5c5894ddd-j4phf"
Nov 04 01:02:28.258245 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.258207    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ae02b148-c5ba-4db0-b817-9be4a488c708-stats-auth\") pod \"router-default-5569c8658c-v7txl\" (UID: \"ae02b148-c5ba-4db0-b817-9be4a488c708\") " pod="openshift-ingress/router-default-5569c8658c-v7txl"
Nov 04 01:02:28.258245 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.258225    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6tkr\" (UniqueName: \"kubernetes.io/projected/fc0b0e48-8adf-4d0d-80dd-ca76846f824a-kube-api-access-r6tkr\") pod \"service-ca-operator-544d749c8c-8jlt4\" (UID: \"fc0b0e48-8adf-4d0d-80dd-ca76846f824a\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-8jlt4"
Nov 04 01:02:28.258374 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.258283    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/21d1d23b-7b6e-4ab5-9593-cf7784fffea2-serving-cert\") pod \"console-operator-7cd56b8dbc-tr8b2\" (UID: \"21d1d23b-7b6e-4ab5-9593-cf7784fffea2\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tr8b2"
Nov 04 01:02:28.258408 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.258367    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/28a8304b-9226-422d-9044-b50678d52971-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-6vxbv\" (UID: \"28a8304b-9226-422d-9044-b50678d52971\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6vxbv"
Nov 04 01:02:28.258436 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.258408    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/20f4989c-4fe8-49d2-9430-5030a4274b60-registry-tls\") pod \"image-registry-6574d8498f-74vmj\" (UID: \"20f4989c-4fe8-49d2-9430-5030a4274b60\") " pod="openshift-image-registry/image-registry-6574d8498f-74vmj"
Nov 04 01:02:28.258475 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.258457    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6t28\" (UniqueName: \"kubernetes.io/projected/a0c9abfd-2f47-4cb9-9d3c-479a6231be97-kube-api-access-p6t28\") pod \"kas-connection-checker-685895c7c9-hhfmk\" (UID: \"a0c9abfd-2f47-4cb9-9d3c-479a6231be97\") " pod="kube-system/kas-connection-checker-685895c7c9-hhfmk"
Nov 04 01:02:28.258514 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.258498    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/20f4989c-4fe8-49d2-9430-5030a4274b60-ca-trust-extracted\") pod \"image-registry-6574d8498f-74vmj\" (UID: \"20f4989c-4fe8-49d2-9430-5030a4274b60\") " pod="openshift-image-registry/image-registry-6574d8498f-74vmj"
Nov 04 01:02:28.258557 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.258539    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ae02b148-c5ba-4db0-b817-9be4a488c708-default-certificate\") pod \"router-default-5569c8658c-v7txl\" (UID: \"ae02b148-c5ba-4db0-b817-9be4a488c708\") " pod="openshift-ingress/router-default-5569c8658c-v7txl"
Nov 04 01:02:28.258586 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.258568    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e41517b6-7ce4-43a6-89a9-76931457ca50-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-jmxvd\" (UID: \"e41517b6-7ce4-43a6-89a9-76931457ca50\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jmxvd"
Nov 04 01:02:28.258634 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.258596    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3d2d0e05-a773-4442-ab38-d460ad32f51e-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-58wxt\" (UID: \"3d2d0e05-a773-4442-ab38-d460ad32f51e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-58wxt"
Nov 04 01:02:28.258666 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.258648    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-installation-pull-secrets\") pod \"image-registry-5c5894ddd-j4phf\" (UID: \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\") " pod="openshift-image-registry/image-registry-5c5894ddd-j4phf"
Nov 04 01:02:28.258714 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.258687    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3d2d0e05-a773-4442-ab38-d460ad32f51e-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-58wxt\" (UID: \"3d2d0e05-a773-4442-ab38-d460ad32f51e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-58wxt"
Nov 04 01:02:28.258767 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.258731    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nfw8\" (UniqueName: \"kubernetes.io/projected/3d2d0e05-a773-4442-ab38-d460ad32f51e-kube-api-access-8nfw8\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-58wxt\" (UID: \"3d2d0e05-a773-4442-ab38-d460ad32f51e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-58wxt"
Nov 04 01:02:28.258866 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.258848    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/28a8304b-9226-422d-9044-b50678d52971-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-6vxbv\" (UID: \"28a8304b-9226-422d-9044-b50678d52971\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6vxbv"
Nov 04 01:02:28.258926 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.258912    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/de97ada4-9cae-4775-a8f8-014004e17d25-serving-cert\") pod \"insights-operator-7c4986d449-sbjxr\" (UID: \"de97ada4-9cae-4775-a8f8-014004e17d25\") " pod="openshift-insights/insights-operator-7c4986d449-sbjxr"
Nov 04 01:02:28.258999 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.258987    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/20f4989c-4fe8-49d2-9430-5030a4274b60-registry-certificates\") pod \"image-registry-6574d8498f-74vmj\" (UID: \"20f4989c-4fe8-49d2-9430-5030a4274b60\") " pod="openshift-image-registry/image-registry-6574d8498f-74vmj"
Nov 04 01:02:28.259058 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.259020    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/de97ada4-9cae-4775-a8f8-014004e17d25-tmp\") pod \"insights-operator-7c4986d449-sbjxr\" (UID: \"de97ada4-9cae-4775-a8f8-014004e17d25\") " pod="openshift-insights/insights-operator-7c4986d449-sbjxr"
Nov 04 01:02:28.259112 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.259067    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/e41517b6-7ce4-43a6-89a9-76931457ca50-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-jmxvd\" (UID: \"e41517b6-7ce4-43a6-89a9-76931457ca50\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jmxvd"
Nov 04 01:02:28.259112 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.259102    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae02b148-c5ba-4db0-b817-9be4a488c708-metrics-certs\") pod \"router-default-5569c8658c-v7txl\" (UID: \"ae02b148-c5ba-4db0-b817-9be4a488c708\") " pod="openshift-ingress/router-default-5569c8658c-v7txl"
Nov 04 01:02:28.259197 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.259128    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvzfb\" (UniqueName: \"kubernetes.io/projected/de97ada4-9cae-4775-a8f8-014004e17d25-kube-api-access-zvzfb\") pod \"insights-operator-7c4986d449-sbjxr\" (UID: \"de97ada4-9cae-4775-a8f8-014004e17d25\") " pod="openshift-insights/insights-operator-7c4986d449-sbjxr"
Nov 04 01:02:28.259233 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.259202    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7lgb\" (UniqueName: \"kubernetes.io/projected/21d1d23b-7b6e-4ab5-9593-cf7784fffea2-kube-api-access-h7lgb\") pod \"console-operator-7cd56b8dbc-tr8b2\" (UID: \"21d1d23b-7b6e-4ab5-9593-cf7784fffea2\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tr8b2"
Nov 04 01:02:28.259331 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.259309    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/20f4989c-4fe8-49d2-9430-5030a4274b60-installation-pull-secrets\") pod \"image-registry-6574d8498f-74vmj\" (UID: \"20f4989c-4fe8-49d2-9430-5030a4274b60\") " pod="openshift-image-registry/image-registry-6574d8498f-74vmj"
Nov 04 01:02:28.259383 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.259348    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fc0b0e48-8adf-4d0d-80dd-ca76846f824a-config\") pod \"service-ca-operator-544d749c8c-8jlt4\" (UID: \"fc0b0e48-8adf-4d0d-80dd-ca76846f824a\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-8jlt4"
Nov 04 01:02:28.259414 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.259379    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rct8p\" (UniqueName: \"kubernetes.io/projected/20f4989c-4fe8-49d2-9430-5030a4274b60-kube-api-access-rct8p\") pod \"image-registry-6574d8498f-74vmj\" (UID: \"20f4989c-4fe8-49d2-9430-5030a4274b60\") " pod="openshift-image-registry/image-registry-6574d8498f-74vmj"
Nov 04 01:02:28.259414 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.259401    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-bound-sa-token\") pod \"image-registry-5c5894ddd-j4phf\" (UID: \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\") " pod="openshift-image-registry/image-registry-5c5894ddd-j4phf"
Nov 04 01:02:28.259520 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.259428    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/21d1d23b-7b6e-4ab5-9593-cf7784fffea2-config\") pod \"console-operator-7cd56b8dbc-tr8b2\" (UID: \"21d1d23b-7b6e-4ab5-9593-cf7784fffea2\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tr8b2"
Nov 04 01:02:28.259520 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.259484    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-registry-certificates\") pod \"image-registry-5c5894ddd-j4phf\" (UID: \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\") " pod="openshift-image-registry/image-registry-5c5894ddd-j4phf"
Nov 04 01:02:28.259520 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.259513    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9hjn\" (UniqueName: \"kubernetes.io/projected/ae02b148-c5ba-4db0-b817-9be4a488c708-kube-api-access-b9hjn\") pod \"router-default-5569c8658c-v7txl\" (UID: \"ae02b148-c5ba-4db0-b817-9be4a488c708\") " pod="openshift-ingress/router-default-5569c8658c-v7txl"
Nov 04 01:02:28.259664 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.259537    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fc0b0e48-8adf-4d0d-80dd-ca76846f824a-serving-cert\") pod \"service-ca-operator-544d749c8c-8jlt4\" (UID: \"fc0b0e48-8adf-4d0d-80dd-ca76846f824a\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-8jlt4"
Nov 04 01:02:28.259664 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.259577    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-registry-tls\") pod \"image-registry-5c5894ddd-j4phf\" (UID: \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\") " pod="openshift-image-registry/image-registry-5c5894ddd-j4phf"
Nov 04 01:02:28.259664 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.259656    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de97ada4-9cae-4775-a8f8-014004e17d25-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-sbjxr\" (UID: \"de97ada4-9cae-4775-a8f8-014004e17d25\") " pod="openshift-insights/insights-operator-7c4986d449-sbjxr"
Nov 04 01:02:28.259768 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.259681    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5f285\" (UniqueName: \"kubernetes.io/projected/e41517b6-7ce4-43a6-89a9-76931457ca50-kube-api-access-5f285\") pod \"cluster-samples-operator-6ddb6c8b94-jmxvd\" (UID: \"e41517b6-7ce4-43a6-89a9-76931457ca50\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jmxvd"
Nov 04 01:02:28.259768 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.259721    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dr2p4\" (UniqueName: \"kubernetes.io/projected/d4efaef2-c145-4839-b32f-ecf7a1c76a99-kube-api-access-dr2p4\") pod \"volume-data-source-validator-58c9f5f46d-gdqjb\" (UID: \"d4efaef2-c145-4839-b32f-ecf7a1c76a99\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-gdqjb"
Nov 04 01:02:28.259768 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.259740    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72dk9\" (UniqueName: \"kubernetes.io/projected/4a79348d-8cf7-4429-8c5a-4084cf85fcc5-kube-api-access-72dk9\") pod \"kas-connection-checker-685895c7c9-rff54\" (UID: \"4a79348d-8cf7-4429-8c5a-4084cf85fcc5\") " pod="kube-system/kas-connection-checker-685895c7c9-rff54"
Nov 04 01:02:28.259768 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.259765    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/20f4989c-4fe8-49d2-9430-5030a4274b60-bound-sa-token\") pod \"image-registry-6574d8498f-74vmj\" (UID: \"20f4989c-4fe8-49d2-9430-5030a4274b60\") " pod="openshift-image-registry/image-registry-6574d8498f-74vmj"
Nov 04 01:02:28.259955 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.259783    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae02b148-c5ba-4db0-b817-9be4a488c708-service-ca-bundle\") pod \"router-default-5569c8658c-v7txl\" (UID: \"ae02b148-c5ba-4db0-b817-9be4a488c708\") " pod="openshift-ingress/router-default-5569c8658c-v7txl"
Nov 04 01:02:28.259955 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.259818    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de97ada4-9cae-4775-a8f8-014004e17d25-service-ca-bundle\") pod \"insights-operator-7c4986d449-sbjxr\" (UID: \"de97ada4-9cae-4775-a8f8-014004e17d25\") " pod="openshift-insights/insights-operator-7c4986d449-sbjxr"
Nov 04 01:02:28.259955 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.259850    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncqm5\" (UniqueName: \"kubernetes.io/projected/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-kube-api-access-ncqm5\") pod \"image-registry-5c5894ddd-j4phf\" (UID: \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\") " pod="openshift-image-registry/image-registry-5c5894ddd-j4phf"
Nov 04 01:02:28.360210 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.360141    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae02b148-c5ba-4db0-b817-9be4a488c708-metrics-certs\") pod \"router-default-5569c8658c-v7txl\" (UID: \"ae02b148-c5ba-4db0-b817-9be4a488c708\") " pod="openshift-ingress/router-default-5569c8658c-v7txl"
Nov 04 01:02:28.360210 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.360171    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvzfb\" (UniqueName: \"kubernetes.io/projected/de97ada4-9cae-4775-a8f8-014004e17d25-kube-api-access-zvzfb\") pod \"insights-operator-7c4986d449-sbjxr\" (UID: \"de97ada4-9cae-4775-a8f8-014004e17d25\") " pod="openshift-insights/insights-operator-7c4986d449-sbjxr"
Nov 04 01:02:28.360210 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.360190    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7lgb\" (UniqueName: \"kubernetes.io/projected/21d1d23b-7b6e-4ab5-9593-cf7784fffea2-kube-api-access-h7lgb\") pod \"console-operator-7cd56b8dbc-tr8b2\" (UID: \"21d1d23b-7b6e-4ab5-9593-cf7784fffea2\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tr8b2"
Nov 04 01:02:28.360429 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.360216    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kp4q\" (UniqueName: \"kubernetes.io/projected/2f68cb5a-3e22-4dd6-98d0-f28988ea86a2-kube-api-access-6kp4q\") pod \"dns-default-klq54\" (UID: \"2f68cb5a-3e22-4dd6-98d0-f28988ea86a2\") " pod="openshift-dns/dns-default-klq54"
Nov 04 01:02:28.360429 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.360247    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/20f4989c-4fe8-49d2-9430-5030a4274b60-installation-pull-secrets\") pod \"image-registry-6574d8498f-74vmj\" (UID: \"20f4989c-4fe8-49d2-9430-5030a4274b60\") " pod="openshift-image-registry/image-registry-6574d8498f-74vmj"
Nov 04 01:02:28.360429 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:28.360295    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 04 01:02:28.360429 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:28.360351    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae02b148-c5ba-4db0-b817-9be4a488c708-metrics-certs podName:ae02b148-c5ba-4db0-b817-9be4a488c708 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:28.860331181 +0000 UTC m=+32.086179360 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ae02b148-c5ba-4db0-b817-9be4a488c708-metrics-certs") pod "router-default-5569c8658c-v7txl" (UID: "ae02b148-c5ba-4db0-b817-9be4a488c708") : secret "router-metrics-certs-default" not found
Nov 04 01:02:28.360429 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.360384    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fc0b0e48-8adf-4d0d-80dd-ca76846f824a-config\") pod \"service-ca-operator-544d749c8c-8jlt4\" (UID: \"fc0b0e48-8adf-4d0d-80dd-ca76846f824a\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-8jlt4"
Nov 04 01:02:28.360429 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.360415    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rct8p\" (UniqueName: \"kubernetes.io/projected/20f4989c-4fe8-49d2-9430-5030a4274b60-kube-api-access-rct8p\") pod \"image-registry-6574d8498f-74vmj\" (UID: \"20f4989c-4fe8-49d2-9430-5030a4274b60\") " pod="openshift-image-registry/image-registry-6574d8498f-74vmj"
Nov 04 01:02:28.360759 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.360442    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-bound-sa-token\") pod \"image-registry-5c5894ddd-j4phf\" (UID: \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\") " pod="openshift-image-registry/image-registry-5c5894ddd-j4phf"
Nov 04 01:02:28.360759 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.360465    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/21d1d23b-7b6e-4ab5-9593-cf7784fffea2-config\") pod \"console-operator-7cd56b8dbc-tr8b2\" (UID: \"21d1d23b-7b6e-4ab5-9593-cf7784fffea2\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tr8b2"
Nov 04 01:02:28.360759 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.360492    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-registry-certificates\") pod \"image-registry-5c5894ddd-j4phf\" (UID: \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\") " pod="openshift-image-registry/image-registry-5c5894ddd-j4phf"
Nov 04 01:02:28.360759 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.360530    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lr5kt\" (UniqueName: \"kubernetes.io/projected/4d3480a2-034c-4959-8aee-b0759dbfc24f-kube-api-access-lr5kt\") pod \"network-check-source-6f78d59f99-zc968\" (UID: \"4d3480a2-034c-4959-8aee-b0759dbfc24f\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-zc968"
Nov 04 01:02:28.360759 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.360559    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9hjn\" (UniqueName: \"kubernetes.io/projected/ae02b148-c5ba-4db0-b817-9be4a488c708-kube-api-access-b9hjn\") pod \"router-default-5569c8658c-v7txl\" (UID: \"ae02b148-c5ba-4db0-b817-9be4a488c708\") " pod="openshift-ingress/router-default-5569c8658c-v7txl"
Nov 04 01:02:28.360759 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.360591    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fc0b0e48-8adf-4d0d-80dd-ca76846f824a-serving-cert\") pod \"service-ca-operator-544d749c8c-8jlt4\" (UID: \"fc0b0e48-8adf-4d0d-80dd-ca76846f824a\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-8jlt4"
Nov 04 01:02:28.361045 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.360778    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-registry-tls\") pod \"image-registry-5c5894ddd-j4phf\" (UID: \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\") " pod="openshift-image-registry/image-registry-5c5894ddd-j4phf"
Nov 04 01:02:28.361045 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.360817    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de97ada4-9cae-4775-a8f8-014004e17d25-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-sbjxr\" (UID: \"de97ada4-9cae-4775-a8f8-014004e17d25\") " pod="openshift-insights/insights-operator-7c4986d449-sbjxr"
Nov 04 01:02:28.361045 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.360846    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5f285\" (UniqueName: \"kubernetes.io/projected/e41517b6-7ce4-43a6-89a9-76931457ca50-kube-api-access-5f285\") pod \"cluster-samples-operator-6ddb6c8b94-jmxvd\" (UID: \"e41517b6-7ce4-43a6-89a9-76931457ca50\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jmxvd"
Nov 04 01:02:28.361045 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.360879    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dr2p4\" (UniqueName: \"kubernetes.io/projected/d4efaef2-c145-4839-b32f-ecf7a1c76a99-kube-api-access-dr2p4\") pod \"volume-data-source-validator-58c9f5f46d-gdqjb\" (UID: \"d4efaef2-c145-4839-b32f-ecf7a1c76a99\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-gdqjb"
Nov 04 01:02:28.361045 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.360913    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72dk9\" (UniqueName: \"kubernetes.io/projected/4a79348d-8cf7-4429-8c5a-4084cf85fcc5-kube-api-access-72dk9\") pod \"kas-connection-checker-685895c7c9-rff54\" (UID: \"4a79348d-8cf7-4429-8c5a-4084cf85fcc5\") " pod="kube-system/kas-connection-checker-685895c7c9-rff54"
Nov 04 01:02:28.361045 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:28.360921    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 01:02:28.361045 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:28.360937    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5c5894ddd-j4phf: secret "image-registry-tls" not found
Nov 04 01:02:28.361045 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.360951    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/20f4989c-4fe8-49d2-9430-5030a4274b60-bound-sa-token\") pod \"image-registry-6574d8498f-74vmj\" (UID: \"20f4989c-4fe8-49d2-9430-5030a4274b60\") " pod="openshift-image-registry/image-registry-6574d8498f-74vmj"
Nov 04 01:02:28.361045 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:28.360976    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-registry-tls podName:0eedbf45-4e7d-47b7-806f-a6abf9aa311f nodeName:}" failed. No retries permitted until 2026-11-04 01:02:28.860963424 +0000 UTC m=+32.086811616 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-registry-tls") pod "image-registry-5c5894ddd-j4phf" (UID: "0eedbf45-4e7d-47b7-806f-a6abf9aa311f") : secret "image-registry-tls" not found
Nov 04 01:02:28.361045 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.360994    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae02b148-c5ba-4db0-b817-9be4a488c708-service-ca-bundle\") pod \"router-default-5569c8658c-v7txl\" (UID: \"ae02b148-c5ba-4db0-b817-9be4a488c708\") " pod="openshift-ingress/router-default-5569c8658c-v7txl"
Nov 04 01:02:28.361045 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.361021    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de97ada4-9cae-4775-a8f8-014004e17d25-service-ca-bundle\") pod \"insights-operator-7c4986d449-sbjxr\" (UID: \"de97ada4-9cae-4775-a8f8-014004e17d25\") " pod="openshift-insights/insights-operator-7c4986d449-sbjxr"
Nov 04 01:02:28.361045 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.361042    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncqm5\" (UniqueName: \"kubernetes.io/projected/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-kube-api-access-ncqm5\") pod \"image-registry-5c5894ddd-j4phf\" (UID: \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\") " pod="openshift-image-registry/image-registry-5c5894ddd-j4phf"
Nov 04 01:02:28.361588 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.361070    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/21d1d23b-7b6e-4ab5-9593-cf7784fffea2-trusted-ca\") pod \"console-operator-7cd56b8dbc-tr8b2\" (UID: \"21d1d23b-7b6e-4ab5-9593-cf7784fffea2\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tr8b2"
Nov 04 01:02:28.361588 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.361086    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fc0b0e48-8adf-4d0d-80dd-ca76846f824a-config\") pod \"service-ca-operator-544d749c8c-8jlt4\" (UID: \"fc0b0e48-8adf-4d0d-80dd-ca76846f824a\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-8jlt4"
Nov 04 01:02:28.361588 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.361093    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f68cb5a-3e22-4dd6-98d0-f28988ea86a2-tmp-dir\") pod \"dns-default-klq54\" (UID: \"2f68cb5a-3e22-4dd6-98d0-f28988ea86a2\") " pod="openshift-dns/dns-default-klq54"
Nov 04 01:02:28.361588 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.361146    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-registry-certificates\") pod \"image-registry-5c5894ddd-j4phf\" (UID: \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\") " pod="openshift-image-registry/image-registry-5c5894ddd-j4phf"
Nov 04 01:02:28.361588 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.361174    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-ca-trust-extracted\") pod \"image-registry-5c5894ddd-j4phf\" (UID: \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\") " pod="openshift-image-registry/image-registry-5c5894ddd-j4phf"
Nov 04 01:02:28.361588 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.361202    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-trusted-ca\") pod \"image-registry-5c5894ddd-j4phf\" (UID: \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\") " pod="openshift-image-registry/image-registry-5c5894ddd-j4phf"
Nov 04 01:02:28.361588 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.361234    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnhzq\" (UniqueName: \"kubernetes.io/projected/23cb8b1a-61f6-4045-9f5a-3ccc1dc346a5-kube-api-access-wnhzq\") pod \"kas-connection-checker-685895c7c9-6lcfv\" (UID: \"23cb8b1a-61f6-4045-9f5a-3ccc1dc346a5\") " pod="kube-system/kas-connection-checker-685895c7c9-6lcfv"
Nov 04 01:02:28.361588 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.361264    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/20f4989c-4fe8-49d2-9430-5030a4274b60-image-registry-private-configuration\") pod \"image-registry-6574d8498f-74vmj\" (UID: \"20f4989c-4fe8-49d2-9430-5030a4274b60\") " pod="openshift-image-registry/image-registry-6574d8498f-74vmj"
Nov 04 01:02:28.361588 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.361293    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/20f4989c-4fe8-49d2-9430-5030a4274b60-trusted-ca\") pod \"image-registry-6574d8498f-74vmj\" (UID: \"20f4989c-4fe8-49d2-9430-5030a4274b60\") " pod="openshift-image-registry/image-registry-6574d8498f-74vmj"
Nov 04 01:02:28.361588 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.361322    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/82ac4b70-1c01-478c-ae1b-4a60dad06677-cert\") pod \"ingress-canary-4827b\" (UID: \"82ac4b70-1c01-478c-ae1b-4a60dad06677\") " pod="openshift-ingress-canary/ingress-canary-4827b"
Nov 04 01:02:28.361588 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.361353    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fk9hd\" (UniqueName: \"kubernetes.io/projected/82ac4b70-1c01-478c-ae1b-4a60dad06677-kube-api-access-fk9hd\") pod \"ingress-canary-4827b\" (UID: \"82ac4b70-1c01-478c-ae1b-4a60dad06677\") " pod="openshift-ingress-canary/ingress-canary-4827b"
Nov 04 01:02:28.361588 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.361398    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pt8q7\" (UniqueName: \"kubernetes.io/projected/28a8304b-9226-422d-9044-b50678d52971-kube-api-access-pt8q7\") pod \"cluster-monitoring-operator-884576c57-6vxbv\" (UID: \"28a8304b-9226-422d-9044-b50678d52971\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6vxbv"
Nov 04 01:02:28.361588 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.361426    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/de97ada4-9cae-4775-a8f8-014004e17d25-snapshots\") pod \"insights-operator-7c4986d449-sbjxr\" (UID: \"de97ada4-9cae-4775-a8f8-014004e17d25\") " pod="openshift-insights/insights-operator-7c4986d449-sbjxr"
Nov 04 01:02:28.361588 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.361450    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-image-registry-private-configuration\") pod \"image-registry-5c5894ddd-j4phf\" (UID: \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\") " pod="openshift-image-registry/image-registry-5c5894ddd-j4phf"
Nov 04 01:02:28.361588 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.361478    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2f68cb5a-3e22-4dd6-98d0-f28988ea86a2-config-volume\") pod \"dns-default-klq54\" (UID: \"2f68cb5a-3e22-4dd6-98d0-f28988ea86a2\") " pod="openshift-dns/dns-default-klq54"
Nov 04 01:02:28.361588 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.361526    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ae02b148-c5ba-4db0-b817-9be4a488c708-stats-auth\") pod \"router-default-5569c8658c-v7txl\" (UID: \"ae02b148-c5ba-4db0-b817-9be4a488c708\") " pod="openshift-ingress/router-default-5569c8658c-v7txl"
Nov 04 01:02:28.362374 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.361555    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6tkr\" (UniqueName: \"kubernetes.io/projected/fc0b0e48-8adf-4d0d-80dd-ca76846f824a-kube-api-access-r6tkr\") pod \"service-ca-operator-544d749c8c-8jlt4\" (UID: \"fc0b0e48-8adf-4d0d-80dd-ca76846f824a\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-8jlt4"
Nov 04 01:02:28.362374 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.361583    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/21d1d23b-7b6e-4ab5-9593-cf7784fffea2-serving-cert\") pod \"console-operator-7cd56b8dbc-tr8b2\" (UID: \"21d1d23b-7b6e-4ab5-9593-cf7784fffea2\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tr8b2"
Nov 04 01:02:28.362374 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.361636    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/28a8304b-9226-422d-9044-b50678d52971-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-6vxbv\" (UID: \"28a8304b-9226-422d-9044-b50678d52971\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6vxbv"
Nov 04 01:02:28.362374 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.361663    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/20f4989c-4fe8-49d2-9430-5030a4274b60-registry-tls\") pod \"image-registry-6574d8498f-74vmj\" (UID: \"20f4989c-4fe8-49d2-9430-5030a4274b60\") " pod="openshift-image-registry/image-registry-6574d8498f-74vmj"
Nov 04 01:02:28.362374 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.361690    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6t28\" (UniqueName: \"kubernetes.io/projected/a0c9abfd-2f47-4cb9-9d3c-479a6231be97-kube-api-access-p6t28\") pod \"kas-connection-checker-685895c7c9-hhfmk\" (UID: \"a0c9abfd-2f47-4cb9-9d3c-479a6231be97\") " pod="kube-system/kas-connection-checker-685895c7c9-hhfmk"
Nov 04 01:02:28.362374 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.361716    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/20f4989c-4fe8-49d2-9430-5030a4274b60-ca-trust-extracted\") pod \"image-registry-6574d8498f-74vmj\" (UID: \"20f4989c-4fe8-49d2-9430-5030a4274b60\") " pod="openshift-image-registry/image-registry-6574d8498f-74vmj"
Nov 04 01:02:28.362374 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.361743    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ae02b148-c5ba-4db0-b817-9be4a488c708-default-certificate\") pod \"router-default-5569c8658c-v7txl\" (UID: \"ae02b148-c5ba-4db0-b817-9be4a488c708\") " pod="openshift-ingress/router-default-5569c8658c-v7txl"
Nov 04 01:02:28.362374 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.361772    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e41517b6-7ce4-43a6-89a9-76931457ca50-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-jmxvd\" (UID: \"e41517b6-7ce4-43a6-89a9-76931457ca50\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jmxvd"
Nov 04 01:02:28.362374 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.361788    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-ca-trust-extracted\") pod \"image-registry-5c5894ddd-j4phf\" (UID: \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\") " pod="openshift-image-registry/image-registry-5c5894ddd-j4phf"
Nov 04 01:02:28.362374 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.361801    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3d2d0e05-a773-4442-ab38-d460ad32f51e-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-58wxt\" (UID: \"3d2d0e05-a773-4442-ab38-d460ad32f51e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-58wxt"
Nov 04 01:02:28.362374 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.361829    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f68cb5a-3e22-4dd6-98d0-f28988ea86a2-metrics-tls\") pod \"dns-default-klq54\" (UID: \"2f68cb5a-3e22-4dd6-98d0-f28988ea86a2\") " pod="openshift-dns/dns-default-klq54"
Nov 04 01:02:28.362374 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.361791    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/21d1d23b-7b6e-4ab5-9593-cf7784fffea2-trusted-ca\") pod \"console-operator-7cd56b8dbc-tr8b2\" (UID: \"21d1d23b-7b6e-4ab5-9593-cf7784fffea2\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tr8b2"
Nov 04 01:02:28.362374 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.361857    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-installation-pull-secrets\") pod \"image-registry-5c5894ddd-j4phf\" (UID: \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\") " pod="openshift-image-registry/image-registry-5c5894ddd-j4phf"
Nov 04 01:02:28.362374 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.361267    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/21d1d23b-7b6e-4ab5-9593-cf7784fffea2-config\") pod \"console-operator-7cd56b8dbc-tr8b2\" (UID: \"21d1d23b-7b6e-4ab5-9593-cf7784fffea2\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tr8b2"
Nov 04 01:02:28.362374 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.361885    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3739befd-6689-4197-b793-6f5b4b796880-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-27rmf\" (UID: \"3739befd-6689-4197-b793-6f5b4b796880\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-27rmf"
Nov 04 01:02:28.363112 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.361917    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3d2d0e05-a773-4442-ab38-d460ad32f51e-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-58wxt\" (UID: \"3d2d0e05-a773-4442-ab38-d460ad32f51e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-58wxt"
Nov 04 01:02:28.363112 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.361945    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3739befd-6689-4197-b793-6f5b4b796880-nginx-conf\") pod \"networking-console-plugin-86c478bb88-27rmf\" (UID: \"3739befd-6689-4197-b793-6f5b4b796880\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-27rmf"
Nov 04 01:02:28.363112 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.361977    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nfw8\" (UniqueName: \"kubernetes.io/projected/3d2d0e05-a773-4442-ab38-d460ad32f51e-kube-api-access-8nfw8\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-58wxt\" (UID: \"3d2d0e05-a773-4442-ab38-d460ad32f51e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-58wxt"
Nov 04 01:02:28.363112 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.362019    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/28a8304b-9226-422d-9044-b50678d52971-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-6vxbv\" (UID: \"28a8304b-9226-422d-9044-b50678d52971\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6vxbv"
Nov 04 01:02:28.363112 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.362046    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/de97ada4-9cae-4775-a8f8-014004e17d25-serving-cert\") pod \"insights-operator-7c4986d449-sbjxr\" (UID: \"de97ada4-9cae-4775-a8f8-014004e17d25\") " pod="openshift-insights/insights-operator-7c4986d449-sbjxr"
Nov 04 01:02:28.363112 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.362077    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/20f4989c-4fe8-49d2-9430-5030a4274b60-registry-certificates\") pod \"image-registry-6574d8498f-74vmj\" (UID: \"20f4989c-4fe8-49d2-9430-5030a4274b60\") " pod="openshift-image-registry/image-registry-6574d8498f-74vmj"
Nov 04 01:02:28.363112 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.362100    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/de97ada4-9cae-4775-a8f8-014004e17d25-tmp\") pod \"insights-operator-7c4986d449-sbjxr\" (UID: \"de97ada4-9cae-4775-a8f8-014004e17d25\") " pod="openshift-insights/insights-operator-7c4986d449-sbjxr"
Nov 04 01:02:28.363112 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.362170    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/e41517b6-7ce4-43a6-89a9-76931457ca50-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-jmxvd\" (UID: \"e41517b6-7ce4-43a6-89a9-76931457ca50\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jmxvd"
Nov 04 01:02:28.363112 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.362217    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de97ada4-9cae-4775-a8f8-014004e17d25-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-sbjxr\" (UID: \"de97ada4-9cae-4775-a8f8-014004e17d25\") " pod="openshift-insights/insights-operator-7c4986d449-sbjxr"
Nov 04 01:02:28.363112 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:28.362307    2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 04 01:02:28.363112 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:28.362354    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e41517b6-7ce4-43a6-89a9-76931457ca50-samples-operator-tls podName:e41517b6-7ce4-43a6-89a9-76931457ca50 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:28.86234039 +0000 UTC m=+32.088188555 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e41517b6-7ce4-43a6-89a9-76931457ca50-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-jmxvd" (UID: "e41517b6-7ce4-43a6-89a9-76931457ca50") : secret "samples-operator-tls" not found
Nov 04 01:02:28.363112 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.362534    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-trusted-ca\") pod \"image-registry-5c5894ddd-j4phf\" (UID: \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\") " pod="openshift-image-registry/image-registry-5c5894ddd-j4phf"
Nov 04 01:02:28.363112 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.362869    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3d2d0e05-a773-4442-ab38-d460ad32f51e-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-58wxt\" (UID: \"3d2d0e05-a773-4442-ab38-d460ad32f51e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-58wxt"
Nov 04 01:02:28.363745 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.363135    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/20f4989c-4fe8-49d2-9430-5030a4274b60-trusted-ca\") pod \"image-registry-6574d8498f-74vmj\" (UID: \"20f4989c-4fe8-49d2-9430-5030a4274b60\") " pod="openshift-image-registry/image-registry-6574d8498f-74vmj"
Nov 04 01:02:28.363745 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:28.363271    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ae02b148-c5ba-4db0-b817-9be4a488c708-service-ca-bundle podName:ae02b148-c5ba-4db0-b817-9be4a488c708 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:28.863251872 +0000 UTC m=+32.089100042 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ae02b148-c5ba-4db0-b817-9be4a488c708-service-ca-bundle") pod "router-default-5569c8658c-v7txl" (UID: "ae02b148-c5ba-4db0-b817-9be4a488c708") : configmap references non-existent config key: service-ca.crt
Nov 04 01:02:28.363745 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.363338    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/20f4989c-4fe8-49d2-9430-5030a4274b60-registry-certificates\") pod \"image-registry-6574d8498f-74vmj\" (UID: \"20f4989c-4fe8-49d2-9430-5030a4274b60\") " pod="openshift-image-registry/image-registry-6574d8498f-74vmj"
Nov 04 01:02:28.363745 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.363454    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/de97ada4-9cae-4775-a8f8-014004e17d25-tmp\") pod \"insights-operator-7c4986d449-sbjxr\" (UID: \"de97ada4-9cae-4775-a8f8-014004e17d25\") " pod="openshift-insights/insights-operator-7c4986d449-sbjxr"
Nov 04 01:02:28.365258 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.365225    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fc0b0e48-8adf-4d0d-80dd-ca76846f824a-serving-cert\") pod \"service-ca-operator-544d749c8c-8jlt4\" (UID: \"fc0b0e48-8adf-4d0d-80dd-ca76846f824a\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-8jlt4"
Nov 04 01:02:28.365396 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.365341    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/28a8304b-9226-422d-9044-b50678d52971-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-6vxbv\" (UID: \"28a8304b-9226-422d-9044-b50678d52971\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6vxbv"
Nov 04 01:02:28.365396 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:28.365387    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 01:02:28.365509 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:28.365405    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6574d8498f-74vmj: secret "image-registry-tls" not found
Nov 04 01:02:28.365509 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:28.365472    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/20f4989c-4fe8-49d2-9430-5030a4274b60-registry-tls podName:20f4989c-4fe8-49d2-9430-5030a4274b60 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:28.86545448 +0000 UTC m=+32.091302663 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/20f4989c-4fe8-49d2-9430-5030a4274b60-registry-tls") pod "image-registry-6574d8498f-74vmj" (UID: "20f4989c-4fe8-49d2-9430-5030a4274b60") : secret "image-registry-tls" not found
Nov 04 01:02:28.365691 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.365668    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ae02b148-c5ba-4db0-b817-9be4a488c708-default-certificate\") pod \"router-default-5569c8658c-v7txl\" (UID: \"ae02b148-c5ba-4db0-b817-9be4a488c708\") " pod="openshift-ingress/router-default-5569c8658c-v7txl"
Nov 04 01:02:28.366515 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.365997    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/20f4989c-4fe8-49d2-9430-5030a4274b60-ca-trust-extracted\") pod \"image-registry-6574d8498f-74vmj\" (UID: \"20f4989c-4fe8-49d2-9430-5030a4274b60\") " pod="openshift-image-registry/image-registry-6574d8498f-74vmj"
Nov 04 01:02:28.366515 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.366068    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/de97ada4-9cae-4775-a8f8-014004e17d25-snapshots\") pod \"insights-operator-7c4986d449-sbjxr\" (UID: \"de97ada4-9cae-4775-a8f8-014004e17d25\") " pod="openshift-insights/insights-operator-7c4986d449-sbjxr"
Nov 04 01:02:28.366515 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.366338    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/20f4989c-4fe8-49d2-9430-5030a4274b60-image-registry-private-configuration\") pod \"image-registry-6574d8498f-74vmj\" (UID: \"20f4989c-4fe8-49d2-9430-5030a4274b60\") " pod="openshift-image-registry/image-registry-6574d8498f-74vmj"
Nov 04 01:02:28.366515 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.366464    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de97ada4-9cae-4775-a8f8-014004e17d25-service-ca-bundle\") pod \"insights-operator-7c4986d449-sbjxr\" (UID: \"de97ada4-9cae-4775-a8f8-014004e17d25\") " pod="openshift-insights/insights-operator-7c4986d449-sbjxr"
Nov 04 01:02:28.366515 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.366508    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-installation-pull-secrets\") pod \"image-registry-5c5894ddd-j4phf\" (UID: \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\") " pod="openshift-image-registry/image-registry-5c5894ddd-j4phf"
Nov 04 01:02:28.370799 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.367141    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/e41517b6-7ce4-43a6-89a9-76931457ca50-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-jmxvd\" (UID: \"e41517b6-7ce4-43a6-89a9-76931457ca50\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jmxvd"
Nov 04 01:02:28.370799 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.367243    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/20f4989c-4fe8-49d2-9430-5030a4274b60-installation-pull-secrets\") pod \"image-registry-6574d8498f-74vmj\" (UID: \"20f4989c-4fe8-49d2-9430-5030a4274b60\") " pod="openshift-image-registry/image-registry-6574d8498f-74vmj"
Nov 04 01:02:28.370799 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.367892    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/21d1d23b-7b6e-4ab5-9593-cf7784fffea2-serving-cert\") pod \"console-operator-7cd56b8dbc-tr8b2\" (UID: \"21d1d23b-7b6e-4ab5-9593-cf7784fffea2\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tr8b2"
Nov 04 01:02:28.371256 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:28.371199    2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 04 01:02:28.371335 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:28.371288    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28a8304b-9226-422d-9044-b50678d52971-cluster-monitoring-operator-tls podName:28a8304b-9226-422d-9044-b50678d52971 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:28.871269737 +0000 UTC m=+32.097117915 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/28a8304b-9226-422d-9044-b50678d52971-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-6vxbv" (UID: "28a8304b-9226-422d-9044-b50678d52971") : secret "cluster-monitoring-operator-tls" not found
Nov 04 01:02:28.372109 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.372024    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/de97ada4-9cae-4775-a8f8-014004e17d25-serving-cert\") pod \"insights-operator-7c4986d449-sbjxr\" (UID: \"de97ada4-9cae-4775-a8f8-014004e17d25\") " pod="openshift-insights/insights-operator-7c4986d449-sbjxr"
Nov 04 01:02:28.373084 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.373059    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-image-registry-private-configuration\") pod \"image-registry-5c5894ddd-j4phf\" (UID: \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\") " pod="openshift-image-registry/image-registry-5c5894ddd-j4phf"
Nov 04 01:02:28.373622 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.373592    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3d2d0e05-a773-4442-ab38-d460ad32f51e-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-58wxt\" (UID: \"3d2d0e05-a773-4442-ab38-d460ad32f51e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-58wxt"
Nov 04 01:02:28.373845 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.373825    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ae02b148-c5ba-4db0-b817-9be4a488c708-stats-auth\") pod \"router-default-5569c8658c-v7txl\" (UID: \"ae02b148-c5ba-4db0-b817-9be4a488c708\") " pod="openshift-ingress/router-default-5569c8658c-v7txl"
Nov 04 01:02:28.373936 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.373922    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvzfb\" (UniqueName: \"kubernetes.io/projected/de97ada4-9cae-4775-a8f8-014004e17d25-kube-api-access-zvzfb\") pod \"insights-operator-7c4986d449-sbjxr\" (UID: \"de97ada4-9cae-4775-a8f8-014004e17d25\") " pod="openshift-insights/insights-operator-7c4986d449-sbjxr"
Nov 04 01:02:28.377603 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.377573    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/20f4989c-4fe8-49d2-9430-5030a4274b60-bound-sa-token\") pod \"image-registry-6574d8498f-74vmj\" (UID: \"20f4989c-4fe8-49d2-9430-5030a4274b60\") " pod="openshift-image-registry/image-registry-6574d8498f-74vmj"
Nov 04 01:02:28.377849 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.377824    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5f285\" (UniqueName: \"kubernetes.io/projected/e41517b6-7ce4-43a6-89a9-76931457ca50-kube-api-access-5f285\") pod \"cluster-samples-operator-6ddb6c8b94-jmxvd\" (UID: \"e41517b6-7ce4-43a6-89a9-76931457ca50\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jmxvd"
Nov 04 01:02:28.377849 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.377840    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-72dk9\" (UniqueName: \"kubernetes.io/projected/4a79348d-8cf7-4429-8c5a-4084cf85fcc5-kube-api-access-72dk9\") pod \"kas-connection-checker-685895c7c9-rff54\" (UID: \"4a79348d-8cf7-4429-8c5a-4084cf85fcc5\") " pod="kube-system/kas-connection-checker-685895c7c9-rff54"
Nov 04 01:02:28.377988 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.377857    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-bound-sa-token\") pod \"image-registry-5c5894ddd-j4phf\" (UID: \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\") " pod="openshift-image-registry/image-registry-5c5894ddd-j4phf"
Nov 04 01:02:28.378561 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.378519    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9hjn\" (UniqueName: \"kubernetes.io/projected/ae02b148-c5ba-4db0-b817-9be4a488c708-kube-api-access-b9hjn\") pod \"router-default-5569c8658c-v7txl\" (UID: \"ae02b148-c5ba-4db0-b817-9be4a488c708\") " pod="openshift-ingress/router-default-5569c8658c-v7txl"
Nov 04 01:02:28.378831 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.378813    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnhzq\" (UniqueName: \"kubernetes.io/projected/23cb8b1a-61f6-4045-9f5a-3ccc1dc346a5-kube-api-access-wnhzq\") pod \"kas-connection-checker-685895c7c9-6lcfv\" (UID: \"23cb8b1a-61f6-4045-9f5a-3ccc1dc346a5\") " pod="kube-system/kas-connection-checker-685895c7c9-6lcfv"
Nov 04 01:02:28.379065 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.379042    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rct8p\" (UniqueName: \"kubernetes.io/projected/20f4989c-4fe8-49d2-9430-5030a4274b60-kube-api-access-rct8p\") pod \"image-registry-6574d8498f-74vmj\" (UID: \"20f4989c-4fe8-49d2-9430-5030a4274b60\") " pod="openshift-image-registry/image-registry-6574d8498f-74vmj"
Nov 04 01:02:28.379990 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.379970    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dr2p4\" (UniqueName: \"kubernetes.io/projected/d4efaef2-c145-4839-b32f-ecf7a1c76a99-kube-api-access-dr2p4\") pod \"volume-data-source-validator-58c9f5f46d-gdqjb\" (UID: \"d4efaef2-c145-4839-b32f-ecf7a1c76a99\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-gdqjb"
Nov 04 01:02:28.380220 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.380200    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncqm5\" (UniqueName: \"kubernetes.io/projected/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-kube-api-access-ncqm5\") pod \"image-registry-5c5894ddd-j4phf\" (UID: \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\") " pod="openshift-image-registry/image-registry-5c5894ddd-j4phf"
Nov 04 01:02:28.380758 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.380740    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nfw8\" (UniqueName: \"kubernetes.io/projected/3d2d0e05-a773-4442-ab38-d460ad32f51e-kube-api-access-8nfw8\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-58wxt\" (UID: \"3d2d0e05-a773-4442-ab38-d460ad32f51e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-58wxt"
Nov 04 01:02:28.381104 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.381082    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7lgb\" (UniqueName: \"kubernetes.io/projected/21d1d23b-7b6e-4ab5-9593-cf7784fffea2-kube-api-access-h7lgb\") pod \"console-operator-7cd56b8dbc-tr8b2\" (UID: \"21d1d23b-7b6e-4ab5-9593-cf7784fffea2\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tr8b2"
Nov 04 01:02:28.381845 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.381825    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pt8q7\" (UniqueName: \"kubernetes.io/projected/28a8304b-9226-422d-9044-b50678d52971-kube-api-access-pt8q7\") pod \"cluster-monitoring-operator-884576c57-6vxbv\" (UID: \"28a8304b-9226-422d-9044-b50678d52971\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6vxbv"
Nov 04 01:02:28.382377 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.382351    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6t28\" (UniqueName: \"kubernetes.io/projected/a0c9abfd-2f47-4cb9-9d3c-479a6231be97-kube-api-access-p6t28\") pod \"kas-connection-checker-685895c7c9-hhfmk\" (UID: \"a0c9abfd-2f47-4cb9-9d3c-479a6231be97\") " pod="kube-system/kas-connection-checker-685895c7c9-hhfmk"
Nov 04 01:02:28.382447 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.382391    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6tkr\" (UniqueName: \"kubernetes.io/projected/fc0b0e48-8adf-4d0d-80dd-ca76846f824a-kube-api-access-r6tkr\") pod \"service-ca-operator-544d749c8c-8jlt4\" (UID: \"fc0b0e48-8adf-4d0d-80dd-ca76846f824a\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-8jlt4"
Nov 04 01:02:28.442250 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.442226    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-hhfmk"
Nov 04 01:02:28.449591 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.449573    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-6lcfv"
Nov 04 01:02:28.462996 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.462971    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kp4q\" (UniqueName: \"kubernetes.io/projected/2f68cb5a-3e22-4dd6-98d0-f28988ea86a2-kube-api-access-6kp4q\") pod \"dns-default-klq54\" (UID: \"2f68cb5a-3e22-4dd6-98d0-f28988ea86a2\") " pod="openshift-dns/dns-default-klq54"
Nov 04 01:02:28.463097 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.463031    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lr5kt\" (UniqueName: \"kubernetes.io/projected/4d3480a2-034c-4959-8aee-b0759dbfc24f-kube-api-access-lr5kt\") pod \"network-check-source-6f78d59f99-zc968\" (UID: \"4d3480a2-034c-4959-8aee-b0759dbfc24f\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-zc968"
Nov 04 01:02:28.463152 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.463106    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f68cb5a-3e22-4dd6-98d0-f28988ea86a2-tmp-dir\") pod \"dns-default-klq54\" (UID: \"2f68cb5a-3e22-4dd6-98d0-f28988ea86a2\") " pod="openshift-dns/dns-default-klq54"
Nov 04 01:02:28.463207 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.463151    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/82ac4b70-1c01-478c-ae1b-4a60dad06677-cert\") pod \"ingress-canary-4827b\" (UID: \"82ac4b70-1c01-478c-ae1b-4a60dad06677\") " pod="openshift-ingress-canary/ingress-canary-4827b"
Nov 04 01:02:28.463207 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.463175    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fk9hd\" (UniqueName: \"kubernetes.io/projected/82ac4b70-1c01-478c-ae1b-4a60dad06677-kube-api-access-fk9hd\") pod \"ingress-canary-4827b\" (UID: \"82ac4b70-1c01-478c-ae1b-4a60dad06677\") " pod="openshift-ingress-canary/ingress-canary-4827b"
Nov 04 01:02:28.463307 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.463217    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2f68cb5a-3e22-4dd6-98d0-f28988ea86a2-config-volume\") pod \"dns-default-klq54\" (UID: \"2f68cb5a-3e22-4dd6-98d0-f28988ea86a2\") " pod="openshift-dns/dns-default-klq54"
Nov 04 01:02:28.463307 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.463296    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f68cb5a-3e22-4dd6-98d0-f28988ea86a2-metrics-tls\") pod \"dns-default-klq54\" (UID: \"2f68cb5a-3e22-4dd6-98d0-f28988ea86a2\") " pod="openshift-dns/dns-default-klq54"
Nov 04 01:02:28.463396 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.463325    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3739befd-6689-4197-b793-6f5b4b796880-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-27rmf\" (UID: \"3739befd-6689-4197-b793-6f5b4b796880\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-27rmf"
Nov 04 01:02:28.463396 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.463355    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3739befd-6689-4197-b793-6f5b4b796880-nginx-conf\") pod \"networking-console-plugin-86c478bb88-27rmf\" (UID: \"3739befd-6689-4197-b793-6f5b4b796880\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-27rmf"
Nov 04 01:02:28.463493 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:28.463398    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 01:02:28.463493 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:28.463470    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f68cb5a-3e22-4dd6-98d0-f28988ea86a2-metrics-tls podName:2f68cb5a-3e22-4dd6-98d0-f28988ea86a2 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:28.963450567 +0000 UTC m=+32.189298740 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2f68cb5a-3e22-4dd6-98d0-f28988ea86a2-metrics-tls") pod "dns-default-klq54" (UID: "2f68cb5a-3e22-4dd6-98d0-f28988ea86a2") : secret "dns-default-metrics-tls" not found
Nov 04 01:02:28.463493 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.463470    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f68cb5a-3e22-4dd6-98d0-f28988ea86a2-tmp-dir\") pod \"dns-default-klq54\" (UID: \"2f68cb5a-3e22-4dd6-98d0-f28988ea86a2\") " pod="openshift-dns/dns-default-klq54"
Nov 04 01:02:28.463663 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:28.463531    2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 04 01:02:28.463663 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:28.463587    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3739befd-6689-4197-b793-6f5b4b796880-networking-console-plugin-cert podName:3739befd-6689-4197-b793-6f5b4b796880 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:28.963572253 +0000 UTC m=+32.189420432 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3739befd-6689-4197-b793-6f5b4b796880-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-27rmf" (UID: "3739befd-6689-4197-b793-6f5b4b796880") : secret "networking-console-plugin-cert" not found
Nov 04 01:02:28.464046 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.464022    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2f68cb5a-3e22-4dd6-98d0-f28988ea86a2-config-volume\") pod \"dns-default-klq54\" (UID: \"2f68cb5a-3e22-4dd6-98d0-f28988ea86a2\") " pod="openshift-dns/dns-default-klq54"
Nov 04 01:02:28.464224 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.464200    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3739befd-6689-4197-b793-6f5b4b796880-nginx-conf\") pod \"networking-console-plugin-86c478bb88-27rmf\" (UID: \"3739befd-6689-4197-b793-6f5b4b796880\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-27rmf"
Nov 04 01:02:28.465423 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.465401    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-rff54"
Nov 04 01:02:28.465515 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.465424    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/82ac4b70-1c01-478c-ae1b-4a60dad06677-cert\") pod \"ingress-canary-4827b\" (UID: \"82ac4b70-1c01-478c-ae1b-4a60dad06677\") " pod="openshift-ingress-canary/ingress-canary-4827b"
Nov 04 01:02:28.482646 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.474633    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-gdqjb"
Nov 04 01:02:28.482646 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.475968    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fk9hd\" (UniqueName: \"kubernetes.io/projected/82ac4b70-1c01-478c-ae1b-4a60dad06677-kube-api-access-fk9hd\") pod \"ingress-canary-4827b\" (UID: \"82ac4b70-1c01-478c-ae1b-4a60dad06677\") " pod="openshift-ingress-canary/ingress-canary-4827b"
Nov 04 01:02:28.482646 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.480936    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kp4q\" (UniqueName: \"kubernetes.io/projected/2f68cb5a-3e22-4dd6-98d0-f28988ea86a2-kube-api-access-6kp4q\") pod \"dns-default-klq54\" (UID: \"2f68cb5a-3e22-4dd6-98d0-f28988ea86a2\") " pod="openshift-dns/dns-default-klq54"
Nov 04 01:02:28.482646 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.482324    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lr5kt\" (UniqueName: \"kubernetes.io/projected/4d3480a2-034c-4959-8aee-b0759dbfc24f-kube-api-access-lr5kt\") pod \"network-check-source-6f78d59f99-zc968\" (UID: \"4d3480a2-034c-4959-8aee-b0759dbfc24f\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-zc968"
Nov 04 01:02:28.498960 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.498928    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-58wxt"
Nov 04 01:02:28.507775 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.507394    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-sbjxr"
Nov 04 01:02:28.517649 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.517376    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-tr8b2"
Nov 04 01:02:28.522822 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.522709    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-8jlt4"
Nov 04 01:02:28.556442 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.556368    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-zc968"
Nov 04 01:02:28.563121 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.563063    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4827b"
Nov 04 01:02:28.591732 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.590754    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gnrzw"
Nov 04 01:02:28.593634 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.591007    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wxhng"
Nov 04 01:02:28.602336 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.602305    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-k9wxw\""
Nov 04 01:02:28.603873 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.602457    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-9c9zb\""
Nov 04 01:02:28.603873 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.602595    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 04 01:02:28.747892 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.747803    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-hhfmk"]
Nov 04 01:02:28.751243 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:02:28.751217    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0c9abfd_2f47_4cb9_9d3c_479a6231be97.slice/crio-9e10a941535f0a270713e2870bafb42f54750a19213e1f8a55fd29ff93f72c15 WatchSource:0}: Error finding container 9e10a941535f0a270713e2870bafb42f54750a19213e1f8a55fd29ff93f72c15: Status 404 returned error can't find the container with id 9e10a941535f0a270713e2870bafb42f54750a19213e1f8a55fd29ff93f72c15
Nov 04 01:02:28.751677 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.751593    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-6lcfv"]
Nov 04 01:02:28.753848 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.753825    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-rff54"]
Nov 04 01:02:28.754090 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:02:28.754044    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23cb8b1a_61f6_4045_9f5a_3ccc1dc346a5.slice/crio-7e3cdeb00325e347f39ce1412a25035ef8094d1edef60d2ab1e69b2929f41095 WatchSource:0}: Error finding container 7e3cdeb00325e347f39ce1412a25035ef8094d1edef60d2ab1e69b2929f41095: Status 404 returned error can't find the container with id 7e3cdeb00325e347f39ce1412a25035ef8094d1edef60d2ab1e69b2929f41095
Nov 04 01:02:28.796764 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.796665    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-6lcfv" event={"ID":"23cb8b1a-61f6-4045-9f5a-3ccc1dc346a5","Type":"ContainerStarted","Data":"7e3cdeb00325e347f39ce1412a25035ef8094d1edef60d2ab1e69b2929f41095"}
Nov 04 01:02:28.798245 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.798207    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-hhfmk" event={"ID":"a0c9abfd-2f47-4cb9-9d3c-479a6231be97","Type":"ContainerStarted","Data":"9e10a941535f0a270713e2870bafb42f54750a19213e1f8a55fd29ff93f72c15"}
Nov 04 01:02:28.799722 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.799693    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-rff54" event={"ID":"4a79348d-8cf7-4429-8c5a-4084cf85fcc5","Type":"ContainerStarted","Data":"dfb26f56a760af7b098af88b4251f984082532654cac9d3d7219fdf618cfd1ca"}
Nov 04 01:02:28.804301 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.804042    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-gdqjb"]
Nov 04 01:02:28.806364 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:02:28.806344    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4efaef2_c145_4839_b32f_ecf7a1c76a99.slice/crio-64c1b58d0019a5ac8bd199acdabfb9bde69dc2b8d33819bb4d5ad0cf732bbfdc WatchSource:0}: Error finding container 64c1b58d0019a5ac8bd199acdabfb9bde69dc2b8d33819bb4d5ad0cf732bbfdc: Status 404 returned error can't find the container with id 64c1b58d0019a5ac8bd199acdabfb9bde69dc2b8d33819bb4d5ad0cf732bbfdc
Nov 04 01:02:28.865075 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.865050    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-58wxt"]
Nov 04 01:02:28.869441 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.868595    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae02b148-c5ba-4db0-b817-9be4a488c708-metrics-certs\") pod \"router-default-5569c8658c-v7txl\" (UID: \"ae02b148-c5ba-4db0-b817-9be4a488c708\") " pod="openshift-ingress/router-default-5569c8658c-v7txl"
Nov 04 01:02:28.869441 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.868686    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-registry-tls\") pod \"image-registry-5c5894ddd-j4phf\" (UID: \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\") " pod="openshift-image-registry/image-registry-5c5894ddd-j4phf"
Nov 04 01:02:28.869441 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:28.868851    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 04 01:02:28.869441 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:28.868911    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae02b148-c5ba-4db0-b817-9be4a488c708-metrics-certs podName:ae02b148-c5ba-4db0-b817-9be4a488c708 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:29.868893395 +0000 UTC m=+33.094741577 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ae02b148-c5ba-4db0-b817-9be4a488c708-metrics-certs") pod "router-default-5569c8658c-v7txl" (UID: "ae02b148-c5ba-4db0-b817-9be4a488c708") : secret "router-metrics-certs-default" not found
Nov 04 01:02:28.869701 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:28.869586    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 01:02:28.869701 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:28.869603    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5c5894ddd-j4phf: secret "image-registry-tls" not found
Nov 04 01:02:28.869701 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:28.869671    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-registry-tls podName:0eedbf45-4e7d-47b7-806f-a6abf9aa311f nodeName:}" failed. No retries permitted until 2026-11-04 01:02:29.869654607 +0000 UTC m=+33.095502773 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-registry-tls") pod "image-registry-5c5894ddd-j4phf" (UID: "0eedbf45-4e7d-47b7-806f-a6abf9aa311f") : secret "image-registry-tls" not found
Nov 04 01:02:28.869845 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:02:28.869731    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d2d0e05_a773_4442_ab38_d460ad32f51e.slice/crio-b1f9979e91f13674381686bc6d8787ce98d980e2ee7acce22758576bc04fe489 WatchSource:0}: Error finding container b1f9979e91f13674381686bc6d8787ce98d980e2ee7acce22758576bc04fe489: Status 404 returned error can't find the container with id b1f9979e91f13674381686bc6d8787ce98d980e2ee7acce22758576bc04fe489
Nov 04 01:02:28.869970 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.869948    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae02b148-c5ba-4db0-b817-9be4a488c708-service-ca-bundle\") pod \"router-default-5569c8658c-v7txl\" (UID: \"ae02b148-c5ba-4db0-b817-9be4a488c708\") " pod="openshift-ingress/router-default-5569c8658c-v7txl"
Nov 04 01:02:28.870080 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.870053    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/20f4989c-4fe8-49d2-9430-5030a4274b60-registry-tls\") pod \"image-registry-6574d8498f-74vmj\" (UID: \"20f4989c-4fe8-49d2-9430-5030a4274b60\") " pod="openshift-image-registry/image-registry-6574d8498f-74vmj"
Nov 04 01:02:28.870141 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.870078    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e41517b6-7ce4-43a6-89a9-76931457ca50-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-jmxvd\" (UID: \"e41517b6-7ce4-43a6-89a9-76931457ca50\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jmxvd"
Nov 04 01:02:28.870313 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:28.870216    2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 04 01:02:28.870313 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:28.870265    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e41517b6-7ce4-43a6-89a9-76931457ca50-samples-operator-tls podName:e41517b6-7ce4-43a6-89a9-76931457ca50 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:29.870252163 +0000 UTC m=+33.096100334 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e41517b6-7ce4-43a6-89a9-76931457ca50-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-jmxvd" (UID: "e41517b6-7ce4-43a6-89a9-76931457ca50") : secret "samples-operator-tls" not found
Nov 04 01:02:28.870313 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:28.870290    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ae02b148-c5ba-4db0-b817-9be4a488c708-service-ca-bundle podName:ae02b148-c5ba-4db0-b817-9be4a488c708 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:29.870275888 +0000 UTC m=+33.096124058 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ae02b148-c5ba-4db0-b817-9be4a488c708-service-ca-bundle") pod "router-default-5569c8658c-v7txl" (UID: "ae02b148-c5ba-4db0-b817-9be4a488c708") : configmap references non-existent config key: service-ca.crt
Nov 04 01:02:28.870473 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:28.870326    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 01:02:28.870473 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:28.870336    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6574d8498f-74vmj: secret "image-registry-tls" not found
Nov 04 01:02:28.870473 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:28.870372    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/20f4989c-4fe8-49d2-9430-5030a4274b60-registry-tls podName:20f4989c-4fe8-49d2-9430-5030a4274b60 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:29.870362908 +0000 UTC m=+33.096211075 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/20f4989c-4fe8-49d2-9430-5030a4274b60-registry-tls") pod "image-registry-6574d8498f-74vmj" (UID: "20f4989c-4fe8-49d2-9430-5030a4274b60") : secret "image-registry-tls" not found
Nov 04 01:02:28.871538 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.871478    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-sbjxr"]
Nov 04 01:02:28.872644 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.872622    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-tr8b2"]
Nov 04 01:02:28.888410 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:02:28.888387    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21d1d23b_7b6e_4ab5_9593_cf7784fffea2.slice/crio-fdee7cbd173f345475fe82fffb45ac2aa6ca0a47294faa98fd31d1e0363d4843 WatchSource:0}: Error finding container fdee7cbd173f345475fe82fffb45ac2aa6ca0a47294faa98fd31d1e0363d4843: Status 404 returned error can't find the container with id fdee7cbd173f345475fe82fffb45ac2aa6ca0a47294faa98fd31d1e0363d4843
Nov 04 01:02:28.890167 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:02:28.889745    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde97ada4_9cae_4775_a8f8_014004e17d25.slice/crio-3e160b420233830856efd74439197cb2faeb82d999dd903b68336089f6f85a7b WatchSource:0}: Error finding container 3e160b420233830856efd74439197cb2faeb82d999dd903b68336089f6f85a7b: Status 404 returned error can't find the container with id 3e160b420233830856efd74439197cb2faeb82d999dd903b68336089f6f85a7b
Nov 04 01:02:28.926468 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.926443    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-zc968"]
Nov 04 01:02:28.929876 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.929850    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4827b"]
Nov 04 01:02:28.931319 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.931276    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-8jlt4"]
Nov 04 01:02:28.932728 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:02:28.932709    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82ac4b70_1c01_478c_ae1b_4a60dad06677.slice/crio-36784fac827113d028c4492f22911e0c8328324c80e663aefccb8c8f4f19bddc WatchSource:0}: Error finding container 36784fac827113d028c4492f22911e0c8328324c80e663aefccb8c8f4f19bddc: Status 404 returned error can't find the container with id 36784fac827113d028c4492f22911e0c8328324c80e663aefccb8c8f4f19bddc
Nov 04 01:02:28.934428 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:02:28.934378    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc0b0e48_8adf_4d0d_80dd_ca76846f824a.slice/crio-70d19664f15ffde6ce00439daf61c56b10552248f9291702fdf875ab2702c2d4 WatchSource:0}: Error finding container 70d19664f15ffde6ce00439daf61c56b10552248f9291702fdf875ab2702c2d4: Status 404 returned error can't find the container with id 70d19664f15ffde6ce00439daf61c56b10552248f9291702fdf875ab2702c2d4
Nov 04 01:02:28.971367 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.971341    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/28a8304b-9226-422d-9044-b50678d52971-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-6vxbv\" (UID: \"28a8304b-9226-422d-9044-b50678d52971\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6vxbv"
Nov 04 01:02:28.971523 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.971389    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f68cb5a-3e22-4dd6-98d0-f28988ea86a2-metrics-tls\") pod \"dns-default-klq54\" (UID: \"2f68cb5a-3e22-4dd6-98d0-f28988ea86a2\") " pod="openshift-dns/dns-default-klq54"
Nov 04 01:02:28.971523 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:28.971416    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3739befd-6689-4197-b793-6f5b4b796880-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-27rmf\" (UID: \"3739befd-6689-4197-b793-6f5b4b796880\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-27rmf"
Nov 04 01:02:28.971594 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:28.971548    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 01:02:28.971594 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:28.971548    2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 04 01:02:28.971681 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:28.971618    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f68cb5a-3e22-4dd6-98d0-f28988ea86a2-metrics-tls podName:2f68cb5a-3e22-4dd6-98d0-f28988ea86a2 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:29.971587141 +0000 UTC m=+33.197435337 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2f68cb5a-3e22-4dd6-98d0-f28988ea86a2-metrics-tls") pod "dns-default-klq54" (UID: "2f68cb5a-3e22-4dd6-98d0-f28988ea86a2") : secret "dns-default-metrics-tls" not found
Nov 04 01:02:28.971681 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:28.971664    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28a8304b-9226-422d-9044-b50678d52971-cluster-monitoring-operator-tls podName:28a8304b-9226-422d-9044-b50678d52971 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:29.971648558 +0000 UTC m=+33.197496722 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/28a8304b-9226-422d-9044-b50678d52971-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-6vxbv" (UID: "28a8304b-9226-422d-9044-b50678d52971") : secret "cluster-monitoring-operator-tls" not found
Nov 04 01:02:28.971752 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:28.971550    2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 04 01:02:28.971752 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:28.971732    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3739befd-6689-4197-b793-6f5b4b796880-networking-console-plugin-cert podName:3739befd-6689-4197-b793-6f5b4b796880 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:29.971721766 +0000 UTC m=+33.197569930 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3739befd-6689-4197-b793-6f5b4b796880-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-27rmf" (UID: "3739befd-6689-4197-b793-6f5b4b796880") : secret "networking-console-plugin-cert" not found
Nov 04 01:02:29.590747 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:29.590712    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx7gq"
Nov 04 01:02:29.594798 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:29.594768    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 04 01:02:29.595153 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:29.594804    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5jp5j\""
Nov 04 01:02:29.805494 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:29.805434    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-zc968" event={"ID":"4d3480a2-034c-4959-8aee-b0759dbfc24f","Type":"ContainerStarted","Data":"fae04b0bf9d49a12c75dcfd62c0839079327cb83ec3d16a6c40e70f967be95de"}
Nov 04 01:02:29.807403 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:29.807368    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-58wxt" event={"ID":"3d2d0e05-a773-4442-ab38-d460ad32f51e","Type":"ContainerStarted","Data":"b1f9979e91f13674381686bc6d8787ce98d980e2ee7acce22758576bc04fe489"}
Nov 04 01:02:29.809386 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:29.809361    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-6lcfv" event={"ID":"23cb8b1a-61f6-4045-9f5a-3ccc1dc346a5","Type":"ContainerStarted","Data":"572fa2a5e4be78e55f9c56874c13996fb754de1d22fefcf7f41dc210a7c55155"}
Nov 04 01:02:29.811783 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:29.811754    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-hhfmk" event={"ID":"a0c9abfd-2f47-4cb9-9d3c-479a6231be97","Type":"ContainerStarted","Data":"b0c59c544871c0457a6816645114bc0c0c656752207cef44f9128485fd614794"}
Nov 04 01:02:29.814237 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:29.814177    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4827b" event={"ID":"82ac4b70-1c01-478c-ae1b-4a60dad06677","Type":"ContainerStarted","Data":"36784fac827113d028c4492f22911e0c8328324c80e663aefccb8c8f4f19bddc"}
Nov 04 01:02:29.816858 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:29.816830    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-rff54" event={"ID":"4a79348d-8cf7-4429-8c5a-4084cf85fcc5","Type":"ContainerStarted","Data":"ad98dd57e919217f3be1e43eae62c2153816af52f66cbb1445d3165c9a816d4a"}
Nov 04 01:02:29.818964 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:29.818873    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-8jlt4" event={"ID":"fc0b0e48-8adf-4d0d-80dd-ca76846f824a","Type":"ContainerStarted","Data":"70d19664f15ffde6ce00439daf61c56b10552248f9291702fdf875ab2702c2d4"}
Nov 04 01:02:29.821263 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:29.821237    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-sbjxr" event={"ID":"de97ada4-9cae-4775-a8f8-014004e17d25","Type":"ContainerStarted","Data":"3e160b420233830856efd74439197cb2faeb82d999dd903b68336089f6f85a7b"}
Nov 04 01:02:29.823564 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:29.823544    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-tr8b2" event={"ID":"21d1d23b-7b6e-4ab5-9593-cf7784fffea2","Type":"ContainerStarted","Data":"fdee7cbd173f345475fe82fffb45ac2aa6ca0a47294faa98fd31d1e0363d4843"}
Nov 04 01:02:29.825721 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:29.825678    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-gdqjb" event={"ID":"d4efaef2-c145-4839-b32f-ecf7a1c76a99","Type":"ContainerStarted","Data":"64c1b58d0019a5ac8bd199acdabfb9bde69dc2b8d33819bb4d5ad0cf732bbfdc"}
Nov 04 01:02:29.829041 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:29.828356    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-6lcfv" podStartSLOduration=435.828340156 podStartE2EDuration="7m15.828340156s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 00:55:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:02:29.827466484 +0000 UTC m=+33.053314673" watchObservedRunningTime="2026-11-04 01:02:29.828340156 +0000 UTC m=+33.054188344"
Nov 04 01:02:29.866510 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:29.866413    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-rff54" podStartSLOduration=435.866395121 podStartE2EDuration="7m15.866395121s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 00:55:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:02:29.847109335 +0000 UTC m=+33.072957522" watchObservedRunningTime="2026-11-04 01:02:29.866395121 +0000 UTC m=+33.092243315"
Nov 04 01:02:29.881342 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:29.880235    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-registry-tls\") pod \"image-registry-5c5894ddd-j4phf\" (UID: \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\") " pod="openshift-image-registry/image-registry-5c5894ddd-j4phf"
Nov 04 01:02:29.881342 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:29.880301    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae02b148-c5ba-4db0-b817-9be4a488c708-service-ca-bundle\") pod \"router-default-5569c8658c-v7txl\" (UID: \"ae02b148-c5ba-4db0-b817-9be4a488c708\") " pod="openshift-ingress/router-default-5569c8658c-v7txl"
Nov 04 01:02:29.881342 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:29.880398    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/20f4989c-4fe8-49d2-9430-5030a4274b60-registry-tls\") pod \"image-registry-6574d8498f-74vmj\" (UID: \"20f4989c-4fe8-49d2-9430-5030a4274b60\") " pod="openshift-image-registry/image-registry-6574d8498f-74vmj"
Nov 04 01:02:29.881342 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:29.880430    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e41517b6-7ce4-43a6-89a9-76931457ca50-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-jmxvd\" (UID: \"e41517b6-7ce4-43a6-89a9-76931457ca50\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jmxvd"
Nov 04 01:02:29.881342 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:29.880498    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae02b148-c5ba-4db0-b817-9be4a488c708-metrics-certs\") pod \"router-default-5569c8658c-v7txl\" (UID: \"ae02b148-c5ba-4db0-b817-9be4a488c708\") " pod="openshift-ingress/router-default-5569c8658c-v7txl"
Nov 04 01:02:29.881342 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:29.880658    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 04 01:02:29.881342 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:29.880714    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae02b148-c5ba-4db0-b817-9be4a488c708-metrics-certs podName:ae02b148-c5ba-4db0-b817-9be4a488c708 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:31.88069814 +0000 UTC m=+35.106546318 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ae02b148-c5ba-4db0-b817-9be4a488c708-metrics-certs") pod "router-default-5569c8658c-v7txl" (UID: "ae02b148-c5ba-4db0-b817-9be4a488c708") : secret "router-metrics-certs-default" not found
Nov 04 01:02:29.881342 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:29.881078    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 01:02:29.881342 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:29.881093    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5c5894ddd-j4phf: secret "image-registry-tls" not found
Nov 04 01:02:29.881342 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:29.881190    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 01:02:29.881342 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:29.881204    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6574d8498f-74vmj: secret "image-registry-tls" not found
Nov 04 01:02:29.881342 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:29.881246    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/20f4989c-4fe8-49d2-9430-5030a4274b60-registry-tls podName:20f4989c-4fe8-49d2-9430-5030a4274b60 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:31.881232024 +0000 UTC m=+35.107080194 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/20f4989c-4fe8-49d2-9430-5030a4274b60-registry-tls") pod "image-registry-6574d8498f-74vmj" (UID: "20f4989c-4fe8-49d2-9430-5030a4274b60") : secret "image-registry-tls" not found
Nov 04 01:02:29.881342 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:29.881315    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ae02b148-c5ba-4db0-b817-9be4a488c708-service-ca-bundle podName:ae02b148-c5ba-4db0-b817-9be4a488c708 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:31.881305129 +0000 UTC m=+35.107153307 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ae02b148-c5ba-4db0-b817-9be4a488c708-service-ca-bundle") pod "router-default-5569c8658c-v7txl" (UID: "ae02b148-c5ba-4db0-b817-9be4a488c708") : configmap references non-existent config key: service-ca.crt
Nov 04 01:02:29.881342 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:29.881315    2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 04 01:02:29.882187 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:29.881379    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e41517b6-7ce4-43a6-89a9-76931457ca50-samples-operator-tls podName:e41517b6-7ce4-43a6-89a9-76931457ca50 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:31.881360969 +0000 UTC m=+35.107209139 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e41517b6-7ce4-43a6-89a9-76931457ca50-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-jmxvd" (UID: "e41517b6-7ce4-43a6-89a9-76931457ca50") : secret "samples-operator-tls" not found
Nov 04 01:02:29.882316 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:29.882296    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-registry-tls podName:0eedbf45-4e7d-47b7-806f-a6abf9aa311f nodeName:}" failed. No retries permitted until 2026-11-04 01:02:31.882278536 +0000 UTC m=+35.108126715 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-registry-tls") pod "image-registry-5c5894ddd-j4phf" (UID: "0eedbf45-4e7d-47b7-806f-a6abf9aa311f") : secret "image-registry-tls" not found
Nov 04 01:02:29.981087 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:29.981023    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/28a8304b-9226-422d-9044-b50678d52971-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-6vxbv\" (UID: \"28a8304b-9226-422d-9044-b50678d52971\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6vxbv"
Nov 04 01:02:29.981259 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:29.981168    2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 04 01:02:29.981259 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:29.981242    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28a8304b-9226-422d-9044-b50678d52971-cluster-monitoring-operator-tls podName:28a8304b-9226-422d-9044-b50678d52971 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:31.981223266 +0000 UTC m=+35.207071444 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/28a8304b-9226-422d-9044-b50678d52971-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-6vxbv" (UID: "28a8304b-9226-422d-9044-b50678d52971") : secret "cluster-monitoring-operator-tls" not found
Nov 04 01:02:29.981747 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:29.981470    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f68cb5a-3e22-4dd6-98d0-f28988ea86a2-metrics-tls\") pod \"dns-default-klq54\" (UID: \"2f68cb5a-3e22-4dd6-98d0-f28988ea86a2\") " pod="openshift-dns/dns-default-klq54"
Nov 04 01:02:29.981747 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:29.981509    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3739befd-6689-4197-b793-6f5b4b796880-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-27rmf\" (UID: \"3739befd-6689-4197-b793-6f5b4b796880\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-27rmf"
Nov 04 01:02:29.982401 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:29.982380    2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 04 01:02:29.982502 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:29.982433    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3739befd-6689-4197-b793-6f5b4b796880-networking-console-plugin-cert podName:3739befd-6689-4197-b793-6f5b4b796880 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:31.982418599 +0000 UTC m=+35.208266777 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3739befd-6689-4197-b793-6f5b4b796880-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-27rmf" (UID: "3739befd-6689-4197-b793-6f5b4b796880") : secret "networking-console-plugin-cert" not found
Nov 04 01:02:29.982502 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:29.982494    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 01:02:29.982640 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:29.982528    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f68cb5a-3e22-4dd6-98d0-f28988ea86a2-metrics-tls podName:2f68cb5a-3e22-4dd6-98d0-f28988ea86a2 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:31.982516748 +0000 UTC m=+35.208364926 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2f68cb5a-3e22-4dd6-98d0-f28988ea86a2-metrics-tls") pod "dns-default-klq54" (UID: "2f68cb5a-3e22-4dd6-98d0-f28988ea86a2") : secret "dns-default-metrics-tls" not found
Nov 04 01:02:30.994951 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:30.994915    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6be44550-8625-4296-aeb9-da833aaa4b2f-original-pull-secret\") pod \"global-pull-secret-syncer-gnrzw\" (UID: \"6be44550-8625-4296-aeb9-da833aaa4b2f\") " pod="kube-system/global-pull-secret-syncer-gnrzw"
Nov 04 01:02:30.995526 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:30.995152    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a409c58b-b4eb-428f-9177-a09b8a229ec7-metrics-certs\") pod \"network-metrics-daemon-rx7gq\" (UID: \"a409c58b-b4eb-428f-9177-a09b8a229ec7\") " pod="openshift-multus/network-metrics-daemon-rx7gq"
Nov 04 01:02:30.996858 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:30.996538    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 04 01:02:30.996858 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:30.996622    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a409c58b-b4eb-428f-9177-a09b8a229ec7-metrics-certs podName:a409c58b-b4eb-428f-9177-a09b8a229ec7 nodeName:}" failed. No retries permitted until 2026-11-04 01:03:02.996586057 +0000 UTC m=+66.222434234 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a409c58b-b4eb-428f-9177-a09b8a229ec7-metrics-certs") pod "network-metrics-daemon-rx7gq" (UID: "a409c58b-b4eb-428f-9177-a09b8a229ec7") : secret "metrics-daemon-secret" not found
Nov 04 01:02:31.006374 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:31.006344    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6be44550-8625-4296-aeb9-da833aaa4b2f-original-pull-secret\") pod \"global-pull-secret-syncer-gnrzw\" (UID: \"6be44550-8625-4296-aeb9-da833aaa4b2f\") " pod="kube-system/global-pull-secret-syncer-gnrzw"
Nov 04 01:02:31.032383 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:31.032172    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gnrzw"
Nov 04 01:02:31.096535 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:31.096028    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wsn9\" (UniqueName: \"kubernetes.io/projected/588c643c-b16a-4d5c-ad85-ccb8d486dd43-kube-api-access-4wsn9\") pod \"network-check-target-wxhng\" (UID: \"588c643c-b16a-4d5c-ad85-ccb8d486dd43\") " pod="openshift-network-diagnostics/network-check-target-wxhng"
Nov 04 01:02:31.099444 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:31.099269    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wsn9\" (UniqueName: \"kubernetes.io/projected/588c643c-b16a-4d5c-ad85-ccb8d486dd43-kube-api-access-4wsn9\") pod \"network-check-target-wxhng\" (UID: \"588c643c-b16a-4d5c-ad85-ccb8d486dd43\") " pod="openshift-network-diagnostics/network-check-target-wxhng"
Nov 04 01:02:31.338365 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:31.338329    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wxhng"
Nov 04 01:02:31.371397 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:31.371345    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-hhfmk" podStartSLOduration=437.371326519 podStartE2EDuration="7m17.371326519s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 00:55:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:02:29.867755775 +0000 UTC m=+33.093603959" watchObservedRunningTime="2026-11-04 01:02:31.371326519 +0000 UTC m=+34.597174707"
Nov 04 01:02:31.371805 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:31.371786    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-gnrzw"]
Nov 04 01:02:31.768460 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:31.767024    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-wxhng"]
Nov 04 01:02:31.839419 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:31.838644    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-gdqjb" event={"ID":"d4efaef2-c145-4839-b32f-ecf7a1c76a99","Type":"ContainerStarted","Data":"7f122c524a33d4ee01d5d903ca4c96d859debc5ee6e700e66ee54333c4ae1425"}
Nov 04 01:02:31.842556 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:31.842479    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-wxhng" event={"ID":"588c643c-b16a-4d5c-ad85-ccb8d486dd43","Type":"ContainerStarted","Data":"808c2afb0f40ee6e188632e613a8eaadc9341eb3b191e2ee4fafb51b356bac7a"}
Nov 04 01:02:31.845039 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:31.844978    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-gnrzw" event={"ID":"6be44550-8625-4296-aeb9-da833aaa4b2f","Type":"ContainerStarted","Data":"fbbf633f62763e69cdd345d3148ac2eb1ba1952752256288060f9610675a3f05"}
Nov 04 01:02:31.860348 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:31.860111    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-gdqjb" podStartSLOduration=406.183037467 podStartE2EDuration="6m48.860093628s" totalImagesPullingTime="2.677056161s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 00:55:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:02:31.858465278 +0000 UTC m=+35.084313462" watchObservedRunningTime="2026-11-04 01:02:31.860093628 +0000 UTC m=+35.085941816"
Nov 04 01:02:31.913186 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:31.911229    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae02b148-c5ba-4db0-b817-9be4a488c708-metrics-certs\") pod \"router-default-5569c8658c-v7txl\" (UID: \"ae02b148-c5ba-4db0-b817-9be4a488c708\") " pod="openshift-ingress/router-default-5569c8658c-v7txl"
Nov 04 01:02:31.913186 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:31.911293    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-registry-tls\") pod \"image-registry-5c5894ddd-j4phf\" (UID: \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\") " pod="openshift-image-registry/image-registry-5c5894ddd-j4phf"
Nov 04 01:02:31.913186 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:31.911368    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae02b148-c5ba-4db0-b817-9be4a488c708-service-ca-bundle\") pod \"router-default-5569c8658c-v7txl\" (UID: \"ae02b148-c5ba-4db0-b817-9be4a488c708\") " pod="openshift-ingress/router-default-5569c8658c-v7txl"
Nov 04 01:02:31.913186 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:31.911439    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/20f4989c-4fe8-49d2-9430-5030a4274b60-registry-tls\") pod \"image-registry-6574d8498f-74vmj\" (UID: \"20f4989c-4fe8-49d2-9430-5030a4274b60\") " pod="openshift-image-registry/image-registry-6574d8498f-74vmj"
Nov 04 01:02:31.913186 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:31.911468    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e41517b6-7ce4-43a6-89a9-76931457ca50-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-jmxvd\" (UID: \"e41517b6-7ce4-43a6-89a9-76931457ca50\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jmxvd"
Nov 04 01:02:31.913186 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:31.911656    2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 04 01:02:31.913186 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:31.911716    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e41517b6-7ce4-43a6-89a9-76931457ca50-samples-operator-tls podName:e41517b6-7ce4-43a6-89a9-76931457ca50 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:35.911698634 +0000 UTC m=+39.137546804 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e41517b6-7ce4-43a6-89a9-76931457ca50-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-jmxvd" (UID: "e41517b6-7ce4-43a6-89a9-76931457ca50") : secret "samples-operator-tls" not found
Nov 04 01:02:31.913186 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:31.912096    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 04 01:02:31.913186 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:31.912144    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae02b148-c5ba-4db0-b817-9be4a488c708-metrics-certs podName:ae02b148-c5ba-4db0-b817-9be4a488c708 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:35.91213061 +0000 UTC m=+39.137978788 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ae02b148-c5ba-4db0-b817-9be4a488c708-metrics-certs") pod "router-default-5569c8658c-v7txl" (UID: "ae02b148-c5ba-4db0-b817-9be4a488c708") : secret "router-metrics-certs-default" not found
Nov 04 01:02:31.913186 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:31.912208    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 01:02:31.913186 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:31.912218    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5c5894ddd-j4phf: secret "image-registry-tls" not found
Nov 04 01:02:31.913186 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:31.912249    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-registry-tls podName:0eedbf45-4e7d-47b7-806f-a6abf9aa311f nodeName:}" failed. No retries permitted until 2026-11-04 01:02:35.912239453 +0000 UTC m=+39.138087625 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-registry-tls") pod "image-registry-5c5894ddd-j4phf" (UID: "0eedbf45-4e7d-47b7-806f-a6abf9aa311f") : secret "image-registry-tls" not found
Nov 04 01:02:31.913186 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:31.913031    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ae02b148-c5ba-4db0-b817-9be4a488c708-service-ca-bundle podName:ae02b148-c5ba-4db0-b817-9be4a488c708 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:35.913017049 +0000 UTC m=+39.138865227 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ae02b148-c5ba-4db0-b817-9be4a488c708-service-ca-bundle") pod "router-default-5569c8658c-v7txl" (UID: "ae02b148-c5ba-4db0-b817-9be4a488c708") : configmap references non-existent config key: service-ca.crt
Nov 04 01:02:31.913186 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:31.913102    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 01:02:31.913186 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:31.913115    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6574d8498f-74vmj: secret "image-registry-tls" not found
Nov 04 01:02:31.913186 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:31.913148    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/20f4989c-4fe8-49d2-9430-5030a4274b60-registry-tls podName:20f4989c-4fe8-49d2-9430-5030a4274b60 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:35.913136896 +0000 UTC m=+39.138985073 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/20f4989c-4fe8-49d2-9430-5030a4274b60-registry-tls") pod "image-registry-6574d8498f-74vmj" (UID: "20f4989c-4fe8-49d2-9430-5030a4274b60") : secret "image-registry-tls" not found
Nov 04 01:02:32.013645 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:32.012639    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/28a8304b-9226-422d-9044-b50678d52971-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-6vxbv\" (UID: \"28a8304b-9226-422d-9044-b50678d52971\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6vxbv"
Nov 04 01:02:32.013645 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:32.012720    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f68cb5a-3e22-4dd6-98d0-f28988ea86a2-metrics-tls\") pod \"dns-default-klq54\" (UID: \"2f68cb5a-3e22-4dd6-98d0-f28988ea86a2\") " pod="openshift-dns/dns-default-klq54"
Nov 04 01:02:32.013645 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:32.012751    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3739befd-6689-4197-b793-6f5b4b796880-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-27rmf\" (UID: \"3739befd-6689-4197-b793-6f5b4b796880\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-27rmf"
Nov 04 01:02:32.013645 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:32.013039    2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 04 01:02:32.013645 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:32.013107    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3739befd-6689-4197-b793-6f5b4b796880-networking-console-plugin-cert podName:3739befd-6689-4197-b793-6f5b4b796880 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:36.013088166 +0000 UTC m=+39.238936348 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3739befd-6689-4197-b793-6f5b4b796880-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-27rmf" (UID: "3739befd-6689-4197-b793-6f5b4b796880") : secret "networking-console-plugin-cert" not found
Nov 04 01:02:32.013645 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:32.013178    2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 04 01:02:32.013645 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:32.013212    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28a8304b-9226-422d-9044-b50678d52971-cluster-monitoring-operator-tls podName:28a8304b-9226-422d-9044-b50678d52971 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:36.013199502 +0000 UTC m=+39.239047679 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/28a8304b-9226-422d-9044-b50678d52971-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-6vxbv" (UID: "28a8304b-9226-422d-9044-b50678d52971") : secret "cluster-monitoring-operator-tls" not found
Nov 04 01:02:32.013645 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:32.013267    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 01:02:32.013645 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:32.013295    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f68cb5a-3e22-4dd6-98d0-f28988ea86a2-metrics-tls podName:2f68cb5a-3e22-4dd6-98d0-f28988ea86a2 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:36.013284877 +0000 UTC m=+39.239133055 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2f68cb5a-3e22-4dd6-98d0-f28988ea86a2-metrics-tls") pod "dns-default-klq54" (UID: "2f68cb5a-3e22-4dd6-98d0-f28988ea86a2") : secret "dns-default-metrics-tls" not found
Nov 04 01:02:33.856641 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:33.856304    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tr8b2_21d1d23b-7b6e-4ab5-9593-cf7784fffea2/console-operator/0.log"
Nov 04 01:02:33.856641 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:33.856352    2582 generic.go:356] "Generic (PLEG): container finished" podID="21d1d23b-7b6e-4ab5-9593-cf7784fffea2" containerID="c8678d9376267afc711725c065b7e04a4af21afca6d8b41bc0389877516da1b1" exitCode=255
Nov 04 01:02:33.856641 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:33.856424    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-tr8b2" event={"ID":"21d1d23b-7b6e-4ab5-9593-cf7784fffea2","Type":"ContainerDied","Data":"c8678d9376267afc711725c065b7e04a4af21afca6d8b41bc0389877516da1b1"}
Nov 04 01:02:33.858070 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:33.857752    2582 scope.go:122] "RemoveContainer" containerID="c8678d9376267afc711725c065b7e04a4af21afca6d8b41bc0389877516da1b1"
Nov 04 01:02:34.863368 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:34.863325    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-8jlt4" event={"ID":"fc0b0e48-8adf-4d0d-80dd-ca76846f824a","Type":"ContainerStarted","Data":"7beb1d6b3427d0bacd043e31c4976f8d8b9e841992e7bf36be4efd7d2d9fc294"}
Nov 04 01:02:34.865991 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:34.865961    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tr8b2_21d1d23b-7b6e-4ab5-9593-cf7784fffea2/console-operator/1.log"
Nov 04 01:02:34.868129 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:34.866440    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tr8b2_21d1d23b-7b6e-4ab5-9593-cf7784fffea2/console-operator/0.log"
Nov 04 01:02:34.868129 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:34.866475    2582 generic.go:356] "Generic (PLEG): container finished" podID="21d1d23b-7b6e-4ab5-9593-cf7784fffea2" containerID="9d9855e9203df085f919c24b870c736a1b366e38fdfa9b8aea39bcaedc7f5ded" exitCode=255
Nov 04 01:02:34.868129 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:34.866534    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-tr8b2" event={"ID":"21d1d23b-7b6e-4ab5-9593-cf7784fffea2","Type":"ContainerDied","Data":"9d9855e9203df085f919c24b870c736a1b366e38fdfa9b8aea39bcaedc7f5ded"}
Nov 04 01:02:34.868129 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:34.866573    2582 scope.go:122] "RemoveContainer" containerID="c8678d9376267afc711725c065b7e04a4af21afca6d8b41bc0389877516da1b1"
Nov 04 01:02:34.868129 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:34.866954    2582 scope.go:122] "RemoveContainer" containerID="9d9855e9203df085f919c24b870c736a1b366e38fdfa9b8aea39bcaedc7f5ded"
Nov 04 01:02:34.868129 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:34.867178    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-tr8b2_openshift-console-operator(21d1d23b-7b6e-4ab5-9593-cf7784fffea2)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-tr8b2" podUID="21d1d23b-7b6e-4ab5-9593-cf7784fffea2"
Nov 04 01:02:34.884004 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:34.882736    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-8jlt4" podStartSLOduration=405.386447208 podStartE2EDuration="6m50.882719768s" totalImagesPullingTime="5.49627256s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 00:55:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:02:34.881052297 +0000 UTC m=+38.106900484" watchObservedRunningTime="2026-11-04 01:02:34.882719768 +0000 UTC m=+38.108567955"
Nov 04 01:02:35.746174 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.746146    2582 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-sg4cw"]
Nov 04 01:02:35.763454 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.763421    2582 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-t6tpg-6r4ws"]
Nov 04 01:02:35.765366 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.764828    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-sg4cw"
Nov 04 01:02:35.774571 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.774381    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 04 01:02:35.774571 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.774434    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 04 01:02:35.774571 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.774541    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-7ffl4\""
Nov 04 01:02:35.780541 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.780517    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-sg4cw"]
Nov 04 01:02:35.780668 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.780593    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-t6tpg-6r4ws"]
Nov 04 01:02:35.780740 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.780725    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-t6tpg-6r4ws"
Nov 04 01:02:35.787950 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.787053    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-f8m8g\""
Nov 04 01:02:35.857636 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.857598    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sg4cw\" (UID: \"ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774\") " pod="openshift-insights/insights-runtime-extractor-sg4cw"
Nov 04 01:02:35.857801 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.857776    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-data-volume\") pod \"insights-runtime-extractor-sg4cw\" (UID: \"ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774\") " pod="openshift-insights/insights-runtime-extractor-sg4cw"
Nov 04 01:02:35.857859 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.857832    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-crio-socket\") pod \"insights-runtime-extractor-sg4cw\" (UID: \"ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774\") " pod="openshift-insights/insights-runtime-extractor-sg4cw"
Nov 04 01:02:35.857997 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.857908    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/e2b1aee4-49c5-47a0-a79f-a3fe032df2d7-archives-path\") pod \"periodic-gathering-t6tpg-6r4ws\" (UID: \"e2b1aee4-49c5-47a0-a79f-a3fe032df2d7\") " pod="openshift-insights/periodic-gathering-t6tpg-6r4ws"
Nov 04 01:02:35.857997 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.857946    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6b7vh\" (UniqueName: \"kubernetes.io/projected/e2b1aee4-49c5-47a0-a79f-a3fe032df2d7-kube-api-access-6b7vh\") pod \"periodic-gathering-t6tpg-6r4ws\" (UID: \"e2b1aee4-49c5-47a0-a79f-a3fe032df2d7\") " pod="openshift-insights/periodic-gathering-t6tpg-6r4ws"
Nov 04 01:02:35.858132 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.858029    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2b1aee4-49c5-47a0-a79f-a3fe032df2d7-service-ca-bundle\") pod \"periodic-gathering-t6tpg-6r4ws\" (UID: \"e2b1aee4-49c5-47a0-a79f-a3fe032df2d7\") " pod="openshift-insights/periodic-gathering-t6tpg-6r4ws"
Nov 04 01:02:35.858132 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.858099    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mh966\" (UniqueName: \"kubernetes.io/projected/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-kube-api-access-mh966\") pod \"insights-runtime-extractor-sg4cw\" (UID: \"ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774\") " pod="openshift-insights/insights-runtime-extractor-sg4cw"
Nov 04 01:02:35.858231 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.858129    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sg4cw\" (UID: \"ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774\") " pod="openshift-insights/insights-runtime-extractor-sg4cw"
Nov 04 01:02:35.858231 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.858169    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2b1aee4-49c5-47a0-a79f-a3fe032df2d7-trusted-ca-bundle\") pod \"periodic-gathering-t6tpg-6r4ws\" (UID: \"e2b1aee4-49c5-47a0-a79f-a3fe032df2d7\") " pod="openshift-insights/periodic-gathering-t6tpg-6r4ws"
Nov 04 01:02:35.874215 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.874154    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4827b" event={"ID":"82ac4b70-1c01-478c-ae1b-4a60dad06677","Type":"ContainerStarted","Data":"0d035fb61ab286e3e3ea0269316e892b27f65ebea24d3383427a41b2f22de8f8"}
Nov 04 01:02:35.880371 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.880314    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-sbjxr" event={"ID":"de97ada4-9cae-4775-a8f8-014004e17d25","Type":"ContainerStarted","Data":"cbd04d364f48b61094def5501e13336aa314a5067464fd1e0cd9862d9890aea9"}
Nov 04 01:02:35.885075 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.885052    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tr8b2_21d1d23b-7b6e-4ab5-9593-cf7784fffea2/console-operator/1.log"
Nov 04 01:02:35.885949 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.885417    2582 scope.go:122] "RemoveContainer" containerID="9d9855e9203df085f919c24b870c736a1b366e38fdfa9b8aea39bcaedc7f5ded"
Nov 04 01:02:35.885949 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:35.885637    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-tr8b2_openshift-console-operator(21d1d23b-7b6e-4ab5-9593-cf7784fffea2)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-tr8b2" podUID="21d1d23b-7b6e-4ab5-9593-cf7784fffea2"
Nov 04 01:02:35.888004 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.887981    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-zc968" event={"ID":"4d3480a2-034c-4959-8aee-b0759dbfc24f","Type":"ContainerStarted","Data":"f0cf4b376eb2882a60cd013965ee377a57dd00a6748937c52c5e836b3719c5cf"}
Nov 04 01:02:35.889576 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.889558    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-58wxt" event={"ID":"3d2d0e05-a773-4442-ab38-d460ad32f51e","Type":"ContainerStarted","Data":"d7fd030cf5489a0ae815519cb9b3727d097138490b776df059c60af9b48cf191"}
Nov 04 01:02:35.896814 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.896778    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-4827b" podStartSLOduration=1.577494616 podStartE2EDuration="7.896766828s" totalImagesPullingTime="6.319272212s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:02:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:02:35.895679132 +0000 UTC m=+39.121527316" watchObservedRunningTime="2026-11-04 01:02:35.896766828 +0000 UTC m=+39.122615014"
Nov 04 01:02:35.924445 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.924366    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-operator-7c4986d449-sbjxr" podStartSLOduration=406.57848644 podStartE2EDuration="6m52.92435226s" totalImagesPullingTime="6.34586582s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 00:55:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:02:35.923281173 +0000 UTC m=+39.149129362" watchObservedRunningTime="2026-11-04 01:02:35.92435226 +0000 UTC m=+39.150200496"
Nov 04 01:02:35.931032 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.929801    2582 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-fp864"]
Nov 04 01:02:35.950710 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.950688    2582 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-b9rdk"]
Nov 04 01:02:35.950864 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.950847    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-fp864"
Nov 04 01:02:35.954646 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.954384    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-58wxt" podStartSLOduration=406.58981815 podStartE2EDuration="6m52.954366769s" totalImagesPullingTime="6.364548619s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 00:55:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:02:35.951860267 +0000 UTC m=+39.177708456" watchObservedRunningTime="2026-11-04 01:02:35.954366769 +0000 UTC m=+39.180214955"
Nov 04 01:02:35.958906 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.958817    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Nov 04 01:02:35.960341 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.959005    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/20f4989c-4fe8-49d2-9430-5030a4274b60-registry-tls\") pod \"image-registry-6574d8498f-74vmj\" (UID: \"20f4989c-4fe8-49d2-9430-5030a4274b60\") " pod="openshift-image-registry/image-registry-6574d8498f-74vmj"
Nov 04 01:02:35.960341 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.959040    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e41517b6-7ce4-43a6-89a9-76931457ca50-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-jmxvd\" (UID: \"e41517b6-7ce4-43a6-89a9-76931457ca50\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jmxvd"
Nov 04 01:02:35.960341 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.959073    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-data-volume\") pod \"insights-runtime-extractor-sg4cw\" (UID: \"ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774\") " pod="openshift-insights/insights-runtime-extractor-sg4cw"
Nov 04 01:02:35.960341 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.959108    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-crio-socket\") pod \"insights-runtime-extractor-sg4cw\" (UID: \"ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774\") " pod="openshift-insights/insights-runtime-extractor-sg4cw"
Nov 04 01:02:35.960341 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.959133    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/e2b1aee4-49c5-47a0-a79f-a3fe032df2d7-archives-path\") pod \"periodic-gathering-t6tpg-6r4ws\" (UID: \"e2b1aee4-49c5-47a0-a79f-a3fe032df2d7\") " pod="openshift-insights/periodic-gathering-t6tpg-6r4ws"
Nov 04 01:02:35.960341 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.959146    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6b7vh\" (UniqueName: \"kubernetes.io/projected/e2b1aee4-49c5-47a0-a79f-a3fe032df2d7-kube-api-access-6b7vh\") pod \"periodic-gathering-t6tpg-6r4ws\" (UID: \"e2b1aee4-49c5-47a0-a79f-a3fe032df2d7\") " pod="openshift-insights/periodic-gathering-t6tpg-6r4ws"
Nov 04 01:02:35.960341 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.959164    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2b1aee4-49c5-47a0-a79f-a3fe032df2d7-service-ca-bundle\") pod \"periodic-gathering-t6tpg-6r4ws\" (UID: \"e2b1aee4-49c5-47a0-a79f-a3fe032df2d7\") " pod="openshift-insights/periodic-gathering-t6tpg-6r4ws"
Nov 04 01:02:35.960341 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.959198    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae02b148-c5ba-4db0-b817-9be4a488c708-metrics-certs\") pod \"router-default-5569c8658c-v7txl\" (UID: \"ae02b148-c5ba-4db0-b817-9be4a488c708\") " pod="openshift-ingress/router-default-5569c8658c-v7txl"
Nov 04 01:02:35.960341 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.959258    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mh966\" (UniqueName: \"kubernetes.io/projected/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-kube-api-access-mh966\") pod \"insights-runtime-extractor-sg4cw\" (UID: \"ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774\") " pod="openshift-insights/insights-runtime-extractor-sg4cw"
Nov 04 01:02:35.960341 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.959297    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sg4cw\" (UID: \"ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774\") " pod="openshift-insights/insights-runtime-extractor-sg4cw"
Nov 04 01:02:35.960341 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.959323    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-registry-tls\") pod \"image-registry-5c5894ddd-j4phf\" (UID: \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\") " pod="openshift-image-registry/image-registry-5c5894ddd-j4phf"
Nov 04 01:02:35.960341 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.959348    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2b1aee4-49c5-47a0-a79f-a3fe032df2d7-trusted-ca-bundle\") pod \"periodic-gathering-t6tpg-6r4ws\" (UID: \"e2b1aee4-49c5-47a0-a79f-a3fe032df2d7\") " pod="openshift-insights/periodic-gathering-t6tpg-6r4ws"
Nov 04 01:02:35.960341 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.959376    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae02b148-c5ba-4db0-b817-9be4a488c708-service-ca-bundle\") pod \"router-default-5569c8658c-v7txl\" (UID: \"ae02b148-c5ba-4db0-b817-9be4a488c708\") " pod="openshift-ingress/router-default-5569c8658c-v7txl"
Nov 04 01:02:35.960341 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.959434    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sg4cw\" (UID: \"ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774\") " pod="openshift-insights/insights-runtime-extractor-sg4cw"
Nov 04 01:02:35.960341 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.959862    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2b1aee4-49c5-47a0-a79f-a3fe032df2d7-service-ca-bundle\") pod \"periodic-gathering-t6tpg-6r4ws\" (UID: \"e2b1aee4-49c5-47a0-a79f-a3fe032df2d7\") " pod="openshift-insights/periodic-gathering-t6tpg-6r4ws"
Nov 04 01:02:35.960341 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:35.960063    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 01:02:35.960341 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:35.960076    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6574d8498f-74vmj: secret "image-registry-tls" not found
Nov 04 01:02:35.964640 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:35.960120    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/20f4989c-4fe8-49d2-9430-5030a4274b60-registry-tls podName:20f4989c-4fe8-49d2-9430-5030a4274b60 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:43.960105835 +0000 UTC m=+47.185954006 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/20f4989c-4fe8-49d2-9430-5030a4274b60-registry-tls") pod "image-registry-6574d8498f-74vmj" (UID: "20f4989c-4fe8-49d2-9430-5030a4274b60") : secret "image-registry-tls" not found
Nov 04 01:02:35.964640 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:35.960443    2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 04 01:02:35.964640 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:35.960491    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e41517b6-7ce4-43a6-89a9-76931457ca50-samples-operator-tls podName:e41517b6-7ce4-43a6-89a9-76931457ca50 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:43.960475141 +0000 UTC m=+47.186323334 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e41517b6-7ce4-43a6-89a9-76931457ca50-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-jmxvd" (UID: "e41517b6-7ce4-43a6-89a9-76931457ca50") : secret "samples-operator-tls" not found
Nov 04 01:02:35.964640 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:35.960551    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 01:02:35.964640 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:35.960560    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5c5894ddd-j4phf: secret "image-registry-tls" not found
Nov 04 01:02:35.964640 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:35.960593    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-registry-tls podName:0eedbf45-4e7d-47b7-806f-a6abf9aa311f nodeName:}" failed. No retries permitted until 2026-11-04 01:02:43.960582028 +0000 UTC m=+47.186430205 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-registry-tls") pod "image-registry-5c5894ddd-j4phf" (UID: "0eedbf45-4e7d-47b7-806f-a6abf9aa311f") : secret "image-registry-tls" not found
Nov 04 01:02:35.964640 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:35.960663    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 04 01:02:35.964640 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:35.960712    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae02b148-c5ba-4db0-b817-9be4a488c708-metrics-certs podName:ae02b148-c5ba-4db0-b817-9be4a488c708 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:43.960695499 +0000 UTC m=+47.186543668 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ae02b148-c5ba-4db0-b817-9be4a488c708-metrics-certs") pod "router-default-5569c8658c-v7txl" (UID: "ae02b148-c5ba-4db0-b817-9be4a488c708") : secret "router-metrics-certs-default" not found
Nov 04 01:02:35.964640 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.961561    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2b1aee4-49c5-47a0-a79f-a3fe032df2d7-trusted-ca-bundle\") pod \"periodic-gathering-t6tpg-6r4ws\" (UID: \"e2b1aee4-49c5-47a0-a79f-a3fe032df2d7\") " pod="openshift-insights/periodic-gathering-t6tpg-6r4ws"
Nov 04 01:02:35.964640 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:35.961691    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ae02b148-c5ba-4db0-b817-9be4a488c708-service-ca-bundle podName:ae02b148-c5ba-4db0-b817-9be4a488c708 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:43.961673942 +0000 UTC m=+47.187522125 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ae02b148-c5ba-4db0-b817-9be4a488c708-service-ca-bundle") pod "router-default-5569c8658c-v7txl" (UID: "ae02b148-c5ba-4db0-b817-9be4a488c708") : configmap references non-existent config key: service-ca.crt
Nov 04 01:02:35.964640 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:35.962134    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 01:02:35.964640 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:35.962180    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-insights-runtime-extractor-tls podName:ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:36.462165576 +0000 UTC m=+39.688013741 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-insights-runtime-extractor-tls") pod "insights-runtime-extractor-sg4cw" (UID: "ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774") : secret "insights-runtime-extractor-tls" not found
Nov 04 01:02:35.964640 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:35.962860    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 04 01:02:35.964640 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:35.962897    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-kube-rbac-proxy-cm podName:ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:36.462885042 +0000 UTC m=+39.688733206 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-kube-rbac-proxy-cm") pod "insights-runtime-extractor-sg4cw" (UID: "ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774") : configmap "kube-rbac-proxy" not found
Nov 04 01:02:35.964640 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.963472    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-data-volume\") pod \"insights-runtime-extractor-sg4cw\" (UID: \"ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774\") " pod="openshift-insights/insights-runtime-extractor-sg4cw"
Nov 04 01:02:35.965416 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.963738    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-crio-socket\") pod \"insights-runtime-extractor-sg4cw\" (UID: \"ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774\") " pod="openshift-insights/insights-runtime-extractor-sg4cw"
Nov 04 01:02:35.965416 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.963771    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/e2b1aee4-49c5-47a0-a79f-a3fe032df2d7-archives-path\") pod \"periodic-gathering-t6tpg-6r4ws\" (UID: \"e2b1aee4-49c5-47a0-a79f-a3fe032df2d7\") " pod="openshift-insights/periodic-gathering-t6tpg-6r4ws"
Nov 04 01:02:35.965916 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.965734    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Nov 04 01:02:35.966395 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.966161    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-fp864"]
Nov 04 01:02:35.966395 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.966205    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-b9rdk"]
Nov 04 01:02:35.966395 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.966287    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-b9rdk"
Nov 04 01:02:35.968113 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.968096    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-r2cph\""
Nov 04 01:02:35.980081 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.980061    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mh966\" (UniqueName: \"kubernetes.io/projected/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-kube-api-access-mh966\") pod \"insights-runtime-extractor-sg4cw\" (UID: \"ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774\") " pod="openshift-insights/insights-runtime-extractor-sg4cw"
Nov 04 01:02:35.980584 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:35.980537    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6b7vh\" (UniqueName: \"kubernetes.io/projected/e2b1aee4-49c5-47a0-a79f-a3fe032df2d7-kube-api-access-6b7vh\") pod \"periodic-gathering-t6tpg-6r4ws\" (UID: \"e2b1aee4-49c5-47a0-a79f-a3fe032df2d7\") " pod="openshift-insights/periodic-gathering-t6tpg-6r4ws"
Nov 04 01:02:36.009147 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:36.009102    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-zc968" podStartSLOduration=403.309870965 podStartE2EDuration="6m50.009086324s" totalImagesPullingTime="6.699215359s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 00:55:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:02:36.006778656 +0000 UTC m=+39.232626840" watchObservedRunningTime="2026-11-04 01:02:36.009086324 +0000 UTC m=+39.234934513"
Nov 04 01:02:36.042227 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:36.042200    2582 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-2zdns"]
Nov 04 01:02:36.060711 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:36.060681    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/28a8304b-9226-422d-9044-b50678d52971-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-6vxbv\" (UID: \"28a8304b-9226-422d-9044-b50678d52971\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6vxbv"
Nov 04 01:02:36.060829 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:36.060793    2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 04 01:02:36.060829 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:36.060816    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f68cb5a-3e22-4dd6-98d0-f28988ea86a2-metrics-tls\") pod \"dns-default-klq54\" (UID: \"2f68cb5a-3e22-4dd6-98d0-f28988ea86a2\") " pod="openshift-dns/dns-default-klq54"
Nov 04 01:02:36.060946 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:36.060848    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p664g\" (UniqueName: \"kubernetes.io/projected/8f92b52c-22de-4ce7-b716-fdfa22d7b52f-kube-api-access-p664g\") pod \"migrator-66df5d496b-fp864\" (UID: \"8f92b52c-22de-4ce7-b716-fdfa22d7b52f\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-fp864"
Nov 04 01:02:36.060946 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:36.060855    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28a8304b-9226-422d-9044-b50678d52971-cluster-monitoring-operator-tls podName:28a8304b-9226-422d-9044-b50678d52971 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:44.060841073 +0000 UTC m=+47.286689240 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/28a8304b-9226-422d-9044-b50678d52971-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-6vxbv" (UID: "28a8304b-9226-422d-9044-b50678d52971") : secret "cluster-monitoring-operator-tls" not found
Nov 04 01:02:36.060946 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:36.060907    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3739befd-6689-4197-b793-6f5b4b796880-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-27rmf\" (UID: \"3739befd-6689-4197-b793-6f5b4b796880\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-27rmf"
Nov 04 01:02:36.061106 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:36.061016    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6tv2\" (UniqueName: \"kubernetes.io/projected/9d694048-4b09-480b-ad26-8084c7775b24-kube-api-access-b6tv2\") pod \"migrator-66df5d496b-b9rdk\" (UID: \"9d694048-4b09-480b-ad26-8084c7775b24\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-b9rdk"
Nov 04 01:02:36.061158 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:36.061122    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 01:02:36.061210 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:36.061161    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f68cb5a-3e22-4dd6-98d0-f28988ea86a2-metrics-tls podName:2f68cb5a-3e22-4dd6-98d0-f28988ea86a2 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:44.061148052 +0000 UTC m=+47.286996217 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2f68cb5a-3e22-4dd6-98d0-f28988ea86a2-metrics-tls") pod "dns-default-klq54" (UID: "2f68cb5a-3e22-4dd6-98d0-f28988ea86a2") : secret "dns-default-metrics-tls" not found
Nov 04 01:02:36.061210 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:36.061193    2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 04 01:02:36.061316 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:36.061243    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3739befd-6689-4197-b793-6f5b4b796880-networking-console-plugin-cert podName:3739befd-6689-4197-b793-6f5b4b796880 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:44.06122917 +0000 UTC m=+47.287077338 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3739befd-6689-4197-b793-6f5b4b796880-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-27rmf" (UID: "3739befd-6689-4197-b793-6f5b4b796880") : secret "networking-console-plugin-cert" not found
Nov 04 01:02:36.061316 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:36.061283    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-2zdns"]
Nov 04 01:02:36.061417 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:36.061353    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-2zdns"
Nov 04 01:02:36.065412 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:36.065390    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Nov 04 01:02:36.065412 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:36.065408    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Nov 04 01:02:36.065638 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:36.065399    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Nov 04 01:02:36.065638 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:36.065481    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-jjj2l\""
Nov 04 01:02:36.065746 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:36.065399    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Nov 04 01:02:36.148239 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:36.148171    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-t6tpg-6r4ws"
Nov 04 01:02:36.161564 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:36.161538    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b4dcd02c-9a77-43da-9923-3ea68740b318-signing-cabundle\") pod \"service-ca-5fffc97bdf-2zdns\" (UID: \"b4dcd02c-9a77-43da-9923-3ea68740b318\") " pod="openshift-service-ca/service-ca-5fffc97bdf-2zdns"
Nov 04 01:02:36.161749 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:36.161590    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b4dcd02c-9a77-43da-9923-3ea68740b318-signing-key\") pod \"service-ca-5fffc97bdf-2zdns\" (UID: \"b4dcd02c-9a77-43da-9923-3ea68740b318\") " pod="openshift-service-ca/service-ca-5fffc97bdf-2zdns"
Nov 04 01:02:36.161824 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:36.161744    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p664g\" (UniqueName: \"kubernetes.io/projected/8f92b52c-22de-4ce7-b716-fdfa22d7b52f-kube-api-access-p664g\") pod \"migrator-66df5d496b-fp864\" (UID: \"8f92b52c-22de-4ce7-b716-fdfa22d7b52f\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-fp864"
Nov 04 01:02:36.161911 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:36.161886    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45zdl\" (UniqueName: \"kubernetes.io/projected/b4dcd02c-9a77-43da-9923-3ea68740b318-kube-api-access-45zdl\") pod \"service-ca-5fffc97bdf-2zdns\" (UID: \"b4dcd02c-9a77-43da-9923-3ea68740b318\") " pod="openshift-service-ca/service-ca-5fffc97bdf-2zdns"
Nov 04 01:02:36.161993 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:36.161931    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6tv2\" (UniqueName: \"kubernetes.io/projected/9d694048-4b09-480b-ad26-8084c7775b24-kube-api-access-b6tv2\") pod \"migrator-66df5d496b-b9rdk\" (UID: \"9d694048-4b09-480b-ad26-8084c7775b24\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-b9rdk"
Nov 04 01:02:36.174748 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:36.174730    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6tv2\" (UniqueName: \"kubernetes.io/projected/9d694048-4b09-480b-ad26-8084c7775b24-kube-api-access-b6tv2\") pod \"migrator-66df5d496b-b9rdk\" (UID: \"9d694048-4b09-480b-ad26-8084c7775b24\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-b9rdk"
Nov 04 01:02:36.175019 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:36.174996    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p664g\" (UniqueName: \"kubernetes.io/projected/8f92b52c-22de-4ce7-b716-fdfa22d7b52f-kube-api-access-p664g\") pod \"migrator-66df5d496b-fp864\" (UID: \"8f92b52c-22de-4ce7-b716-fdfa22d7b52f\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-fp864"
Nov 04 01:02:36.262811 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:36.262783    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b4dcd02c-9a77-43da-9923-3ea68740b318-signing-cabundle\") pod \"service-ca-5fffc97bdf-2zdns\" (UID: \"b4dcd02c-9a77-43da-9923-3ea68740b318\") " pod="openshift-service-ca/service-ca-5fffc97bdf-2zdns"
Nov 04 01:02:36.262956 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:36.262824    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b4dcd02c-9a77-43da-9923-3ea68740b318-signing-key\") pod \"service-ca-5fffc97bdf-2zdns\" (UID: \"b4dcd02c-9a77-43da-9923-3ea68740b318\") " pod="openshift-service-ca/service-ca-5fffc97bdf-2zdns"
Nov 04 01:02:36.262956 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:36.262924    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45zdl\" (UniqueName: \"kubernetes.io/projected/b4dcd02c-9a77-43da-9923-3ea68740b318-kube-api-access-45zdl\") pod \"service-ca-5fffc97bdf-2zdns\" (UID: \"b4dcd02c-9a77-43da-9923-3ea68740b318\") " pod="openshift-service-ca/service-ca-5fffc97bdf-2zdns"
Nov 04 01:02:36.263720 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:36.263699    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b4dcd02c-9a77-43da-9923-3ea68740b318-signing-cabundle\") pod \"service-ca-5fffc97bdf-2zdns\" (UID: \"b4dcd02c-9a77-43da-9923-3ea68740b318\") " pod="openshift-service-ca/service-ca-5fffc97bdf-2zdns"
Nov 04 01:02:36.266148 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:36.266129    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b4dcd02c-9a77-43da-9923-3ea68740b318-signing-key\") pod \"service-ca-5fffc97bdf-2zdns\" (UID: \"b4dcd02c-9a77-43da-9923-3ea68740b318\") " pod="openshift-service-ca/service-ca-5fffc97bdf-2zdns"
Nov 04 01:02:36.276171 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:36.276152    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-45zdl\" (UniqueName: \"kubernetes.io/projected/b4dcd02c-9a77-43da-9923-3ea68740b318-kube-api-access-45zdl\") pod \"service-ca-5fffc97bdf-2zdns\" (UID: \"b4dcd02c-9a77-43da-9923-3ea68740b318\") " pod="openshift-service-ca/service-ca-5fffc97bdf-2zdns"
Nov 04 01:02:36.301516 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:36.301489    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-fp864"
Nov 04 01:02:36.324115 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:36.324086    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-b9rdk"
Nov 04 01:02:36.386086 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:36.386050    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-2zdns"
Nov 04 01:02:36.449013 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:36.448042    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-t6tpg-6r4ws"]
Nov 04 01:02:36.465505 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:36.465296    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sg4cw\" (UID: \"ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774\") " pod="openshift-insights/insights-runtime-extractor-sg4cw"
Nov 04 01:02:36.465505 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:36.465424    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sg4cw\" (UID: \"ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774\") " pod="openshift-insights/insights-runtime-extractor-sg4cw"
Nov 04 01:02:36.466362 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:36.465889    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 01:02:36.466362 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:36.465960    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-insights-runtime-extractor-tls podName:ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:37.46594281 +0000 UTC m=+40.691790988 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-insights-runtime-extractor-tls") pod "insights-runtime-extractor-sg4cw" (UID: "ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774") : secret "insights-runtime-extractor-tls" not found
Nov 04 01:02:36.466362 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:36.466292    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 04 01:02:36.466362 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:36.466340    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-kube-rbac-proxy-cm podName:ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:37.466326018 +0000 UTC m=+40.692174189 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-kube-rbac-proxy-cm") pod "insights-runtime-extractor-sg4cw" (UID: "ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774") : configmap "kube-rbac-proxy" not found
Nov 04 01:02:36.625867 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:36.622894    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-fp864"]
Nov 04 01:02:36.639123 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:36.639095    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-b9rdk"]
Nov 04 01:02:36.643048 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:02:36.643022    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d694048_4b09_480b_ad26_8084c7775b24.slice/crio-00c93c0ab8afbab1c8b30f3162eed2558e8d2cc243e654b7f8276a693dd3331a WatchSource:0}: Error finding container 00c93c0ab8afbab1c8b30f3162eed2558e8d2cc243e654b7f8276a693dd3331a: Status 404 returned error can't find the container with id 00c93c0ab8afbab1c8b30f3162eed2558e8d2cc243e654b7f8276a693dd3331a
Nov 04 01:02:36.710276 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:36.710249    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-2zdns"]
Nov 04 01:02:36.713186 ip-10-0-132-67 kubenswrapper[2582]: W1104 01:02:36.713146    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4dcd02c_9a77_43da_9923_3ea68740b318.slice/crio-221b52fd58c2e1a2d63f2e5d720cb19394a30e5038eadbfead24a3c9bbc10394 WatchSource:0}: Error finding container 221b52fd58c2e1a2d63f2e5d720cb19394a30e5038eadbfead24a3c9bbc10394: Status 404 returned error can't find the container with id 221b52fd58c2e1a2d63f2e5d720cb19394a30e5038eadbfead24a3c9bbc10394
Nov 04 01:02:36.894068 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:36.894040    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-fp864" event={"ID":"8f92b52c-22de-4ce7-b716-fdfa22d7b52f","Type":"ContainerStarted","Data":"d3753e582fe061c82729cc6f22b8faeefcc91fc5ce845e1308069b4347e092ba"}
Nov 04 01:02:36.895676 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:36.895645    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-2zdns" event={"ID":"b4dcd02c-9a77-43da-9923-3ea68740b318","Type":"ContainerStarted","Data":"4f7118faf288e0ce9ef7c970afa81d0f0d1185983d0bf88fbfd2971aa9d5ba6f"}
Nov 04 01:02:36.895800 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:36.895693    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-2zdns" event={"ID":"b4dcd02c-9a77-43da-9923-3ea68740b318","Type":"ContainerStarted","Data":"221b52fd58c2e1a2d63f2e5d720cb19394a30e5038eadbfead24a3c9bbc10394"}
Nov 04 01:02:36.896838 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:36.896814    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-b9rdk" event={"ID":"9d694048-4b09-480b-ad26-8084c7775b24","Type":"ContainerStarted","Data":"00c93c0ab8afbab1c8b30f3162eed2558e8d2cc243e654b7f8276a693dd3331a"}
Nov 04 01:02:36.898260 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:36.898237    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-wxhng" event={"ID":"588c643c-b16a-4d5c-ad85-ccb8d486dd43","Type":"ContainerStarted","Data":"a7c7c7910b3795bd0fb9da8033e7e0b24fb0e1b9d961296c0549b586adf7785c"}
Nov 04 01:02:36.898364 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:36.898314    2582 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-wxhng"
Nov 04 01:02:36.899969 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:36.899942    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-t6tpg-6r4ws" event={"ID":"e2b1aee4-49c5-47a0-a79f-a3fe032df2d7","Type":"ContainerStarted","Data":"a0fce730bf11eaa5b9e2bb472b0945a0aa56467f69bb1c716e870daa069b6abb"}
Nov 04 01:02:36.900065 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:36.899989    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-t6tpg-6r4ws" event={"ID":"e2b1aee4-49c5-47a0-a79f-a3fe032df2d7","Type":"ContainerStarted","Data":"0acd953dd0d10f6aab6269a568a4bfe682d862b978047b926a7a8e3fd42e2bc6"}
Nov 04 01:02:36.914154 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:36.914108    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca/service-ca-5fffc97bdf-2zdns" podStartSLOduration=0.91409332 podStartE2EDuration="914.09332ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:02:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:02:36.913020951 +0000 UTC m=+40.138869137" watchObservedRunningTime="2026-11-04 01:02:36.91409332 +0000 UTC m=+40.139941508"
Nov 04 01:02:36.930160 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:36.930090    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-wxhng" podStartSLOduration=34.885549692 podStartE2EDuration="38.930074981s" totalImagesPullingTime="4.044525289s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:01:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:02:36.92982147 +0000 UTC m=+40.155669655" watchObservedRunningTime="2026-11-04 01:02:36.930074981 +0000 UTC m=+40.155923169"
Nov 04 01:02:36.949981 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:36.949944    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-t6tpg-6r4ws" podStartSLOduration=1.949931702 podStartE2EDuration="1.949931702s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:02:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:02:36.947805293 +0000 UTC m=+40.173653483" watchObservedRunningTime="2026-11-04 01:02:36.949931702 +0000 UTC m=+40.175779888"
Nov 04 01:02:37.474922 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:37.474890    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sg4cw\" (UID: \"ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774\") " pod="openshift-insights/insights-runtime-extractor-sg4cw"
Nov 04 01:02:37.475109 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:37.475008    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sg4cw\" (UID: \"ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774\") " pod="openshift-insights/insights-runtime-extractor-sg4cw"
Nov 04 01:02:37.475200 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:37.475107    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 04 01:02:37.475200 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:37.475155    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-kube-rbac-proxy-cm podName:ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:39.475143126 +0000 UTC m=+42.700991291 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-kube-rbac-proxy-cm") pod "insights-runtime-extractor-sg4cw" (UID: "ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774") : configmap "kube-rbac-proxy" not found
Nov 04 01:02:37.475511 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:37.475493    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 01:02:37.475595 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:37.475539    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-insights-runtime-extractor-tls podName:ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:39.475528169 +0000 UTC m=+42.701376347 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-insights-runtime-extractor-tls") pod "insights-runtime-extractor-sg4cw" (UID: "ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774") : secret "insights-runtime-extractor-tls" not found
Nov 04 01:02:38.517497 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:38.517457    2582 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-tr8b2"
Nov 04 01:02:38.517906 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:38.517886    2582 scope.go:122] "RemoveContainer" containerID="9d9855e9203df085f919c24b870c736a1b366e38fdfa9b8aea39bcaedc7f5ded"
Nov 04 01:02:38.518101 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:38.518079    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-tr8b2_openshift-console-operator(21d1d23b-7b6e-4ab5-9593-cf7784fffea2)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-tr8b2" podUID="21d1d23b-7b6e-4ab5-9593-cf7784fffea2"
Nov 04 01:02:38.518160 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:38.518110    2582 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-tr8b2"
Nov 04 01:02:38.728970 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:38.728940    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8gvq8_d6a4c158-e115-4d2f-af0d-1d9249dc7384/dns-node-resolver/0.log"
Nov 04 01:02:38.907886 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:38.907817    2582 scope.go:122] "RemoveContainer" containerID="9d9855e9203df085f919c24b870c736a1b366e38fdfa9b8aea39bcaedc7f5ded"
Nov 04 01:02:38.908031 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:38.908001    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-tr8b2_openshift-console-operator(21d1d23b-7b6e-4ab5-9593-cf7784fffea2)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-tr8b2" podUID="21d1d23b-7b6e-4ab5-9593-cf7784fffea2"
Nov 04 01:02:39.498100 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:39.498065    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sg4cw\" (UID: \"ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774\") " pod="openshift-insights/insights-runtime-extractor-sg4cw"
Nov 04 01:02:39.498245 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:39.498179    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sg4cw\" (UID: \"ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774\") " pod="openshift-insights/insights-runtime-extractor-sg4cw"
Nov 04 01:02:39.498245 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:39.498209    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 01:02:39.498345 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:39.498254    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 04 01:02:39.498345 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:39.498266    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-insights-runtime-extractor-tls podName:ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:43.49825182 +0000 UTC m=+46.724099984 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-insights-runtime-extractor-tls") pod "insights-runtime-extractor-sg4cw" (UID: "ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774") : secret "insights-runtime-extractor-tls" not found
Nov 04 01:02:39.498345 ip-10-0-132-67 kubenswrapper[2582]: E1104 01:02:39.498320    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-kube-rbac-proxy-cm podName:ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:43.498308572 +0000 UTC m=+46.724156736 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-kube-rbac-proxy-cm") pod "insights-runtime-extractor-sg4cw" (UID: "ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774") : configmap "kube-rbac-proxy" not found
Nov 04 01:02:39.729332 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:39.729309    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-fvjq7_7b044258-a241-4bd2-a6eb-9ed586a0f2cc/node-ca/0.log"
Nov 04 01:02:39.891399 ip-10-0-132-67 kubenswrapper[2582]: I1104 01:02:39.891287    2582 dynamic_cafile_content.go:175] "Shutting down controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 04 01:02:39.891458 ip-10-0-132-67 systemd[1]: Stopping Kubernetes Kubelet...
Nov 04 01:02:39.905959 ip-10-0-132-67 systemd[1]: kubelet.service: Deactivated successfully.
Nov 04 01:02:39.906121 ip-10-0-132-67 systemd[1]: Stopped Kubernetes Kubelet.
Nov 04 01:02:39.906173 ip-10-0-132-67 systemd[1]: kubelet.service: Consumed 3.087s CPU time.
Nov 04 01:02:39.915185 ip-10-0-132-67 systemd[1]: Starting Kubernetes Kubelet...
Nov 04 01:02:40.025035 ip-10-0-132-67 kubenswrapper[5068]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 01:02:40.025035 ip-10-0-132-67 kubenswrapper[5068]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 04 01:02:40.025035 ip-10-0-132-67 kubenswrapper[5068]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 01:02:40.025035 ip-10-0-132-67 kubenswrapper[5068]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 01:02:40.028662 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028568    5068 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:02:40.028662 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028660    5068 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:02:40.028662 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028664    5068 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:02:40.028662 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028668    5068 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:02:40.028817 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028671    5068 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:02:40.028817 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028674    5068 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:02:40.028817 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028677    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:02:40.028817 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028679    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:02:40.028817 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028682    5068 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:02:40.028817 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028685    5068 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:02:40.028817 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028688    5068 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:02:40.028817 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028692    5068 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:02:40.028817 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028694    5068 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:02:40.028817 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028697    5068 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:02:40.028817 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028700    5068 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:02:40.028817 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028702    5068 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:02:40.028817 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028705    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:02:40.028817 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028708    5068 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:02:40.028817 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028711    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:02:40.028817 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028713    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:02:40.028817 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028716    5068 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:02:40.028817 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028719    5068 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:02:40.028817 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028723    5068 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:02:40.028817 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028726    5068 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:02:40.029350 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028738    5068 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:02:40.029350 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028741    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:02:40.029350 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028743    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:02:40.029350 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028746    5068 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:02:40.029350 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028748    5068 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:02:40.029350 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028751    5068 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:02:40.029350 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028754    5068 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:02:40.029350 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028763    5068 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:02:40.029350 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028766    5068 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:02:40.029350 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028769    5068 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:02:40.029350 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028775    5068 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:02:40.029350 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028778    5068 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:02:40.029350 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028781    5068 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:02:40.029350 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028784    5068 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:02:40.029350 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028786    5068 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:02:40.029350 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028790    5068 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:02:40.029350 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028794    5068 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:02:40.029350 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028797    5068 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:02:40.029811 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028800    5068 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:02:40.029811 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028803    5068 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:02:40.029811 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028805    5068 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:02:40.029811 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028808    5068 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:02:40.029811 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028810    5068 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:02:40.029811 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028813    5068 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:02:40.029811 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028815    5068 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:02:40.029811 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028818    5068 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:02:40.029811 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028821    5068 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:02:40.029811 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028824    5068 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:02:40.029811 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028826    5068 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:02:40.029811 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028829    5068 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:02:40.029811 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028832    5068 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:02:40.029811 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028835    5068 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:02:40.029811 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028837    5068 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:02:40.029811 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028840    5068 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:02:40.029811 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028843    5068 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:02:40.029811 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028846    5068 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:02:40.029811 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028848    5068 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:02:40.029811 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028851    5068 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:02:40.030294 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028855    5068 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:02:40.030294 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028860    5068 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:02:40.030294 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028863    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:02:40.030294 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028865    5068 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:02:40.030294 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028869    5068 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:02:40.030294 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028871    5068 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:02:40.030294 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028874    5068 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:02:40.030294 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028877    5068 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:02:40.030294 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028879    5068 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:02:40.030294 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028882    5068 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:02:40.030294 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028884    5068 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:02:40.030294 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028887    5068 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:02:40.030294 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028889    5068 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:02:40.030294 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028892    5068 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:02:40.030294 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028896    5068 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:02:40.030294 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028898    5068 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:02:40.030294 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028901    5068 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:02:40.030294 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028904    5068 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:02:40.030294 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028906    5068 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:02:40.030766 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028908    5068 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:02:40.030766 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028911    5068 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:02:40.030766 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028919    5068 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:02:40.030766 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028922    5068 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:02:40.030766 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028925    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:02:40.030766 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028927    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:02:40.030766 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028930    5068 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:02:40.030766 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028933    5068 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:02:40.030766 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028935    5068 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:02:40.030766 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028938    5068 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:02:40.030766 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028940    5068 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:02:40.030766 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028943    5068 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:02:40.030766 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028946    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:02:40.030766 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028949    5068 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:02:40.030766 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028951    5068 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:02:40.030766 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028954    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:02:40.030766 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028956    5068 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:02:40.030766 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028959    5068 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:02:40.030766 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028963    5068 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:02:40.030766 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028966    5068 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:02:40.031265 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028969    5068 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:02:40.031265 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028971    5068 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:02:40.031265 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028974    5068 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:02:40.031265 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028976    5068 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:02:40.031265 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.028979    5068 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:02:40.031265 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029747    5068 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:02:40.031265 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029754    5068 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:02:40.031265 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029758    5068 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:02:40.031265 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029761    5068 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:02:40.031265 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029764    5068 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:02:40.031265 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029766    5068 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:02:40.031265 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029769    5068 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:02:40.031265 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029772    5068 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:02:40.031265 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029774    5068 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:02:40.031265 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029777    5068 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:02:40.031265 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029781    5068 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:02:40.031265 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029785    5068 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:02:40.031265 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029788    5068 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:02:40.031265 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029790    5068 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:02:40.031265 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029793    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:02:40.031764 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029796    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:02:40.031764 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029799    5068 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:02:40.031764 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029801    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:02:40.031764 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029804    5068 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:02:40.031764 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029807    5068 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:02:40.031764 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029809    5068 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:02:40.031764 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029812    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:02:40.031764 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029815    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:02:40.031764 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029817    5068 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:02:40.031764 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029820    5068 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:02:40.031764 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029823    5068 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:02:40.031764 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029825    5068 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:02:40.031764 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029828    5068 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:02:40.031764 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029831    5068 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:02:40.031764 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029833    5068 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:02:40.031764 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029836    5068 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:02:40.031764 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029838    5068 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:02:40.031764 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029841    5068 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:02:40.031764 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029843    5068 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:02:40.032237 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029846    5068 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:02:40.032237 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029849    5068 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:02:40.032237 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029851    5068 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:02:40.032237 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029854    5068 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:02:40.032237 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029857    5068 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:02:40.032237 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029859    5068 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:02:40.032237 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029862    5068 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:02:40.032237 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029864    5068 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:02:40.032237 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029866    5068 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:02:40.032237 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029871    5068 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:02:40.032237 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029874    5068 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:02:40.032237 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029876    5068 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:02:40.032237 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029879    5068 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:02:40.032237 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029882    5068 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:02:40.032237 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029884    5068 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:02:40.032237 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029887    5068 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:02:40.032237 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029889    5068 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:02:40.032237 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029892    5068 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:02:40.032237 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029894    5068 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:02:40.032237 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029897    5068 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:02:40.032758 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029899    5068 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:02:40.032758 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029902    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:02:40.032758 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029904    5068 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:02:40.032758 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029907    5068 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:02:40.032758 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029909    5068 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:02:40.032758 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029912    5068 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:02:40.032758 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029914    5068 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:02:40.032758 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029917    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:02:40.032758 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029921    5068 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:02:40.032758 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029925    5068 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:02:40.032758 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029929    5068 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:02:40.032758 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029932    5068 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:02:40.032758 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029935    5068 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:02:40.032758 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029938    5068 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:02:40.032758 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029940    5068 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:02:40.032758 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029943    5068 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:02:40.032758 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029946    5068 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:02:40.032758 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029948    5068 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:02:40.032758 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029951    5068 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:02:40.033234 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029953    5068 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:02:40.033234 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029956    5068 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:02:40.033234 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029959    5068 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:02:40.033234 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029964    5068 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:02:40.033234 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029967    5068 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:02:40.033234 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029969    5068 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:02:40.033234 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029972    5068 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:02:40.033234 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029974    5068 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:02:40.033234 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029977    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:02:40.033234 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029980    5068 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:02:40.033234 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029982    5068 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:02:40.033234 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029985    5068 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:02:40.033234 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029987    5068 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:02:40.033234 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029990    5068 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:02:40.033234 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029993    5068 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:02:40.033234 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029995    5068 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:02:40.033234 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.029998    5068 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:02:40.033234 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030000    5068 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:02:40.033234 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030003    5068 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:02:40.033234 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030005    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:02:40.033735 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030008    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:02:40.033735 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030011    5068 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:02:40.033735 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030013    5068 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:02:40.033735 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030016    5068 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:02:40.033735 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030018    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:02:40.033735 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030021    5068 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:02:40.033735 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030023    5068 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:02:40.033735 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030028    5068 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:02:40.033735 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030031    5068 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:02:40.033735 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030034    5068 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:02:40.033735 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030037    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:02:40.033735 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030039    5068 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:02:40.033735 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030042    5068 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:02:40.033735 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030504    5068 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:02:40.033735 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030509    5068 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:02:40.033735 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030515    5068 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:02:40.033735 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030518    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:02:40.033735 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030520    5068 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:02:40.033735 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030523    5068 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:02:40.034204 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030526    5068 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:02:40.034204 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030528    5068 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:02:40.034204 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030531    5068 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:02:40.034204 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030534    5068 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:02:40.034204 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030537    5068 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:02:40.034204 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030539    5068 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:02:40.034204 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030542    5068 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:02:40.034204 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030545    5068 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:02:40.034204 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030547    5068 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:02:40.034204 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030550    5068 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:02:40.034204 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030552    5068 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:02:40.034204 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030555    5068 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:02:40.034204 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030557    5068 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:02:40.034204 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030560    5068 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:02:40.034204 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030563    5068 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:02:40.034204 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030565    5068 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:02:40.034204 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030567    5068 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:02:40.034204 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030571    5068 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:02:40.034204 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030575    5068 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:02:40.034708 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030578    5068 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:02:40.034708 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030582    5068 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:02:40.034708 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030585    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:02:40.034708 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030588    5068 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:02:40.034708 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030591    5068 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:02:40.034708 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030595    5068 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:02:40.034708 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030598    5068 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:02:40.034708 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030602    5068 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:02:40.034708 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030605    5068 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:02:40.034708 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030625    5068 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:02:40.034708 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030630    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:02:40.034708 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030633    5068 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:02:40.034708 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030636    5068 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:02:40.034708 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030639    5068 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:02:40.034708 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030641    5068 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:02:40.034708 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030644    5068 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:02:40.034708 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030647    5068 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:02:40.034708 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030649    5068 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:02:40.034708 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030652    5068 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:02:40.034708 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030654    5068 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:02:40.035179 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030657    5068 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:02:40.035179 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030659    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:02:40.035179 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030662    5068 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:02:40.035179 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030665    5068 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:02:40.035179 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030668    5068 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:02:40.035179 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030671    5068 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:02:40.035179 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030674    5068 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:02:40.035179 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030677    5068 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:02:40.035179 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030679    5068 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:02:40.035179 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030682    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:02:40.035179 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030684    5068 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:02:40.035179 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030687    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:02:40.035179 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030689    5068 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:02:40.035179 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030692    5068 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:02:40.035179 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030695    5068 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:02:40.035179 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030697    5068 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:02:40.035179 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030700    5068 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:02:40.035179 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030703    5068 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:02:40.035179 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030706    5068 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:02:40.035667 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030708    5068 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:02:40.035667 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030711    5068 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:02:40.035667 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030715    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:02:40.035667 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030717    5068 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:02:40.035667 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030722    5068 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:02:40.035667 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030724    5068 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:02:40.035667 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030727    5068 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:02:40.035667 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030730    5068 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:02:40.035667 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030732    5068 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:02:40.035667 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030735    5068 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:02:40.035667 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030737    5068 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:02:40.035667 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030740    5068 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:02:40.035667 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030742    5068 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:02:40.035667 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030745    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:02:40.035667 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030747    5068 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:02:40.035667 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030750    5068 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:02:40.035667 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030752    5068 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:02:40.035667 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030755    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:02:40.035667 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030760    5068 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:02:40.035667 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030763    5068 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:02:40.036134 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030766    5068 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:02:40.036134 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030769    5068 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:02:40.036134 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030772    5068 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:02:40.036134 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030775    5068 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:02:40.036134 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030777    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:02:40.036134 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030780    5068 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:02:40.036134 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030783    5068 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:02:40.036134 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030785    5068 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:02:40.036134 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030788    5068 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:02:40.036134 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030791    5068 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:02:40.036134 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030793    5068 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:02:40.036134 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030796    5068 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:02:40.036134 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030799    5068 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:02:40.036134 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030801    5068 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:02:40.036134 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030806    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:02:40.036134 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030808    5068 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:02:40.036134 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030811    5068 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:02:40.036134 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030814    5068 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:02:40.036134 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030817    5068 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:02:40.036580 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030819    5068 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:02:40.036580 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030822    5068 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:02:40.036580 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.030824    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:02:40.036580 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.030914    5068 flags.go:64] FLAG: --address="0.0.0.0"
Nov 04 01:02:40.036580 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.030926    5068 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 04 01:02:40.036580 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.030931    5068 flags.go:64] FLAG: --anonymous-auth="true"
Nov 04 01:02:40.036580 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.030936    5068 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 04 01:02:40.036580 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.030940    5068 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 04 01:02:40.036580 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.030943    5068 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 04 01:02:40.036580 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.030948    5068 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 04 01:02:40.036580 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.030952    5068 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 04 01:02:40.036580 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.030957    5068 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 04 01:02:40.036580 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.030960    5068 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 04 01:02:40.036580 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.030964    5068 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 04 01:02:40.036580 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.030967    5068 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 04 01:02:40.036580 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.030970    5068 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 04 01:02:40.036580 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.030973    5068 flags.go:64] FLAG: --cgroup-root=""
Nov 04 01:02:40.036580 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.030976    5068 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 04 01:02:40.036580 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.030979    5068 flags.go:64] FLAG: --client-ca-file=""
Nov 04 01:02:40.036580 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.030982    5068 flags.go:64] FLAG: --cloud-provider="external"
Nov 04 01:02:40.036580 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.030985    5068 flags.go:64] FLAG: --cluster-dns="[]"
Nov 04 01:02:40.036580 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.030990    5068 flags.go:64] FLAG: --cluster-domain=""
Nov 04 01:02:40.036580 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.030993    5068 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 04 01:02:40.036580 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.030996    5068 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 04 01:02:40.037161 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.030999    5068 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 04 01:02:40.037161 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031003    5068 flags.go:64] FLAG: --container-log-max-files="5"
Nov 04 01:02:40.037161 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031007    5068 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 04 01:02:40.037161 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031011    5068 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 04 01:02:40.037161 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031014    5068 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 04 01:02:40.037161 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031018    5068 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 04 01:02:40.037161 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031021    5068 flags.go:64] FLAG: --contention-profiling="false"
Nov 04 01:02:40.037161 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031024    5068 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 04 01:02:40.037161 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031027    5068 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 04 01:02:40.037161 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031030    5068 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 04 01:02:40.037161 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031033    5068 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 04 01:02:40.037161 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031037    5068 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 04 01:02:40.037161 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031040    5068 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 04 01:02:40.037161 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031043    5068 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 04 01:02:40.037161 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031046    5068 flags.go:64] FLAG: --enable-load-reader="false"
Nov 04 01:02:40.037161 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031049    5068 flags.go:64] FLAG: --enable-server="true"
Nov 04 01:02:40.037161 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031052    5068 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 04 01:02:40.037161 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031056    5068 flags.go:64] FLAG: --event-burst="100"
Nov 04 01:02:40.037161 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031059    5068 flags.go:64] FLAG: --event-qps="50"
Nov 04 01:02:40.037161 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031063    5068 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 04 01:02:40.037161 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031066    5068 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 04 01:02:40.037161 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031069    5068 flags.go:64] FLAG: --eviction-hard=""
Nov 04 01:02:40.037161 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031073    5068 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 04 01:02:40.037161 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031076    5068 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 04 01:02:40.037785 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031079    5068 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 04 01:02:40.037785 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031082    5068 flags.go:64] FLAG: --eviction-soft=""
Nov 04 01:02:40.037785 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031085    5068 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 04 01:02:40.037785 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031088    5068 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 04 01:02:40.037785 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031091    5068 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 04 01:02:40.037785 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031094    5068 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 04 01:02:40.037785 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031097    5068 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 04 01:02:40.037785 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031100    5068 flags.go:64] FLAG: --fail-swap-on="true"
Nov 04 01:02:40.037785 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031103    5068 flags.go:64] FLAG: --feature-gates=""
Nov 04 01:02:40.037785 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031108    5068 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 04 01:02:40.037785 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031111    5068 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 04 01:02:40.037785 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031115    5068 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 04 01:02:40.037785 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031119    5068 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 04 01:02:40.037785 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031122    5068 flags.go:64] FLAG: --healthz-port="10248"
Nov 04 01:02:40.037785 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031125    5068 flags.go:64] FLAG: --help="false"
Nov 04 01:02:40.037785 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031128    5068 flags.go:64] FLAG: --hostname-override="ip-10-0-132-67.us-east-2.compute.internal"
Nov 04 01:02:40.037785 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031135    5068 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 04 01:02:40.037785 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031138    5068 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 04 01:02:40.037785 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031141    5068 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 04 01:02:40.037785 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031145    5068 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 04 01:02:40.037785 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031148    5068 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 04 01:02:40.037785 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031151    5068 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 04 01:02:40.037785 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031154    5068 flags.go:64] FLAG: --image-service-endpoint=""
Nov 04 01:02:40.038344 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031157    5068 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 04 01:02:40.038344 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031160    5068 flags.go:64] FLAG: --kube-api-burst="100"
Nov 04 01:02:40.038344 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031163    5068 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 04 01:02:40.038344 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031166    5068 flags.go:64] FLAG: --kube-api-qps="50"
Nov 04 01:02:40.038344 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031170    5068 flags.go:64] FLAG: --kube-reserved=""
Nov 04 01:02:40.038344 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031174    5068 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 04 01:02:40.038344 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031177    5068 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 04 01:02:40.038344 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031180    5068 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 04 01:02:40.038344 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031183    5068 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 04 01:02:40.038344 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031185    5068 flags.go:64] FLAG: --lock-file=""
Nov 04 01:02:40.038344 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031188    5068 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 04 01:02:40.038344 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031191    5068 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 04 01:02:40.038344 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031194    5068 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 04 01:02:40.038344 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031200    5068 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 04 01:02:40.038344 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031203    5068 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 04 01:02:40.038344 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031206    5068 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 04 01:02:40.038344 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031209    5068 flags.go:64] FLAG: --logging-format="text"
Nov 04 01:02:40.038344 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031212    5068 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 04 01:02:40.038344 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031215    5068 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 04 01:02:40.038344 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031218    5068 flags.go:64] FLAG: --manifest-url=""
Nov 04 01:02:40.038344 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031222    5068 flags.go:64] FLAG: --manifest-url-header=""
Nov 04 01:02:40.038344 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031227    5068 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 04 01:02:40.038344 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031230    5068 flags.go:64] FLAG: --max-open-files="1000000"
Nov 04 01:02:40.038344 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031234    5068 flags.go:64] FLAG: --max-pods="110"
Nov 04 01:02:40.038344 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031237    5068 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 04 01:02:40.038960 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031240    5068 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 04 01:02:40.038960 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031243    5068 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 04 01:02:40.038960 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031246    5068 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 04 01:02:40.038960 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031249    5068 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 04 01:02:40.038960 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031252    5068 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 04 01:02:40.038960 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031255    5068 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 04 01:02:40.038960 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031262    5068 flags.go:64] FLAG: --node-status-max-images="50"
Nov 04 01:02:40.038960 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031266    5068 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 04 01:02:40.038960 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031269    5068 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 04 01:02:40.038960 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031272    5068 flags.go:64] FLAG: --pod-cidr=""
Nov 04 01:02:40.038960 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031275    5068 flags.go:64] FLAG: --pod-manifest-path=""
Nov 04 01:02:40.038960 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031280    5068 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 04 01:02:40.038960 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031283    5068 flags.go:64] FLAG: --pods-per-core="0"
Nov 04 01:02:40.038960 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031286    5068 flags.go:64] FLAG: --port="10250"
Nov 04 01:02:40.038960 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031289    5068 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 04 01:02:40.038960 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031292    5068 flags.go:64] FLAG: --provider-id="aws:///us-east-2a/i-06148214c3363a096"
Nov 04 01:02:40.038960 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031296    5068 flags.go:64] FLAG: --qos-reserved=""
Nov 04 01:02:40.038960 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031299    5068 flags.go:64] FLAG: --read-only-port="10255"
Nov 04 01:02:40.038960 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031302    5068 flags.go:64] FLAG: --register-node="true"
Nov 04 01:02:40.038960 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031305    5068 flags.go:64] FLAG: --register-with-taints=""
Nov 04 01:02:40.038960 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031309    5068 flags.go:64] FLAG: --registry-burst="10"
Nov 04 01:02:40.038960 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031312    5068 flags.go:64] FLAG: --registry-qps="5"
Nov 04 01:02:40.038960 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031314    5068 flags.go:64] FLAG: --reserved-cpus=""
Nov 04 01:02:40.038960 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031318    5068 flags.go:64] FLAG: --reserved-memory=""
Nov 04 01:02:40.038960 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031321    5068 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 04 01:02:40.038960 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031325    5068 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 04 01:02:40.039625 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031328    5068 flags.go:64] FLAG: --rotate-certificates="false"
Nov 04 01:02:40.039625 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031331    5068 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 04 01:02:40.039625 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031334    5068 flags.go:64] FLAG: --runonce="false"
Nov 04 01:02:40.039625 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031337    5068 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 04 01:02:40.039625 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031340    5068 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 04 01:02:40.039625 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031343    5068 flags.go:64] FLAG: --seccomp-default="false"
Nov 04 01:02:40.039625 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031346    5068 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 04 01:02:40.039625 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031349    5068 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 04 01:02:40.039625 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031352    5068 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 04 01:02:40.039625 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031355    5068 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 04 01:02:40.039625 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031358    5068 flags.go:64] FLAG: --storage-driver-password="root"
Nov 04 01:02:40.039625 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031361    5068 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 04 01:02:40.039625 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031364    5068 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 04 01:02:40.039625 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031367    5068 flags.go:64] FLAG: --storage-driver-user="root"
Nov 04 01:02:40.039625 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031370    5068 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 04 01:02:40.039625 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031372    5068 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 04 01:02:40.039625 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031375    5068 flags.go:64] FLAG: --system-cgroups=""
Nov 04 01:02:40.039625 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031379    5068 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 04 01:02:40.039625 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031384    5068 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 04 01:02:40.039625 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031387    5068 flags.go:64] FLAG: --tls-cert-file=""
Nov 04 01:02:40.039625 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031390    5068 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 04 01:02:40.039625 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031394    5068 flags.go:64] FLAG: --tls-min-version=""
Nov 04 01:02:40.039625 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031397    5068 flags.go:64] FLAG: --tls-private-key-file=""
Nov 04 01:02:40.039625 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031400    5068 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 04 01:02:40.039625 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031403    5068 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 04 01:02:40.040529 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031406    5068 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 04 01:02:40.040529 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031409    5068 flags.go:64] FLAG: --v="2"
Nov 04 01:02:40.040529 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031413    5068 flags.go:64] FLAG: --version="false"
Nov 04 01:02:40.040529 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031417    5068 flags.go:64] FLAG: --vmodule=""
Nov 04 01:02:40.040529 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031421    5068 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 04 01:02:40.040529 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031424    5068 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 04 01:02:40.040529 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031565    5068 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:02:40.040529 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031568    5068 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:02:40.040529 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031572    5068 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:02:40.040529 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031574    5068 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:02:40.040529 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031577    5068 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:02:40.040529 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031580    5068 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:02:40.040529 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031582    5068 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:02:40.040529 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031585    5068 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:02:40.040529 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031588    5068 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:02:40.040529 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031590    5068 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:02:40.040529 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031593    5068 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:02:40.040529 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031596    5068 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:02:40.040529 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031620    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:02:40.040529 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031625    5068 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:02:40.040529 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031629    5068 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:02:40.041354 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031632    5068 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:02:40.041354 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031635    5068 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:02:40.041354 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031637    5068 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:02:40.041354 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031642    5068 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:02:40.041354 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031644    5068 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:02:40.041354 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031647    5068 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:02:40.041354 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031650    5068 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:02:40.041354 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031652    5068 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:02:40.041354 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031655    5068 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:02:40.041354 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031658    5068 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:02:40.041354 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031660    5068 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:02:40.041354 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031663    5068 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:02:40.041354 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031666    5068 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:02:40.041354 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031670    5068 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:02:40.041354 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031674    5068 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:02:40.041354 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031677    5068 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:02:40.041354 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031679    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:02:40.041354 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031682    5068 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:02:40.041354 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031685    5068 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:02:40.041872 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031687    5068 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:02:40.041872 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031690    5068 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:02:40.041872 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031692    5068 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:02:40.041872 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031695    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:02:40.041872 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031697    5068 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:02:40.041872 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031700    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:02:40.041872 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031702    5068 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:02:40.041872 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031705    5068 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:02:40.041872 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031707    5068 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:02:40.041872 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031711    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:02:40.041872 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031714    5068 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:02:40.041872 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031716    5068 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:02:40.041872 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031719    5068 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:02:40.041872 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031721    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:02:40.041872 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031724    5068 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:02:40.041872 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031727    5068 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:02:40.041872 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031730    5068 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:02:40.041872 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031733    5068 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:02:40.041872 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031736    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:02:40.041872 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031738    5068 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:02:40.042400 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031741    5068 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:02:40.042400 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031744    5068 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:02:40.042400 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031746    5068 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:02:40.042400 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031749    5068 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:02:40.042400 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031751    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:02:40.042400 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031754    5068 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:02:40.042400 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031756    5068 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:02:40.042400 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031759    5068 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:02:40.042400 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031761    5068 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:02:40.042400 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031764    5068 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:02:40.042400 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031766    5068 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:02:40.042400 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031769    5068 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:02:40.042400 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031772    5068 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:02:40.042400 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031774    5068 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:02:40.042400 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031777    5068 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:02:40.042400 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031780    5068 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:02:40.042400 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031782    5068 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:02:40.042400 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031785    5068 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:02:40.042400 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031788    5068 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:02:40.042400 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031790    5068 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:02:40.042970 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031792    5068 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:02:40.042970 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031795    5068 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:02:40.042970 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031798    5068 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:02:40.042970 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031801    5068 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:02:40.042970 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031804    5068 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:02:40.042970 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031806    5068 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:02:40.042970 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031809    5068 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:02:40.042970 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031812    5068 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:02:40.042970 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031816    5068 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:02:40.042970 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031820    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:02:40.042970 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031823    5068 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:02:40.042970 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031825    5068 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:02:40.042970 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031828    5068 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:02:40.042970 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031831    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:02:40.042970 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031833    5068 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:02:40.042970 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031836    5068 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:02:40.042970 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031839    5068 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:02:40.042970 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031841    5068 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:02:40.042970 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031845    5068 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:02:40.043429 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031848    5068 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:02:40.043429 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031851    5068 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:02:40.043429 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031853    5068 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:02:40.043429 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031856    5068 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:02:40.043429 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031859    5068 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:02:40.043429 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031861    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:02:40.043429 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031864    5068 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:02:40.043429 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031866    5068 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:02:40.043429 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031869    5068 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:02:40.043429 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031872    5068 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:02:40.043429 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031875    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:02:40.043429 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031877    5068 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:02:40.043429 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031880    5068 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:02:40.043429 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.031905    5068 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 01:02:40.043429 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031940    5068 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:02:40.043429 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031944    5068 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:02:40.043904 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031947    5068 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:02:40.043904 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031950    5068 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:02:40.043904 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031953    5068 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:02:40.043904 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031955    5068 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:02:40.043904 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031958    5068 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:02:40.043904 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031963    5068 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:02:40.043904 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031968    5068 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:02:40.043904 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031972    5068 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:02:40.043904 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031975    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:02:40.043904 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031978    5068 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:02:40.043904 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031981    5068 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:02:40.043904 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031984    5068 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:02:40.043904 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031987    5068 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:02:40.043904 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031989    5068 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:02:40.043904 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031992    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:02:40.043904 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031994    5068 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:02:40.043904 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031997    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:02:40.043904 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.031999    5068 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:02:40.043904 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032002    5068 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:02:40.044535 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032005    5068 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:02:40.044535 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032007    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:02:40.044535 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032010    5068 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:02:40.044535 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032012    5068 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:02:40.044535 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032015    5068 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:02:40.044535 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032017    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:02:40.044535 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032020    5068 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:02:40.044535 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032022    5068 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:02:40.044535 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032025    5068 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:02:40.044535 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032027    5068 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:02:40.044535 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032030    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:02:40.044535 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032033    5068 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:02:40.044535 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032035    5068 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:02:40.044535 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032038    5068 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:02:40.044535 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032041    5068 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:02:40.044535 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032044    5068 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:02:40.044535 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032046    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:02:40.044535 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032048    5068 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:02:40.044535 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032052    5068 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:02:40.044535 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032055    5068 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:02:40.045236 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032060    5068 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:02:40.045236 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032063    5068 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:02:40.045236 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032065    5068 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:02:40.045236 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032068    5068 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:02:40.045236 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032071    5068 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:02:40.045236 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032073    5068 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:02:40.045236 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032076    5068 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:02:40.045236 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032078    5068 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:02:40.045236 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032081    5068 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:02:40.045236 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032083    5068 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:02:40.045236 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032086    5068 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:02:40.045236 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032089    5068 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:02:40.045236 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032091    5068 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:02:40.045236 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032094    5068 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:02:40.045236 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032096    5068 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:02:40.045236 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032099    5068 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:02:40.045236 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032102    5068 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:02:40.045236 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032104    5068 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:02:40.045236 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032107    5068 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:02:40.045236 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032109    5068 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:02:40.045736 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032112    5068 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:02:40.045736 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032115    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:02:40.045736 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032117    5068 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:02:40.045736 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032120    5068 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:02:40.045736 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032122    5068 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:02:40.045736 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032125    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:02:40.045736 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032127    5068 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:02:40.045736 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032130    5068 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:02:40.045736 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032133    5068 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:02:40.045736 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032135    5068 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:02:40.045736 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032140    5068 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:02:40.045736 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032144    5068 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:02:40.045736 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032149    5068 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:02:40.045736 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032152    5068 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:02:40.045736 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032154    5068 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:02:40.045736 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032157    5068 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:02:40.045736 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032159    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:02:40.045736 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032162    5068 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:02:40.045736 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032164    5068 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:02:40.046215 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032167    5068 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:02:40.046215 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032170    5068 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:02:40.046215 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032173    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:02:40.046215 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032175    5068 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:02:40.046215 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032178    5068 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:02:40.046215 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032181    5068 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:02:40.046215 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032183    5068 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:02:40.046215 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032186    5068 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:02:40.046215 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032188    5068 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:02:40.046215 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032191    5068 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:02:40.046215 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032194    5068 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:02:40.046215 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032196    5068 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:02:40.046215 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032199    5068 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:02:40.046215 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032202    5068 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:02:40.046215 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032204    5068 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:02:40.046215 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032207    5068 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:02:40.046215 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032210    5068 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:02:40.046215 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032212    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:02:40.046215 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032219    5068 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:02:40.046686 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032222    5068 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:02:40.046686 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032225    5068 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:02:40.046686 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032228    5068 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:02:40.046686 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032230    5068 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:02:40.046686 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032234    5068 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:02:40.046686 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032237    5068 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:02:40.046686 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.032241    5068 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:02:40.046686 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.035716    5068 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 04 01:02:40.046686 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.035733    5068 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 04 01:02:40.046686 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.035750    5068 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 04 01:02:40.046686 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.035755    5068 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 04 01:02:40.046686 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035810    5068 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:02:40.046686 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035815    5068 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:02:40.046686 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035818    5068 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:02:40.046686 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035821    5068 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:02:40.046686 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035824    5068 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:02:40.046686 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035827    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:02:40.046686 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035831    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:02:40.046686 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035833    5068 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:02:40.047156 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035836    5068 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:02:40.047156 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035839    5068 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:02:40.047156 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035841    5068 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:02:40.047156 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035844    5068 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:02:40.047156 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035847    5068 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:02:40.047156 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035850    5068 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:02:40.047156 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035853    5068 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:02:40.047156 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035858    5068 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:02:40.047156 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035861    5068 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:02:40.047156 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035865    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:02:40.047156 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035869    5068 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:02:40.047156 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035871    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:02:40.047156 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035874    5068 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:02:40.047156 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035877    5068 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:02:40.047156 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035879    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:02:40.047156 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035882    5068 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:02:40.047156 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035885    5068 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:02:40.047156 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035888    5068 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:02:40.047156 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035890    5068 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:02:40.047668 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035893    5068 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:02:40.047668 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035896    5068 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:02:40.047668 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035899    5068 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:02:40.047668 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035902    5068 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:02:40.047668 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035904    5068 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:02:40.047668 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035907    5068 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:02:40.047668 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035909    5068 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:02:40.047668 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035912    5068 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:02:40.047668 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035916    5068 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:02:40.047668 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035920    5068 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:02:40.047668 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035924    5068 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:02:40.047668 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035927    5068 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:02:40.047668 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035929    5068 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:02:40.047668 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035932    5068 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:02:40.047668 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035935    5068 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:02:40.047668 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035938    5068 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:02:40.047668 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035941    5068 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:02:40.047668 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035943    5068 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:02:40.047668 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035946    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:02:40.048131 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035948    5068 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:02:40.048131 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035951    5068 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:02:40.048131 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035955    5068 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:02:40.048131 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035957    5068 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:02:40.048131 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035960    5068 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:02:40.048131 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035962    5068 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:02:40.048131 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035965    5068 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:02:40.048131 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035968    5068 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:02:40.048131 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035970    5068 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:02:40.048131 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035972    5068 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:02:40.048131 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035975    5068 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:02:40.048131 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035977    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:02:40.048131 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035980    5068 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:02:40.048131 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035983    5068 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:02:40.048131 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035985    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:02:40.048131 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035988    5068 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:02:40.048131 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035991    5068 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:02:40.048131 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035994    5068 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:02:40.048131 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035997    5068 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:02:40.048131 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.035999    5068 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:02:40.048787 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036002    5068 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:02:40.048787 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036004    5068 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:02:40.048787 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036007    5068 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:02:40.048787 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036010    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:02:40.048787 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036012    5068 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:02:40.048787 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036015    5068 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:02:40.048787 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036017    5068 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:02:40.048787 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036020    5068 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:02:40.048787 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036023    5068 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:02:40.048787 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036026    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:02:40.048787 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036028    5068 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:02:40.048787 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036031    5068 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:02:40.048787 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036033    5068 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:02:40.048787 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036036    5068 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:02:40.048787 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036039    5068 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:02:40.048787 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036041    5068 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:02:40.048787 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036044    5068 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:02:40.048787 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036046    5068 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:02:40.048787 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036049    5068 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:02:40.048787 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036051    5068 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:02:40.049295 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036054    5068 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:02:40.049295 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036056    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:02:40.049295 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036059    5068 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:02:40.049295 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036061    5068 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:02:40.049295 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036064    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:02:40.049295 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036066    5068 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:02:40.049295 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036069    5068 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:02:40.049295 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036071    5068 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:02:40.049295 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036074    5068 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:02:40.049295 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036077    5068 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:02:40.049295 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036079    5068 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:02:40.049295 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036082    5068 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:02:40.049295 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036084    5068 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:02:40.049295 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036087    5068 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:02:40.049295 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036090    5068 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:02:40.049295 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036092    5068 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:02:40.049295 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036095    5068 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:02:40.049295 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036097    5068 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:02:40.049295 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036100    5068 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:02:40.049295 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036103    5068 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:02:40.049881 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.036131    5068 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 01:02:40.049881 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036259    5068 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:02:40.049881 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036263    5068 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:02:40.049881 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036266    5068 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:02:40.049881 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036269    5068 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:02:40.049881 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036272    5068 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:02:40.049881 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036274    5068 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:02:40.049881 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036277    5068 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:02:40.049881 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036280    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:02:40.049881 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036282    5068 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:02:40.049881 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036286    5068 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:02:40.049881 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036291    5068 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:02:40.049881 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036294    5068 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:02:40.049881 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036297    5068 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:02:40.049881 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036300    5068 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:02:40.049881 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036303    5068 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:02:40.050307 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036306    5068 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:02:40.050307 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036308    5068 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:02:40.050307 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036311    5068 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:02:40.050307 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036313    5068 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:02:40.050307 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036316    5068 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:02:40.050307 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036318    5068 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:02:40.050307 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036321    5068 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:02:40.050307 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036324    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:02:40.050307 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036327    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:02:40.050307 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036329    5068 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:02:40.050307 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036332    5068 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:02:40.050307 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036334    5068 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:02:40.050307 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036337    5068 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:02:40.050307 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036339    5068 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:02:40.050307 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036342    5068 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:02:40.050307 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036344    5068 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:02:40.050307 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036347    5068 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:02:40.050307 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036350    5068 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:02:40.050307 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036352    5068 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:02:40.050307 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036355    5068 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:02:40.050898 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036358    5068 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:02:40.050898 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036360    5068 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:02:40.050898 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036363    5068 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:02:40.050898 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036365    5068 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:02:40.050898 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036368    5068 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:02:40.050898 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036370    5068 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:02:40.050898 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036373    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:02:40.050898 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036375    5068 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:02:40.050898 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036378    5068 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:02:40.050898 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036380    5068 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:02:40.050898 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036383    5068 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:02:40.050898 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036385    5068 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:02:40.050898 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036388    5068 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:02:40.050898 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036390    5068 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:02:40.050898 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036393    5068 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:02:40.050898 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036396    5068 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:02:40.050898 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036398    5068 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:02:40.050898 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036400    5068 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:02:40.050898 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036403    5068 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:02:40.051350 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036409    5068 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:02:40.051350 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036412    5068 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:02:40.051350 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036415    5068 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:02:40.051350 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036417    5068 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:02:40.051350 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036420    5068 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:02:40.051350 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036424    5068 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:02:40.051350 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036427    5068 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:02:40.051350 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036430    5068 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:02:40.051350 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036433    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:02:40.051350 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036436    5068 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:02:40.051350 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036438    5068 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:02:40.051350 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036441    5068 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:02:40.051350 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036444    5068 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:02:40.051350 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036446    5068 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:02:40.051350 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036449    5068 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:02:40.051350 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036451    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:02:40.051350 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036454    5068 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:02:40.051350 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036457    5068 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:02:40.051350 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036459    5068 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:02:40.051834 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036462    5068 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:02:40.051834 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036464    5068 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:02:40.051834 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036467    5068 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:02:40.051834 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036470    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:02:40.051834 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036473    5068 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:02:40.051834 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036475    5068 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:02:40.051834 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036477    5068 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:02:40.051834 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036480    5068 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:02:40.051834 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036483    5068 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:02:40.051834 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036485    5068 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:02:40.051834 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036488    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:02:40.051834 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036490    5068 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:02:40.051834 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036493    5068 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:02:40.051834 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036496    5068 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:02:40.051834 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036499    5068 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:02:40.051834 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036501    5068 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:02:40.051834 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036504    5068 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:02:40.051834 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036506    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:02:40.051834 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036509    5068 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:02:40.052307 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036511    5068 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:02:40.052307 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036514    5068 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:02:40.052307 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036516    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:02:40.052307 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036521    5068 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:02:40.052307 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036523    5068 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:02:40.052307 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036526    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:02:40.052307 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036529    5068 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:02:40.052307 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036531    5068 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:02:40.052307 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036535    5068 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:02:40.052307 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036537    5068 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:02:40.052307 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036540    5068 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:02:40.052307 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036542    5068 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:02:40.052307 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036545    5068 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:02:40.052307 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036547    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:02:40.052307 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.036570    5068 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 01:02:40.052307 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036629    5068 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:02:40.052696 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036633    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:02:40.052696 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036637    5068 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:02:40.052696 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036640    5068 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:02:40.052696 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036642    5068 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:02:40.052696 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036645    5068 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:02:40.052696 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036648    5068 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:02:40.052696 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036650    5068 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:02:40.052696 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036653    5068 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:02:40.052696 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036655    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:02:40.052696 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036658    5068 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:02:40.052696 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036662    5068 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:02:40.052696 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036665    5068 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:02:40.052696 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036669    5068 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:02:40.052696 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036672    5068 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:02:40.052696 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036675    5068 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:02:40.052696 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036677    5068 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:02:40.052696 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036680    5068 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:02:40.052696 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036683    5068 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:02:40.052696 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036686    5068 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:02:40.053209 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036688    5068 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:02:40.053209 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036693    5068 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:02:40.053209 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036695    5068 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:02:40.053209 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036698    5068 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:02:40.053209 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036701    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:02:40.053209 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036704    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:02:40.053209 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036707    5068 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:02:40.053209 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036709    5068 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:02:40.053209 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036712    5068 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:02:40.053209 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036714    5068 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:02:40.053209 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036717    5068 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:02:40.053209 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036719    5068 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:02:40.053209 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036722    5068 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:02:40.053209 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036724    5068 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:02:40.053209 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036727    5068 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:02:40.053209 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036730    5068 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:02:40.053209 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036732    5068 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:02:40.053209 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036735    5068 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:02:40.053209 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036737    5068 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:02:40.053738 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036740    5068 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:02:40.053738 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036742    5068 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:02:40.053738 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036745    5068 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:02:40.053738 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036748    5068 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:02:40.053738 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036750    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:02:40.053738 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036753    5068 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:02:40.053738 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036756    5068 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:02:40.053738 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036759    5068 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:02:40.053738 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036761    5068 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:02:40.053738 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036764    5068 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:02:40.053738 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036766    5068 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:02:40.053738 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036769    5068 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:02:40.053738 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036771    5068 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:02:40.053738 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036774    5068 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:02:40.053738 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036778    5068 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:02:40.053738 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036781    5068 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:02:40.053738 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036783    5068 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:02:40.053738 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036786    5068 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:02:40.053738 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036788    5068 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:02:40.054195 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036791    5068 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:02:40.054195 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036794    5068 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:02:40.054195 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036796    5068 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:02:40.054195 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036800    5068 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:02:40.054195 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036803    5068 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:02:40.054195 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036806    5068 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:02:40.054195 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036809    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:02:40.054195 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036811    5068 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:02:40.054195 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036814    5068 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:02:40.054195 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036817    5068 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:02:40.054195 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036819    5068 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:02:40.054195 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036822    5068 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:02:40.054195 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036824    5068 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:02:40.054195 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036827    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:02:40.054195 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036829    5068 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:02:40.054195 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036832    5068 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:02:40.054195 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036834    5068 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:02:40.054195 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036837    5068 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:02:40.054195 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036840    5068 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:02:40.054195 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036842    5068 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:02:40.054694 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036845    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:02:40.054694 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036848    5068 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:02:40.054694 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036850    5068 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:02:40.054694 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036853    5068 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:02:40.054694 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036855    5068 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:02:40.054694 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036858    5068 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:02:40.054694 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036861    5068 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:02:40.054694 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036866    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:02:40.054694 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036869    5068 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:02:40.054694 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036872    5068 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:02:40.054694 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036875    5068 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:02:40.054694 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036878    5068 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:02:40.054694 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036881    5068 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:02:40.054694 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036883    5068 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:02:40.054694 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036886    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:02:40.054694 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036888    5068 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:02:40.054694 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036891    5068 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:02:40.054694 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036894    5068 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:02:40.054694 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036897    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:02:40.054694 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036899    5068 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:02:40.055494 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036901    5068 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:02:40.055494 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036904    5068 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:02:40.055494 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036907    5068 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:02:40.055494 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036909    5068 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:02:40.055494 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036912    5068 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:02:40.055494 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036915    5068 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:02:40.055494 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036917    5068 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:02:40.055494 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:40.036920    5068 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:02:40.055494 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.036969    5068 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 04 01:02:40.055494 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.037903    5068 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Nov 04 01:02:40.055494 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.037952    5068 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Nov 04 01:02:40.055494 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.038131    5068 server.go:1014] "Starting client certificate rotation"
Nov 04 01:02:40.055494 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.038264    5068 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 04 01:02:40.055494 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.038310    5068 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-03 00:56:57 +0000 UTC" deadline="2028-05-14 00:05:08.660694742 +0000 UTC"
Nov 04 01:02:40.055494 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.038346    5068 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13367h2m28.62235099s"
Nov 04 01:02:40.055494 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.038597    5068 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 04 01:02:40.056187 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.039786    5068 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 04 01:02:40.056187 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.042195    5068 log.go:25] "Validated CRI v1 runtime API"
Nov 04 01:02:40.056187 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.043874    5068 log.go:25] "Validated CRI v1 image API"
Nov 04 01:02:40.056187 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.044597    5068 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 04 01:02:40.056187 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.046465    5068 fs.go:135] Filesystem UUIDs: map[2f245827-4bad-472e-b665-4f970b464ebf:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 917c0278-c1e3-466a-b9cc-4bd2d67be92e:/dev/nvme0n1p3]
Nov 04 01:02:40.056447 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.046486    5068 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/00c93c0ab8afbab1c8b30f3162eed2558e8d2cc243e654b7f8276a693dd3331a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/00c93c0ab8afbab1c8b30f3162eed2558e8d2cc243e654b7f8276a693dd3331a/userdata/shm major:0 minor:348 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/08e42593c7cf86de79c059dab595951e8d13b366390903d025ad4b53c8719f8c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/08e42593c7cf86de79c059dab595951e8d13b366390903d025ad4b53c8719f8c/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0acd953dd0d10f6aab6269a568a4bfe682d862b978047b926a7a8e3fd42e2bc6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0acd953dd0d10f6aab6269a568a4bfe682d862b978047b926a7a8e3fd42e2bc6/userdata/shm major:0 minor:340 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/221b52fd58c2e1a2d63f2e5d720cb19394a30e5038eadbfead24a3c9bbc10394/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/221b52fd58c2e1a2d63f2e5d720cb19394a30e5038eadbfead24a3c9bbc10394/userdata/shm major:0 minor:350 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2aa8127eda532fa7ee4c334ca641dc30fb42d41497fb07b2e4a0376a0ea3c416/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2aa8127eda532fa7ee4c334ca641dc30fb42d41497fb07b2e4a0376a0ea3c416/userdata/shm major:0 minor:76 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/36784fac827113d028c4492f22911e0c8328324c80e663aefccb8c8f4f19bddc/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/36784fac827113d028c4492f22911e0c8328324c80e663aefccb8c8f4f19bddc/userdata/shm major:0 minor:245 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3e160b420233830856efd74439197cb2faeb82d999dd903b68336089f6f85a7b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3e160b420233830856efd74439197cb2faeb82d999dd903b68336089f6f85a7b/userdata/shm major:0 minor:237 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4a430137277d3d191db07d8b5f20cb298c9ff6674f04c7635fd64f98c46fc91c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4a430137277d3d191db07d8b5f20cb298c9ff6674f04c7635fd64f98c46fc91c/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/64c1b58d0019a5ac8bd199acdabfb9bde69dc2b8d33819bb4d5ad0cf732bbfdc/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/64c1b58d0019a5ac8bd199acdabfb9bde69dc2b8d33819bb4d5ad0cf732bbfdc/userdata/shm major:0 minor:234 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/70d19664f15ffde6ce00439daf61c56b10552248f9291702fdf875ab2702c2d4/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/70d19664f15ffde6ce00439daf61c56b10552248f9291702fdf875ab2702c2d4/userdata/shm major:0 minor:242 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7da1e32f59d67ca747c01f1ce91c2973944962cd032beabd2ccd2c13c342366a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7da1e32f59d67ca747c01f1ce91c2973944962cd032beabd2ccd2c13c342366a/userdata/shm major:0 minor:79 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7e3cdeb00325e347f39ce1412a25035ef8094d1edef60d2ab1e69b2929f41095/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7e3cdeb00325e347f39ce1412a25035ef8094d1edef60d2ab1e69b2929f41095/userdata/shm major:0 minor:226 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/808c2afb0f40ee6e188632e613a8eaadc9341eb3b191e2ee4fafb51b356bac7a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/808c2afb0f40ee6e188632e613a8eaadc9341eb3b191e2ee4fafb51b356bac7a/userdata/shm major:0 minor:292 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9e10a941535f0a270713e2870bafb42f54750a19213e1f8a55fd29ff93f72c15/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9e10a941535f0a270713e2870bafb42f54750a19213e1f8a55fd29ff93f72c15/userdata/shm major:0 minor:224 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b1f9979e91f13674381686bc6d8787ce98d980e2ee7acce22758576bc04fe489/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b1f9979e91f13674381686bc6d8787ce98d980e2ee7acce22758576bc04fe489/userdata/shm major:0 minor:236 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d3753e582fe061c82729cc6f22b8faeefcc91fc5ce845e1308069b4347e092ba/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d3753e582fe061c82729cc6f22b8faeefcc91fc5ce845e1308069b4347e092ba/userdata/shm major:0 minor:346 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/da0ea2a4d569949cd653e044b028fe259960f7770a377c653ae58477e2547f30/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/da0ea2a4d569949cd653e044b028fe259960f7770a377c653ae58477e2547f30/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/dfb26f56a760af7b098af88b4251f984082532654cac9d3d7219fdf618cfd1ca/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/dfb26f56a760af7b098af88b4251f984082532654cac9d3d7219fdf618cfd1ca/userdata/shm major:0 minor:229 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f0dffe660314752a2e102bfc7eed66425cddd61b4ca0027f4132de7c475c8af2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f0dffe660314752a2e102bfc7eed66425cddd61b4ca0027f4132de7c475c8af2/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f616a7b67b469b0b7393a53e6376c7f3f28b86c54b76a8f0f35df11a947e268b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f616a7b67b469b0b7393a53e6376c7f3f28b86c54b76a8f0f35df11a947e268b/userdata/shm major:0 minor:84 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f6645f2c05b3c7556458c7f2bd08199e2f168775c846c5b5cf2aa3593a6916f0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f6645f2c05b3c7556458c7f2bd08199e2f168775c846c5b5cf2aa3593a6916f0/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f9d36abdc505c14e0014478293acdb283ec8f81dde22697d6ffe71873e2c62f2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f9d36abdc505c14e0014478293acdb283ec8f81dde22697d6ffe71873e2c62f2/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/fae04b0bf9d49a12c75dcfd62c0839079327cb83ec3d16a6c40e70f967be95de/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/fae04b0bf9d49a12c75dcfd62c0839079327cb83ec3d16a6c40e70f967be95de/userdata/shm major:0 minor:243 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/fbbf633f62763e69cdd345d3148ac2eb1ba1952752256288060f9610675a3f05/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/fbbf633f62763e69cdd345d3148ac2eb1ba1952752256288060f9610675a3f05/userdata/shm major:0 minor:287 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/fdee7cbd173f345475fe82fffb45ac2aa6ca0a47294faa98fd31d1e0363d4843/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/fdee7cbd173f345475fe82fffb45ac2aa6ca0a47294faa98fd31d1e0363d4843/userdata/shm major:0 minor:239 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ff43c868abd19cc8cc3756d6c7b1acaeafa2c2dc60a90562bb5df20889f60d6b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ff43c868abd19cc8cc3756d6c7b1acaeafa2c2dc60a90562bb5df20889f60d6b/userdata/shm major:0 minor:51 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0eedbf45-4e7d-47b7-806f-a6abf9aa311f/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/0eedbf45-4e7d-47b7-806f-a6abf9aa311f/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:213 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0eedbf45-4e7d-47b7-806f-a6abf9aa311f/volumes/kubernetes.io~projected/kube-api-access-ncqm5:{mountpoint:/var/lib/kubelet/pods/0eedbf45-4e7d-47b7-806f-a6abf9aa311f/volumes/kubernetes.io~projected/kube-api-access-ncqm5 major:0 minor:218 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0eedbf45-4e7d-47b7-806f-a6abf9aa311f/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/0eedbf45-4e7d-47b7-806f-a6abf9aa311f/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:206 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0eedbf45-4e7d-47b7-806f-a6abf9aa311f/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/0eedbf45-4e7d-47b7-806f-a6abf9aa311f/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/20f4989c-4fe8-49d2-9430-5030a4274b60/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/20f4989c-4fe8-49d2-9430-5030a4274b60/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:210 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/20f4989c-4fe8-49d2-9430-5030a4274b60/volumes/kubernetes.io~projected/kube-api-access-rct8p:{mountpoint:/var/lib/kubelet/pods/20f4989c-4fe8-49d2-9430-5030a4274b60/volumes/kubernetes.io~projected/kube-api-access-rct8p major:0 minor:216 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/20f4989c-4fe8-49d2-9430-5030a4274b60/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/20f4989c-4fe8-49d2-9430-5030a4274b60/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/20f4989c-4fe8-49d2-9430-5030a4274b60/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/20f4989c-4fe8-49d2-9430-5030a4274b60/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/21d1d23b-7b6e-4ab5-9593-cf7784fffea2/volumes/kubernetes.io~projected/kube-api-access-h7lgb:{mountpoint:/var/lib/kubelet/pods/21d1d23b-7b6e-4ab5-9593-cf7784fffea2/volumes/kubernetes.io~projected/kube-api-access-h7lgb major:0 minor:220 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/21d1d23b-7b6e-4ab5-9593-cf7784fffea2/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/21d1d23b-7b6e-4ab5-9593-cf7784fffea2/volumes/kubernetes.io~secret/serving-cert major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/23cb8b1a-61f6-4045-9f5a-3ccc1dc346a5/volumes/kubernetes.io~projected/kube-api-access-wnhzq:{mountpoint:/var/lib/kubelet/pods/23cb8b1a-61f6-4045-9f5a-3ccc1dc346a5/volumes/kubernetes.io~projected/kube-api-access-wnhzq major:0 minor:215 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/28a8304b-9226-422d-9044-b50678d52971/volumes/kubernetes.io~projected/kube-api-access-pt8q7:{mountpoint:/var/lib/kubelet/pods/28a8304b-9226-422d-9044-b50678d52971/volumes/kubernetes.io~projected/kube-api-access-pt8q7 major:0 minor:221 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2f68cb5a-3e22-4dd6-98d0-f28988ea86a2/volumes/kubernetes.io~projected/kube-api-access-6kp4q:{mountpoint:/var/lib/kubelet/pods/2f68cb5a-3e22-4dd6-98d0-f28988ea86a2/volumes/kubernetes.io~projected/kube-api-access-6kp4q major:0 minor:232 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3a53d2f9-8ce8-4ae5-9974-a8c865065ff3/volumes/kubernetes.io~projected/kube-api-access-mpjd9:{mountpoint:/var/lib/kubelet/pods/3a53d2f9-8ce8-4ae5-9974-a8c865065ff3/volumes/kubernetes.io~projected/kube-api-access-mpjd9 major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3d2d0e05-a773-4442-ab38-d460ad32f51e/volumes/kubernetes.io~projected/kube-api-access-8nfw8:{mountpoint:/var/lib/kubelet/pods/3d2d0e05-a773-4442-ab38-d460ad32f51e/volumes/kubernetes.io~projected/kube-api-access-8nfw8 major:0 minor:219 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3d2d0e05-a773-4442-ab38-d460ad32f51e/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/3d2d0e05-a773-4442-ab38-d460ad32f51e/volumes/kubernetes.io~secret/serving-cert major:0 minor:207 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3f74f502-9d11-4533-8bdf-cf880a0fbe0c/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/3f74f502-9d11-4533-8bdf-cf880a0fbe0c/volumes/kubernetes.io~secret/agent-certs major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4965a588-c16b-488b-82c7-b53bb390f7d0/volumes/kubernetes.io~projected/kube-api-access-xqnfn:{mountpoint:/var/lib/kubelet/pods/4965a588-c16b-488b-82c7-b53bb390f7d0/volumes/kubernetes.io~projected/kube-api-access-xqnfn major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4a79348d-8cf7-4429-8c5a-4084cf85fcc5/volumes/kubernetes.io~projected/kube-api-access-72dk9:{mountpoint:/var/lib/kubelet/pods/4a79348d-8cf7-4429-8c5a-4084cf85fcc5/volumes/kubernetes.io~projected/kube-api-access-72dk9 major:0 minor:212 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4d3480a2-034c-4959-8aee-b0759dbfc24f/volumes/kubernetes.io~projected/kube-api-access-lr5kt:{mountpoint:/var/lib/kubelet/pods/4d3480a2-034c-4959-8aee-b0759dbfc24f/volumes/kubernetes.io~projected/kube-api-access-lr5kt major:0 minor:233 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/588c643c-b16a-4d5c-ad85-ccb8d486dd43/volumes/kubernetes.io~projected/kube-api-access-4wsn9:{mountpoint:/var/lib/kubelet/pods/588c643c-b16a-4d5c-ad85-ccb8d486dd43/volumes/kubernetes.io~projected/kube-api-access-4wsn9 major:0 minor:289 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6be44550-8625-4296-aeb9-da833aaa4b2f/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/6be44550-8625-4296-aeb9-da833aaa4b2f/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:286 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd/volumes/kubernetes.io~empty-dir/tmp major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd/volumes/kubernetes.io~projected/kube-api-access-p559g:{mountpoint:/var/lib/kubelet/pods/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd/volumes/kubernetes.io~projected/kube-api-access-p559g major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7b044258-a241-4bd2-a6eb-9ed586a0f2cc/volumes/kubernetes.io~projected/kube-api-access-pksw4:{mountpoint:/var/lib/kubelet/pods/7b044258-a241-4bd2-a6eb-9ed586a0f2cc/volumes/kubernetes.io~projected/kube-api-access-pksw4 major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/80a681c1-e74e-4887-a56d-6be313103581/volumes/kubernetes.io~projected/kube-api-access-499cg:{mountpoint:/var/lib/kubelet/pods/80a681c1-e74e-4887-a56d-6be313103581/volumes/kubernetes.io~projected/kube-api-access-499cg major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/82ac4b70-1c01-478c-ae1b-4a60dad06677/volumes/kubernetes.io~projected/kube-api-access-fk9hd:{mountpoint:/var/lib/kubelet/pods/82ac4b70-1c01-478c-ae1b-4a60dad06677/volumes/kubernetes.io~projected/kube-api-access-fk9hd major:0 minor:230 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/82ac4b70-1c01-478c-ae1b-4a60dad06677/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/82ac4b70-1c01-478c-ae1b-4a60dad06677/volumes/kubernetes.io~secret/cert major:0 minor:228 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8f92b52c-22de-4ce7-b716-fdfa22d7b52f/volumes/kubernetes.io~projected/kube-api-access-p664g:{mountpoint:/var/lib/kubelet/pods/8f92b52c-22de-4ce7-b716-fdfa22d7b52f/volumes/kubernetes.io~projected/kube-api-access-p664g major:0 minor:343 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9d694048-4b09-480b-ad26-8084c7775b24/volumes/kubernetes.io~projected/kube-api-access-b6tv2:{mountpoint:/var/lib/kubelet/pods/9d694048-4b09-480b-ad26-8084c7775b24/volumes/kubernetes.io~projected/kube-api-access-b6tv2 major:0 minor:342 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a0c9abfd-2f47-4cb9-9d3c-479a6231be97/volumes/kubernetes.io~projected/kube-api-access-p6t28:{mountpoint:/var/lib/kubelet/pods/a0c9abfd-2f47-4cb9-9d3c-479a6231be97/volumes/kubernetes.io~projected/kube-api-access-p6t28 major:0 minor:222 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a409c58b-b4eb-428f-9177-a09b8a229ec7/volumes/kubernetes.io~projected/kube-api-access-8r85c:{mountpoint:/var/lib/kubelet/pods/a409c58b-b4eb-428f-9177-a09b8a229ec7/volumes/kubernetes.io~projected/kube-api-access-8r85c major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/abaec7c0-0005-4bdd-9d8f-0ef0998629a6/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/abaec7c0-0005-4bdd-9d8f-0ef0998629a6/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/abaec7c0-0005-4bdd-9d8f-0ef0998629a6/volumes/kubernetes.io~projected/kube-api-access-8vgx6:{mountpoint:/var/lib/kubelet/pods/abaec7c0-0005-4bdd-9d8f-0ef0998629a6/volumes/kubernetes.io~projected/kube-api-access-8vgx6 major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/abaec7c0-0005-4bdd-9d8f-0ef0998629a6/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/abaec7c0-0005-4bdd-9d8f-0ef0998629a6/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774/volumes/kubernetes.io~projected/kube-api-access-mh966:{mountpoint:/var/lib/kubelet/pods/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774/volumes/kubernetes.io~projected/kube-api-access-mh966 major:0 minor:339 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ae02b148-c5ba-4db0-b817-9be4a488c708/volumes/kubernetes.io~projected/kube-api-access-b9hjn:{mountpoint:/var/lib/kubelet/pods/ae02b148-c5ba-4db0-b817-9be4a488c708/volumes/kubernetes.io~projected/kube-api-access-b9hjn major:0 minor:214 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ae02b148-c5ba-4db0-b817-9be4a488c708/volumes/kubernetes.io~secret/default-certificate:{mountpoint:/var/lib/kubelet/pods/ae02b148-c5ba-4db0-b817-9be4a488c708/volumes/kubernetes.io~secret/default-certificate major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ae02b148-c5ba-4db0-b817-9be4a488c708/volumes/kubernetes.io~secret/stats-auth:{mountpoint:/var/lib/kubelet/pods/ae02b148-c5ba-4db0-b817-9be4a488c708/volumes/kubernetes.io~secret/stats-auth major:0 minor:208 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b3bd337e-c495-4349-b56d-c16ebe3a9b11/volumes/kubernetes.io~projected/kube-api-access-9n9lx:{mountpoint:/var/lib/kubelet/pods/b3bd337e-c495-4349-b56d-c16ebe3a9b11/volumes/kubernetes.io~projected/kube-api-access-9n9lx major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b4dcd02c-9a77-43da-9923-3ea68740b318/volumes/kubernetes.io~projected/kube-api-access-45zdl:{mountpoint:/var/lib/kubelet/pods/b4dcd02c-9a77-43da-9923-3ea68740b318/volumes/kubernetes.io~projected/kube-api-access-45zdl major:0 minor:345 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b4dcd02c-9a77-43da-9923-3ea68740b318/volumes/kubernetes.io~secret/signing-key:{mountpoint:/var/lib/kubelet/pods/b4dcd02c-9a77-43da-9923-3ea68740b318/volumes/kubernetes.io~secret/signing-key major:0 minor:344 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d4efaef2-c145-4839-b32f-ecf7a1c76a99/volumes/kubernetes.io~projected/kube-api-access-dr2p4:{mountpoint:/var/lib/kubelet/pods/d4efaef2-c145-4839-b32f-ecf7a1c76a99/volumes/kubernetes.io~projected/kube-api-access-dr2p4 major:0 minor:217 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d6a4c158-e115-4d2f-af0d-1d9249dc7384/volumes/kubernetes.io~projected/kube-api-access-wj8p2:{mountpoint:/var/lib/kubelet/pods/d6a4c158-e115-4d2f-af0d-1d9249dc7384/volumes/kubernetes.io~projected/kube-api-access-wj8p2 major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/de97ada4-9cae-4775-a8f8-014004e17d25/volumes/kubernetes.io~projected/kube-api-access-zvzfb:{mountpoint:/var/lib/kubelet/pods/de97ada4-9cae-4775-a8f8-014004e17d25/volumes/kubernetes.io~projected/kube-api-access-zvzfb major:0 minor:209 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/de97ada4-9cae-4775-a8f8-014004e17d25/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/de97ada4-9cae-4775-a8f8-014004e17d25/volumes/kubernetes.io~secret/serving-cert major:0 minor:205 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e2b1aee4-49c5-47a0-a79f-a3fe032df2d7/volumes/kubernetes.io~projected/kube-api-access-6b7vh:{mountpoint:/var/lib/kubelet/pods/e2b1aee4-49c5-47a0-a79f-a3fe032df2d7/volumes/kubernetes.io~projected/kube-api-access-6b7vh major:0 minor:334 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e41517b6-7ce4-43a6-89a9-76931457ca50/volumes/kubernetes.io~projected/kube-api-access-5f285:{mountpoint:/var/lib/kubelet/pods/e41517b6-7ce4-43a6-89a9-76931457ca50/volumes/kubernetes.io~projected/kube-api-access-5f285 major:0 minor:211 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fc0b0e48-8adf-4d0d-80dd-ca76846f824a/volumes/kubernetes.io~projected/kube-api-access-r6tkr:{mountpoint:/var/lib/kubelet/pods/fc0b0e48-8adf-4d0d-80dd-ca76846f824a/volumes/kubernetes.io~projected/kube-api-access-r6tkr major:0 minor:223 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fc0b0e48-8adf-4d0d-80dd-ca76846f824a/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/fc0b0e48-8adf-4d0d-80dd-ca76846f824a/volumes/kubernetes.io~secret/serving-cert major:0 minor:45 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/68531872243b02c734bbe7fd2c82febfbc9a3529356930ee580a7f22cffabe6a/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/0e9ec6b70c8f811f401199f1b9aed8b055bb5bc7b87a10be5afbeeeebd936c68/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/c2950ef120a7117436ba834afc72dc73db8d0285b6ecda6799a868c3f80fea6e/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/c626bd6d6384eeeddd803f6eb175f0785b8b86cdfed237fc32de5fda25edcd2d/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/1da309ea0b090b7bad62331a37efecdf1df50882c25fbe19a174b08734ee545b/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/ae4d57944cf53b1a5c6e43822dcb7e207e23bad0d96dcace7be44923f7779a70/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/df505b36b549aab336a67e17f840c384950fc94f38252b1fa79932638b6e72fe/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/950b2c1920afcaf631fc1903645d80f67d30c0c4e7e09ed1caf1772029787171/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/b7ac799607b534c3487f2043684021e7d566f694bbb793c67e94f2211035627f/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-135:{mountpoint:/var/lib/containers/storage/overlay/e74b26163f35c282d896b87022e24d736ba72901113a93ad5d5905bba397f853/merged major:0 minor:135 fsType:overlay blockSize:0} overlay_0-144:{mountpoint:/var/lib/containers/storage/overlay/5059551d6dc7ca31406b38cf267496151c7fc1b754e3bccaf1aeb124f9ea301c/merged major:0 minor:144 fsType:overlay blockSize:0} overlay_0-147:{mountpoint:/var/lib/containers/storage/overlay/452ad2933b05cada08bdfe413c2770582ff614deefbc120ff36b9583114ede9b/merged major:0 minor:147 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/59b24ac8abe565207e2bb8c9b66bedb60b253f610266f0d99193eacacdc3af97/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/24c7804824ef9c5adcdbe01f1ba16627b1d69bec5c65bf41810428b8667c117a/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/229cb281aa4e6261b7557babad3409cf18771b0533e7009177f8bf37ceb00c92/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/67295479cff50a65b48ca29825df8ce24920c37b01bd1363bf740fc199cfebdd/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/421aada2f5f939ec1fcc2ff13a595ecbb91fb96e7e420d0b4dabc11fc73006d6/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/8a2f97ff9bf4ff894402f2dd44881c7d1abed1f6d84ccfa496205109f00dfd95/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/b3d72df0b75da3686f4751ecc40220b44fbea7437b0fe744447ba184822f4f44/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/a6d08b3f3535d3ec08de3ff55d15e0200531a2aaba77fa9a628b297d91edc282/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/c966752aed55007d8d8e42f481a923b41c237b83dbf9217ea8802e47a5f3d4f5/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-248:{mountpoint:/var/lib/containers/storage/overlay/264202ae0a460aeacb6aa6b14eeec8b8a944a92a78330c35ed4c28c0c61c1f96/merged major:0 minor:248 fsType:overlay blockSize:0} overlay_0-250:{mountpoint:/var/lib/containers/storage/overlay/be8118f17b1a4c62736bc3930966ca9e4fd0a9742961b423e9920fe665b9c31a/merged major:0 minor:250 fsType:overlay blockSize:0} overlay_0-252:{mountpoint:/var/lib/containers/storage/overlay/94018d6d2627e321d7d6b54e3e0df0af1641cb38a1de80907c7debb9b4e4e788/merged major:0 minor:252 fsType:overlay blockSize:0} overlay_0-254:{mountpoint:/var/lib/containers/storage/overlay/b6e284043e54c0efeb716db4c43aca12cedcdf09b2fabafe53ba3bc52c61ba1c/merged major:0 minor:254 fsType:overlay blockSize:0} overlay_0-256:{mountpoint:/var/lib/containers/storage/overlay/0a9652948a6aef374c9cf79d3b079d8fbd021b7f7dc7185d7d1214f2f1bd7787/merged major:0 minor:256 fsType:overlay blockSize:0} overlay_0-258:{mountpoint:/var/lib/containers/storage/overlay/e067e952a154ae55511a1bc8292f43c6a79bffe648e81b20548be4c0697f5c56/merged major:0 minor:258 fsType:overlay blockSize:0} overlay_0-260:{mountpoint:/var/lib/containers/storage/overlay/8671bdc14142f08f51d940efff32fdb77b436c281a06189a039e8bf70478407f/merged major:0 minor:260 fsType:overlay blockSize:0} overlay_0-262:{mountpoint:/var/lib/containers/storage/overlay/1f981fe04118c67961d5d464fcfcae9a633db8ce01b94ba37fc22094515a9b87/merged major:0 minor:262 fsType:overlay blockSize:0} overlay_0-264:{mountpoint:/var/lib/containers/storage/overlay/e12ef2f0e13873c67b2ecf89275d7ac164fa88df818ee49f17ffbc219b6a0b25/merged major:0 minor:264 fsType:overlay blockSize:0} overlay_0-266:{mountpoint:/var/lib/containers/storage/overlay/1578b6c5e700f4ba0fae76fb655ae442d0d76f3b24d41a8b84573de64b8787b5/merged major:0 minor:266 fsType:overlay blockSize:0} overlay_0-267:{mountpoint:/var/lib/containers/storage/overlay/d0b6f141201598301c32b90d215a0dd4faa0b6fb203598a09d5eb7e6d44139e7/merged major:0 minor:267 fsType:overlay blockSize:0} overlay_0-269:{mountpoint:/var/lib/containers/storage/overlay/34feb4c48fc00b95f7dfe934dcc10bacc46992d30211023b8d430e47a85441f3/merged major:0 minor:269 fsType:overlay blockSize:0} overlay_0-284:{mountpoint:/var/lib/containers/storage/overlay/f3dbdc802161be53f921f49e5b5d5f736125dcab1721f4f1a8009cbcff0bed76/merged major:0 minor:284 fsType:overlay blockSize:0} overlay_0-290:{mountpoint:/var/lib/containers/storage/overlay/a8d61b7fe02c7dbf1871532028589c59a7077d07ad622982bb8b194b7efb30e9/merged major:0 minor:290 fsType:overlay blockSize:0} overlay_0-294:{mountpoint:/var/lib/containers/storage/overlay/44cd35305b3507cfe8d0d574cdda1570bfc0f5f03c0d67c8269edbb2e56e3a43/merged major:0 minor:294 fsType:overlay blockSize:0} overlay_0-296:{mountpoint:/var/lib/containers/storage/overlay/23da693c4cc91d0d530071bc137af250f1e725ff89cc718fb4bdaf99dc056d40/merged major:0 minor:296 fsType:overlay blockSize:0} overlay_0-302:{mountpoint:/var/lib/containers/storage/overlay/0e8607e0f170686927ec57c6c0c811692423124de6243f6a1b9388627daa7fb5/merged major:0 minor:302 fsType:overlay blockSize:0} overlay_0-304:{mountpoint:/var/lib/containers/storage/overlay/744175d2d89642e3485572457b4c9413e7406b9bc989dbde8a62f14889e5b4c4/merged major:0 minor:304 fsType:overlay blockSize:0} overlay_0-310:{mountpoint:/var/lib/containers/storage/overlay/9076fae9fd49037e4bbaa194680cf682db782390231e5ecc2ce036c6667b8aae/merged major:0 minor:310 fsType:overlay blockSize:0} overlay_0-312:{mountpoint:/var/lib/containers/storage/overlay/28108f574bdd2e6c1bbd97b20ec3d654eb9a6b240e36e178c493c7c6276baae3/merged major:0 minor:312 fsType:overlay blockSize:0} overlay_0-314:{mountpoint:/var/lib/containers/storage/overlay/b97fc83a0662559ab872d43a897a2741089cdba7bb54c608a09f9207195d1ccf/merged major:0 minor:314 fsType:overlay blockSize:0} overlay_0-316:{mountpoint:/var/lib/containers/storage/overlay/5e0e093ed2390dff0e9e9bd159a5d437e693eadfe14713e85634a1f0f925e62a/merged major:0 minor:316 fsType:overlay blockSize:0} overlay_0-352:{mountpoint:/var/lib/containers/storage/overlay/e39ebb9bfe43805b69fe7a16fa483398341d66c730229911c82b96548343f302/merged major:0 minor:352 fsType:overlay blockSize:0} overlay_0-354:{mountpoint:/var/lib/containers/storage/overlay/4efa90bd3a0c4d21ffcf4e8d2f600238e2ba9e28a195c2d1acdd164f2f60ce33/merged major:0 minor:354 fsType:overlay blockSize:0} overlay_0-356:{mountpoint:/var/lib/containers/storage/overlay/05171e51cfabf5e0d1dfd4a2da1636c1dc1241725675f50669c9a037bc55511b/merged major:0 minor:356 fsType:overlay blockSize:0} overlay_0-362:{mountpoint:/var/lib/containers/storage/overlay/190a205312e7eaaae133947b4e9c7d3f642eeac59d15d979704c8917dedc6c53/merged major:0 minor:362 fsType:overlay blockSize:0} overlay_0-364:{mountpoint:/var/lib/containers/storage/overlay/5aff141d34a6f9d15a9cc4f5d04ed92290b7ebcea9f9c82995b502beed362472/merged major:0 minor:364 fsType:overlay blockSize:0} overlay_0-366:{mountpoint:/var/lib/containers/storage/overlay/aaac2e92ce84ad9e1a69c80198f165aed6633c57bcd572f7b99bac76e0424db9/merged major:0 minor:366 fsType:overlay blockSize:0} overlay_0-368:{mountpoint:/var/lib/containers/storage/overlay/e997e8b52ec615c278372c893c2e98f58cca43e7e1b25fa495f117e849a0414d/merged major:0 minor:368 fsType:overlay blockSize:0} overlay_0-374:{mountpoint:/var/lib/containers/storage/overlay/9551a716626fa04a1c57816c3a3836e22d938e36672e7d0b328b9d00cdec69b1/merged major:0 minor:374 fsType:overlay blockSize:0} overlay_0-376:{mountpoint:/var/lib/containers/storage/overlay/866b5c0a113e3f263fcfc486d65e03bb6059fa2c8b9f126da57500f877d17698/merged major:0 minor:376 fsType:overlay blockSize:0} overlay_0-49:{mountpoint:/var/lib/containers/storage/overlay/9eeab0d905bcfafdadc42797b8d20244191638e04293a2ed8ee00fcd40e719ae/merged major:0 minor:49 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/7e3aa313e659229a25c052997f0b6d120287e5c8a09afe584fb2c3eb681d99bc/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/9bb43464d35a781534e99d4490fa42356af91cc86dbf476ad0b129b6df4e499c/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/47875f4d4199c440c10875f2d4e720033f7e1094e3315a7ab5570741307e388e/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-78:{mountpoint:/var/lib/containers/storage/overlay/7211f612b415df4abd807f9df3241c7317a8b17fad000a413feab155024bda2d/merged major:0 minor:78 fsType:overlay blockSize:0} overlay_0-82:{mountpoint:/var/lib/containers/storage/overlay/d58c7ffb5730dd61cc1950d52723e774a6fefa5e5750a0a126892f866297be0c/merged major:0 minor:82 fsType:overlay blockSize:0} overlay_0-86:{mountpoint:/var/lib/containers/storage/overlay/fe500628c4c7d4d8b71b151b45a3272fe315a43e14f8a7d7a930f4254bb016bd/merged major:0 minor:86 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/b0e23da13b6a58fddc2bd901c789931083fba0a75bf161468fc049ca5d9bfc65/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/055d6ba3969ff4f375c3a37084b67616bca345112ecf03cb8ca561f847205d9d/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/0b572852c963d00c1f41df96957e914cbbbca85016cbecddf4659ba68b8aad12/merged major:0 minor:96 fsType:overlay blockSize:0}]
Nov 04 01:02:40.056863 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.054313    5068 manager.go:217] Machine: {Timestamp:2026-11-04 01:02:40.053872495 +0000 UTC m=+0.100433295 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099076 MemoryCapacity:33164341248 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e12c6543deb065ef9c9649fccaefe SystemUUID:ec2e12c6-543d-eb06-5ef9-c9649fccaefe BootID:4583952d-5bf2-4dfd-be58-c8de4cec02ad Filesystems:[{Device:/var/lib/kubelet/pods/3d2d0e05-a773-4442-ab38-d460ad32f51e/volumes/kubernetes.io~projected/kube-api-access-8nfw8 DeviceMajor:0 DeviceMinor:219 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/fc0b0e48-8adf-4d0d-80dd-ca76846f824a/volumes/kubernetes.io~projected/kube-api-access-r6tkr DeviceMajor:0 DeviceMinor:223 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-248 DeviceMajor:0 DeviceMinor:248 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-254 DeviceMajor:0 DeviceMinor:254 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-256 DeviceMajor:0 DeviceMinor:256 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3e160b420233830856efd74439197cb2faeb82d999dd903b68336089f6f85a7b/userdata/shm DeviceMajor:0 DeviceMinor:237 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/588c643c-b16a-4d5c-ad85-ccb8d486dd43/volumes/kubernetes.io~projected/kube-api-access-4wsn9 DeviceMajor:0 DeviceMinor:289 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774/volumes/kubernetes.io~projected/kube-api-access-mh966 DeviceMajor:0 DeviceMinor:339 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/b4dcd02c-9a77-43da-9923-3ea68740b318/volumes/kubernetes.io~projected/kube-api-access-45zdl DeviceMajor:0 DeviceMinor:345 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/00c93c0ab8afbab1c8b30f3162eed2558e8d2cc243e654b7f8276a693dd3331a/userdata/shm DeviceMajor:0 DeviceMinor:348 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ae02b148-c5ba-4db0-b817-9be4a488c708/volumes/kubernetes.io~projected/kube-api-access-b9hjn DeviceMajor:0 DeviceMinor:214 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:61 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/fae04b0bf9d49a12c75dcfd62c0839079327cb83ec3d16a6c40e70f967be95de/userdata/shm DeviceMajor:0 DeviceMinor:243 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-250 DeviceMajor:0 DeviceMinor:250 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-49 DeviceMajor:0 DeviceMinor:49 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ae02b148-c5ba-4db0-b817-9be4a488c708/volumes/kubernetes.io~secret/default-certificate DeviceMajor:0 DeviceMinor:203 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/0eedbf45-4e7d-47b7-806f-a6abf9aa311f/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:206 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-290 DeviceMajor:0 DeviceMinor:290 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0acd953dd0d10f6aab6269a568a4bfe682d862b978047b926a7a8e3fd42e2bc6/userdata/shm DeviceMajor:0 DeviceMinor:340 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/3f74f502-9d11-4533-8bdf-cf880a0fbe0c/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:55 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f6645f2c05b3c7556458c7f2bd08199e2f168775c846c5b5cf2aa3593a6916f0/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-354 DeviceMajor:0 DeviceMinor:354 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fc0b0e48-8adf-4d0d-80dd-ca76846f824a/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:45 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-266 DeviceMajor:0 DeviceMinor:266 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-316 DeviceMajor:0 DeviceMinor:316 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4a430137277d3d191db07d8b5f20cb298c9ff6674f04c7635fd64f98c46fc91c/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/da0ea2a4d569949cd653e044b028fe259960f7770a377c653ae58477e2547f30/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/0eedbf45-4e7d-47b7-806f-a6abf9aa311f/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:213 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/21d1d23b-7b6e-4ab5-9593-cf7784fffea2/volumes/kubernetes.io~projected/kube-api-access-h7lgb DeviceMajor:0 DeviceMinor:220 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/3d2d0e05-a773-4442-ab38-d460ad32f51e/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:207 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/d4efaef2-c145-4839-b32f-ecf7a1c76a99/volumes/kubernetes.io~projected/kube-api-access-dr2p4 DeviceMajor:0 DeviceMinor:217 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/82ac4b70-1c01-478c-ae1b-4a60dad06677/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:228 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-296 DeviceMajor:0 DeviceMinor:296 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-368 DeviceMajor:0 DeviceMinor:368 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd/volumes/kubernetes.io~projected/kube-api-access-p559g DeviceMajor:0 DeviceMinor:64 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-144 DeviceMajor:0 DeviceMinor:144 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/21d1d23b-7b6e-4ab5-9593-cf7784fffea2/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:204 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-304 DeviceMajor:0 DeviceMinor:304 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2aa8127eda532fa7ee4c334ca641dc30fb42d41497fb07b2e4a0376a0ea3c416/userdata/shm DeviceMajor:0 DeviceMinor:76 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/82ac4b70-1c01-478c-ae1b-4a60dad06677/volumes/kubernetes.io~projected/kube-api-access-fk9hd DeviceMajor:0 DeviceMinor:230 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/808c2afb0f40ee6e188632e613a8eaadc9341eb3b191e2ee4fafb51b356bac7a/userdata/shm DeviceMajor:0 DeviceMinor:292 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/de97ada4-9cae-4775-a8f8-014004e17d25/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:205 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-267 DeviceMajor:0 DeviceMinor:267 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-310 DeviceMajor:0 DeviceMinor:310 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/20f4989c-4fe8-49d2-9430-5030a4274b60/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:202 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/0eedbf45-4e7d-47b7-806f-a6abf9aa311f/volumes/kubernetes.io~projected/kube-api-access-ncqm5 DeviceMajor:0 DeviceMinor:218 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/b4dcd02c-9a77-43da-9923-3ea68740b318/volumes/kubernetes.io~secret/signing-key DeviceMajor:0 DeviceMinor:344 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/abaec7c0-0005-4bdd-9d8f-0ef0998629a6/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:56 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-82 DeviceMajor:0 DeviceMinor:82 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/20f4989c-4fe8-49d2-9430-5030a4274b60/volumes/kubernetes.io~projected/kube-api-access-rct8p DeviceMajor:0 DeviceMinor:216 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/36784fac827113d028c4492f22911e0c8328324c80e663aefccb8c8f4f19bddc/userdata/shm DeviceMajor:0 DeviceMinor:245 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-284 DeviceMajor:0 DeviceMinor:284 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-302 DeviceMajor:0 DeviceMinor:302 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-352 DeviceMajor:0 DeviceMinor:352 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b3bd337e-c495-4349-b56d-c16ebe3a9b11/volumes/kubernetes.io~projected/kube-api-access-9n9lx DeviceMajor:0 DeviceMinor:65 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9e10a941535f0a270713e2870bafb42f54750a19213e1f8a55fd29ff93f72c15/userdata/shm DeviceMajor:0 DeviceMinor:224 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/20f4989c-4fe8-49d2-9430-5030a4274b60/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:210 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/2f68cb5a-3e22-4dd6-98d0-f28988ea86a2/volumes/kubernetes.io~projected/kube-api-access-6kp4q DeviceMajor:0 DeviceMinor:232 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-258 DeviceMajor:0 DeviceMinor:258 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d3753e582fe061c82729cc6f22b8faeefcc91fc5ce845e1308069b4347e092ba/userdata/shm DeviceMajor:0 DeviceMinor:346 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/9d694048-4b09-480b-ad26-8084c7775b24/volumes/kubernetes.io~projected/kube-api-access-b6tv2 DeviceMajor:0 DeviceMinor:342 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/a409c58b-b4eb-428f-9177-a09b8a229ec7/volumes/kubernetes.io~projected/kube-api-access-8r85c DeviceMajor:0 DeviceMinor:58 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:62 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/28a8304b-9226-422d-9044-b50678d52971/volumes/kubernetes.io~projected/kube-api-access-pt8q7 DeviceMajor:0 DeviceMinor:221 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/var/lib/kubelet/pods/4965a588-c16b-488b-82c7-b53bb390f7d0/volumes/kubernetes.io~projected/kube-api-access-xqnfn DeviceMajor:0 DeviceMinor:66 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b1f9979e91f13674381686bc6d8787ce98d980e2ee7acce22758576bc04fe489/userdata/shm DeviceMajor:0 DeviceMinor:236 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-364 DeviceMajor:0 DeviceMinor:364 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f0dffe660314752a2e102bfc7eed66425cddd61b4ca0027f4132de7c475c8af2/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/0eedbf45-4e7d-47b7-806f-a6abf9aa311f/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:201 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/ae02b148-c5ba-4db0-b817-9be4a488c708/volumes/kubernetes.io~secret/stats-auth DeviceMajor:0 DeviceMinor:208 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/7b044258-a241-4bd2-a6eb-9ed586a0f2cc/volumes/kubernetes.io~projected/kube-api-access-pksw4 DeviceMajor:0 DeviceMinor:67 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f616a7b67b469b0b7393a53e6376c7f3f28b86c54b76a8f0f35df11a947e268b/userdata/shm DeviceMajor:0 DeviceMinor:84 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-86 DeviceMajor:0 DeviceMinor:86 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-147 DeviceMajor:0 DeviceMinor:147 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/23cb8b1a-61f6-4045-9f5a-3ccc1dc346a5/volumes/kubernetes.io~projected/kube-api-access-wnhzq DeviceMajor:0 DeviceMinor:215 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-356 DeviceMajor:0 DeviceMinor:356 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-366 DeviceMajor:0 DeviceMinor:366 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582172672 Type:vfs Inodes:1048576 HasInodes:true} {Device:overlay_0-78 DeviceMajor:0 DeviceMinor:78 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/20f4989c-4fe8-49d2-9430-5030a4274b60/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:197 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/de97ada4-9cae-4775-a8f8-014004e17d25/volumes/kubernetes.io~projected/kube-api-access-zvzfb DeviceMajor:0 DeviceMinor:209 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/e41517b6-7ce4-43a6-89a9-76931457ca50/volumes/kubernetes.io~projected/kube-api-access-5f285 DeviceMajor:0 DeviceMinor:211 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/dfb26f56a760af7b098af88b4251f984082532654cac9d3d7219fdf618cfd1ca/userdata/shm DeviceMajor:0 DeviceMinor:229 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/4d3480a2-034c-4959-8aee-b0759dbfc24f/volumes/kubernetes.io~projected/kube-api-access-lr5kt DeviceMajor:0 DeviceMinor:233 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/d6a4c158-e115-4d2f-af0d-1d9249dc7384/volumes/kubernetes.io~projected/kube-api-access-wj8p2 DeviceMajor:0 DeviceMinor:59 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-269 DeviceMajor:0 DeviceMinor:269 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/6be44550-8625-4296-aeb9-da833aaa4b2f/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:286 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-294 DeviceMajor:0 DeviceMinor:294 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-312 DeviceMajor:0 DeviceMinor:312 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-135 DeviceMajor:0 DeviceMinor:135 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/64c1b58d0019a5ac8bd199acdabfb9bde69dc2b8d33819bb4d5ad0cf732bbfdc/userdata/shm DeviceMajor:0 DeviceMinor:234 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-252 DeviceMajor:0 DeviceMinor:252 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e2b1aee4-49c5-47a0-a79f-a3fe032df2d7/volumes/kubernetes.io~projected/kube-api-access-6b7vh DeviceMajor:0 DeviceMinor:334 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/221b52fd58c2e1a2d63f2e5d720cb19394a30e5038eadbfead24a3c9bbc10394/userdata/shm DeviceMajor:0 DeviceMinor:350 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-374 DeviceMajor:0 DeviceMinor:374 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-376 DeviceMajor:0 DeviceMinor:376 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/abaec7c0-0005-4bdd-9d8f-0ef0998629a6/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-262 DeviceMajor:0 DeviceMinor:262 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-314 DeviceMajor:0 DeviceMinor:314 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3a53d2f9-8ce8-4ae5-9974-a8c865065ff3/volumes/kubernetes.io~projected/kube-api-access-mpjd9 DeviceMajor:0 DeviceMinor:57 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7da1e32f59d67ca747c01f1ce91c2973944962cd032beabd2ccd2c13c342366a/userdata/shm DeviceMajor:0 DeviceMinor:79 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/4a79348d-8cf7-4429-8c5a-4084cf85fcc5/volumes/kubernetes.io~projected/kube-api-access-72dk9 DeviceMajor:0 DeviceMinor:212 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/70d19664f15ffde6ce00439daf61c56b10552248f9291702fdf875ab2702c2d4/userdata/shm DeviceMajor:0 DeviceMinor:242 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-362 DeviceMajor:0 DeviceMinor:362 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/80a681c1-e74e-4887-a56d-6be313103581/volumes/kubernetes.io~projected/kube-api-access-499cg DeviceMajor:0 DeviceMinor:63 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/08e42593c7cf86de79c059dab595951e8d13b366390903d025ad4b53c8719f8c/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-264 DeviceMajor:0 DeviceMinor:264 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8f92b52c-22de-4ce7-b716-fdfa22d7b52f/volumes/kubernetes.io~projected/kube-api-access-p664g DeviceMajor:0 DeviceMinor:343 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/fdee7cbd173f345475fe82fffb45ac2aa6ca0a47294faa98fd31d1e0363d4843/userdata/shm DeviceMajor:0 DeviceMinor:239 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/fbbf633f62763e69cdd345d3148ac2eb1ba1952752256288060f9610675a3f05/userdata/shm DeviceMajor:0 DeviceMinor:287 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ff43c868abd19cc8cc3756d6c7b1acaeafa2c2dc60a90562bb5df20889f60d6b/userdata/shm DeviceMajor:0 DeviceMinor:51 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/abaec7c0-0005-4bdd-9d8f-0ef0998629a6/volumes/kubernetes.io~projected/kube-api-access-8vgx6 DeviceMajor:0 DeviceMinor:60 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f9d36abdc505c14e0014478293acdb283ec8f81dde22697d6ffe71873e2c62f2/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a0c9abfd-2f47-4cb9-9d3c-479a6231be97/volumes/kubernetes.io~projected/kube-api-access-p6t28 DeviceMajor:0 DeviceMinor:222 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7e3cdeb00325e347f39ce1412a25035ef8094d1edef60d2ab1e69b2929f41095/userdata/shm DeviceMajor:0 DeviceMinor:226 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-260 DeviceMajor:0 DeviceMinor:260 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:00c93c0ab8afbab MacAddress:a2:cd:e3:8c:af:db Speed:10000 Mtu:8901} {Name:0acd953dd0d10f6 MacAddress:a2:c5:2a:97:36:be Speed:10000 Mtu:8901} {Name:221b52fd58c2e1a MacAddress:92:b3:5c:12:3e:03 Speed:10000 Mtu:8901} {Name:36784fac827113d MacAddress:8a:e9:f1:9f:86:c2 Speed:10000 Mtu:8901} {Name:3e160b420233830 MacAddress:06:20:40:dd:1f:85 Speed:10000 Mtu:8901} {Name:64c1b58d0019a5a MacAddress:b6:9a:6b:13:e5:10 Speed:10000 Mtu:8901} {Name:70d19664f15ffde MacAddress:76:05:61:dd:45:27 Speed:10000 Mtu:8901} {Name:7e3cdeb00325e34 MacAddress:96:f5:f6:b5:54:7e Speed:10000 Mtu:8901} {Name:808c2afb0f40ee6 MacAddress:56:98:3f:c4:f9:d6 Speed:10000 Mtu:8901} {Name:9e10a941535f0a2 MacAddress:5e:51:5f:fa:c6:b9 Speed:10000 Mtu:8901} {Name:b1f9979e91f1367 MacAddress:f2:00:b6:16:d8:bb Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:ff:c6:b1:41:ef Speed:0 Mtu:9001} {Name:br-int MacAddress:62:51:13:c2:9f:aa Speed:0 Mtu:8901} {Name:d3753e582fe061c MacAddress:8a:3e:12:a9:3d:52 Speed:10000 Mtu:8901} {Name:dfb26f56a760af7 MacAddress:e2:17:f9:c5:41:fb Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:ff:c6:b1:41:ef Speed:0 Mtu:9001} {Name:fae04b0bf9d49a1 MacAddress:92:2d:47:23:8f:5f Speed:10000 Mtu:8901} {Name:fbbf633f62763e6 MacAddress:c6:98:aa:e2:df:59 Speed:10000 Mtu:8901} {Name:fdee7cbd173f345 MacAddress:ba:1b:5a:8b:f9:a4 Speed:10000 Mtu:8901} {Name:genev_sys_6081 MacAddress:8e:c7:64:fc:b4:a7 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:84:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:6e:29:90:5d:8c:22 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164341248 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 04 01:02:40.056863 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.054689    5068 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 04 01:02:40.056863 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.054730    5068 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 04 01:02:40.056863 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.055103    5068 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 04 01:02:40.056863 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.055128    5068 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-67.us-east-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 04 01:02:40.056863 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.055308    5068 topology_manager.go:143] "Creating topology manager with none policy"
Nov 04 01:02:40.056863 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.055320    5068 container_manager_linux.go:311] "Creating device plugin manager"
Nov 04 01:02:40.056863 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.055338    5068 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 04 01:02:40.056863 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.055366    5068 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 04 01:02:40.056863 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.055392    5068 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 04 01:02:40.056863 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.055626    5068 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 04 01:02:40.056863 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.055768    5068 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 04 01:02:40.056863 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.055813    5068 kubelet.go:493] "Attempting to sync node with API server"
Nov 04 01:02:40.056863 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.055831    5068 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 04 01:02:40.056863 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.055845    5068 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 04 01:02:40.056863 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.055858    5068 kubelet.go:405] "Adding apiserver pod source"
Nov 04 01:02:40.056863 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.055869    5068 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 04 01:02:40.057818 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.057057    5068 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 04 01:02:40.057818 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.057073    5068 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 04 01:02:40.057818 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.057637    5068 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 04 01:02:40.061687 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.061591    5068 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 04 01:02:40.061687 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.061627    5068 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 04 01:02:40.061803 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.061744    5068 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Nov 04 01:02:40.062003 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.061989    5068 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 04 01:02:40.062037 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.062021    5068 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 04 01:02:40.062132 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.062122    5068 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 04 01:02:40.062163 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.062137    5068 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 04 01:02:40.062163 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.062143    5068 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 04 01:02:40.062163 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.062149    5068 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 04 01:02:40.062163 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.062158    5068 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 04 01:02:40.062268 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.062167    5068 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 04 01:02:40.062268 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.062175    5068 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 04 01:02:40.062268 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.062180    5068 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 04 01:02:40.062268 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.062187    5068 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 04 01:02:40.062268 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.062193    5068 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 04 01:02:40.062268 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.062202    5068 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 04 01:02:40.062268 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.062210    5068 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 04 01:02:40.062268 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.062235    5068 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 04 01:02:40.062268 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.062243    5068 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 04 01:02:40.065654 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.065640    5068 server.go:1263] "Started kubelet"
Nov 04 01:02:40.065717 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.065674    5068 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 04 01:02:40.065822 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.065774    5068 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 04 01:02:40.065875 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.065844    5068 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 04 01:02:40.066198 ip-10-0-132-67 systemd[1]: Started Kubernetes Kubelet.
Nov 04 01:02:40.067308 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.066242    5068 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 04 01:02:40.067370 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.067313    5068 server.go:317] "Adding debug handlers to kubelet server"
Nov 04 01:02:40.070286 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.070267    5068 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 04 01:02:40.070410 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.070393    5068 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 04 01:02:40.070470 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:02:40.070443    5068 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-67.us-east-2.compute.internal\" not found"
Nov 04 01:02:40.070521 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.070493    5068 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 04 01:02:40.070521 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.070502    5068 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 04 01:02:40.076480 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.073302    5068 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 04 01:02:40.076480 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.075234    5068 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 01:02:40.077780 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.076980    5068 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 00:56:59 +0000 UTC" deadline="2028-06-04 11:16:37.487824863 +0000 UTC"
Nov 04 01:02:40.077780 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.077027    5068 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13882h13m57.410801675s"
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.078181    5068 factory.go:55] Registering systemd factory
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.078201    5068 factory.go:223] Registration of the systemd container factory successfully
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.078436    5068 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.078651    5068 factory.go:153] Registering CRI-O factory
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.078667    5068 factory.go:223] Registration of the crio container factory successfully
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.078736    5068 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.078754    5068 factory.go:103] Registering Raw factory
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.078771    5068 manager.go:1199] Started watching for new ooms in manager
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079044    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7b044258-a241-4bd2-a6eb-9ed586a0f2cc" volumeName="kubernetes.io/configmap/7b044258-a241-4bd2-a6eb-9ed586a0f2cc-serviceca" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079068    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b4dcd02c-9a77-43da-9923-3ea68740b318" volumeName="kubernetes.io/configmap/b4dcd02c-9a77-43da-9923-3ea68740b318-signing-cabundle" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079077    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d6a4c158-e115-4d2f-af0d-1d9249dc7384" volumeName="kubernetes.io/empty-dir/d6a4c158-e115-4d2f-af0d-1d9249dc7384-tmp-dir" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079085    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e2b1aee4-49c5-47a0-a79f-a3fe032df2d7" volumeName="kubernetes.io/projected/e2b1aee4-49c5-47a0-a79f-a3fe032df2d7-kube-api-access-6b7vh" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079094    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e41517b6-7ce4-43a6-89a9-76931457ca50" volumeName="kubernetes.io/configmap/e41517b6-7ce4-43a6-89a9-76931457ca50-samples-operator-config" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079101    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fc0b0e48-8adf-4d0d-80dd-ca76846f824a" volumeName="kubernetes.io/secret/fc0b0e48-8adf-4d0d-80dd-ca76846f824a-serving-cert" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079108    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="28a8304b-9226-422d-9044-b50678d52971" volumeName="kubernetes.io/configmap/28a8304b-9226-422d-9044-b50678d52971-telemetry-config" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079117    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="28a8304b-9226-422d-9044-b50678d52971" volumeName="kubernetes.io/projected/28a8304b-9226-422d-9044-b50678d52971-kube-api-access-pt8q7" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079126    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a409c58b-b4eb-428f-9177-a09b8a229ec7" volumeName="kubernetes.io/projected/a409c58b-b4eb-428f-9177-a09b8a229ec7-kube-api-access-8r85c" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079133    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b3bd337e-c495-4349-b56d-c16ebe3a9b11" volumeName="kubernetes.io/configmap/b3bd337e-c495-4349-b56d-c16ebe3a9b11-cni-binary-copy" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079142    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e2b1aee4-49c5-47a0-a79f-a3fe032df2d7" volumeName="kubernetes.io/configmap/e2b1aee4-49c5-47a0-a79f-a3fe032df2d7-service-ca-bundle" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079150    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="20f4989c-4fe8-49d2-9430-5030a4274b60" volumeName="kubernetes.io/secret/20f4989c-4fe8-49d2-9430-5030a4274b60-installation-pull-secrets" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079158    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4965a588-c16b-488b-82c7-b53bb390f7d0" volumeName="kubernetes.io/projected/4965a588-c16b-488b-82c7-b53bb390f7d0-kube-api-access-xqnfn" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079178    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e2b1aee4-49c5-47a0-a79f-a3fe032df2d7" volumeName="kubernetes.io/empty-dir/e2b1aee4-49c5-47a0-a79f-a3fe032df2d7-archives-path" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079188    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0eedbf45-4e7d-47b7-806f-a6abf9aa311f" volumeName="kubernetes.io/configmap/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-trusted-ca" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079195    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="20f4989c-4fe8-49d2-9430-5030a4274b60" volumeName="kubernetes.io/configmap/20f4989c-4fe8-49d2-9430-5030a4274b60-registry-certificates" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079203    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="20f4989c-4fe8-49d2-9430-5030a4274b60" volumeName="kubernetes.io/projected/20f4989c-4fe8-49d2-9430-5030a4274b60-kube-api-access-rct8p" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079210    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4d3480a2-034c-4959-8aee-b0759dbfc24f" volumeName="kubernetes.io/projected/4d3480a2-034c-4959-8aee-b0759dbfc24f-kube-api-access-lr5kt" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079217    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd" volumeName="kubernetes.io/projected/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-kube-api-access-p559g" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079224    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8f92b52c-22de-4ce7-b716-fdfa22d7b52f" volumeName="kubernetes.io/projected/8f92b52c-22de-4ce7-b716-fdfa22d7b52f-kube-api-access-p664g" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079231    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="abaec7c0-0005-4bdd-9d8f-0ef0998629a6" volumeName="kubernetes.io/configmap/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-env-overrides" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079238    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ae02b148-c5ba-4db0-b817-9be4a488c708" volumeName="kubernetes.io/secret/ae02b148-c5ba-4db0-b817-9be4a488c708-stats-auth" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079247    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="20f4989c-4fe8-49d2-9430-5030a4274b60" volumeName="kubernetes.io/empty-dir/20f4989c-4fe8-49d2-9430-5030a4274b60-ca-trust-extracted" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079254    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="20f4989c-4fe8-49d2-9430-5030a4274b60" volumeName="kubernetes.io/projected/20f4989c-4fe8-49d2-9430-5030a4274b60-bound-sa-token" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079271    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="20f4989c-4fe8-49d2-9430-5030a4274b60" volumeName="kubernetes.io/secret/20f4989c-4fe8-49d2-9430-5030a4274b60-image-registry-private-configuration" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079279    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2f68cb5a-3e22-4dd6-98d0-f28988ea86a2" volumeName="kubernetes.io/configmap/2f68cb5a-3e22-4dd6-98d0-f28988ea86a2-config-volume" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079327    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2f68cb5a-3e22-4dd6-98d0-f28988ea86a2" volumeName="kubernetes.io/empty-dir/2f68cb5a-3e22-4dd6-98d0-f28988ea86a2-tmp-dir" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079335    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3739befd-6689-4197-b793-6f5b4b796880" volumeName="kubernetes.io/configmap/3739befd-6689-4197-b793-6f5b4b796880-nginx-conf" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079347    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3d2d0e05-a773-4442-ab38-d460ad32f51e" volumeName="kubernetes.io/projected/3d2d0e05-a773-4442-ab38-d460ad32f51e-kube-api-access-8nfw8" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079355    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ae02b148-c5ba-4db0-b817-9be4a488c708" volumeName="kubernetes.io/secret/ae02b148-c5ba-4db0-b817-9be4a488c708-default-certificate" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079362    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="21d1d23b-7b6e-4ab5-9593-cf7784fffea2" volumeName="kubernetes.io/configmap/21d1d23b-7b6e-4ab5-9593-cf7784fffea2-config" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079383    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="de97ada4-9cae-4775-a8f8-014004e17d25" volumeName="kubernetes.io/empty-dir/de97ada4-9cae-4775-a8f8-014004e17d25-snapshots" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079390    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="de97ada4-9cae-4775-a8f8-014004e17d25" volumeName="kubernetes.io/projected/de97ada4-9cae-4775-a8f8-014004e17d25-kube-api-access-zvzfb" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079397    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="82ac4b70-1c01-478c-ae1b-4a60dad06677" volumeName="kubernetes.io/projected/82ac4b70-1c01-478c-ae1b-4a60dad06677-kube-api-access-fk9hd" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079405    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="abaec7c0-0005-4bdd-9d8f-0ef0998629a6" volumeName="kubernetes.io/configmap/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-ovnkube-config" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079412    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774" volumeName="kubernetes.io/projected/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-kube-api-access-mh966" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079419    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="23cb8b1a-61f6-4045-9f5a-3ccc1dc346a5" volumeName="kubernetes.io/projected/23cb8b1a-61f6-4045-9f5a-3ccc1dc346a5-kube-api-access-wnhzq" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079429    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3d2d0e05-a773-4442-ab38-d460ad32f51e" volumeName="kubernetes.io/configmap/3d2d0e05-a773-4442-ab38-d460ad32f51e-config" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079436    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0eedbf45-4e7d-47b7-806f-a6abf9aa311f" volumeName="kubernetes.io/secret/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-installation-pull-secrets" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079444    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b3bd337e-c495-4349-b56d-c16ebe3a9b11" volumeName="kubernetes.io/projected/b3bd337e-c495-4349-b56d-c16ebe3a9b11-kube-api-access-9n9lx" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079452    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d6a4c158-e115-4d2f-af0d-1d9249dc7384" volumeName="kubernetes.io/projected/d6a4c158-e115-4d2f-af0d-1d9249dc7384-kube-api-access-wj8p2" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079458    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e2b1aee4-49c5-47a0-a79f-a3fe032df2d7" volumeName="kubernetes.io/configmap/e2b1aee4-49c5-47a0-a79f-a3fe032df2d7-trusted-ca-bundle" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079466    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0eedbf45-4e7d-47b7-806f-a6abf9aa311f" volumeName="kubernetes.io/configmap/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-registry-certificates" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079473    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0eedbf45-4e7d-47b7-806f-a6abf9aa311f" volumeName="kubernetes.io/empty-dir/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-ca-trust-extracted" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079480    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0eedbf45-4e7d-47b7-806f-a6abf9aa311f" volumeName="kubernetes.io/secret/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-image-registry-private-configuration" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079487    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4a79348d-8cf7-4429-8c5a-4084cf85fcc5" volumeName="kubernetes.io/projected/4a79348d-8cf7-4429-8c5a-4084cf85fcc5-kube-api-access-72dk9" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079494    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd" volumeName="kubernetes.io/empty-dir/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-tmp" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079501    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="80a681c1-e74e-4887-a56d-6be313103581" volumeName="kubernetes.io/configmap/80a681c1-e74e-4887-a56d-6be313103581-multus-daemon-config" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079508    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774" volumeName="kubernetes.io/empty-dir/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-data-volume" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079515    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b4dcd02c-9a77-43da-9923-3ea68740b318" volumeName="kubernetes.io/projected/b4dcd02c-9a77-43da-9923-3ea68740b318-kube-api-access-45zdl" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079522    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="abaec7c0-0005-4bdd-9d8f-0ef0998629a6" volumeName="kubernetes.io/projected/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-kube-api-access-8vgx6" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079531    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="20f4989c-4fe8-49d2-9430-5030a4274b60" volumeName="kubernetes.io/configmap/20f4989c-4fe8-49d2-9430-5030a4274b60-trusted-ca" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079539    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="21d1d23b-7b6e-4ab5-9593-cf7784fffea2" volumeName="kubernetes.io/configmap/21d1d23b-7b6e-4ab5-9593-cf7784fffea2-trusted-ca" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079554    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="21d1d23b-7b6e-4ab5-9593-cf7784fffea2" volumeName="kubernetes.io/projected/21d1d23b-7b6e-4ab5-9593-cf7784fffea2-kube-api-access-h7lgb" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079562    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6be44550-8625-4296-aeb9-da833aaa4b2f" volumeName="kubernetes.io/secret/6be44550-8625-4296-aeb9-da833aaa4b2f-original-pull-secret" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079571    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="abaec7c0-0005-4bdd-9d8f-0ef0998629a6" volumeName="kubernetes.io/configmap/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-ovnkube-script-lib" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079583    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b3bd337e-c495-4349-b56d-c16ebe3a9b11" volumeName="kubernetes.io/configmap/b3bd337e-c495-4349-b56d-c16ebe3a9b11-cni-sysctl-allowlist" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079593    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="de97ada4-9cae-4775-a8f8-014004e17d25" volumeName="kubernetes.io/empty-dir/de97ada4-9cae-4775-a8f8-014004e17d25-tmp" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079639    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b3bd337e-c495-4349-b56d-c16ebe3a9b11" volumeName="kubernetes.io/configmap/b3bd337e-c495-4349-b56d-c16ebe3a9b11-whereabouts-flatfile-configmap" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079650    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b4dcd02c-9a77-43da-9923-3ea68740b318" volumeName="kubernetes.io/secret/b4dcd02c-9a77-43da-9923-3ea68740b318-signing-key" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079660    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d4efaef2-c145-4839-b32f-ecf7a1c76a99" volumeName="kubernetes.io/projected/d4efaef2-c145-4839-b32f-ecf7a1c76a99-kube-api-access-dr2p4" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079667    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3f74f502-9d11-4533-8bdf-cf880a0fbe0c" volumeName="kubernetes.io/configmap/3f74f502-9d11-4533-8bdf-cf880a0fbe0c-konnectivity-ca" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079674    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3f74f502-9d11-4533-8bdf-cf880a0fbe0c" volumeName="kubernetes.io/secret/3f74f502-9d11-4533-8bdf-cf880a0fbe0c-agent-certs" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079683    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7b044258-a241-4bd2-a6eb-9ed586a0f2cc" volumeName="kubernetes.io/projected/7b044258-a241-4bd2-a6eb-9ed586a0f2cc-kube-api-access-pksw4" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079690    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="de97ada4-9cae-4775-a8f8-014004e17d25" volumeName="kubernetes.io/configmap/de97ada4-9cae-4775-a8f8-014004e17d25-trusted-ca-bundle" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079697    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="de97ada4-9cae-4775-a8f8-014004e17d25" volumeName="kubernetes.io/secret/de97ada4-9cae-4775-a8f8-014004e17d25-serving-cert" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079704    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e41517b6-7ce4-43a6-89a9-76931457ca50" volumeName="kubernetes.io/projected/e41517b6-7ce4-43a6-89a9-76931457ca50-kube-api-access-5f285" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079711    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fc0b0e48-8adf-4d0d-80dd-ca76846f824a" volumeName="kubernetes.io/configmap/fc0b0e48-8adf-4d0d-80dd-ca76846f824a-config" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079719    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="80a681c1-e74e-4887-a56d-6be313103581" volumeName="kubernetes.io/projected/80a681c1-e74e-4887-a56d-6be313103581-kube-api-access-499cg" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079726    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="82ac4b70-1c01-478c-ae1b-4a60dad06677" volumeName="kubernetes.io/secret/82ac4b70-1c01-478c-ae1b-4a60dad06677-cert" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079735    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9d694048-4b09-480b-ad26-8084c7775b24" volumeName="kubernetes.io/projected/9d694048-4b09-480b-ad26-8084c7775b24-kube-api-access-b6tv2" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079743    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a0c9abfd-2f47-4cb9-9d3c-479a6231be97" volumeName="kubernetes.io/projected/a0c9abfd-2f47-4cb9-9d3c-479a6231be97-kube-api-access-p6t28" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079749    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="de97ada4-9cae-4775-a8f8-014004e17d25" volumeName="kubernetes.io/configmap/de97ada4-9cae-4775-a8f8-014004e17d25-service-ca-bundle" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079757    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0eedbf45-4e7d-47b7-806f-a6abf9aa311f" volumeName="kubernetes.io/projected/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-kube-api-access-ncqm5" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079765    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2f68cb5a-3e22-4dd6-98d0-f28988ea86a2" volumeName="kubernetes.io/projected/2f68cb5a-3e22-4dd6-98d0-f28988ea86a2-kube-api-access-6kp4q" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079772    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3a53d2f9-8ce8-4ae5-9974-a8c865065ff3" volumeName="kubernetes.io/projected/3a53d2f9-8ce8-4ae5-9974-a8c865065ff3-kube-api-access-mpjd9" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079779    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="588c643c-b16a-4d5c-ad85-ccb8d486dd43" volumeName="kubernetes.io/projected/588c643c-b16a-4d5c-ad85-ccb8d486dd43-kube-api-access-4wsn9" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079786    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="80a681c1-e74e-4887-a56d-6be313103581" volumeName="kubernetes.io/configmap/80a681c1-e74e-4887-a56d-6be313103581-cni-binary-copy" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079794    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="abaec7c0-0005-4bdd-9d8f-0ef0998629a6" volumeName="kubernetes.io/secret/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-ovn-node-metrics-cert" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079802    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fc0b0e48-8adf-4d0d-80dd-ca76846f824a" volumeName="kubernetes.io/projected/fc0b0e48-8adf-4d0d-80dd-ca76846f824a-kube-api-access-r6tkr" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079811    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="21d1d23b-7b6e-4ab5-9593-cf7784fffea2" volumeName="kubernetes.io/secret/21d1d23b-7b6e-4ab5-9593-cf7784fffea2-serving-cert" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079818    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3a53d2f9-8ce8-4ae5-9974-a8c865065ff3" volumeName="kubernetes.io/configmap/3a53d2f9-8ce8-4ae5-9974-a8c865065ff3-iptables-alerter-script" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079827    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3d2d0e05-a773-4442-ab38-d460ad32f51e" volumeName="kubernetes.io/secret/3d2d0e05-a773-4442-ab38-d460ad32f51e-serving-cert" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079834    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ae02b148-c5ba-4db0-b817-9be4a488c708" volumeName="kubernetes.io/projected/ae02b148-c5ba-4db0-b817-9be4a488c708-kube-api-access-b9hjn" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079841    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0eedbf45-4e7d-47b7-806f-a6abf9aa311f" volumeName="kubernetes.io/projected/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-bound-sa-token" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079848    5068 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd" volumeName="kubernetes.io/empty-dir/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-etc-tuned" seLinuxMountContext=""
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079857    5068 reconstruct.go:97] "Volume reconstruction finished"
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079862    5068 reconciler.go:29] "Reconciler: start to sync state"
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079904    5068 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 01:02:40.080817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.079087    5068 manager.go:319] Starting recovery of all containers
Nov 04 01:02:40.084932 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.084840    5068 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Nov 04 01:02:40.091667 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.091583    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tr8b2_21d1d23b-7b6e-4ab5-9593-cf7784fffea2/console-operator/1.log"
Nov 04 01:02:40.091667 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.091660    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tr8b2_21d1d23b-7b6e-4ab5-9593-cf7784fffea2/console-operator/1.log"
Nov 04 01:02:40.103305 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.103266    5068 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 04 01:02:40.104906 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.104883    5068 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 04 01:02:40.104906 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.104904    5068 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 04 01:02:40.105048 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.104919    5068 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 04 01:02:40.105048 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:02:40.104951    5068 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Nov 04 01:02:40.108918 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.108896    5068 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 01:02:40.124750 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.124726    5068 generic.go:356] "Generic (PLEG): container finished" podID="b3bd337e-c495-4349-b56d-c16ebe3a9b11" containerID="64da235aa48ce0025b528cca6f283dc408d2f0d043240515b18a66c35ebcf232" exitCode=0
Nov 04 01:02:40.124750 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.124747    5068 generic.go:356] "Generic (PLEG): container finished" podID="b3bd337e-c495-4349-b56d-c16ebe3a9b11" containerID="a86f747a6f2075432e47541b1846c8075ad7f4426ac0596a5d46923a1ca5383f" exitCode=0
Nov 04 01:02:40.124750 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.124753    5068 generic.go:356] "Generic (PLEG): container finished" podID="b3bd337e-c495-4349-b56d-c16ebe3a9b11" containerID="891c44a2cbd939e78ef6b0c631e1ef7e08373df28106b068356a2bafe8179db3" exitCode=0
Nov 04 01:02:40.124958 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.124759    5068 generic.go:356] "Generic (PLEG): container finished" podID="b3bd337e-c495-4349-b56d-c16ebe3a9b11" containerID="badfa2f0633d7138f9da22eca143668068d34ea6e7c245cb46e8cdbfcf4659d5" exitCode=0
Nov 04 01:02:40.124958 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.124765    5068 generic.go:356] "Generic (PLEG): container finished" podID="b3bd337e-c495-4349-b56d-c16ebe3a9b11" containerID="ac5e28a5c11ea0d56a2e397f89743f1ffa843571a9bb5dc89a57b112d378749f" exitCode=0
Nov 04 01:02:40.124958 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.124770    5068 generic.go:356] "Generic (PLEG): container finished" podID="b3bd337e-c495-4349-b56d-c16ebe3a9b11" containerID="98c46866a770d6bad4c965fb1cd3a4d69ea6a720a56febeacdde21d7d4b688da" exitCode=0
Nov 04 01:02:40.152838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.152776    5068 generic.go:356] "Generic (PLEG): container finished" podID="210e3b81ce84c7df763e2a095c51a6d5" containerID="acb80777421a025a1efc66bd483cb1690ba6307d1e0d34cab7479a93c1fa8495" exitCode=0
Nov 04 01:02:40.154415 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.154399    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tr8b2_21d1d23b-7b6e-4ab5-9593-cf7784fffea2/console-operator/1.log"
Nov 04 01:02:40.154544 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.154420    5068 generic.go:356] "Generic (PLEG): container finished" podID="21d1d23b-7b6e-4ab5-9593-cf7784fffea2" containerID="9d9855e9203df085f919c24b870c736a1b366e38fdfa9b8aea39bcaedc7f5ded" exitCode=255
Nov 04 01:02:40.174297 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.174279    5068 manager.go:324] Recovery completed
Nov 04 01:02:40.195874 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.195848    5068 cpu_manager.go:226] "Starting" policy="none"
Nov 04 01:02:40.195874 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.195865    5068 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 04 01:02:40.196016 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.195886    5068 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 04 01:02:40.196069 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.196033    5068 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Nov 04 01:02:40.196494 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.196048    5068 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Nov 04 01:02:40.196570 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.196496    5068 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Nov 04 01:02:40.196570 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.196506    5068 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Nov 04 01:02:40.196570 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.196514    5068 policy_none.go:50] "Start"
Nov 04 01:02:40.196570 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.196526    5068 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 04 01:02:40.196570 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.196541    5068 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 04 01:02:40.196816 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.196738    5068 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Nov 04 01:02:40.196816 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.196749    5068 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Nov 04 01:02:40.196816 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.196758    5068 policy_none.go:44] "Start"
Nov 04 01:02:40.201158 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.201133    5068 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 04 01:02:40.203693 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.203681    5068 manager.go:341] "Starting Device Plugin manager"
Nov 04 01:02:40.203778 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:02:40.203706    5068 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 04 01:02:40.203778 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.203718    5068 server.go:87] "Starting device plugin registration server"
Nov 04 01:02:40.204002 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.203988    5068 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 04 01:02:40.204072 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.204002    5068 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 04 01:02:40.204121 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.204103    5068 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 04 01:02:40.204267 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.204228    5068 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 04 01:02:40.204342 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.204274    5068 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 04 01:02:40.204342 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.204289    5068 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 04 01:02:40.204443 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.204340    5068 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-04T01:02:40.204262896Z","UUID":"4c228154-f061-431c-bd6b-e4682c0f758a","Handler":null,"Name":"","Endpoint":""}
Nov 04 01:02:40.205070 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.205026    5068 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-67.us-east-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-67.us-east-2.compute.internal"]
Nov 04 01:02:40.205801 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.205783    5068 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 04 01:02:40.205884 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.205802    5068 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-67.us-east-2.compute.internal"
Nov 04 01:02:40.205951 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.205807    5068 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 04 01:02:40.206003 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.205945    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-67.us-east-2.compute.internal" event={"ID":"9589190378fb255196269e8a3bdf442c","Type":"ContainerStarted","Data":"d12e32b345211851ccc3a1f44adc1a1683b0c0d46661274955118ae7a7a4b5cc"}
Nov 04 01:02:40.206003 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.205990    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-67.us-east-2.compute.internal" event={"ID":"9589190378fb255196269e8a3bdf442c","Type":"ContainerStarted","Data":"ff43c868abd19cc8cc3756d6c7b1acaeafa2c2dc60a90562bb5df20889f60d6b"}
Nov 04 01:02:40.206003 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.205998    5068 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-67.us-east-2.compute.internal"
Nov 04 01:02:40.206162 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.206011    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-67.us-east-2.compute.internal" event={"ID":"210e3b81ce84c7df763e2a095c51a6d5","Type":"ContainerStarted","Data":"2706c4453c2d26c4260b528d49ecb541481c44394c07099ea1d95ad7c6203c12"}
Nov 04 01:02:40.206162 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.206030    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-67.us-east-2.compute.internal" event={"ID":"210e3b81ce84c7df763e2a095c51a6d5","Type":"ContainerDied","Data":"acb80777421a025a1efc66bd483cb1690ba6307d1e0d34cab7479a93c1fa8495"}
Nov 04 01:02:40.206162 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.206046    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-67.us-east-2.compute.internal" event={"ID":"210e3b81ce84c7df763e2a095c51a6d5","Type":"ContainerStarted","Data":"f0dffe660314752a2e102bfc7eed66425cddd61b4ca0027f4132de7c475c8af2"}
Nov 04 01:02:40.206162 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:02:40.206144    5068 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 04 01:02:40.219738 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.219718    5068 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 01:02:40.219828 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:02:40.219763    5068 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-132-67.us-east-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-67.us-east-2.compute.internal"
Nov 04 01:02:40.220255 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.220239    5068 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 01:02:40.220322 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:02:40.220281    5068 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-132-67.us-east-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-67.us-east-2.compute.internal"
Nov 04 01:02:40.286598 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.286566    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/210e3b81ce84c7df763e2a095c51a6d5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-67.us-east-2.compute.internal\" (UID: \"210e3b81ce84c7df763e2a095c51a6d5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-67.us-east-2.compute.internal"
Nov 04 01:02:40.286762 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.286604    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/210e3b81ce84c7df763e2a095c51a6d5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-67.us-east-2.compute.internal\" (UID: \"210e3b81ce84c7df763e2a095c51a6d5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-67.us-east-2.compute.internal"
Nov 04 01:02:40.286762 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.286640    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9589190378fb255196269e8a3bdf442c-config\") pod \"kube-apiserver-proxy-ip-10-0-132-67.us-east-2.compute.internal\" (UID: \"9589190378fb255196269e8a3bdf442c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-67.us-east-2.compute.internal"
Nov 04 01:02:40.304681 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.304659    5068 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:02:40.306420 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.306404    5068 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-67.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:02:40.306488 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.306431    5068 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-67.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:02:40.306488 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.306442    5068 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-67.us-east-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:02:40.306488 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.306481    5068 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-132-67.us-east-2.compute.internal"
Nov 04 01:02:40.319182 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.319164    5068 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-132-67.us-east-2.compute.internal"
Nov 04 01:02:40.319238 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.319217    5068 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-132-67.us-east-2.compute.internal"
Nov 04 01:02:40.387001 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.386973    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/210e3b81ce84c7df763e2a095c51a6d5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-67.us-east-2.compute.internal\" (UID: \"210e3b81ce84c7df763e2a095c51a6d5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-67.us-east-2.compute.internal"
Nov 04 01:02:40.387121 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.387005    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/210e3b81ce84c7df763e2a095c51a6d5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-67.us-east-2.compute.internal\" (UID: \"210e3b81ce84c7df763e2a095c51a6d5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-67.us-east-2.compute.internal"
Nov 04 01:02:40.387121 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.387022    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9589190378fb255196269e8a3bdf442c-config\") pod \"kube-apiserver-proxy-ip-10-0-132-67.us-east-2.compute.internal\" (UID: \"9589190378fb255196269e8a3bdf442c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-67.us-east-2.compute.internal"
Nov 04 01:02:40.387121 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.387076    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/210e3b81ce84c7df763e2a095c51a6d5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-67.us-east-2.compute.internal\" (UID: \"210e3b81ce84c7df763e2a095c51a6d5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-67.us-east-2.compute.internal"
Nov 04 01:02:40.387245 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.387078    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9589190378fb255196269e8a3bdf442c-config\") pod \"kube-apiserver-proxy-ip-10-0-132-67.us-east-2.compute.internal\" (UID: \"9589190378fb255196269e8a3bdf442c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-67.us-east-2.compute.internal"
Nov 04 01:02:40.387245 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:40.387079    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/210e3b81ce84c7df763e2a095c51a6d5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-67.us-east-2.compute.internal\" (UID: \"210e3b81ce84c7df763e2a095c51a6d5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-67.us-east-2.compute.internal"
Nov 04 01:02:41.056351 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.056325    5068 apiserver.go:52] "Watching apiserver"
Nov 04 01:02:41.075872 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.075846    5068 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 04 01:02:41.076504 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.076481    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-rff54","openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jmxvd","openshift-dns/node-resolver-8gvq8","openshift-image-registry/image-registry-5c5894ddd-j4phf","openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-58wxt","openshift-multus/multus-additional-cni-plugins-qlr2s","openshift-kube-storage-version-migrator/migrator-66df5d496b-b9rdk","openshift-ingress/router-default-5569c8658c-v7txl","kube-system/kas-connection-checker-685895c7c9-6lcfv","openshift-multus/network-metrics-daemon-rx7gq","openshift-monitoring/cluster-monitoring-operator-884576c57-6vxbv","openshift-insights/insights-operator-7c4986d449-sbjxr","openshift-multus/multus-jfd2m","openshift-service-ca-operator/service-ca-operator-544d749c8c-8jlt4","openshift-insights/insights-runtime-extractor-sg4cw","openshift-network-diagnostics/network-check-source-6f78d59f99-zc968","openshift-insights/periodic-gathering-t6tpg-6r4ws","openshift-kube-storage-version-migrator/migrator-66df5d496b-fp864","openshift-image-registry/image-registry-6574d8498f-74vmj","openshift-network-console/networking-console-plugin-86c478bb88-27rmf","kube-system/global-pull-secret-syncer-gnrzw","openshift-console-operator/console-operator-7cd56b8dbc-tr8b2","openshift-image-registry/node-ca-fvjq7","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt2x6","openshift-ovn-kubernetes/ovnkube-node-sz7gg","openshift-ingress-canary/ingress-canary-4827b","kube-system/kube-apiserver-proxy-ip-10-0-132-67.us-east-2.compute.internal","openshift-network-operator/iptables-alerter-l7n65","openshift-dns/dns-default-klq54","openshift-service-ca/service-ca-5fffc97bdf-2zdns","openshift-cluster-node-tuning-operator/tuned-pwv64","openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-gdqjb","kube-system/kas-connection-checker-685895c7c9-hhfmk","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-67.us-east-2.compute.internal","kube-system/konnectivity-agent-sjcm7","openshift-network-diagnostics/network-check-target-wxhng"]
Nov 04 01:02:41.078225 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.078204    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5569c8658c-v7txl"
Nov 04 01:02:41.078564 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.078547    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6vxbv"
Nov 04 01:02:41.079756 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.079738    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jmxvd"
Nov 04 01:02:41.080056 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.080042    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6574d8498f-74vmj"
Nov 04 01:02:41.081227 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.081213    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-27rmf"
Nov 04 01:02:41.081703 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.081690    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx7gq"
Nov 04 01:02:41.084731 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.084708    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\""
Nov 04 01:02:41.084851 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.084736    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Nov 04 01:02:41.084940 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.084913    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Nov 04 01:02:41.085112 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.084755    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Nov 04 01:02:41.085241 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.085219    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\""
Nov 04 01:02:41.085353 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.085106    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Nov 04 01:02:41.085542 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.085520    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Nov 04 01:02:41.085592 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.085367    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Nov 04 01:02:41.085672 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.085596    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Nov 04 01:02:41.085722 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.085531    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Nov 04 01:02:41.085858 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.085837    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Nov 04 01:02:41.085934 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.085912    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Nov 04 01:02:41.085995 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.085349    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Nov 04 01:02:41.085995 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.085841    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Nov 04 01:02:41.086094 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.086045    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Nov 04 01:02:41.086141 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.086109    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Nov 04 01:02:41.087891 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.087873    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Nov 04 01:02:41.087989 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.087960    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Nov 04 01:02:41.087989 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.087985    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 04 01:02:41.088672 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.088650    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Nov 04 01:02:41.091294 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.091278    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5c5894ddd-j4phf"
Nov 04 01:02:41.091694 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.091679    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Nov 04 01:02:41.091764 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.091680    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Nov 04 01:02:41.092409 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.092394    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-klq54"
Nov 04 01:02:41.093004 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.092989    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-sg4cw"
Nov 04 01:02:41.093393 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.093372    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Nov 04 01:02:41.093474 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.093402    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Nov 04 01:02:41.093544 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.093526    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Nov 04 01:02:41.093676 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.093653    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Nov 04 01:02:41.093676 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.093656    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Nov 04 01:02:41.093676 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.093671    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 04 01:02:41.093808 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.093675    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Nov 04 01:02:41.093808 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.093740    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Nov 04 01:02:41.093808 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.093786    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Nov 04 01:02:41.095723 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.095704    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\""
Nov 04 01:02:41.095826 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.095754    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 04 01:02:41.095826 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.095758    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Nov 04 01:02:41.095826 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.095773    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 04 01:02:41.095826 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.095804    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 04 01:02:41.095826 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.095810    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 04 01:02:41.095826 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.095825    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Nov 04 01:02:41.095826 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.095831    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 04 01:02:41.096096 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.095834    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 04 01:02:41.096096 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.095838    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 04 01:02:41.096096 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.095842    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Nov 04 01:02:41.096096 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.095866    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 04 01:02:41.096096 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.095809    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Nov 04 01:02:41.096096 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.095878    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Nov 04 01:02:41.096096 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.095875    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Nov 04 01:02:41.096096 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.095884    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 04 01:02:41.096096 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.095906    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 04 01:02:41.096849 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.096834    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 04 01:02:41.096849 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.096846    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 04 01:02:41.096950 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.096909    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 04 01:02:41.098357 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.098342    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Nov 04 01:02:41.098357 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.098352    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-c48q9\""
Nov 04 01:02:41.098424 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.098352    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-78n7j\""
Nov 04 01:02:41.098424 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.098369    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5jp5j\""
Nov 04 01:02:41.098918 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.098896    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-k9wxw\""
Nov 04 01:02:41.098970 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.098929    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-n5n45\""
Nov 04 01:02:41.098970 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.098934    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-cfm9p\""
Nov 04 01:02:41.098970 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.098945    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 04 01:02:41.098970 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.098968    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 04 01:02:41.099158 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.099140    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 04 01:02:41.100292 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.100276    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Nov 04 01:02:41.100292 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.100291    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-pvmvv\""
Nov 04 01:02:41.100788 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.100774    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-9c9zb\""
Nov 04 01:02:41.101968 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.101950    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-rq94r\""
Nov 04 01:02:41.102038 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.101958    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 04 01:02:41.102038 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.102000    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 04 01:02:41.102437 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.102411    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 04 01:02:41.102437 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.102433    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 04 01:02:41.102575 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.102433    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 04 01:02:41.102575 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.102448    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 04 01:02:41.104048 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.104029    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 04 01:02:41.104130 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.104070    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 04 01:02:41.104130 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.104083    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 04 01:02:41.104130 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.104098    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-h8pc2\""
Nov 04 01:02:41.104130 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.104102    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 04 01:02:41.104130 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.104102    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zw2l8\""
Nov 04 01:02:41.104561 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.104549    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 04 01:02:41.108349 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.108330    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 04 01:02:41.108500 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.108480    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 04 01:02:41.108578 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.108527    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 04 01:02:41.108578 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.108559    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Nov 04 01:02:41.108578 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.108569    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 04 01:02:41.108578 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.108566    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 04 01:02:41.108735 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.108569    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-mhcr5\""
Nov 04 01:02:41.108735 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.108603    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-dh9vh\""
Nov 04 01:02:41.108792 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.108746    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 04 01:02:41.108887 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.108869    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 04 01:02:41.109712 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.109695    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 04 01:02:41.109712 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.109708    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 04 01:02:41.109811 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.109730    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Nov 04 01:02:41.109811 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.109758    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-r2cph\""
Nov 04 01:02:41.109811 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.109767    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 04 01:02:41.109935 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.109841    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-6kj2d\""
Nov 04 01:02:41.109976 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.109967    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-f8m8g\""
Nov 04 01:02:41.111438 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.111424    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 04 01:02:41.111491 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.111439    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-w5mjn\""
Nov 04 01:02:41.111537 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.111529    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vklqk\""
Nov 04 01:02:41.111643 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.111606    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Nov 04 01:02:41.111810 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.111795    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Nov 04 01:02:41.111867 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.111853    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Nov 04 01:02:41.113193 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.113178    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-7ffl4\""
Nov 04 01:02:41.113397 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.113386    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Nov 04 01:02:41.113446 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.113392    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-jjj2l\""
Nov 04 01:02:41.113810 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.113794    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Nov 04 01:02:41.113894 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.113860    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Nov 04 01:02:41.163461 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.163444    5068 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-67.us-east-2.compute.internal"
Nov 04 01:02:41.163570 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.163557    5068 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-67.us-east-2.compute.internal"
Nov 04 01:02:41.174215 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.174194    5068 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 01:02:41.174290 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:02:41.174245    5068 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-132-67.us-east-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-67.us-east-2.compute.internal"
Nov 04 01:02:41.174389 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.174373    5068 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 01:02:41.174438 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:02:41.174416    5068 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-132-67.us-east-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-67.us-east-2.compute.internal"
Nov 04 01:02:41.175839 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.175825    5068 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 04 01:02:41.190895 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.190869    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/21d1d23b-7b6e-4ab5-9593-cf7784fffea2-config\") pod \"console-operator-7cd56b8dbc-tr8b2\" (UID: \"21d1d23b-7b6e-4ab5-9593-cf7784fffea2\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tr8b2"
Nov 04 01:02:41.190985 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.190904    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pt8q7\" (UniqueName: \"kubernetes.io/projected/28a8304b-9226-422d-9044-b50678d52971-kube-api-access-pt8q7\") pod \"cluster-monitoring-operator-884576c57-6vxbv\" (UID: \"28a8304b-9226-422d-9044-b50678d52971\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6vxbv"
Nov 04 01:02:41.190985 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.190924    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-etc-sysconfig\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:02:41.190985 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.190946    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7b044258-a241-4bd2-a6eb-9ed586a0f2cc-serviceca\") pod \"node-ca-fvjq7\" (UID: \"7b044258-a241-4bd2-a6eb-9ed586a0f2cc\") " pod="openshift-image-registry/node-ca-fvjq7"
Nov 04 01:02:41.190985 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.190968    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-run-ovn\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:41.191178 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.191049    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6be44550-8625-4296-aeb9-da833aaa4b2f-dbus\") pod \"global-pull-secret-syncer-gnrzw\" (UID: \"6be44550-8625-4296-aeb9-da833aaa4b2f\") " pod="kube-system/global-pull-secret-syncer-gnrzw"
Nov 04 01:02:41.191178 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.191090    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/e2b1aee4-49c5-47a0-a79f-a3fe032df2d7-archives-path\") pod \"periodic-gathering-t6tpg-6r4ws\" (UID: \"e2b1aee4-49c5-47a0-a79f-a3fe032df2d7\") " pod="openshift-insights/periodic-gathering-t6tpg-6r4ws"
Nov 04 01:02:41.191178 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.191106    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3f74f502-9d11-4533-8bdf-cf880a0fbe0c-konnectivity-ca\") pod \"konnectivity-agent-sjcm7\" (UID: \"3f74f502-9d11-4533-8bdf-cf880a0fbe0c\") " pod="kube-system/konnectivity-agent-sjcm7"
Nov 04 01:02:41.191178 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.191139    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/21d1d23b-7b6e-4ab5-9593-cf7784fffea2-config\") pod \"console-operator-7cd56b8dbc-tr8b2\" (UID: \"21d1d23b-7b6e-4ab5-9593-cf7784fffea2\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tr8b2"
Nov 04 01:02:41.191351 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.191234    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/e2b1aee4-49c5-47a0-a79f-a3fe032df2d7-archives-path\") pod \"periodic-gathering-t6tpg-6r4ws\" (UID: \"e2b1aee4-49c5-47a0-a79f-a3fe032df2d7\") " pod="openshift-insights/periodic-gathering-t6tpg-6r4ws"
Nov 04 01:02:41.191351 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.191281    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mh966\" (UniqueName: \"kubernetes.io/projected/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-kube-api-access-mh966\") pod \"insights-runtime-extractor-sg4cw\" (UID: \"ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774\") " pod="openshift-insights/insights-runtime-extractor-sg4cw"
Nov 04 01:02:41.191351 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.191315    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fc0b0e48-8adf-4d0d-80dd-ca76846f824a-serving-cert\") pod \"service-ca-operator-544d749c8c-8jlt4\" (UID: \"fc0b0e48-8adf-4d0d-80dd-ca76846f824a\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-8jlt4"
Nov 04 01:02:41.191351 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.191339    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/82ac4b70-1c01-478c-ae1b-4a60dad06677-cert\") pod \"ingress-canary-4827b\" (UID: \"82ac4b70-1c01-478c-ae1b-4a60dad06677\") " pod="openshift-ingress-canary/ingress-canary-4827b"
Nov 04 01:02:41.191528 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.191373    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3f74f502-9d11-4533-8bdf-cf880a0fbe0c-konnectivity-ca\") pod \"konnectivity-agent-sjcm7\" (UID: \"3f74f502-9d11-4533-8bdf-cf880a0fbe0c\") " pod="kube-system/konnectivity-agent-sjcm7"
Nov 04 01:02:41.191528 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.191438    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-var-lib-openvswitch\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:41.191649 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.191537    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-image-registry-private-configuration\") pod \"image-registry-5c5894ddd-j4phf\" (UID: \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\") " pod="openshift-image-registry/image-registry-5c5894ddd-j4phf"
Nov 04 01:02:41.191649 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.191568    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-trusted-ca\") pod \"image-registry-5c5894ddd-j4phf\" (UID: \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\") " pod="openshift-image-registry/image-registry-5c5894ddd-j4phf"
Nov 04 01:02:41.191649 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.191632    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae02b148-c5ba-4db0-b817-9be4a488c708-service-ca-bundle\") pod \"router-default-5569c8658c-v7txl\" (UID: \"ae02b148-c5ba-4db0-b817-9be4a488c708\") " pod="openshift-ingress/router-default-5569c8658c-v7txl"
Nov 04 01:02:41.191788 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.191650    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/82ac4b70-1c01-478c-ae1b-4a60dad06677-cert\") pod \"ingress-canary-4827b\" (UID: \"82ac4b70-1c01-478c-ae1b-4a60dad06677\") " pod="openshift-ingress-canary/ingress-canary-4827b"
Nov 04 01:02:41.191788 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.191686    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-cnibin\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:02:41.191788 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.191713    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4965a588-c16b-488b-82c7-b53bb390f7d0-socket-dir\") pod \"aws-ebs-csi-driver-node-tt2x6\" (UID: \"4965a588-c16b-488b-82c7-b53bb390f7d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt2x6"
Nov 04 01:02:41.191788 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.191738    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-log-socket\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:41.191788 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.191756    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fc0b0e48-8adf-4d0d-80dd-ca76846f824a-serving-cert\") pod \"service-ca-operator-544d749c8c-8jlt4\" (UID: \"fc0b0e48-8adf-4d0d-80dd-ca76846f824a\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-8jlt4"
Nov 04 01:02:41.191788 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.191764    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3a53d2f9-8ce8-4ae5-9974-a8c865065ff3-host-slash\") pod \"iptables-alerter-l7n65\" (UID: \"3a53d2f9-8ce8-4ae5-9974-a8c865065ff3\") " pod="openshift-network-operator/iptables-alerter-l7n65"
Nov 04 01:02:41.191788 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.191753    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-image-registry-private-configuration\") pod \"image-registry-5c5894ddd-j4phf\" (UID: \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\") " pod="openshift-image-registry/image-registry-5c5894ddd-j4phf"
Nov 04 01:02:41.192085 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.191792    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b3bd337e-c495-4349-b56d-c16ebe3a9b11-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qlr2s\" (UID: \"b3bd337e-c495-4349-b56d-c16ebe3a9b11\") " pod="openshift-multus/multus-additional-cni-plugins-qlr2s"
Nov 04 01:02:41.192085 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.191823    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f68cb5a-3e22-4dd6-98d0-f28988ea86a2-tmp-dir\") pod \"dns-default-klq54\" (UID: \"2f68cb5a-3e22-4dd6-98d0-f28988ea86a2\") " pod="openshift-dns/dns-default-klq54"
Nov 04 01:02:41.192085 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.191888    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b4dcd02c-9a77-43da-9923-3ea68740b318-signing-cabundle\") pod \"service-ca-5fffc97bdf-2zdns\" (UID: \"b4dcd02c-9a77-43da-9923-3ea68740b318\") " pod="openshift-service-ca/service-ca-5fffc97bdf-2zdns"
Nov 04 01:02:41.192085 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.191892    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7b044258-a241-4bd2-a6eb-9ed586a0f2cc-serviceca\") pod \"node-ca-fvjq7\" (UID: \"7b044258-a241-4bd2-a6eb-9ed586a0f2cc\") " pod="openshift-image-registry/node-ca-fvjq7"
Nov 04 01:02:41.192085 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.191937    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f68cb5a-3e22-4dd6-98d0-f28988ea86a2-tmp-dir\") pod \"dns-default-klq54\" (UID: \"2f68cb5a-3e22-4dd6-98d0-f28988ea86a2\") " pod="openshift-dns/dns-default-klq54"
Nov 04 01:02:41.192085 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.191948    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e41517b6-7ce4-43a6-89a9-76931457ca50-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-jmxvd\" (UID: \"e41517b6-7ce4-43a6-89a9-76931457ca50\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jmxvd"
Nov 04 01:02:41.192085 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.192040    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4965a588-c16b-488b-82c7-b53bb390f7d0-registration-dir\") pod \"aws-ebs-csi-driver-node-tt2x6\" (UID: \"4965a588-c16b-488b-82c7-b53bb390f7d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt2x6"
Nov 04 01:02:41.192085 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.192057    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b4dcd02c-9a77-43da-9923-3ea68740b318-signing-cabundle\") pod \"service-ca-5fffc97bdf-2zdns\" (UID: \"b4dcd02c-9a77-43da-9923-3ea68740b318\") " pod="openshift-service-ca/service-ca-5fffc97bdf-2zdns"
Nov 04 01:02:41.192085 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.192070    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dr2p4\" (UniqueName: \"kubernetes.io/projected/d4efaef2-c145-4839-b32f-ecf7a1c76a99-kube-api-access-dr2p4\") pod \"volume-data-source-validator-58c9f5f46d-gdqjb\" (UID: \"d4efaef2-c145-4839-b32f-ecf7a1c76a99\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-gdqjb"
Nov 04 01:02:41.192085 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.192087    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/20f4989c-4fe8-49d2-9430-5030a4274b60-bound-sa-token\") pod \"image-registry-6574d8498f-74vmj\" (UID: \"20f4989c-4fe8-49d2-9430-5030a4274b60\") " pod="openshift-image-registry/image-registry-6574d8498f-74vmj"
Nov 04 01:02:41.192569 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.192077    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b3bd337e-c495-4349-b56d-c16ebe3a9b11-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qlr2s\" (UID: \"b3bd337e-c495-4349-b56d-c16ebe3a9b11\") " pod="openshift-multus/multus-additional-cni-plugins-qlr2s"
Nov 04 01:02:41.192569 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.192162    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7lgb\" (UniqueName: \"kubernetes.io/projected/21d1d23b-7b6e-4ab5-9593-cf7784fffea2-kube-api-access-h7lgb\") pod \"console-operator-7cd56b8dbc-tr8b2\" (UID: \"21d1d23b-7b6e-4ab5-9593-cf7784fffea2\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tr8b2"
Nov 04 01:02:41.192569 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.192307    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sg4cw\" (UID: \"ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774\") " pod="openshift-insights/insights-runtime-extractor-sg4cw"
Nov 04 01:02:41.192569 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.192331    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8r85c\" (UniqueName: \"kubernetes.io/projected/a409c58b-b4eb-428f-9177-a09b8a229ec7-kube-api-access-8r85c\") pod \"network-metrics-daemon-rx7gq\" (UID: \"a409c58b-b4eb-428f-9177-a09b8a229ec7\") " pod="openshift-multus/network-metrics-daemon-rx7gq"
Nov 04 01:02:41.192569 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.192347    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-etc-kubernetes\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:02:41.192569 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.192362    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-multus-socket-dir-parent\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:02:41.192569 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.192375    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b3bd337e-c495-4349-b56d-c16ebe3a9b11-system-cni-dir\") pod \"multus-additional-cni-plugins-qlr2s\" (UID: \"b3bd337e-c495-4349-b56d-c16ebe3a9b11\") " pod="openshift-multus/multus-additional-cni-plugins-qlr2s"
Nov 04 01:02:41.192569 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.192392    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b3bd337e-c495-4349-b56d-c16ebe3a9b11-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qlr2s\" (UID: \"b3bd337e-c495-4349-b56d-c16ebe3a9b11\") " pod="openshift-multus/multus-additional-cni-plugins-qlr2s"
Nov 04 01:02:41.192569 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.192452    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-trusted-ca\") pod \"image-registry-5c5894ddd-j4phf\" (UID: \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\") " pod="openshift-image-registry/image-registry-5c5894ddd-j4phf"
Nov 04 01:02:41.192569 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.192456    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pksw4\" (UniqueName: \"kubernetes.io/projected/7b044258-a241-4bd2-a6eb-9ed586a0f2cc-kube-api-access-pksw4\") pod \"node-ca-fvjq7\" (UID: \"7b044258-a241-4bd2-a6eb-9ed586a0f2cc\") " pod="openshift-image-registry/node-ca-fvjq7"
Nov 04 01:02:41.192569 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.192484    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de97ada4-9cae-4775-a8f8-014004e17d25-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-sbjxr\" (UID: \"de97ada4-9cae-4775-a8f8-014004e17d25\") " pod="openshift-insights/insights-operator-7c4986d449-sbjxr"
Nov 04 01:02:41.192569 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.192535    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b3bd337e-c495-4349-b56d-c16ebe3a9b11-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qlr2s\" (UID: \"b3bd337e-c495-4349-b56d-c16ebe3a9b11\") " pod="openshift-multus/multus-additional-cni-plugins-qlr2s"
Nov 04 01:02:41.192569 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.192567    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2b1aee4-49c5-47a0-a79f-a3fe032df2d7-service-ca-bundle\") pod \"periodic-gathering-t6tpg-6r4ws\" (UID: \"e2b1aee4-49c5-47a0-a79f-a3fe032df2d7\") " pod="openshift-insights/periodic-gathering-t6tpg-6r4ws"
Nov 04 01:02:41.193132 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.192650    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-node-log\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:41.193132 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.192699    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:41.193132 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.192723    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2b1aee4-49c5-47a0-a79f-a3fe032df2d7-service-ca-bundle\") pod \"periodic-gathering-t6tpg-6r4ws\" (UID: \"e2b1aee4-49c5-47a0-a79f-a3fe032df2d7\") " pod="openshift-insights/periodic-gathering-t6tpg-6r4ws"
Nov 04 01:02:41.193132 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.192729    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rct8p\" (UniqueName: \"kubernetes.io/projected/20f4989c-4fe8-49d2-9430-5030a4274b60-kube-api-access-rct8p\") pod \"image-registry-6574d8498f-74vmj\" (UID: \"20f4989c-4fe8-49d2-9430-5030a4274b60\") " pod="openshift-image-registry/image-registry-6574d8498f-74vmj"
Nov 04 01:02:41.193132 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.192755    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-multus-conf-dir\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:02:41.193132 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.192757    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de97ada4-9cae-4775-a8f8-014004e17d25-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-sbjxr\" (UID: \"de97ada4-9cae-4775-a8f8-014004e17d25\") " pod="openshift-insights/insights-operator-7c4986d449-sbjxr"
Nov 04 01:02:41.193132 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.192791    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/20f4989c-4fe8-49d2-9430-5030a4274b60-trusted-ca\") pod \"image-registry-6574d8498f-74vmj\" (UID: \"20f4989c-4fe8-49d2-9430-5030a4274b60\") " pod="openshift-image-registry/image-registry-6574d8498f-74vmj"
Nov 04 01:02:41.193132 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.192806    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-data-volume\") pod \"insights-runtime-extractor-sg4cw\" (UID: \"ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774\") " pod="openshift-insights/insights-runtime-extractor-sg4cw"
Nov 04 01:02:41.193132 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.192838    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-tmp\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:02:41.193132 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.192899    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-tmp\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:02:41.193132 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.192899    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-host-var-lib-kubelet\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:02:41.193132 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.192914    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-data-volume\") pod \"insights-runtime-extractor-sg4cw\" (UID: \"ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774\") " pod="openshift-insights/insights-runtime-extractor-sg4cw"
Nov 04 01:02:41.193132 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.192930    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-host-kubelet\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:41.193132 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.192952    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-host-cni-bin\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:41.193132 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.192999    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3a53d2f9-8ce8-4ae5-9974-a8c865065ff3-iptables-alerter-script\") pod \"iptables-alerter-l7n65\" (UID: \"3a53d2f9-8ce8-4ae5-9974-a8c865065ff3\") " pod="openshift-network-operator/iptables-alerter-l7n65"
Nov 04 01:02:41.193132 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.193033    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-ca-trust-extracted\") pod \"image-registry-5c5894ddd-j4phf\" (UID: \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\") " pod="openshift-image-registry/image-registry-5c5894ddd-j4phf"
Nov 04 01:02:41.193132 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.193054    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7b044258-a241-4bd2-a6eb-9ed586a0f2cc-host\") pod \"node-ca-fvjq7\" (UID: \"7b044258-a241-4bd2-a6eb-9ed586a0f2cc\") " pod="openshift-image-registry/node-ca-fvjq7"
Nov 04 01:02:41.193132 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.193069    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wsn9\" (UniqueName: \"kubernetes.io/projected/588c643c-b16a-4d5c-ad85-ccb8d486dd43-kube-api-access-4wsn9\") pod \"network-check-target-wxhng\" (UID: \"588c643c-b16a-4d5c-ad85-ccb8d486dd43\") " pod="openshift-network-diagnostics/network-check-target-wxhng"
Nov 04 01:02:41.193132 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.193084    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4965a588-c16b-488b-82c7-b53bb390f7d0-sys-fs\") pod \"aws-ebs-csi-driver-node-tt2x6\" (UID: \"4965a588-c16b-488b-82c7-b53bb390f7d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt2x6"
Nov 04 01:02:41.193132 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.193103    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3d2d0e05-a773-4442-ab38-d460ad32f51e-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-58wxt\" (UID: \"3d2d0e05-a773-4442-ab38-d460ad32f51e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-58wxt"
Nov 04 01:02:41.193132 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.193128    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6tv2\" (UniqueName: \"kubernetes.io/projected/9d694048-4b09-480b-ad26-8084c7775b24-kube-api-access-b6tv2\") pod \"migrator-66df5d496b-b9rdk\" (UID: \"9d694048-4b09-480b-ad26-8084c7775b24\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-b9rdk"
Nov 04 01:02:41.193132 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.193144    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-ca-trust-extracted\") pod \"image-registry-5c5894ddd-j4phf\" (UID: \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\") " pod="openshift-image-registry/image-registry-5c5894ddd-j4phf"
Nov 04 01:02:41.194086 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.193142    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/20f4989c-4fe8-49d2-9430-5030a4274b60-trusted-ca\") pod \"image-registry-6574d8498f-74vmj\" (UID: \"20f4989c-4fe8-49d2-9430-5030a4274b60\") " pod="openshift-image-registry/image-registry-6574d8498f-74vmj"
Nov 04 01:02:41.194086 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.193150    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/80a681c1-e74e-4887-a56d-6be313103581-cni-binary-copy\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:02:41.194086 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.193234    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3a53d2f9-8ce8-4ae5-9974-a8c865065ff3-iptables-alerter-script\") pod \"iptables-alerter-l7n65\" (UID: \"3a53d2f9-8ce8-4ae5-9974-a8c865065ff3\") " pod="openshift-network-operator/iptables-alerter-l7n65"
Nov 04 01:02:41.194086 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.193278    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3d2d0e05-a773-4442-ab38-d460ad32f51e-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-58wxt\" (UID: \"3d2d0e05-a773-4442-ab38-d460ad32f51e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-58wxt"
Nov 04 01:02:41.194086 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.193319    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d6a4c158-e115-4d2f-af0d-1d9249dc7384-hosts-file\") pod \"node-resolver-8gvq8\" (UID: \"d6a4c158-e115-4d2f-af0d-1d9249dc7384\") " pod="openshift-dns/node-resolver-8gvq8"
Nov 04 01:02:41.194086 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.193362    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/80a681c1-e74e-4887-a56d-6be313103581-cni-binary-copy\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:02:41.194086 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.193366    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/20f4989c-4fe8-49d2-9430-5030a4274b60-registry-certificates\") pod \"image-registry-6574d8498f-74vmj\" (UID: \"20f4989c-4fe8-49d2-9430-5030a4274b60\") " pod="openshift-image-registry/image-registry-6574d8498f-74vmj"
Nov 04 01:02:41.194086 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.193432    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a409c58b-b4eb-428f-9177-a09b8a229ec7-metrics-certs\") pod \"network-metrics-daemon-rx7gq\" (UID: \"a409c58b-b4eb-428f-9177-a09b8a229ec7\") " pod="openshift-multus/network-metrics-daemon-rx7gq"
Nov 04 01:02:41.194086 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.193460    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b4dcd02c-9a77-43da-9923-3ea68740b318-signing-key\") pod \"service-ca-5fffc97bdf-2zdns\" (UID: \"b4dcd02c-9a77-43da-9923-3ea68740b318\") " pod="openshift-service-ca/service-ca-5fffc97bdf-2zdns"
Nov 04 01:02:41.194086 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.193477    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-multus-cni-dir\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:02:41.194086 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.193494    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6t28\" (UniqueName: \"kubernetes.io/projected/a0c9abfd-2f47-4cb9-9d3c-479a6231be97-kube-api-access-p6t28\") pod \"kas-connection-checker-685895c7c9-hhfmk\" (UID: \"a0c9abfd-2f47-4cb9-9d3c-479a6231be97\") " pod="kube-system/kas-connection-checker-685895c7c9-hhfmk"
Nov 04 01:02:41.194086 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.193520    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-registry-certificates\") pod \"image-registry-5c5894ddd-j4phf\" (UID: \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\") " pod="openshift-image-registry/image-registry-5c5894ddd-j4phf"
Nov 04 01:02:41.194086 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.193554    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpjd9\" (UniqueName: \"kubernetes.io/projected/3a53d2f9-8ce8-4ae5-9974-a8c865065ff3-kube-api-access-mpjd9\") pod \"iptables-alerter-l7n65\" (UID: \"3a53d2f9-8ce8-4ae5-9974-a8c865065ff3\") " pod="openshift-network-operator/iptables-alerter-l7n65"
Nov 04 01:02:41.194086 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.193573    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wj8p2\" (UniqueName: \"kubernetes.io/projected/d6a4c158-e115-4d2f-af0d-1d9249dc7384-kube-api-access-wj8p2\") pod \"node-resolver-8gvq8\" (UID: \"d6a4c158-e115-4d2f-af0d-1d9249dc7384\") " pod="openshift-dns/node-resolver-8gvq8"
Nov 04 01:02:41.194086 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.193684    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-etc-sysctl-conf\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:02:41.194086 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.193701    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b4dcd02c-9a77-43da-9923-3ea68740b318-signing-key\") pod \"service-ca-5fffc97bdf-2zdns\" (UID: \"b4dcd02c-9a77-43da-9923-3ea68740b318\") " pod="openshift-service-ca/service-ca-5fffc97bdf-2zdns"
Nov 04 01:02:41.194086 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.193717    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b3bd337e-c495-4349-b56d-c16ebe3a9b11-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qlr2s\" (UID: \"b3bd337e-c495-4349-b56d-c16ebe3a9b11\") " pod="openshift-multus/multus-additional-cni-plugins-qlr2s"
Nov 04 01:02:41.194086 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.193757    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnhzq\" (UniqueName: \"kubernetes.io/projected/23cb8b1a-61f6-4045-9f5a-3ccc1dc346a5-kube-api-access-wnhzq\") pod \"kas-connection-checker-685895c7c9-6lcfv\" (UID: \"23cb8b1a-61f6-4045-9f5a-3ccc1dc346a5\") " pod="kube-system/kas-connection-checker-685895c7c9-6lcfv"
Nov 04 01:02:41.194086 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.193818    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-host-var-lib-cni-bin\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:02:41.194086 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.193939    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-host-slash\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:41.194086 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.193978    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-ovnkube-script-lib\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:41.194086 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.194010    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6be44550-8625-4296-aeb9-da833aaa4b2f-original-pull-secret\") pod \"global-pull-secret-syncer-gnrzw\" (UID: \"6be44550-8625-4296-aeb9-da833aaa4b2f\") " pod="kube-system/global-pull-secret-syncer-gnrzw"
Nov 04 01:02:41.194086 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.194037    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-crio-socket\") pod \"insights-runtime-extractor-sg4cw\" (UID: \"ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774\") " pod="openshift-insights/insights-runtime-extractor-sg4cw"
Nov 04 01:02:41.194086 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.194043    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/20f4989c-4fe8-49d2-9430-5030a4274b60-registry-certificates\") pod \"image-registry-6574d8498f-74vmj\" (UID: \"20f4989c-4fe8-49d2-9430-5030a4274b60\") " pod="openshift-image-registry/image-registry-6574d8498f-74vmj"
Nov 04 01:02:41.195140 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.194143    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9hjn\" (UniqueName: \"kubernetes.io/projected/ae02b148-c5ba-4db0-b817-9be4a488c708-kube-api-access-b9hjn\") pod \"router-default-5569c8658c-v7txl\" (UID: \"ae02b148-c5ba-4db0-b817-9be4a488c708\") " pod="openshift-ingress/router-default-5569c8658c-v7txl"
Nov 04 01:02:41.195140 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.194161    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fk9hd\" (UniqueName: \"kubernetes.io/projected/82ac4b70-1c01-478c-ae1b-4a60dad06677-kube-api-access-fk9hd\") pod \"ingress-canary-4827b\" (UID: \"82ac4b70-1c01-478c-ae1b-4a60dad06677\") " pod="openshift-ingress-canary/ingress-canary-4827b"
Nov 04 01:02:41.195140 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.194175    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/80a681c1-e74e-4887-a56d-6be313103581-multus-daemon-config\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:02:41.195140 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.194190    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-ovnkube-config\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:41.195140 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.194258    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6be44550-8625-4296-aeb9-da833aaa4b2f-original-pull-secret\") pod \"global-pull-secret-syncer-gnrzw\" (UID: \"6be44550-8625-4296-aeb9-da833aaa4b2f\") " pod="kube-system/global-pull-secret-syncer-gnrzw"
Nov 04 01:02:41.195140 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.194273    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-registry-certificates\") pod \"image-registry-5c5894ddd-j4phf\" (UID: \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\") " pod="openshift-image-registry/image-registry-5c5894ddd-j4phf"
Nov 04 01:02:41.195140 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.194287    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-ovnkube-script-lib\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:41.195140 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.194321    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nfw8\" (UniqueName: \"kubernetes.io/projected/3d2d0e05-a773-4442-ab38-d460ad32f51e-kube-api-access-8nfw8\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-58wxt\" (UID: \"3d2d0e05-a773-4442-ab38-d460ad32f51e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-58wxt"
Nov 04 01:02:41.195140 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.194365    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3739befd-6689-4197-b793-6f5b4b796880-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-27rmf\" (UID: \"3739befd-6689-4197-b793-6f5b4b796880\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-27rmf"
Nov 04 01:02:41.195140 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.194399    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/80a681c1-e74e-4887-a56d-6be313103581-multus-daemon-config\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:02:41.195140 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.194404    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-ovnkube-config\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:41.195140 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.194422    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6be44550-8625-4296-aeb9-da833aaa4b2f-kubelet-config\") pod \"global-pull-secret-syncer-gnrzw\" (UID: \"6be44550-8625-4296-aeb9-da833aaa4b2f\") " pod="kube-system/global-pull-secret-syncer-gnrzw"
Nov 04 01:02:41.195140 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.194449    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-bound-sa-token\") pod \"image-registry-5c5894ddd-j4phf\" (UID: \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\") " pod="openshift-image-registry/image-registry-5c5894ddd-j4phf"
Nov 04 01:02:41.195140 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.194487    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fc0b0e48-8adf-4d0d-80dd-ca76846f824a-config\") pod \"service-ca-operator-544d749c8c-8jlt4\" (UID: \"fc0b0e48-8adf-4d0d-80dd-ca76846f824a\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-8jlt4"
Nov 04 01:02:41.195140 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.194508    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-hostroot\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:02:41.195140 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.194524    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-host-run-netns\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:41.195140 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.194640    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6a4c158-e115-4d2f-af0d-1d9249dc7384-tmp-dir\") pod \"node-resolver-8gvq8\" (UID: \"d6a4c158-e115-4d2f-af0d-1d9249dc7384\") " pod="openshift-dns/node-resolver-8gvq8"
Nov 04 01:02:41.195140 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.194670    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae02b148-c5ba-4db0-b817-9be4a488c708-metrics-certs\") pod \"router-default-5569c8658c-v7txl\" (UID: \"ae02b148-c5ba-4db0-b817-9be4a488c708\") " pod="openshift-ingress/router-default-5569c8658c-v7txl"
Nov 04 01:02:41.195140 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.194694    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-etc-modprobe-d\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:02:41.195140 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.194732    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6a4c158-e115-4d2f-af0d-1d9249dc7384-tmp-dir\") pod \"node-resolver-8gvq8\" (UID: \"d6a4c158-e115-4d2f-af0d-1d9249dc7384\") " pod="openshift-dns/node-resolver-8gvq8"
Nov 04 01:02:41.195140 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.194742    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-var-lib-kubelet\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:02:41.195140 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.194775    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45zdl\" (UniqueName: \"kubernetes.io/projected/b4dcd02c-9a77-43da-9923-3ea68740b318-kube-api-access-45zdl\") pod \"service-ca-5fffc97bdf-2zdns\" (UID: \"b4dcd02c-9a77-43da-9923-3ea68740b318\") " pod="openshift-service-ca/service-ca-5fffc97bdf-2zdns"
Nov 04 01:02:41.195140 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.194784    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fc0b0e48-8adf-4d0d-80dd-ca76846f824a-config\") pod \"service-ca-operator-544d749c8c-8jlt4\" (UID: \"fc0b0e48-8adf-4d0d-80dd-ca76846f824a\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-8jlt4"
Nov 04 01:02:41.195140 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.194819    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/20f4989c-4fe8-49d2-9430-5030a4274b60-installation-pull-secrets\") pod \"image-registry-6574d8498f-74vmj\" (UID: \"20f4989c-4fe8-49d2-9430-5030a4274b60\") " pod="openshift-image-registry/image-registry-6574d8498f-74vmj"
Nov 04 01:02:41.195140 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.194900    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-etc-systemd\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:02:41.195140 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.194932    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kp4q\" (UniqueName: \"kubernetes.io/projected/2f68cb5a-3e22-4dd6-98d0-f28988ea86a2-kube-api-access-6kp4q\") pod \"dns-default-klq54\" (UID: \"2f68cb5a-3e22-4dd6-98d0-f28988ea86a2\") " pod="openshift-dns/dns-default-klq54"
Nov 04 01:02:41.195140 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.194960    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvzfb\" (UniqueName: \"kubernetes.io/projected/de97ada4-9cae-4775-a8f8-014004e17d25-kube-api-access-zvzfb\") pod \"insights-operator-7c4986d449-sbjxr\" (UID: \"de97ada4-9cae-4775-a8f8-014004e17d25\") " pod="openshift-insights/insights-operator-7c4986d449-sbjxr"
Nov 04 01:02:41.195140 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.194977    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/20f4989c-4fe8-49d2-9430-5030a4274b60-installation-pull-secrets\") pod \"image-registry-6574d8498f-74vmj\" (UID: \"20f4989c-4fe8-49d2-9430-5030a4274b60\") " pod="openshift-image-registry/image-registry-6574d8498f-74vmj"
Nov 04 01:02:41.195140 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.194994    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4965a588-c16b-488b-82c7-b53bb390f7d0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tt2x6\" (UID: \"4965a588-c16b-488b-82c7-b53bb390f7d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt2x6"
Nov 04 01:02:41.195140 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.195015    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-systemd-units\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:41.195140 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.195030    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-run-systemd\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:41.195140 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.195121    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-etc-openvswitch\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:41.196550 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.195155    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/de97ada4-9cae-4775-a8f8-014004e17d25-serving-cert\") pod \"insights-operator-7c4986d449-sbjxr\" (UID: \"de97ada4-9cae-4775-a8f8-014004e17d25\") " pod="openshift-insights/insights-operator-7c4986d449-sbjxr"
Nov 04 01:02:41.196550 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.195190    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3d2d0e05-a773-4442-ab38-d460ad32f51e-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-58wxt\" (UID: \"3d2d0e05-a773-4442-ab38-d460ad32f51e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-58wxt"
Nov 04 01:02:41.196550 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.195208    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/20f4989c-4fe8-49d2-9430-5030a4274b60-image-registry-private-configuration\") pod \"image-registry-6574d8498f-74vmj\" (UID: \"20f4989c-4fe8-49d2-9430-5030a4274b60\") " pod="openshift-image-registry/image-registry-6574d8498f-74vmj"
Nov 04 01:02:41.196550 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.195239    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/20f4989c-4fe8-49d2-9430-5030a4274b60-registry-tls\") pod \"image-registry-6574d8498f-74vmj\" (UID: \"20f4989c-4fe8-49d2-9430-5030a4274b60\") " pod="openshift-image-registry/image-registry-6574d8498f-74vmj"
Nov 04 01:02:41.196550 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.195265    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/20f4989c-4fe8-49d2-9430-5030a4274b60-ca-trust-extracted\") pod \"image-registry-6574d8498f-74vmj\" (UID: \"20f4989c-4fe8-49d2-9430-5030a4274b60\") " pod="openshift-image-registry/image-registry-6574d8498f-74vmj"
Nov 04 01:02:41.196550 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.195292    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6b7vh\" (UniqueName: \"kubernetes.io/projected/e2b1aee4-49c5-47a0-a79f-a3fe032df2d7-kube-api-access-6b7vh\") pod \"periodic-gathering-t6tpg-6r4ws\" (UID: \"e2b1aee4-49c5-47a0-a79f-a3fe032df2d7\") " pod="openshift-insights/periodic-gathering-t6tpg-6r4ws"
Nov 04 01:02:41.196550 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.195325    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/28a8304b-9226-422d-9044-b50678d52971-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-6vxbv\" (UID: \"28a8304b-9226-422d-9044-b50678d52971\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6vxbv"
Nov 04 01:02:41.196550 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.195382    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72dk9\" (UniqueName: \"kubernetes.io/projected/4a79348d-8cf7-4429-8c5a-4084cf85fcc5-kube-api-access-72dk9\") pod \"kas-connection-checker-685895c7c9-rff54\" (UID: \"4a79348d-8cf7-4429-8c5a-4084cf85fcc5\") " pod="kube-system/kas-connection-checker-685895c7c9-rff54"
Nov 04 01:02:41.196550 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.195416    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/de97ada4-9cae-4775-a8f8-014004e17d25-serving-cert\") pod \"insights-operator-7c4986d449-sbjxr\" (UID: \"de97ada4-9cae-4775-a8f8-014004e17d25\") " pod="openshift-insights/insights-operator-7c4986d449-sbjxr"
Nov 04 01:02:41.196550 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.195429    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/20f4989c-4fe8-49d2-9430-5030a4274b60-ca-trust-extracted\") pod \"image-registry-6574d8498f-74vmj\" (UID: \"20f4989c-4fe8-49d2-9430-5030a4274b60\") " pod="openshift-image-registry/image-registry-6574d8498f-74vmj"
Nov 04 01:02:41.196550 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.195447    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/20f4989c-4fe8-49d2-9430-5030a4274b60-image-registry-private-configuration\") pod \"image-registry-6574d8498f-74vmj\" (UID: \"20f4989c-4fe8-49d2-9430-5030a4274b60\") " pod="openshift-image-registry/image-registry-6574d8498f-74vmj"
Nov 04 01:02:41.196550 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.195505    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-host\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:02:41.196550 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.195538    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/e41517b6-7ce4-43a6-89a9-76931457ca50-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-jmxvd\" (UID: \"e41517b6-7ce4-43a6-89a9-76931457ca50\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jmxvd"
Nov 04 01:02:41.196550 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.195555    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3d2d0e05-a773-4442-ab38-d460ad32f51e-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-58wxt\" (UID: \"3d2d0e05-a773-4442-ab38-d460ad32f51e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-58wxt"
Nov 04 01:02:41.196550 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.195726    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4965a588-c16b-488b-82c7-b53bb390f7d0-device-dir\") pod \"aws-ebs-csi-driver-node-tt2x6\" (UID: \"4965a588-c16b-488b-82c7-b53bb390f7d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt2x6"
Nov 04 01:02:41.196550 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.195758    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-env-overrides\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:41.196550 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.195783    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-ovn-node-metrics-cert\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:41.196550 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.195806    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-run\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:02:41.196550 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.195819    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/e41517b6-7ce4-43a6-89a9-76931457ca50-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-jmxvd\" (UID: \"e41517b6-7ce4-43a6-89a9-76931457ca50\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jmxvd"
Nov 04 01:02:41.196550 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.195835    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5f285\" (UniqueName: \"kubernetes.io/projected/e41517b6-7ce4-43a6-89a9-76931457ca50-kube-api-access-5f285\") pod \"cluster-samples-operator-6ddb6c8b94-jmxvd\" (UID: \"e41517b6-7ce4-43a6-89a9-76931457ca50\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jmxvd"
Nov 04 01:02:41.196550 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.195914    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-host-run-netns\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:02:41.196550 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.195971    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vgx6\" (UniqueName: \"kubernetes.io/projected/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-kube-api-access-8vgx6\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:41.196550 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.196358    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de97ada4-9cae-4775-a8f8-014004e17d25-service-ca-bundle\") pod \"insights-operator-7c4986d449-sbjxr\" (UID: \"de97ada4-9cae-4775-a8f8-014004e17d25\") " pod="openshift-insights/insights-operator-7c4986d449-sbjxr"
Nov 04 01:02:41.196550 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.196406    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-system-cni-dir\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:02:41.196550 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.196452    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lr5kt\" (UniqueName: \"kubernetes.io/projected/4d3480a2-034c-4959-8aee-b0759dbfc24f-kube-api-access-lr5kt\") pod \"network-check-source-6f78d59f99-zc968\" (UID: \"4d3480a2-034c-4959-8aee-b0759dbfc24f\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-zc968"
Nov 04 01:02:41.196550 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.196482    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de97ada4-9cae-4775-a8f8-014004e17d25-service-ca-bundle\") pod \"insights-operator-7c4986d449-sbjxr\" (UID: \"de97ada4-9cae-4775-a8f8-014004e17d25\") " pod="openshift-insights/insights-operator-7c4986d449-sbjxr"
Nov 04 01:02:41.196550 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.196492    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-etc-sysctl-d\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:02:41.196550 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.196545    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-host-run-multus-certs\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:02:41.197761 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.196800    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-env-overrides\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:41.197761 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.196913    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-ovn-node-metrics-cert\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:41.198675 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.198176    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-499cg\" (UniqueName: \"kubernetes.io/projected/80a681c1-e74e-4887-a56d-6be313103581-kube-api-access-499cg\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:02:41.198675 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.198223    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-run-openvswitch\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:41.198675 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.198262    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/28a8304b-9226-422d-9044-b50678d52971-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-6vxbv\" (UID: \"28a8304b-9226-422d-9044-b50678d52971\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6vxbv"
Nov 04 01:02:41.198675 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.198293    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ae02b148-c5ba-4db0-b817-9be4a488c708-default-certificate\") pod \"router-default-5569c8658c-v7txl\" (UID: \"ae02b148-c5ba-4db0-b817-9be4a488c708\") " pod="openshift-ingress/router-default-5569c8658c-v7txl"
Nov 04 01:02:41.198675 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.198326    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ae02b148-c5ba-4db0-b817-9be4a488c708-stats-auth\") pod \"router-default-5569c8658c-v7txl\" (UID: \"ae02b148-c5ba-4db0-b817-9be4a488c708\") " pod="openshift-ingress/router-default-5569c8658c-v7txl"
Nov 04 01:02:41.198675 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.198358    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-etc-tuned\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:02:41.198675 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.198390    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f68cb5a-3e22-4dd6-98d0-f28988ea86a2-metrics-tls\") pod \"dns-default-klq54\" (UID: \"2f68cb5a-3e22-4dd6-98d0-f28988ea86a2\") " pod="openshift-dns/dns-default-klq54"
Nov 04 01:02:41.198675 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.198426    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-host-run-k8s-cni-cncf-io\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:02:41.198675 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.198459    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-host-var-lib-cni-multus\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:02:41.198675 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.198487    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p664g\" (UniqueName: \"kubernetes.io/projected/8f92b52c-22de-4ce7-b716-fdfa22d7b52f-kube-api-access-p664g\") pod \"migrator-66df5d496b-fp864\" (UID: \"8f92b52c-22de-4ce7-b716-fdfa22d7b52f\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-fp864"
Nov 04 01:02:41.198675 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.198521    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-installation-pull-secrets\") pod \"image-registry-5c5894ddd-j4phf\" (UID: \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\") " pod="openshift-image-registry/image-registry-5c5894ddd-j4phf"
Nov 04 01:02:41.198675 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.198552    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/21d1d23b-7b6e-4ab5-9593-cf7784fffea2-serving-cert\") pod \"console-operator-7cd56b8dbc-tr8b2\" (UID: \"21d1d23b-7b6e-4ab5-9593-cf7784fffea2\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tr8b2"
Nov 04 01:02:41.198675 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.198584    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sg4cw\" (UID: \"ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774\") " pod="openshift-insights/insights-runtime-extractor-sg4cw"
Nov 04 01:02:41.199312 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.199083    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2f68cb5a-3e22-4dd6-98d0-f28988ea86a2-config-volume\") pod \"dns-default-klq54\" (UID: \"2f68cb5a-3e22-4dd6-98d0-f28988ea86a2\") " pod="openshift-dns/dns-default-klq54"
Nov 04 01:02:41.199312 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.199127    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4965a588-c16b-488b-82c7-b53bb390f7d0-etc-selinux\") pod \"aws-ebs-csi-driver-node-tt2x6\" (UID: \"4965a588-c16b-488b-82c7-b53bb390f7d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt2x6"
Nov 04 01:02:41.199312 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.199165    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3739befd-6689-4197-b793-6f5b4b796880-nginx-conf\") pod \"networking-console-plugin-86c478bb88-27rmf\" (UID: \"3739befd-6689-4197-b793-6f5b4b796880\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-27rmf"
Nov 04 01:02:41.199312 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.199199    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2b1aee4-49c5-47a0-a79f-a3fe032df2d7-trusted-ca-bundle\") pod \"periodic-gathering-t6tpg-6r4ws\" (UID: \"e2b1aee4-49c5-47a0-a79f-a3fe032df2d7\") " pod="openshift-insights/periodic-gathering-t6tpg-6r4ws"
Nov 04 01:02:41.199312 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.199233    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-lib-modules\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:02:41.199312 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.199270    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b3bd337e-c495-4349-b56d-c16ebe3a9b11-cni-binary-copy\") pod \"multus-additional-cni-plugins-qlr2s\" (UID: \"b3bd337e-c495-4349-b56d-c16ebe3a9b11\") " pod="openshift-multus/multus-additional-cni-plugins-qlr2s"
Nov 04 01:02:41.199312 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.199304    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncqm5\" (UniqueName: \"kubernetes.io/projected/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-kube-api-access-ncqm5\") pod \"image-registry-5c5894ddd-j4phf\" (UID: \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\") " pod="openshift-image-registry/image-registry-5c5894ddd-j4phf"
Nov 04 01:02:41.199654 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.199339    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3f74f502-9d11-4533-8bdf-cf880a0fbe0c-agent-certs\") pod \"konnectivity-agent-sjcm7\" (UID: \"3f74f502-9d11-4533-8bdf-cf880a0fbe0c\") " pod="kube-system/konnectivity-agent-sjcm7"
Nov 04 01:02:41.199654 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.199374    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6tkr\" (UniqueName: \"kubernetes.io/projected/fc0b0e48-8adf-4d0d-80dd-ca76846f824a-kube-api-access-r6tkr\") pod \"service-ca-operator-544d749c8c-8jlt4\" (UID: \"fc0b0e48-8adf-4d0d-80dd-ca76846f824a\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-8jlt4"
Nov 04 01:02:41.199654 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.199409    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b3bd337e-c495-4349-b56d-c16ebe3a9b11-cnibin\") pod \"multus-additional-cni-plugins-qlr2s\" (UID: \"b3bd337e-c495-4349-b56d-c16ebe3a9b11\") " pod="openshift-multus/multus-additional-cni-plugins-qlr2s"
Nov 04 01:02:41.199654 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.199436    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/de97ada4-9cae-4775-a8f8-014004e17d25-tmp\") pod \"insights-operator-7c4986d449-sbjxr\" (UID: \"de97ada4-9cae-4775-a8f8-014004e17d25\") " pod="openshift-insights/insights-operator-7c4986d449-sbjxr"
Nov 04 01:02:41.199654 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.199472    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-host-run-ovn-kubernetes\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:41.199654 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.199506    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-host-cni-netd\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:41.199654 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.199541    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/21d1d23b-7b6e-4ab5-9593-cf7784fffea2-trusted-ca\") pod \"console-operator-7cd56b8dbc-tr8b2\" (UID: \"21d1d23b-7b6e-4ab5-9593-cf7784fffea2\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tr8b2"
Nov 04 01:02:41.199654 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.199576    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9n9lx\" (UniqueName: \"kubernetes.io/projected/b3bd337e-c495-4349-b56d-c16ebe3a9b11-kube-api-access-9n9lx\") pod \"multus-additional-cni-plugins-qlr2s\" (UID: \"b3bd337e-c495-4349-b56d-c16ebe3a9b11\") " pod="openshift-multus/multus-additional-cni-plugins-qlr2s"
Nov 04 01:02:41.199654 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.199632    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/de97ada4-9cae-4775-a8f8-014004e17d25-snapshots\") pod \"insights-operator-7c4986d449-sbjxr\" (UID: \"de97ada4-9cae-4775-a8f8-014004e17d25\") " pod="openshift-insights/insights-operator-7c4986d449-sbjxr"
Nov 04 01:02:41.200058 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.199665    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-etc-kubernetes\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:02:41.200058 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.199696    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-sys\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:02:41.200058 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.199731    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p559g\" (UniqueName: \"kubernetes.io/projected/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-kube-api-access-p559g\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:02:41.200058 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.199779    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqnfn\" (UniqueName: \"kubernetes.io/projected/4965a588-c16b-488b-82c7-b53bb390f7d0-kube-api-access-xqnfn\") pod \"aws-ebs-csi-driver-node-tt2x6\" (UID: \"4965a588-c16b-488b-82c7-b53bb390f7d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt2x6"
Nov 04 01:02:41.200058 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.199810    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-registry-tls\") pod \"image-registry-5c5894ddd-j4phf\" (UID: \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\") " pod="openshift-image-registry/image-registry-5c5894ddd-j4phf"
Nov 04 01:02:41.201738 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.200441    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/28a8304b-9226-422d-9044-b50678d52971-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-6vxbv\" (UID: \"28a8304b-9226-422d-9044-b50678d52971\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6vxbv"
Nov 04 01:02:41.201738 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.200641    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2f68cb5a-3e22-4dd6-98d0-f28988ea86a2-config-volume\") pod \"dns-default-klq54\" (UID: \"2f68cb5a-3e22-4dd6-98d0-f28988ea86a2\") " pod="openshift-dns/dns-default-klq54"
Nov 04 01:02:41.201738 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.200650    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-installation-pull-secrets\") pod \"image-registry-5c5894ddd-j4phf\" (UID: \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\") " pod="openshift-image-registry/image-registry-5c5894ddd-j4phf"
Nov 04 01:02:41.201738 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.200646    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3739befd-6689-4197-b793-6f5b4b796880-nginx-conf\") pod \"networking-console-plugin-86c478bb88-27rmf\" (UID: \"3739befd-6689-4197-b793-6f5b4b796880\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-27rmf"
Nov 04 01:02:41.201738 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.200702    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ae02b148-c5ba-4db0-b817-9be4a488c708-default-certificate\") pod \"router-default-5569c8658c-v7txl\" (UID: \"ae02b148-c5ba-4db0-b817-9be4a488c708\") " pod="openshift-ingress/router-default-5569c8658c-v7txl"
Nov 04 01:02:41.201738 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.200759    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-etc-tuned\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:02:41.201738 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.200894    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ae02b148-c5ba-4db0-b817-9be4a488c708-stats-auth\") pod \"router-default-5569c8658c-v7txl\" (UID: \"ae02b148-c5ba-4db0-b817-9be4a488c708\") " pod="openshift-ingress/router-default-5569c8658c-v7txl"
Nov 04 01:02:41.201738 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.200939    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/de97ada4-9cae-4775-a8f8-014004e17d25-snapshots\") pod \"insights-operator-7c4986d449-sbjxr\" (UID: \"de97ada4-9cae-4775-a8f8-014004e17d25\") " pod="openshift-insights/insights-operator-7c4986d449-sbjxr"
Nov 04 01:02:41.201738 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.201117    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b3bd337e-c495-4349-b56d-c16ebe3a9b11-cni-binary-copy\") pod \"multus-additional-cni-plugins-qlr2s\" (UID: \"b3bd337e-c495-4349-b56d-c16ebe3a9b11\") " pod="openshift-multus/multus-additional-cni-plugins-qlr2s"
Nov 04 01:02:41.201738 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.201282    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3f74f502-9d11-4533-8bdf-cf880a0fbe0c-agent-certs\") pod \"konnectivity-agent-sjcm7\" (UID: \"3f74f502-9d11-4533-8bdf-cf880a0fbe0c\") " pod="kube-system/konnectivity-agent-sjcm7"
Nov 04 01:02:41.201738 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.201562    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2b1aee4-49c5-47a0-a79f-a3fe032df2d7-trusted-ca-bundle\") pod \"periodic-gathering-t6tpg-6r4ws\" (UID: \"e2b1aee4-49c5-47a0-a79f-a3fe032df2d7\") " pod="openshift-insights/periodic-gathering-t6tpg-6r4ws"
Nov 04 01:02:41.201738 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.201715    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/21d1d23b-7b6e-4ab5-9593-cf7784fffea2-serving-cert\") pod \"console-operator-7cd56b8dbc-tr8b2\" (UID: \"21d1d23b-7b6e-4ab5-9593-cf7784fffea2\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tr8b2"
Nov 04 01:02:41.202256 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.201762    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/de97ada4-9cae-4775-a8f8-014004e17d25-tmp\") pod \"insights-operator-7c4986d449-sbjxr\" (UID: \"de97ada4-9cae-4775-a8f8-014004e17d25\") " pod="openshift-insights/insights-operator-7c4986d449-sbjxr"
Nov 04 01:02:41.202256 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.201973    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/21d1d23b-7b6e-4ab5-9593-cf7784fffea2-trusted-ca\") pod \"console-operator-7cd56b8dbc-tr8b2\" (UID: \"21d1d23b-7b6e-4ab5-9593-cf7784fffea2\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tr8b2"
Nov 04 01:02:41.202532 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.202418    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mh966\" (UniqueName: \"kubernetes.io/projected/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-kube-api-access-mh966\") pod \"insights-runtime-extractor-sg4cw\" (UID: \"ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774\") " pod="openshift-insights/insights-runtime-extractor-sg4cw"
Nov 04 01:02:41.202873 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.202854    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pt8q7\" (UniqueName: \"kubernetes.io/projected/28a8304b-9226-422d-9044-b50678d52971-kube-api-access-pt8q7\") pod \"cluster-monitoring-operator-884576c57-6vxbv\" (UID: \"28a8304b-9226-422d-9044-b50678d52971\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6vxbv"
Nov 04 01:02:41.203345 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.203328    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dr2p4\" (UniqueName: \"kubernetes.io/projected/d4efaef2-c145-4839-b32f-ecf7a1c76a99-kube-api-access-dr2p4\") pod \"volume-data-source-validator-58c9f5f46d-gdqjb\" (UID: \"d4efaef2-c145-4839-b32f-ecf7a1c76a99\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-gdqjb"
Nov 04 01:02:41.204084 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.204066    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8r85c\" (UniqueName: \"kubernetes.io/projected/a409c58b-b4eb-428f-9177-a09b8a229ec7-kube-api-access-8r85c\") pod \"network-metrics-daemon-rx7gq\" (UID: \"a409c58b-b4eb-428f-9177-a09b8a229ec7\") " pod="openshift-multus/network-metrics-daemon-rx7gq"
Nov 04 01:02:41.204178 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.204104    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6t28\" (UniqueName: \"kubernetes.io/projected/a0c9abfd-2f47-4cb9-9d3c-479a6231be97-kube-api-access-p6t28\") pod \"kas-connection-checker-685895c7c9-hhfmk\" (UID: \"a0c9abfd-2f47-4cb9-9d3c-479a6231be97\") " pod="kube-system/kas-connection-checker-685895c7c9-hhfmk"
Nov 04 01:02:41.204365 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.204346    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/20f4989c-4fe8-49d2-9430-5030a4274b60-bound-sa-token\") pod \"image-registry-6574d8498f-74vmj\" (UID: \"20f4989c-4fe8-49d2-9430-5030a4274b60\") " pod="openshift-image-registry/image-registry-6574d8498f-74vmj"
Nov 04 01:02:41.204750 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.204731    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wsn9\" (UniqueName: \"kubernetes.io/projected/588c643c-b16a-4d5c-ad85-ccb8d486dd43-kube-api-access-4wsn9\") pod \"network-check-target-wxhng\" (UID: \"588c643c-b16a-4d5c-ad85-ccb8d486dd43\") " pod="openshift-network-diagnostics/network-check-target-wxhng"
Nov 04 01:02:41.210686 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.210670    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6tv2\" (UniqueName: \"kubernetes.io/projected/9d694048-4b09-480b-ad26-8084c7775b24-kube-api-access-b6tv2\") pod \"migrator-66df5d496b-b9rdk\" (UID: \"9d694048-4b09-480b-ad26-8084c7775b24\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-b9rdk"
Nov 04 01:02:41.211594 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.211575    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnhzq\" (UniqueName: \"kubernetes.io/projected/23cb8b1a-61f6-4045-9f5a-3ccc1dc346a5-kube-api-access-wnhzq\") pod \"kas-connection-checker-685895c7c9-6lcfv\" (UID: \"23cb8b1a-61f6-4045-9f5a-3ccc1dc346a5\") " pod="kube-system/kas-connection-checker-685895c7c9-6lcfv"
Nov 04 01:02:41.211709 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.211651    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pksw4\" (UniqueName: \"kubernetes.io/projected/7b044258-a241-4bd2-a6eb-9ed586a0f2cc-kube-api-access-pksw4\") pod \"node-ca-fvjq7\" (UID: \"7b044258-a241-4bd2-a6eb-9ed586a0f2cc\") " pod="openshift-image-registry/node-ca-fvjq7"
Nov 04 01:02:41.212046 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.212026    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpjd9\" (UniqueName: \"kubernetes.io/projected/3a53d2f9-8ce8-4ae5-9974-a8c865065ff3-kube-api-access-mpjd9\") pod \"iptables-alerter-l7n65\" (UID: \"3a53d2f9-8ce8-4ae5-9974-a8c865065ff3\") " pod="openshift-network-operator/iptables-alerter-l7n65"
Nov 04 01:02:41.212104 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.212050    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9hjn\" (UniqueName: \"kubernetes.io/projected/ae02b148-c5ba-4db0-b817-9be4a488c708-kube-api-access-b9hjn\") pod \"router-default-5569c8658c-v7txl\" (UID: \"ae02b148-c5ba-4db0-b817-9be4a488c708\") " pod="openshift-ingress/router-default-5569c8658c-v7txl"
Nov 04 01:02:41.212104 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.212091    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wj8p2\" (UniqueName: \"kubernetes.io/projected/d6a4c158-e115-4d2f-af0d-1d9249dc7384-kube-api-access-wj8p2\") pod \"node-resolver-8gvq8\" (UID: \"d6a4c158-e115-4d2f-af0d-1d9249dc7384\") " pod="openshift-dns/node-resolver-8gvq8"
Nov 04 01:02:41.212754 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.212728    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nfw8\" (UniqueName: \"kubernetes.io/projected/3d2d0e05-a773-4442-ab38-d460ad32f51e-kube-api-access-8nfw8\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-58wxt\" (UID: \"3d2d0e05-a773-4442-ab38-d460ad32f51e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-58wxt"
Nov 04 01:02:41.213250 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.212888    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6b7vh\" (UniqueName: \"kubernetes.io/projected/e2b1aee4-49c5-47a0-a79f-a3fe032df2d7-kube-api-access-6b7vh\") pod \"periodic-gathering-t6tpg-6r4ws\" (UID: \"e2b1aee4-49c5-47a0-a79f-a3fe032df2d7\") " pod="openshift-insights/periodic-gathering-t6tpg-6r4ws"
Nov 04 01:02:41.213250 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.212910    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-bound-sa-token\") pod \"image-registry-5c5894ddd-j4phf\" (UID: \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\") " pod="openshift-image-registry/image-registry-5c5894ddd-j4phf"
Nov 04 01:02:41.213250 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.213022    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rct8p\" (UniqueName: \"kubernetes.io/projected/20f4989c-4fe8-49d2-9430-5030a4274b60-kube-api-access-rct8p\") pod \"image-registry-6574d8498f-74vmj\" (UID: \"20f4989c-4fe8-49d2-9430-5030a4274b60\") " pod="openshift-image-registry/image-registry-6574d8498f-74vmj"
Nov 04 01:02:41.213250 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.213042    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-45zdl\" (UniqueName: \"kubernetes.io/projected/b4dcd02c-9a77-43da-9923-3ea68740b318-kube-api-access-45zdl\") pod \"service-ca-5fffc97bdf-2zdns\" (UID: \"b4dcd02c-9a77-43da-9923-3ea68740b318\") " pod="openshift-service-ca/service-ca-5fffc97bdf-2zdns"
Nov 04 01:02:41.213250 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.213212    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-499cg\" (UniqueName: \"kubernetes.io/projected/80a681c1-e74e-4887-a56d-6be313103581-kube-api-access-499cg\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:02:41.213250 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.213219    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-72dk9\" (UniqueName: \"kubernetes.io/projected/4a79348d-8cf7-4429-8c5a-4084cf85fcc5-kube-api-access-72dk9\") pod \"kas-connection-checker-685895c7c9-rff54\" (UID: \"4a79348d-8cf7-4429-8c5a-4084cf85fcc5\") " pod="kube-system/kas-connection-checker-685895c7c9-rff54"
Nov 04 01:02:41.213651 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.213573    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvzfb\" (UniqueName: \"kubernetes.io/projected/de97ada4-9cae-4775-a8f8-014004e17d25-kube-api-access-zvzfb\") pod \"insights-operator-7c4986d449-sbjxr\" (UID: \"de97ada4-9cae-4775-a8f8-014004e17d25\") " pod="openshift-insights/insights-operator-7c4986d449-sbjxr"
Nov 04 01:02:41.213987 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.213967    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fk9hd\" (UniqueName: \"kubernetes.io/projected/82ac4b70-1c01-478c-ae1b-4a60dad06677-kube-api-access-fk9hd\") pod \"ingress-canary-4827b\" (UID: \"82ac4b70-1c01-478c-ae1b-4a60dad06677\") " pod="openshift-ingress-canary/ingress-canary-4827b"
Nov 04 01:02:41.216859 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.216825    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p664g\" (UniqueName: \"kubernetes.io/projected/8f92b52c-22de-4ce7-b716-fdfa22d7b52f-kube-api-access-p664g\") pod \"migrator-66df5d496b-fp864\" (UID: \"8f92b52c-22de-4ce7-b716-fdfa22d7b52f\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-fp864"
Nov 04 01:02:41.216950 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.216889    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6tkr\" (UniqueName: \"kubernetes.io/projected/fc0b0e48-8adf-4d0d-80dd-ca76846f824a-kube-api-access-r6tkr\") pod \"service-ca-operator-544d749c8c-8jlt4\" (UID: \"fc0b0e48-8adf-4d0d-80dd-ca76846f824a\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-8jlt4"
Nov 04 01:02:41.216950 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.216927    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqnfn\" (UniqueName: \"kubernetes.io/projected/4965a588-c16b-488b-82c7-b53bb390f7d0-kube-api-access-xqnfn\") pod \"aws-ebs-csi-driver-node-tt2x6\" (UID: \"4965a588-c16b-488b-82c7-b53bb390f7d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt2x6"
Nov 04 01:02:41.217241 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.217216    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lr5kt\" (UniqueName: \"kubernetes.io/projected/4d3480a2-034c-4959-8aee-b0759dbfc24f-kube-api-access-lr5kt\") pod \"network-check-source-6f78d59f99-zc968\" (UID: \"4d3480a2-034c-4959-8aee-b0759dbfc24f\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-zc968"
Nov 04 01:02:41.217368 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.217353    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7lgb\" (UniqueName: \"kubernetes.io/projected/21d1d23b-7b6e-4ab5-9593-cf7784fffea2-kube-api-access-h7lgb\") pod \"console-operator-7cd56b8dbc-tr8b2\" (UID: \"21d1d23b-7b6e-4ab5-9593-cf7784fffea2\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tr8b2"
Nov 04 01:02:41.217716 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.217697    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kp4q\" (UniqueName: \"kubernetes.io/projected/2f68cb5a-3e22-4dd6-98d0-f28988ea86a2-kube-api-access-6kp4q\") pod \"dns-default-klq54\" (UID: \"2f68cb5a-3e22-4dd6-98d0-f28988ea86a2\") " pod="openshift-dns/dns-default-klq54"
Nov 04 01:02:41.217933 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.217913    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5f285\" (UniqueName: \"kubernetes.io/projected/e41517b6-7ce4-43a6-89a9-76931457ca50-kube-api-access-5f285\") pod \"cluster-samples-operator-6ddb6c8b94-jmxvd\" (UID: \"e41517b6-7ce4-43a6-89a9-76931457ca50\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jmxvd"
Nov 04 01:02:41.217933 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.217926    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncqm5\" (UniqueName: \"kubernetes.io/projected/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-kube-api-access-ncqm5\") pod \"image-registry-5c5894ddd-j4phf\" (UID: \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\") " pod="openshift-image-registry/image-registry-5c5894ddd-j4phf"
Nov 04 01:02:41.218219 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.218202    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vgx6\" (UniqueName: \"kubernetes.io/projected/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-kube-api-access-8vgx6\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:41.218279 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.218226    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p559g\" (UniqueName: \"kubernetes.io/projected/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-kube-api-access-p559g\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:02:41.218279 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.218268    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9n9lx\" (UniqueName: \"kubernetes.io/projected/b3bd337e-c495-4349-b56d-c16ebe3a9b11-kube-api-access-9n9lx\") pod \"multus-additional-cni-plugins-qlr2s\" (UID: \"b3bd337e-c495-4349-b56d-c16ebe3a9b11\") " pod="openshift-multus/multus-additional-cni-plugins-qlr2s"
Nov 04 01:02:41.300413 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.300374    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f68cb5a-3e22-4dd6-98d0-f28988ea86a2-metrics-tls\") pod \"dns-default-klq54\" (UID: \"2f68cb5a-3e22-4dd6-98d0-f28988ea86a2\") " pod="openshift-dns/dns-default-klq54"
Nov 04 01:02:41.300507 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.300419    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-host-run-k8s-cni-cncf-io\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:02:41.300507 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.300439    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-host-var-lib-cni-multus\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:02:41.300507 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.300462    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sg4cw\" (UID: \"ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774\") " pod="openshift-insights/insights-runtime-extractor-sg4cw"
Nov 04 01:02:41.300507 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.300485    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4965a588-c16b-488b-82c7-b53bb390f7d0-etc-selinux\") pod \"aws-ebs-csi-driver-node-tt2x6\" (UID: \"4965a588-c16b-488b-82c7-b53bb390f7d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt2x6"
Nov 04 01:02:41.300507 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.300498    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-host-var-lib-cni-multus\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:02:41.300713 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.300510    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-lib-modules\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:02:41.300713 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:02:41.300518    5068 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 04 01:02:41.300713 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.300486    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-host-run-k8s-cni-cncf-io\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:02:41.300713 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.300564    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4965a588-c16b-488b-82c7-b53bb390f7d0-etc-selinux\") pod \"aws-ebs-csi-driver-node-tt2x6\" (UID: \"4965a588-c16b-488b-82c7-b53bb390f7d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt2x6"
Nov 04 01:02:41.300713 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.300587    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b3bd337e-c495-4349-b56d-c16ebe3a9b11-cnibin\") pod \"multus-additional-cni-plugins-qlr2s\" (UID: \"b3bd337e-c495-4349-b56d-c16ebe3a9b11\") " pod="openshift-multus/multus-additional-cni-plugins-qlr2s"
Nov 04 01:02:41.300713 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:02:41.300602    5068 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-kube-rbac-proxy-cm podName:ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:41.800587829 +0000 UTC m=+1.847148615 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-kube-rbac-proxy-cm") pod "insights-runtime-extractor-sg4cw" (UID: "ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774") : configmap "kube-rbac-proxy" not found
Nov 04 01:02:41.300713 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.300636    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-lib-modules\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:02:41.300713 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.300645    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b3bd337e-c495-4349-b56d-c16ebe3a9b11-cnibin\") pod \"multus-additional-cni-plugins-qlr2s\" (UID: \"b3bd337e-c495-4349-b56d-c16ebe3a9b11\") " pod="openshift-multus/multus-additional-cni-plugins-qlr2s"
Nov 04 01:02:41.300713 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.300652    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-host-run-ovn-kubernetes\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:41.300713 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.300674    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-host-cni-netd\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:41.300713 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.300689    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-etc-kubernetes\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:02:41.300713 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.300691    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-host-run-ovn-kubernetes\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:41.300713 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.300705    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-sys\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:02:41.301276 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.300729    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-registry-tls\") pod \"image-registry-5c5894ddd-j4phf\" (UID: \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\") " pod="openshift-image-registry/image-registry-5c5894ddd-j4phf"
Nov 04 01:02:41.301276 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.300735    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-host-cni-netd\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:41.301276 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.300748    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-etc-sysconfig\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:02:41.301276 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.300764    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-run-ovn\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:41.301276 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.300773    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-etc-kubernetes\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:02:41.301276 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.300763    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-sys\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:02:41.301276 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.300797    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6be44550-8625-4296-aeb9-da833aaa4b2f-dbus\") pod \"global-pull-secret-syncer-gnrzw\" (UID: \"6be44550-8625-4296-aeb9-da833aaa4b2f\") " pod="kube-system/global-pull-secret-syncer-gnrzw"
Nov 04 01:02:41.301276 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.300773    5068 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 04 01:02:41.301276 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.300807    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-etc-sysconfig\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:02:41.301276 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.300857    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-run-ovn\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:41.301276 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:02:41.300978    5068 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 01:02:41.301276 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:02:41.300993    5068 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5c5894ddd-j4phf: secret "image-registry-tls" not found
Nov 04 01:02:41.301276 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.301000    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-var-lib-openvswitch\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:41.301276 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.301021    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae02b148-c5ba-4db0-b817-9be4a488c708-service-ca-bundle\") pod \"router-default-5569c8658c-v7txl\" (UID: \"ae02b148-c5ba-4db0-b817-9be4a488c708\") " pod="openshift-ingress/router-default-5569c8658c-v7txl"
Nov 04 01:02:41.301276 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.301038    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-cnibin\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:02:41.301276 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.301003    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6be44550-8625-4296-aeb9-da833aaa4b2f-dbus\") pod \"global-pull-secret-syncer-gnrzw\" (UID: \"6be44550-8625-4296-aeb9-da833aaa4b2f\") " pod="kube-system/global-pull-secret-syncer-gnrzw"
Nov 04 01:02:41.301276 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:02:41.301054    5068 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-registry-tls podName:0eedbf45-4e7d-47b7-806f-a6abf9aa311f nodeName:}" failed. No retries permitted until 2026-11-04 01:02:41.801037741 +0000 UTC m=+1.847598523 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-registry-tls") pod "image-registry-5c5894ddd-j4phf" (UID: "0eedbf45-4e7d-47b7-806f-a6abf9aa311f") : secret "image-registry-tls" not found
Nov 04 01:02:41.301276 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.301073    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-var-lib-openvswitch\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:41.301276 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:02:41.301132    5068 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ae02b148-c5ba-4db0-b817-9be4a488c708-service-ca-bundle podName:ae02b148-c5ba-4db0-b817-9be4a488c708 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:41.801117606 +0000 UTC m=+1.847678375 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ae02b148-c5ba-4db0-b817-9be4a488c708-service-ca-bundle") pod "router-default-5569c8658c-v7txl" (UID: "ae02b148-c5ba-4db0-b817-9be4a488c708") : configmap references non-existent config key: service-ca.crt
Nov 04 01:02:41.301276 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.301129    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-cnibin\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:02:41.301276 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.301158    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4965a588-c16b-488b-82c7-b53bb390f7d0-socket-dir\") pod \"aws-ebs-csi-driver-node-tt2x6\" (UID: \"4965a588-c16b-488b-82c7-b53bb390f7d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt2x6"
Nov 04 01:02:41.301276 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.301185    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-log-socket\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:41.301276 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.301220    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3a53d2f9-8ce8-4ae5-9974-a8c865065ff3-host-slash\") pod \"iptables-alerter-l7n65\" (UID: \"3a53d2f9-8ce8-4ae5-9974-a8c865065ff3\") " pod="openshift-network-operator/iptables-alerter-l7n65"
Nov 04 01:02:41.301276 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.301240    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-log-socket\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:41.301276 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.301248    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e41517b6-7ce4-43a6-89a9-76931457ca50-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-jmxvd\" (UID: \"e41517b6-7ce4-43a6-89a9-76931457ca50\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jmxvd"
Nov 04 01:02:41.301276 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.301252    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4965a588-c16b-488b-82c7-b53bb390f7d0-socket-dir\") pod \"aws-ebs-csi-driver-node-tt2x6\" (UID: \"4965a588-c16b-488b-82c7-b53bb390f7d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt2x6"
Nov 04 01:02:41.301276 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.301273    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4965a588-c16b-488b-82c7-b53bb390f7d0-registration-dir\") pod \"aws-ebs-csi-driver-node-tt2x6\" (UID: \"4965a588-c16b-488b-82c7-b53bb390f7d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt2x6"
Nov 04 01:02:41.301276 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.301277    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3a53d2f9-8ce8-4ae5-9974-a8c865065ff3-host-slash\") pod \"iptables-alerter-l7n65\" (UID: \"3a53d2f9-8ce8-4ae5-9974-a8c865065ff3\") " pod="openshift-network-operator/iptables-alerter-l7n65"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.301298    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sg4cw\" (UID: \"ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774\") " pod="openshift-insights/insights-runtime-extractor-sg4cw"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.301318    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4965a588-c16b-488b-82c7-b53bb390f7d0-registration-dir\") pod \"aws-ebs-csi-driver-node-tt2x6\" (UID: \"4965a588-c16b-488b-82c7-b53bb390f7d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt2x6"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.301321    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-etc-kubernetes\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.301346    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-multus-socket-dir-parent\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.301355    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-etc-kubernetes\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:02:41.301325    5068 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.301373    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b3bd337e-c495-4349-b56d-c16ebe3a9b11-system-cni-dir\") pod \"multus-additional-cni-plugins-qlr2s\" (UID: \"b3bd337e-c495-4349-b56d-c16ebe3a9b11\") " pod="openshift-multus/multus-additional-cni-plugins-qlr2s"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:02:41.301397    5068 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e41517b6-7ce4-43a6-89a9-76931457ca50-samples-operator-tls podName:e41517b6-7ce4-43a6-89a9-76931457ca50 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:41.801389673 +0000 UTC m=+1.847950440 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e41517b6-7ce4-43a6-89a9-76931457ca50-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-jmxvd" (UID: "e41517b6-7ce4-43a6-89a9-76931457ca50") : secret "samples-operator-tls" not found
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:02:41.301357    5068 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.301411    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-node-log\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.301395    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-multus-socket-dir-parent\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.301425    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b3bd337e-c495-4349-b56d-c16ebe3a9b11-system-cni-dir\") pod \"multus-additional-cni-plugins-qlr2s\" (UID: \"b3bd337e-c495-4349-b56d-c16ebe3a9b11\") " pod="openshift-multus/multus-additional-cni-plugins-qlr2s"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:02:41.301445    5068 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-insights-runtime-extractor-tls podName:ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:41.801431739 +0000 UTC m=+1.847992521 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-insights-runtime-extractor-tls") pod "insights-runtime-extractor-sg4cw" (UID: "ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774") : secret "insights-runtime-extractor-tls" not found
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.301435    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.301468    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.301469    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-node-log\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.301473    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-multus-conf-dir\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.301502    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-multus-conf-dir\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.301541    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-host-var-lib-kubelet\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.301567    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-host-kubelet\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.301572    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-host-var-lib-kubelet\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.301591    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-host-cni-bin\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.301629    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-host-kubelet\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.301633    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7b044258-a241-4bd2-a6eb-9ed586a0f2cc-host\") pod \"node-ca-fvjq7\" (UID: \"7b044258-a241-4bd2-a6eb-9ed586a0f2cc\") " pod="openshift-image-registry/node-ca-fvjq7"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.301653    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-host-cni-bin\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.301666    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7b044258-a241-4bd2-a6eb-9ed586a0f2cc-host\") pod \"node-ca-fvjq7\" (UID: \"7b044258-a241-4bd2-a6eb-9ed586a0f2cc\") " pod="openshift-image-registry/node-ca-fvjq7"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.301667    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4965a588-c16b-488b-82c7-b53bb390f7d0-sys-fs\") pod \"aws-ebs-csi-driver-node-tt2x6\" (UID: \"4965a588-c16b-488b-82c7-b53bb390f7d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt2x6"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.301699    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d6a4c158-e115-4d2f-af0d-1d9249dc7384-hosts-file\") pod \"node-resolver-8gvq8\" (UID: \"d6a4c158-e115-4d2f-af0d-1d9249dc7384\") " pod="openshift-dns/node-resolver-8gvq8"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.301713    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4965a588-c16b-488b-82c7-b53bb390f7d0-sys-fs\") pod \"aws-ebs-csi-driver-node-tt2x6\" (UID: \"4965a588-c16b-488b-82c7-b53bb390f7d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt2x6"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.301725    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a409c58b-b4eb-428f-9177-a09b8a229ec7-metrics-certs\") pod \"network-metrics-daemon-rx7gq\" (UID: \"a409c58b-b4eb-428f-9177-a09b8a229ec7\") " pod="openshift-multus/network-metrics-daemon-rx7gq"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.301751    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-multus-cni-dir\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.301757    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d6a4c158-e115-4d2f-af0d-1d9249dc7384-hosts-file\") pod \"node-resolver-8gvq8\" (UID: \"d6a4c158-e115-4d2f-af0d-1d9249dc7384\") " pod="openshift-dns/node-resolver-8gvq8"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.301776    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-etc-sysctl-conf\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:02:41.301794    5068 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.301802    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b3bd337e-c495-4349-b56d-c16ebe3a9b11-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qlr2s\" (UID: \"b3bd337e-c495-4349-b56d-c16ebe3a9b11\") " pod="openshift-multus/multus-additional-cni-plugins-qlr2s"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.301809    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-multus-cni-dir\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.301828    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-host-var-lib-cni-bin\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:02:41.301838    5068 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a409c58b-b4eb-428f-9177-a09b8a229ec7-metrics-certs podName:a409c58b-b4eb-428f-9177-a09b8a229ec7 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:41.801824943 +0000 UTC m=+1.848385711 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a409c58b-b4eb-428f-9177-a09b8a229ec7-metrics-certs") pod "network-metrics-daemon-rx7gq" (UID: "a409c58b-b4eb-428f-9177-a09b8a229ec7") : secret "metrics-daemon-secret" not found
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.301857    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-host-var-lib-cni-bin\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.301875    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-host-slash\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.301890    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b3bd337e-c495-4349-b56d-c16ebe3a9b11-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qlr2s\" (UID: \"b3bd337e-c495-4349-b56d-c16ebe3a9b11\") " pod="openshift-multus/multus-additional-cni-plugins-qlr2s"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.301905    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-crio-socket\") pod \"insights-runtime-extractor-sg4cw\" (UID: \"ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774\") " pod="openshift-insights/insights-runtime-extractor-sg4cw"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.301928    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-host-slash\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.301908    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-etc-sysctl-conf\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.301942    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3739befd-6689-4197-b793-6f5b4b796880-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-27rmf\" (UID: \"3739befd-6689-4197-b793-6f5b4b796880\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-27rmf"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.301969    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-crio-socket\") pod \"insights-runtime-extractor-sg4cw\" (UID: \"ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774\") " pod="openshift-insights/insights-runtime-extractor-sg4cw"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.301974    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6be44550-8625-4296-aeb9-da833aaa4b2f-kubelet-config\") pod \"global-pull-secret-syncer-gnrzw\" (UID: \"6be44550-8625-4296-aeb9-da833aaa4b2f\") " pod="kube-system/global-pull-secret-syncer-gnrzw"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.302006    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-hostroot\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.302024    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6be44550-8625-4296-aeb9-da833aaa4b2f-kubelet-config\") pod \"global-pull-secret-syncer-gnrzw\" (UID: \"6be44550-8625-4296-aeb9-da833aaa4b2f\") " pod="kube-system/global-pull-secret-syncer-gnrzw"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.302033    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-host-run-netns\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.302062    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae02b148-c5ba-4db0-b817-9be4a488c708-metrics-certs\") pod \"router-default-5569c8658c-v7txl\" (UID: \"ae02b148-c5ba-4db0-b817-9be4a488c708\") " pod="openshift-ingress/router-default-5569c8658c-v7txl"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.302070    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-host-run-netns\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:02:41.302096    5068 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.302086    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-etc-modprobe-d\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.302112    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-hostroot\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.302128    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-var-lib-kubelet\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.302155    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-etc-systemd\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:02:41.302172    5068 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae02b148-c5ba-4db0-b817-9be4a488c708-metrics-certs podName:ae02b148-c5ba-4db0-b817-9be4a488c708 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:41.802158322 +0000 UTC m=+1.848719091 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ae02b148-c5ba-4db0-b817-9be4a488c708-metrics-certs") pod "router-default-5569c8658c-v7txl" (UID: "ae02b148-c5ba-4db0-b817-9be4a488c708") : secret "router-metrics-certs-default" not found
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.302195    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-etc-systemd\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.302202    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4965a588-c16b-488b-82c7-b53bb390f7d0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tt2x6\" (UID: \"4965a588-c16b-488b-82c7-b53bb390f7d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt2x6"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.302208    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-etc-modprobe-d\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.302210    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-var-lib-kubelet\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.302227    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-systemd-units\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.302261    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-run-systemd\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.302262    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4965a588-c16b-488b-82c7-b53bb390f7d0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tt2x6\" (UID: \"4965a588-c16b-488b-82c7-b53bb390f7d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt2x6"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.302287    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-etc-openvswitch\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.302305    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/20f4989c-4fe8-49d2-9430-5030a4274b60-registry-tls\") pod \"image-registry-6574d8498f-74vmj\" (UID: \"20f4989c-4fe8-49d2-9430-5030a4274b60\") " pod="openshift-image-registry/image-registry-6574d8498f-74vmj"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.302312    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-etc-openvswitch\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.302302    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-systemd-units\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.302302    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-run-systemd\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.302334    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/28a8304b-9226-422d-9044-b50678d52971-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-6vxbv\" (UID: \"28a8304b-9226-422d-9044-b50678d52971\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6vxbv"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.302357    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-host\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:02:41.302368    5068 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.302372    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4965a588-c16b-488b-82c7-b53bb390f7d0-device-dir\") pod \"aws-ebs-csi-driver-node-tt2x6\" (UID: \"4965a588-c16b-488b-82c7-b53bb390f7d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt2x6"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:02:41.302380    5068 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6574d8498f-74vmj: secret "image-registry-tls" not found
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.302394    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-run\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:02:41.302394    5068 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.302410    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-host\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:02:41.302417    5068 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/20f4989c-4fe8-49d2-9430-5030a4274b60-registry-tls podName:20f4989c-4fe8-49d2-9430-5030a4274b60 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:41.802404368 +0000 UTC m=+1.848965155 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/20f4989c-4fe8-49d2-9430-5030a4274b60-registry-tls") pod "image-registry-6574d8498f-74vmj" (UID: "20f4989c-4fe8-49d2-9430-5030a4274b60") : secret "image-registry-tls" not found
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.302430    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4965a588-c16b-488b-82c7-b53bb390f7d0-device-dir\") pod \"aws-ebs-csi-driver-node-tt2x6\" (UID: \"4965a588-c16b-488b-82c7-b53bb390f7d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tt2x6"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.302450    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-run\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.302445    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-host-run-netns\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.302467    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-host-run-netns\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:02:41.302419 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:02:41.302479    5068 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28a8304b-9226-422d-9044-b50678d52971-cluster-monitoring-operator-tls podName:28a8304b-9226-422d-9044-b50678d52971 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:41.802467495 +0000 UTC m=+1.849028262 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/28a8304b-9226-422d-9044-b50678d52971-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-6vxbv" (UID: "28a8304b-9226-422d-9044-b50678d52971") : secret "cluster-monitoring-operator-tls" not found
Nov 04 01:02:41.304962 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.302501    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-system-cni-dir\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:02:41.304962 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.302524    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-etc-sysctl-d\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:02:41.304962 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.302549    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-host-run-multus-certs\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:02:41.304962 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.302557    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-system-cni-dir\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:02:41.304962 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.302572    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-run-openvswitch\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:41.304962 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.302597    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/80a681c1-e74e-4887-a56d-6be313103581-host-run-multus-certs\") pod \"multus-jfd2m\" (UID: \"80a681c1-e74e-4887-a56d-6be313103581\") " pod="openshift-multus/multus-jfd2m"
Nov 04 01:02:41.304962 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.302662    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd-etc-sysctl-d\") pod \"tuned-pwv64\" (UID: \"6f70c6f7-47f0-46fc-b7c6-88a780ae6ddd\") " pod="openshift-cluster-node-tuning-operator/tuned-pwv64"
Nov 04 01:02:41.304962 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.302668    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/abaec7c0-0005-4bdd-9d8f-0ef0998629a6-run-openvswitch\") pod \"ovnkube-node-sz7gg\" (UID: \"abaec7c0-0005-4bdd-9d8f-0ef0998629a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:41.304962 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.303017    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f68cb5a-3e22-4dd6-98d0-f28988ea86a2-metrics-tls\") pod \"dns-default-klq54\" (UID: \"2f68cb5a-3e22-4dd6-98d0-f28988ea86a2\") " pod="openshift-dns/dns-default-klq54"
Nov 04 01:02:41.304962 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.304452    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3739befd-6689-4197-b793-6f5b4b796880-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-27rmf\" (UID: \"3739befd-6689-4197-b793-6f5b4b796880\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-27rmf"
Nov 04 01:02:41.380060 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.379993    5068 scope.go:122] "RemoveContainer" containerID="9d9855e9203df085f919c24b870c736a1b366e38fdfa9b8aea39bcaedc7f5ded"
Nov 04 01:02:41.382082 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.382062    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-27rmf"
Nov 04 01:02:41.392868 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.392848    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-klq54"
Nov 04 01:02:41.700127 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.700061    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-27rmf"]
Nov 04 01:02:41.702881 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:41.702854    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3739befd_6689_4197_b793_6f5b4b796880.slice/crio-2f52f9a32f85b3e7b97dab6ed49a62b11b8b804f1710f8c5bb3f85c14646df9e WatchSource:0}: Error finding container 2f52f9a32f85b3e7b97dab6ed49a62b11b8b804f1710f8c5bb3f85c14646df9e: Status 404 returned error can't find the container with id 2f52f9a32f85b3e7b97dab6ed49a62b11b8b804f1710f8c5bb3f85c14646df9e
Nov 04 01:02:41.704982 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.704963    5068 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 01:02:41.738794 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.738772    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-klq54"]
Nov 04 01:02:41.741546 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:41.741525    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f68cb5a_3e22_4dd6_98d0_f28988ea86a2.slice/crio-d85893a22330628f6f70977d02da1607a333cd45ac74cbcebc435b6ab73dbb9c WatchSource:0}: Error finding container d85893a22330628f6f70977d02da1607a333cd45ac74cbcebc435b6ab73dbb9c: Status 404 returned error can't find the container with id d85893a22330628f6f70977d02da1607a333cd45ac74cbcebc435b6ab73dbb9c
Nov 04 01:02:41.808914 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.808886    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a409c58b-b4eb-428f-9177-a09b8a229ec7-metrics-certs\") pod \"network-metrics-daemon-rx7gq\" (UID: \"a409c58b-b4eb-428f-9177-a09b8a229ec7\") " pod="openshift-multus/network-metrics-daemon-rx7gq"
Nov 04 01:02:41.809033 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.808928    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae02b148-c5ba-4db0-b817-9be4a488c708-metrics-certs\") pod \"router-default-5569c8658c-v7txl\" (UID: \"ae02b148-c5ba-4db0-b817-9be4a488c708\") " pod="openshift-ingress/router-default-5569c8658c-v7txl"
Nov 04 01:02:41.809033 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.808950    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/20f4989c-4fe8-49d2-9430-5030a4274b60-registry-tls\") pod \"image-registry-6574d8498f-74vmj\" (UID: \"20f4989c-4fe8-49d2-9430-5030a4274b60\") " pod="openshift-image-registry/image-registry-6574d8498f-74vmj"
Nov 04 01:02:41.809033 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.808969    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/28a8304b-9226-422d-9044-b50678d52971-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-6vxbv\" (UID: \"28a8304b-9226-422d-9044-b50678d52971\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6vxbv"
Nov 04 01:02:41.809033 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.809005    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sg4cw\" (UID: \"ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774\") " pod="openshift-insights/insights-runtime-extractor-sg4cw"
Nov 04 01:02:41.809194 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:02:41.809051    5068 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 04 01:02:41.809194 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:02:41.809103    5068 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae02b148-c5ba-4db0-b817-9be4a488c708-metrics-certs podName:ae02b148-c5ba-4db0-b817-9be4a488c708 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:42.809090242 +0000 UTC m=+2.855651010 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ae02b148-c5ba-4db0-b817-9be4a488c708-metrics-certs") pod "router-default-5569c8658c-v7txl" (UID: "ae02b148-c5ba-4db0-b817-9be4a488c708") : secret "router-metrics-certs-default" not found
Nov 04 01:02:41.809194 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:02:41.809109    5068 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 04 01:02:41.809194 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.809038    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-registry-tls\") pod \"image-registry-5c5894ddd-j4phf\" (UID: \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\") " pod="openshift-image-registry/image-registry-5c5894ddd-j4phf"
Nov 04 01:02:41.809194 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:02:41.809049    5068 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 04 01:02:41.809194 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:02:41.809161    5068 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28a8304b-9226-422d-9044-b50678d52971-cluster-monitoring-operator-tls podName:28a8304b-9226-422d-9044-b50678d52971 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:42.809146533 +0000 UTC m=+2.855707317 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/28a8304b-9226-422d-9044-b50678d52971-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-6vxbv" (UID: "28a8304b-9226-422d-9044-b50678d52971") : secret "cluster-monitoring-operator-tls" not found
Nov 04 01:02:41.809529 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.809204    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae02b148-c5ba-4db0-b817-9be4a488c708-service-ca-bundle\") pod \"router-default-5569c8658c-v7txl\" (UID: \"ae02b148-c5ba-4db0-b817-9be4a488c708\") " pod="openshift-ingress/router-default-5569c8658c-v7txl"
Nov 04 01:02:41.809529 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.809264    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e41517b6-7ce4-43a6-89a9-76931457ca50-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-jmxvd\" (UID: \"e41517b6-7ce4-43a6-89a9-76931457ca50\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jmxvd"
Nov 04 01:02:41.809529 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:02:41.809290    5068 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ae02b148-c5ba-4db0-b817-9be4a488c708-service-ca-bundle podName:ae02b148-c5ba-4db0-b817-9be4a488c708 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:42.809279515 +0000 UTC m=+2.855840297 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ae02b148-c5ba-4db0-b817-9be4a488c708-service-ca-bundle") pod "router-default-5569c8658c-v7txl" (UID: "ae02b148-c5ba-4db0-b817-9be4a488c708") : configmap references non-existent config key: service-ca.crt
Nov 04 01:02:41.809529 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:02:41.809321    5068 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a409c58b-b4eb-428f-9177-a09b8a229ec7-metrics-certs podName:a409c58b-b4eb-428f-9177-a09b8a229ec7 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:42.809305075 +0000 UTC m=+2.855865847 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a409c58b-b4eb-428f-9177-a09b8a229ec7-metrics-certs") pod "network-metrics-daemon-rx7gq" (UID: "a409c58b-b4eb-428f-9177-a09b8a229ec7") : secret "metrics-daemon-secret" not found
Nov 04 01:02:41.809529 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:02:41.809396    5068 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 04 01:02:41.809529 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:02:41.809448    5068 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e41517b6-7ce4-43a6-89a9-76931457ca50-samples-operator-tls podName:e41517b6-7ce4-43a6-89a9-76931457ca50 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:42.809433078 +0000 UTC m=+2.855993855 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e41517b6-7ce4-43a6-89a9-76931457ca50-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-jmxvd" (UID: "e41517b6-7ce4-43a6-89a9-76931457ca50") : secret "samples-operator-tls" not found
Nov 04 01:02:41.809529 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:02:41.809482    5068 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 01:02:41.809819 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:02:41.809546    5068 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-insights-runtime-extractor-tls podName:ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:42.809519233 +0000 UTC m=+2.856080002 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-insights-runtime-extractor-tls") pod "insights-runtime-extractor-sg4cw" (UID: "ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774") : secret "insights-runtime-extractor-tls" not found
Nov 04 01:02:41.809819 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.809582    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sg4cw\" (UID: \"ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774\") " pod="openshift-insights/insights-runtime-extractor-sg4cw"
Nov 04 01:02:41.809928 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.809909    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sg4cw\" (UID: \"ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774\") " pod="openshift-insights/insights-runtime-extractor-sg4cw"
Nov 04 01:02:41.811342 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.811320    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-registry-tls\") pod \"image-registry-5c5894ddd-j4phf\" (UID: \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\") " pod="openshift-image-registry/image-registry-5c5894ddd-j4phf"
Nov 04 01:02:41.811432 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.811398    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/20f4989c-4fe8-49d2-9430-5030a4274b60-registry-tls\") pod \"image-registry-6574d8498f-74vmj\" (UID: \"20f4989c-4fe8-49d2-9430-5030a4274b60\") " pod="openshift-image-registry/image-registry-6574d8498f-74vmj"
Nov 04 01:02:41.980369 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.980312    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6574d8498f-74vmj"
Nov 04 01:02:41.991719 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:41.991692    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5c5894ddd-j4phf"
Nov 04 01:02:42.168774 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:42.168748    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tr8b2_21d1d23b-7b6e-4ab5-9593-cf7784fffea2/console-operator/2.log"
Nov 04 01:02:42.169218 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:42.169162    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tr8b2_21d1d23b-7b6e-4ab5-9593-cf7784fffea2/console-operator/1.log"
Nov 04 01:02:42.169218 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:42.169191    5068 generic.go:356] "Generic (PLEG): container finished" podID="21d1d23b-7b6e-4ab5-9593-cf7784fffea2" containerID="3f616d7b6159eea672f9e3bf933cf09e5c6002d3832bc7830536cf94f89f3518" exitCode=255
Nov 04 01:02:42.169294 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:42.169264    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-tr8b2" event={"ID":"21d1d23b-7b6e-4ab5-9593-cf7784fffea2","Type":"ContainerDied","Data":"3f616d7b6159eea672f9e3bf933cf09e5c6002d3832bc7830536cf94f89f3518"}
Nov 04 01:02:42.169342 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:42.169330    5068 scope.go:122] "RemoveContainer" containerID="9d9855e9203df085f919c24b870c736a1b366e38fdfa9b8aea39bcaedc7f5ded"
Nov 04 01:02:42.169685 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:42.169650    5068 scope.go:122] "RemoveContainer" containerID="3f616d7b6159eea672f9e3bf933cf09e5c6002d3832bc7830536cf94f89f3518"
Nov 04 01:02:42.169903 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:02:42.169884    5068 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-tr8b2_openshift-console-operator(21d1d23b-7b6e-4ab5-9593-cf7784fffea2)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-tr8b2" podUID="21d1d23b-7b6e-4ab5-9593-cf7784fffea2"
Nov 04 01:02:42.170388 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:42.170335    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-klq54" event={"ID":"2f68cb5a-3e22-4dd6-98d0-f28988ea86a2","Type":"ContainerStarted","Data":"d85893a22330628f6f70977d02da1607a333cd45ac74cbcebc435b6ab73dbb9c"}
Nov 04 01:02:42.171510 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:42.171490    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-27rmf" event={"ID":"3739befd-6689-4197-b793-6f5b4b796880","Type":"ContainerStarted","Data":"2f52f9a32f85b3e7b97dab6ed49a62b11b8b804f1710f8c5bb3f85c14646df9e"}
Nov 04 01:02:42.283432 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:42.283401    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6574d8498f-74vmj"]
Nov 04 01:02:42.286373 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:42.286348    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20f4989c_4fe8_49d2_9430_5030a4274b60.slice/crio-5d8bd94e84de17d7d076b48b189f850d6f0bf0468bb23780f77eb1f4534d57f7 WatchSource:0}: Error finding container 5d8bd94e84de17d7d076b48b189f850d6f0bf0468bb23780f77eb1f4534d57f7: Status 404 returned error can't find the container with id 5d8bd94e84de17d7d076b48b189f850d6f0bf0468bb23780f77eb1f4534d57f7
Nov 04 01:02:42.307134 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:42.307108    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5c5894ddd-j4phf"]
Nov 04 01:02:42.318881 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:42.318855    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0eedbf45_4e7d_47b7_806f_a6abf9aa311f.slice/crio-731654e624407a53caa45304528a500db108bdd6f60024ca9993da7c2ee7a31b WatchSource:0}: Error finding container 731654e624407a53caa45304528a500db108bdd6f60024ca9993da7c2ee7a31b: Status 404 returned error can't find the container with id 731654e624407a53caa45304528a500db108bdd6f60024ca9993da7c2ee7a31b
Nov 04 01:02:42.835478 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:42.835431    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae02b148-c5ba-4db0-b817-9be4a488c708-service-ca-bundle\") pod \"router-default-5569c8658c-v7txl\" (UID: \"ae02b148-c5ba-4db0-b817-9be4a488c708\") " pod="openshift-ingress/router-default-5569c8658c-v7txl"
Nov 04 01:02:42.835667 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:42.835490    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e41517b6-7ce4-43a6-89a9-76931457ca50-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-jmxvd\" (UID: \"e41517b6-7ce4-43a6-89a9-76931457ca50\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jmxvd"
Nov 04 01:02:42.835667 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:42.835522    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sg4cw\" (UID: \"ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774\") " pod="openshift-insights/insights-runtime-extractor-sg4cw"
Nov 04 01:02:42.835667 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:42.835568    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a409c58b-b4eb-428f-9177-a09b8a229ec7-metrics-certs\") pod \"network-metrics-daemon-rx7gq\" (UID: \"a409c58b-b4eb-428f-9177-a09b8a229ec7\") " pod="openshift-multus/network-metrics-daemon-rx7gq"
Nov 04 01:02:42.835667 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:42.835634    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae02b148-c5ba-4db0-b817-9be4a488c708-metrics-certs\") pod \"router-default-5569c8658c-v7txl\" (UID: \"ae02b148-c5ba-4db0-b817-9be4a488c708\") " pod="openshift-ingress/router-default-5569c8658c-v7txl"
Nov 04 01:02:42.835921 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:42.835672    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/28a8304b-9226-422d-9044-b50678d52971-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-6vxbv\" (UID: \"28a8304b-9226-422d-9044-b50678d52971\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6vxbv"
Nov 04 01:02:42.835921 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:02:42.835815    5068 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 04 01:02:42.835921 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:02:42.835876    5068 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28a8304b-9226-422d-9044-b50678d52971-cluster-monitoring-operator-tls podName:28a8304b-9226-422d-9044-b50678d52971 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:44.835858462 +0000 UTC m=+4.882419246 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/28a8304b-9226-422d-9044-b50678d52971-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-6vxbv" (UID: "28a8304b-9226-422d-9044-b50678d52971") : secret "cluster-monitoring-operator-tls" not found
Nov 04 01:02:42.836320 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:02:42.836300    5068 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ae02b148-c5ba-4db0-b817-9be4a488c708-service-ca-bundle podName:ae02b148-c5ba-4db0-b817-9be4a488c708 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:44.836282429 +0000 UTC m=+4.882843211 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ae02b148-c5ba-4db0-b817-9be4a488c708-service-ca-bundle") pod "router-default-5569c8658c-v7txl" (UID: "ae02b148-c5ba-4db0-b817-9be4a488c708") : configmap references non-existent config key: service-ca.crt
Nov 04 01:02:42.837161 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:02:42.837139    5068 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 01:02:42.837250 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:02:42.837200    5068 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-insights-runtime-extractor-tls podName:ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:44.837184167 +0000 UTC m=+4.883744950 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-insights-runtime-extractor-tls") pod "insights-runtime-extractor-sg4cw" (UID: "ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774") : secret "insights-runtime-extractor-tls" not found
Nov 04 01:02:42.837327 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:02:42.837265    5068 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 04 01:02:42.837327 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:02:42.837298    5068 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae02b148-c5ba-4db0-b817-9be4a488c708-metrics-certs podName:ae02b148-c5ba-4db0-b817-9be4a488c708 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:44.837288468 +0000 UTC m=+4.883849248 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ae02b148-c5ba-4db0-b817-9be4a488c708-metrics-certs") pod "router-default-5569c8658c-v7txl" (UID: "ae02b148-c5ba-4db0-b817-9be4a488c708") : secret "router-metrics-certs-default" not found
Nov 04 01:02:42.855387 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:42.840819    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a409c58b-b4eb-428f-9177-a09b8a229ec7-metrics-certs\") pod \"network-metrics-daemon-rx7gq\" (UID: \"a409c58b-b4eb-428f-9177-a09b8a229ec7\") " pod="openshift-multus/network-metrics-daemon-rx7gq"
Nov 04 01:02:42.855387 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:42.854466    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e41517b6-7ce4-43a6-89a9-76931457ca50-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-jmxvd\" (UID: \"e41517b6-7ce4-43a6-89a9-76931457ca50\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jmxvd"
Nov 04 01:02:42.881094 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:42.881060    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jmxvd"
Nov 04 01:02:42.882294 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:42.882269    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx7gq"
Nov 04 01:02:43.117326 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:43.117191    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-tr8b2"
Nov 04 01:02:43.178032 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:43.178000    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tr8b2_21d1d23b-7b6e-4ab5-9593-cf7784fffea2/console-operator/2.log"
Nov 04 01:02:43.178554 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:43.178483    5068 scope.go:122] "RemoveContainer" containerID="3f616d7b6159eea672f9e3bf933cf09e5c6002d3832bc7830536cf94f89f3518"
Nov 04 01:02:43.179096 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:02:43.179051    5068 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-tr8b2_openshift-console-operator(21d1d23b-7b6e-4ab5-9593-cf7784fffea2)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-tr8b2" podUID="21d1d23b-7b6e-4ab5-9593-cf7784fffea2"
Nov 04 01:02:43.180946 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:43.180916    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5c5894ddd-j4phf" event={"ID":"0eedbf45-4e7d-47b7-806f-a6abf9aa311f","Type":"ContainerStarted","Data":"241362af369d65ca558f8de6078270ebcc97edecba2d327df927474c79d903b2"}
Nov 04 01:02:43.181060 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:43.180962    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5c5894ddd-j4phf" event={"ID":"0eedbf45-4e7d-47b7-806f-a6abf9aa311f","Type":"ContainerStarted","Data":"731654e624407a53caa45304528a500db108bdd6f60024ca9993da7c2ee7a31b"}
Nov 04 01:02:43.182737 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:43.182713    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6574d8498f-74vmj" event={"ID":"20f4989c-4fe8-49d2-9430-5030a4274b60","Type":"ContainerStarted","Data":"77af54067f39ec3bd0ed2726ca2ece863a1aac3d452d739a881f4137b4e19809"}
Nov 04 01:02:43.182845 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:43.182758    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6574d8498f-74vmj" event={"ID":"20f4989c-4fe8-49d2-9430-5030a4274b60","Type":"ContainerStarted","Data":"5d8bd94e84de17d7d076b48b189f850d6f0bf0468bb23780f77eb1f4534d57f7"}
Nov 04 01:02:43.215996 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:43.215970    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jmxvd"]
Nov 04 01:02:43.226007 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:43.225981    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rx7gq"]
Nov 04 01:02:43.239892 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:43.239866    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda409c58b_b4eb_428f_9177_a09b8a229ec7.slice/crio-66bb1d8a16d2fb45f190a34bc5839396490e3f276f1162f2c173ec51ac194b3f WatchSource:0}: Error finding container 66bb1d8a16d2fb45f190a34bc5839396490e3f276f1162f2c173ec51ac194b3f: Status 404 returned error can't find the container with id 66bb1d8a16d2fb45f190a34bc5839396490e3f276f1162f2c173ec51ac194b3f
Nov 04 01:02:44.191935 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:44.191004    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-27rmf" event={"ID":"3739befd-6689-4197-b793-6f5b4b796880","Type":"ContainerStarted","Data":"0c74f6cf8391fa0245afc0fce39db624bc5e881b27422dc632ed6e5c3965cacd"}
Nov 04 01:02:44.193051 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:44.193015    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jmxvd" event={"ID":"e41517b6-7ce4-43a6-89a9-76931457ca50","Type":"ContainerStarted","Data":"7ccbcf08b75f6498fb7f51a8b7d56807daa3f96dbb77802371799e719c201892"}
Nov 04 01:02:44.194913 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:44.194385    5068 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 04 01:02:44.194913 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:44.194420    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rx7gq" event={"ID":"a409c58b-b4eb-428f-9177-a09b8a229ec7","Type":"ContainerStarted","Data":"66bb1d8a16d2fb45f190a34bc5839396490e3f276f1162f2c173ec51ac194b3f"}
Nov 04 01:02:44.194913 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:44.194432    5068 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 04 01:02:44.854535 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:44.854501    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sg4cw\" (UID: \"ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774\") " pod="openshift-insights/insights-runtime-extractor-sg4cw"
Nov 04 01:02:44.854732 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:44.854556    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae02b148-c5ba-4db0-b817-9be4a488c708-metrics-certs\") pod \"router-default-5569c8658c-v7txl\" (UID: \"ae02b148-c5ba-4db0-b817-9be4a488c708\") " pod="openshift-ingress/router-default-5569c8658c-v7txl"
Nov 04 01:02:44.854732 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:44.854581    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/28a8304b-9226-422d-9044-b50678d52971-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-6vxbv\" (UID: \"28a8304b-9226-422d-9044-b50678d52971\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6vxbv"
Nov 04 01:02:44.854732 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:02:44.854689    5068 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 04 01:02:44.854732 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:02:44.854691    5068 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 01:02:44.854732 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:02:44.854703    5068 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 04 01:02:44.854915 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:44.854735    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae02b148-c5ba-4db0-b817-9be4a488c708-service-ca-bundle\") pod \"router-default-5569c8658c-v7txl\" (UID: \"ae02b148-c5ba-4db0-b817-9be4a488c708\") " pod="openshift-ingress/router-default-5569c8658c-v7txl"
Nov 04 01:02:44.854915 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:02:44.854744    5068 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28a8304b-9226-422d-9044-b50678d52971-cluster-monitoring-operator-tls podName:28a8304b-9226-422d-9044-b50678d52971 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:48.854729007 +0000 UTC m=+8.901289776 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/28a8304b-9226-422d-9044-b50678d52971-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-6vxbv" (UID: "28a8304b-9226-422d-9044-b50678d52971") : secret "cluster-monitoring-operator-tls" not found
Nov 04 01:02:44.854915 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:02:44.854797    5068 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-insights-runtime-extractor-tls podName:ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:48.854784534 +0000 UTC m=+8.901345305 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-insights-runtime-extractor-tls") pod "insights-runtime-extractor-sg4cw" (UID: "ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774") : secret "insights-runtime-extractor-tls" not found
Nov 04 01:02:44.854915 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:02:44.854815    5068 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae02b148-c5ba-4db0-b817-9be4a488c708-metrics-certs podName:ae02b148-c5ba-4db0-b817-9be4a488c708 nodeName:}" failed. No retries permitted until 2026-11-04 01:02:48.854804 +0000 UTC m=+8.901364784 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ae02b148-c5ba-4db0-b817-9be4a488c708-metrics-certs") pod "router-default-5569c8658c-v7txl" (UID: "ae02b148-c5ba-4db0-b817-9be4a488c708") : secret "router-metrics-certs-default" not found
Nov 04 01:02:44.855381 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:44.855363    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae02b148-c5ba-4db0-b817-9be4a488c708-service-ca-bundle\") pod \"router-default-5569c8658c-v7txl\" (UID: \"ae02b148-c5ba-4db0-b817-9be4a488c708\") " pod="openshift-ingress/router-default-5569c8658c-v7txl"
Nov 04 01:02:45.199981 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:45.199895    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rx7gq" event={"ID":"a409c58b-b4eb-428f-9177-a09b8a229ec7","Type":"ContainerStarted","Data":"71c3eb491f28d42237789a9d01e357b954152bf714261d3de61d09ec094076d2"}
Nov 04 01:02:45.199981 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:45.199958    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rx7gq" event={"ID":"a409c58b-b4eb-428f-9177-a09b8a229ec7","Type":"ContainerStarted","Data":"a27cb94c41ce1c2acf3f0a44ac5897bd157bf6200b6653cbc2fb2f5dad6bb7f4"}
Nov 04 01:02:45.201894 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:45.201865    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-klq54" event={"ID":"2f68cb5a-3e22-4dd6-98d0-f28988ea86a2","Type":"ContainerStarted","Data":"46040d1414cd365036da56ef6ae11357619a0b222f3776099403eeb08c54fdfa"}
Nov 04 01:02:45.202037 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:45.201913    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-klq54" event={"ID":"2f68cb5a-3e22-4dd6-98d0-f28988ea86a2","Type":"ContainerStarted","Data":"09528fc94f8141ad8af8d7f72e2b9891c6a7f3b19ca1886a4f06cf16ef174d3c"}
Nov 04 01:02:45.202037 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:45.201960    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-klq54"
Nov 04 01:02:45.204259 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:45.204237    5068 generic.go:356] "Generic (PLEG): container finished" podID="fc0b0e48-8adf-4d0d-80dd-ca76846f824a" containerID="7beb1d6b3427d0bacd043e31c4976f8d8b9e841992e7bf36be4efd7d2d9fc294" exitCode=0
Nov 04 01:02:45.205010 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:45.204411    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-8jlt4" event={"ID":"fc0b0e48-8adf-4d0d-80dd-ca76846f824a","Type":"ContainerDied","Data":"7beb1d6b3427d0bacd043e31c4976f8d8b9e841992e7bf36be4efd7d2d9fc294"}
Nov 04 01:02:45.205010 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:45.204730    5068 scope.go:122] "RemoveContainer" containerID="7beb1d6b3427d0bacd043e31c4976f8d8b9e841992e7bf36be4efd7d2d9fc294"
Nov 04 01:02:46.209461 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:46.209423    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jmxvd" event={"ID":"e41517b6-7ce4-43a6-89a9-76931457ca50","Type":"ContainerStarted","Data":"855a66eb64aacf7818be87ec250e3394d89aaed4c04cbba8812b9da4b555224d"}
Nov 04 01:02:46.209926 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:46.209480    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jmxvd" event={"ID":"e41517b6-7ce4-43a6-89a9-76931457ca50","Type":"ContainerStarted","Data":"aa8fe9ed7c574bbe257c3b24590dca7d7278355a3cc6420b61dd9234ae8d7efa"}
Nov 04 01:02:46.211288 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:46.211262    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-8jlt4" event={"ID":"fc0b0e48-8adf-4d0d-80dd-ca76846f824a","Type":"ContainerStarted","Data":"cb47035f6a3fce4bd6b794aa40165a5de60df89da0c93947566ee836fdd37d13"}
Nov 04 01:02:47.799678 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:47.799633    5068 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-tr8b2"
Nov 04 01:02:47.800046 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:47.799985    5068 scope.go:122] "RemoveContainer" containerID="3f616d7b6159eea672f9e3bf933cf09e5c6002d3832bc7830536cf94f89f3518"
Nov 04 01:02:47.800165 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:02:47.800148    5068 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-tr8b2_openshift-console-operator(21d1d23b-7b6e-4ab5-9593-cf7784fffea2)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-tr8b2" podUID="21d1d23b-7b6e-4ab5-9593-cf7784fffea2"
Nov 04 01:02:48.193489 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:48.193419    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:48.888828 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:48.888784    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae02b148-c5ba-4db0-b817-9be4a488c708-metrics-certs\") pod \"router-default-5569c8658c-v7txl\" (UID: \"ae02b148-c5ba-4db0-b817-9be4a488c708\") " pod="openshift-ingress/router-default-5569c8658c-v7txl"
Nov 04 01:02:48.888828 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:48.888833    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/28a8304b-9226-422d-9044-b50678d52971-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-6vxbv\" (UID: \"28a8304b-9226-422d-9044-b50678d52971\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6vxbv"
Nov 04 01:02:48.889240 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:48.888877    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sg4cw\" (UID: \"ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774\") " pod="openshift-insights/insights-runtime-extractor-sg4cw"
Nov 04 01:02:48.891189 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:48.891156    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sg4cw\" (UID: \"ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774\") " pod="openshift-insights/insights-runtime-extractor-sg4cw"
Nov 04 01:02:48.891338 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:48.891309    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae02b148-c5ba-4db0-b817-9be4a488c708-metrics-certs\") pod \"router-default-5569c8658c-v7txl\" (UID: \"ae02b148-c5ba-4db0-b817-9be4a488c708\") " pod="openshift-ingress/router-default-5569c8658c-v7txl"
Nov 04 01:02:48.891402 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:48.891355    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/28a8304b-9226-422d-9044-b50678d52971-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-6vxbv\" (UID: \"28a8304b-9226-422d-9044-b50678d52971\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6vxbv"
Nov 04 01:02:48.894203 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:48.894181    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-sg4cw"
Nov 04 01:02:49.165554 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:49.165489    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-sg4cw"]
Nov 04 01:02:49.168783 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:49.168754    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad05b9b1_4ef6_4fec_83fb_b2b10d9d8774.slice/crio-455b3695048eb745883175b00fdaa4a5e8ba42f1b09f652e0fee9cb291952d75 WatchSource:0}: Error finding container 455b3695048eb745883175b00fdaa4a5e8ba42f1b09f652e0fee9cb291952d75: Status 404 returned error can't find the container with id 455b3695048eb745883175b00fdaa4a5e8ba42f1b09f652e0fee9cb291952d75
Nov 04 01:02:49.178711 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:49.178687    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5569c8658c-v7txl"
Nov 04 01:02:49.178834 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:49.178810    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6vxbv"
Nov 04 01:02:49.233109 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:49.233031    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-sg4cw" event={"ID":"ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774","Type":"ContainerStarted","Data":"455b3695048eb745883175b00fdaa4a5e8ba42f1b09f652e0fee9cb291952d75"}
Nov 04 01:02:49.463362 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:49.463296    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-6vxbv"]
Nov 04 01:02:49.466555 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:49.466518    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28a8304b_9226_422d_9044_b50678d52971.slice/crio-3a1c920f91ef6b8cb514284664905645d1d8e724ca4292835aff2aa1c3e54663 WatchSource:0}: Error finding container 3a1c920f91ef6b8cb514284664905645d1d8e724ca4292835aff2aa1c3e54663: Status 404 returned error can't find the container with id 3a1c920f91ef6b8cb514284664905645d1d8e724ca4292835aff2aa1c3e54663
Nov 04 01:02:49.467458 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:49.467432    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5569c8658c-v7txl"]
Nov 04 01:02:49.470963 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:02:49.470939    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae02b148_c5ba_4db0_b817_9be4a488c708.slice/crio-79f03eabf797f0574b9e437502362e544b1f5b897243b1e9a971403a184e8e6a WatchSource:0}: Error finding container 79f03eabf797f0574b9e437502362e544b1f5b897243b1e9a971403a184e8e6a: Status 404 returned error can't find the container with id 79f03eabf797f0574b9e437502362e544b1f5b897243b1e9a971403a184e8e6a
Nov 04 01:02:49.929888 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:49.929857    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5c5894ddd-j4phf"
Nov 04 01:02:50.238097 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:50.238011    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5569c8658c-v7txl" event={"ID":"ae02b148-c5ba-4db0-b817-9be4a488c708","Type":"ContainerStarted","Data":"19d6df427e25534d39f00638a31f08adc6b6486ea50b4fff72003253c17d8076"}
Nov 04 01:02:50.238368 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:50.238326    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5569c8658c-v7txl" event={"ID":"ae02b148-c5ba-4db0-b817-9be4a488c708","Type":"ContainerStarted","Data":"79f03eabf797f0574b9e437502362e544b1f5b897243b1e9a971403a184e8e6a"}
Nov 04 01:02:50.243637 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:50.241021    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-sg4cw" event={"ID":"ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774","Type":"ContainerStarted","Data":"70013c881e2d857b929d3f81d3aa149b29c176f3d0aded846f450a18b79ba610"}
Nov 04 01:02:50.243637 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:50.241142    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-sg4cw" event={"ID":"ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774","Type":"ContainerStarted","Data":"c5e03e78b6f5cffe3a7e766de75148e713d5a892a321da0bb236703efc90c736"}
Nov 04 01:02:50.243637 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:50.242729    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6vxbv" event={"ID":"28a8304b-9226-422d-9044-b50678d52971","Type":"ContainerStarted","Data":"3a1c920f91ef6b8cb514284664905645d1d8e724ca4292835aff2aa1c3e54663"}
Nov 04 01:02:50.763777 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:50.763739    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-wxhng"
Nov 04 01:02:51.178929 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:51.178850    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-5569c8658c-v7txl"
Nov 04 01:02:51.181863 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:51.181837    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-5569c8658c-v7txl"
Nov 04 01:02:51.312210 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:51.311173    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:02:51.312210 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:51.311439    5068 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 04 01:02:51.526195 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:51.525467    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6574d8498f-74vmj"
Nov 04 01:02:52.252769 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:52.252680    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-sg4cw" event={"ID":"ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774","Type":"ContainerStarted","Data":"dd7605b9768e70318574d5bbb3e117638d56211c212d2bbfd95d8e2dbd242da7"}
Nov 04 01:02:52.254224 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:52.254198    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6vxbv" event={"ID":"28a8304b-9226-422d-9044-b50678d52971","Type":"ContainerStarted","Data":"711cbff654e43b582de947252fa62ff708a1c89823c8e10dd2716d1703a22495"}
Nov 04 01:02:52.254360 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:52.254253    5068 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 04 01:02:53.106153 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:53.106126    5068 scope.go:122] "RemoveContainer" containerID="3f616d7b6159eea672f9e3bf933cf09e5c6002d3832bc7830536cf94f89f3518"
Nov 04 01:02:53.260458 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:53.260434    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tr8b2_21d1d23b-7b6e-4ab5-9593-cf7784fffea2/console-operator/2.log"
Nov 04 01:02:53.260844 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:53.260547    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-tr8b2" event={"ID":"21d1d23b-7b6e-4ab5-9593-cf7784fffea2","Type":"ContainerStarted","Data":"373712a885ed41ec6f8283de48031d716bfe3344987f21778d5b2db33a39525f"}
Nov 04 01:02:53.261117 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:53.261089    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-tr8b2"
Nov 04 01:02:53.341626 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:53.341585    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-tr8b2"
Nov 04 01:02:55.213472 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:55.213445    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-klq54"
Nov 04 01:02:56.610570 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:56.610537    5068 patch_prober.go:29] interesting pod/image-registry-6574d8498f-74vmj container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Nov 04 01:02:56.610947 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:56.610594    5068 prober.go:122] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6574d8498f-74vmj" podUID="20f4989c-4fe8-49d2-9430-5030a4274b60" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 01:02:57.301886 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:57.301854    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6574d8498f-74vmj"]
Nov 04 01:02:57.307434 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:57.307403    5068 patch_prober.go:29] interesting pod/image-registry-6574d8498f-74vmj container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Nov 04 01:02:57.307601 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:57.307486    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6574d8498f-74vmj" podUID="20f4989c-4fe8-49d2-9430-5030a4274b60" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 01:02:57.765155 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:57.765117    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-5569c8658c-v7txl"
Nov 04 01:02:57.765533 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:57.765265    5068 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 04 01:02:57.766830 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:57.766804    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-5569c8658c-v7txl"
Nov 04 01:02:59.934298 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:59.934259    5068 patch_prober.go:29] interesting pod/image-registry-5c5894ddd-j4phf container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Nov 04 01:02:59.934700 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:02:59.934315    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-5c5894ddd-j4phf" podUID="0eedbf45-4e7d-47b7-806f-a6abf9aa311f" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 01:03:00.548630 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:00.548573    5068 patch_prober.go:29] interesting pod/image-registry-5c5894ddd-j4phf container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Nov 04 01:03:00.548829 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:00.548642    5068 prober.go:122] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-5c5894ddd-j4phf" podUID="0eedbf45-4e7d-47b7-806f-a6abf9aa311f" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 01:03:06.337858 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:06.337825    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sz7gg"
Nov 04 01:03:07.085189 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:07.085160    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-sjcm7"
Nov 04 01:03:07.306492 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:07.306466    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6574d8498f-74vmj"
Nov 04 01:03:09.934297 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:09.934268    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5c5894ddd-j4phf"
Nov 04 01:03:10.767281 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:10.767253    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-wxhng"
Nov 04 01:03:12.948203 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:12.948154    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-5gf2h"]
Nov 04 01:03:12.985645 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:12.985602    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5gf2h"
Nov 04 01:03:12.996684 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:12.996659    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Nov 04 01:03:12.996806 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:12.996705    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Nov 04 01:03:13.034847 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:13.034816    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Nov 04 01:03:13.034847 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:13.034841    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-t6wds\""
Nov 04 01:03:13.035051 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:13.034846    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Nov 04 01:03:13.091918 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:13.091879    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a623766e-ac81-46ee-8643-cb92eaaf61cb-node-exporter-wtmp\") pod \"node-exporter-5gf2h\" (UID: \"a623766e-ac81-46ee-8643-cb92eaaf61cb\") " pod="openshift-monitoring/node-exporter-5gf2h"
Nov 04 01:03:13.092093 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:13.091930    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a623766e-ac81-46ee-8643-cb92eaaf61cb-metrics-client-ca\") pod \"node-exporter-5gf2h\" (UID: \"a623766e-ac81-46ee-8643-cb92eaaf61cb\") " pod="openshift-monitoring/node-exporter-5gf2h"
Nov 04 01:03:13.092093 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:13.091956    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a623766e-ac81-46ee-8643-cb92eaaf61cb-node-exporter-accelerators-collector-config\") pod \"node-exporter-5gf2h\" (UID: \"a623766e-ac81-46ee-8643-cb92eaaf61cb\") " pod="openshift-monitoring/node-exporter-5gf2h"
Nov 04 01:03:13.092192 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:13.092108    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a623766e-ac81-46ee-8643-cb92eaaf61cb-sys\") pod \"node-exporter-5gf2h\" (UID: \"a623766e-ac81-46ee-8643-cb92eaaf61cb\") " pod="openshift-monitoring/node-exporter-5gf2h"
Nov 04 01:03:13.092192 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:13.092136    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mr788\" (UniqueName: \"kubernetes.io/projected/a623766e-ac81-46ee-8643-cb92eaaf61cb-kube-api-access-mr788\") pod \"node-exporter-5gf2h\" (UID: \"a623766e-ac81-46ee-8643-cb92eaaf61cb\") " pod="openshift-monitoring/node-exporter-5gf2h"
Nov 04 01:03:13.092192 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:13.092161    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a623766e-ac81-46ee-8643-cb92eaaf61cb-node-exporter-textfile\") pod \"node-exporter-5gf2h\" (UID: \"a623766e-ac81-46ee-8643-cb92eaaf61cb\") " pod="openshift-monitoring/node-exporter-5gf2h"
Nov 04 01:03:13.092344 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:13.092252    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a623766e-ac81-46ee-8643-cb92eaaf61cb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5gf2h\" (UID: \"a623766e-ac81-46ee-8643-cb92eaaf61cb\") " pod="openshift-monitoring/node-exporter-5gf2h"
Nov 04 01:03:13.092344 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:13.092321    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a623766e-ac81-46ee-8643-cb92eaaf61cb-node-exporter-tls\") pod \"node-exporter-5gf2h\" (UID: \"a623766e-ac81-46ee-8643-cb92eaaf61cb\") " pod="openshift-monitoring/node-exporter-5gf2h"
Nov 04 01:03:13.092503 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:13.092371    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a623766e-ac81-46ee-8643-cb92eaaf61cb-root\") pod \"node-exporter-5gf2h\" (UID: \"a623766e-ac81-46ee-8643-cb92eaaf61cb\") " pod="openshift-monitoring/node-exporter-5gf2h"
Nov 04 01:03:13.193484 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:13.193453    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a623766e-ac81-46ee-8643-cb92eaaf61cb-metrics-client-ca\") pod \"node-exporter-5gf2h\" (UID: \"a623766e-ac81-46ee-8643-cb92eaaf61cb\") " pod="openshift-monitoring/node-exporter-5gf2h"
Nov 04 01:03:13.193484 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:13.193491    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a623766e-ac81-46ee-8643-cb92eaaf61cb-node-exporter-accelerators-collector-config\") pod \"node-exporter-5gf2h\" (UID: \"a623766e-ac81-46ee-8643-cb92eaaf61cb\") " pod="openshift-monitoring/node-exporter-5gf2h"
Nov 04 01:03:13.193744 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:13.193598    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a623766e-ac81-46ee-8643-cb92eaaf61cb-sys\") pod \"node-exporter-5gf2h\" (UID: \"a623766e-ac81-46ee-8643-cb92eaaf61cb\") " pod="openshift-monitoring/node-exporter-5gf2h"
Nov 04 01:03:13.193744 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:13.193644    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mr788\" (UniqueName: \"kubernetes.io/projected/a623766e-ac81-46ee-8643-cb92eaaf61cb-kube-api-access-mr788\") pod \"node-exporter-5gf2h\" (UID: \"a623766e-ac81-46ee-8643-cb92eaaf61cb\") " pod="openshift-monitoring/node-exporter-5gf2h"
Nov 04 01:03:13.193744 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:13.193662    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a623766e-ac81-46ee-8643-cb92eaaf61cb-node-exporter-textfile\") pod \"node-exporter-5gf2h\" (UID: \"a623766e-ac81-46ee-8643-cb92eaaf61cb\") " pod="openshift-monitoring/node-exporter-5gf2h"
Nov 04 01:03:13.193744 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:13.193679    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a623766e-ac81-46ee-8643-cb92eaaf61cb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5gf2h\" (UID: \"a623766e-ac81-46ee-8643-cb92eaaf61cb\") " pod="openshift-monitoring/node-exporter-5gf2h"
Nov 04 01:03:13.193954 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:13.193804    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a623766e-ac81-46ee-8643-cb92eaaf61cb-sys\") pod \"node-exporter-5gf2h\" (UID: \"a623766e-ac81-46ee-8643-cb92eaaf61cb\") " pod="openshift-monitoring/node-exporter-5gf2h"
Nov 04 01:03:13.193954 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:13.193923    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a623766e-ac81-46ee-8643-cb92eaaf61cb-node-exporter-tls\") pod \"node-exporter-5gf2h\" (UID: \"a623766e-ac81-46ee-8643-cb92eaaf61cb\") " pod="openshift-monitoring/node-exporter-5gf2h"
Nov 04 01:03:13.194053 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:13.193984    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a623766e-ac81-46ee-8643-cb92eaaf61cb-root\") pod \"node-exporter-5gf2h\" (UID: \"a623766e-ac81-46ee-8643-cb92eaaf61cb\") " pod="openshift-monitoring/node-exporter-5gf2h"
Nov 04 01:03:13.194053 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:13.194025    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a623766e-ac81-46ee-8643-cb92eaaf61cb-node-exporter-textfile\") pod \"node-exporter-5gf2h\" (UID: \"a623766e-ac81-46ee-8643-cb92eaaf61cb\") " pod="openshift-monitoring/node-exporter-5gf2h"
Nov 04 01:03:13.194149 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:13.194032    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a623766e-ac81-46ee-8643-cb92eaaf61cb-node-exporter-wtmp\") pod \"node-exporter-5gf2h\" (UID: \"a623766e-ac81-46ee-8643-cb92eaaf61cb\") " pod="openshift-monitoring/node-exporter-5gf2h"
Nov 04 01:03:13.194149 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:13.194141    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a623766e-ac81-46ee-8643-cb92eaaf61cb-node-exporter-wtmp\") pod \"node-exporter-5gf2h\" (UID: \"a623766e-ac81-46ee-8643-cb92eaaf61cb\") " pod="openshift-monitoring/node-exporter-5gf2h"
Nov 04 01:03:13.194251 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:13.194198    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a623766e-ac81-46ee-8643-cb92eaaf61cb-root\") pod \"node-exporter-5gf2h\" (UID: \"a623766e-ac81-46ee-8643-cb92eaaf61cb\") " pod="openshift-monitoring/node-exporter-5gf2h"
Nov 04 01:03:13.194251 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:13.194206    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a623766e-ac81-46ee-8643-cb92eaaf61cb-metrics-client-ca\") pod \"node-exporter-5gf2h\" (UID: \"a623766e-ac81-46ee-8643-cb92eaaf61cb\") " pod="openshift-monitoring/node-exporter-5gf2h"
Nov 04 01:03:13.194251 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:13.194218    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a623766e-ac81-46ee-8643-cb92eaaf61cb-node-exporter-accelerators-collector-config\") pod \"node-exporter-5gf2h\" (UID: \"a623766e-ac81-46ee-8643-cb92eaaf61cb\") " pod="openshift-monitoring/node-exporter-5gf2h"
Nov 04 01:03:13.196123 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:13.196098    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a623766e-ac81-46ee-8643-cb92eaaf61cb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5gf2h\" (UID: \"a623766e-ac81-46ee-8643-cb92eaaf61cb\") " pod="openshift-monitoring/node-exporter-5gf2h"
Nov 04 01:03:13.196279 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:13.196246    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a623766e-ac81-46ee-8643-cb92eaaf61cb-node-exporter-tls\") pod \"node-exporter-5gf2h\" (UID: \"a623766e-ac81-46ee-8643-cb92eaaf61cb\") " pod="openshift-monitoring/node-exporter-5gf2h"
Nov 04 01:03:13.206851 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:13.206799    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mr788\" (UniqueName: \"kubernetes.io/projected/a623766e-ac81-46ee-8643-cb92eaaf61cb-kube-api-access-mr788\") pod \"node-exporter-5gf2h\" (UID: \"a623766e-ac81-46ee-8643-cb92eaaf61cb\") " pod="openshift-monitoring/node-exporter-5gf2h"
Nov 04 01:03:13.332731 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:13.332696    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5gf2h"
Nov 04 01:03:13.367091 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:03:13.367058    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda623766e_ac81_46ee_8643_cb92eaaf61cb.slice/crio-969b1311980fde4179cb104f1ea5dd9a8388cecd232ee60a7f1258f2d7002cfd WatchSource:0}: Error finding container 969b1311980fde4179cb104f1ea5dd9a8388cecd232ee60a7f1258f2d7002cfd: Status 404 returned error can't find the container with id 969b1311980fde4179cb104f1ea5dd9a8388cecd232ee60a7f1258f2d7002cfd
Nov 04 01:03:13.449850 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:03:13.449812    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:03:14.342663 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:14.342625    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5gf2h" event={"ID":"a623766e-ac81-46ee-8643-cb92eaaf61cb","Type":"ContainerStarted","Data":"969b1311980fde4179cb104f1ea5dd9a8388cecd232ee60a7f1258f2d7002cfd"}
Nov 04 01:03:15.349045 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:15.349006    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5gf2h" event={"ID":"a623766e-ac81-46ee-8643-cb92eaaf61cb","Type":"ContainerStarted","Data":"b79e2dede2eec67f0326b4ca7df9257a52f06d1985aaec8d7d9aac5f7e32e8ca"}
Nov 04 01:03:16.354224 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:16.354188    5068 generic.go:356] "Generic (PLEG): container finished" podID="a623766e-ac81-46ee-8643-cb92eaaf61cb" containerID="b79e2dede2eec67f0326b4ca7df9257a52f06d1985aaec8d7d9aac5f7e32e8ca" exitCode=0
Nov 04 01:03:16.354592 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:16.354279    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5gf2h" event={"ID":"a623766e-ac81-46ee-8643-cb92eaaf61cb","Type":"ContainerDied","Data":"b79e2dede2eec67f0326b4ca7df9257a52f06d1985aaec8d7d9aac5f7e32e8ca"}
Nov 04 01:03:17.360686 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:17.360647    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5gf2h" event={"ID":"a623766e-ac81-46ee-8643-cb92eaaf61cb","Type":"ContainerStarted","Data":"6625d4ac3973f7251b59c49ff5923e29b60776c2f4dfddff8ae708f728180204"}
Nov 04 01:03:17.361147 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:17.360736    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5gf2h" event={"ID":"a623766e-ac81-46ee-8643-cb92eaaf61cb","Type":"ContainerStarted","Data":"c90dd0aea1467eb6d8466840b4f848486b048816f87c81a2568767e1e33f0133"}
Nov 04 01:03:17.384255 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:17.384208    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-5gf2h" podStartSLOduration=3.5061172369999998 podStartE2EDuration="5.384196576s" totalImagesPullingTime="1.790795419s" totalInitContainerRuntime="87.28392ms" isStatefulPod=true podCreationTimestamp="2026-11-04 01:03:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:03:17.384069638 +0000 UTC m=+37.430630427" watchObservedRunningTime="2026-11-04 01:03:17.384196576 +0000 UTC m=+37.430757360"
Nov 04 01:03:19.239782 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:19.239748    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5c5894ddd-j4phf"]
Nov 04 01:03:22.323454 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:22.323370    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6574d8498f-74vmj" podUID="20f4989c-4fe8-49d2-9430-5030a4274b60" containerName="registry" containerID="cri-o://77af54067f39ec3bd0ed2726ca2ece863a1aac3d452d739a881f4137b4e19809" gracePeriod=30
Nov 04 01:03:22.580052 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:22.579999    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6574d8498f-74vmj"
Nov 04 01:03:22.669788 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:22.669758    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/20f4989c-4fe8-49d2-9430-5030a4274b60-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/20f4989c-4fe8-49d2-9430-5030a4274b60-image-registry-private-configuration\") pod \"20f4989c-4fe8-49d2-9430-5030a4274b60\" (UID: \"20f4989c-4fe8-49d2-9430-5030a4274b60\") "
Nov 04 01:03:22.669941 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:22.669838    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/20f4989c-4fe8-49d2-9430-5030a4274b60-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/20f4989c-4fe8-49d2-9430-5030a4274b60-registry-certificates\") pod \"20f4989c-4fe8-49d2-9430-5030a4274b60\" (UID: \"20f4989c-4fe8-49d2-9430-5030a4274b60\") "
Nov 04 01:03:22.669941 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:22.669874    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/20f4989c-4fe8-49d2-9430-5030a4274b60-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/20f4989c-4fe8-49d2-9430-5030a4274b60-ca-trust-extracted\") pod \"20f4989c-4fe8-49d2-9430-5030a4274b60\" (UID: \"20f4989c-4fe8-49d2-9430-5030a4274b60\") "
Nov 04 01:03:22.669941 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:22.669903    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/20f4989c-4fe8-49d2-9430-5030a4274b60-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/20f4989c-4fe8-49d2-9430-5030a4274b60-bound-sa-token\") pod \"20f4989c-4fe8-49d2-9430-5030a4274b60\" (UID: \"20f4989c-4fe8-49d2-9430-5030a4274b60\") "
Nov 04 01:03:22.669941 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:22.669934    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/20f4989c-4fe8-49d2-9430-5030a4274b60-registry-tls\" (UniqueName: \"kubernetes.io/projected/20f4989c-4fe8-49d2-9430-5030a4274b60-registry-tls\") pod \"20f4989c-4fe8-49d2-9430-5030a4274b60\" (UID: \"20f4989c-4fe8-49d2-9430-5030a4274b60\") "
Nov 04 01:03:22.670152 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:22.669968    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/20f4989c-4fe8-49d2-9430-5030a4274b60-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/20f4989c-4fe8-49d2-9430-5030a4274b60-trusted-ca\") pod \"20f4989c-4fe8-49d2-9430-5030a4274b60\" (UID: \"20f4989c-4fe8-49d2-9430-5030a4274b60\") "
Nov 04 01:03:22.670152 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:22.669998    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/20f4989c-4fe8-49d2-9430-5030a4274b60-kube-api-access-rct8p\" (UniqueName: \"kubernetes.io/projected/20f4989c-4fe8-49d2-9430-5030a4274b60-kube-api-access-rct8p\") pod \"20f4989c-4fe8-49d2-9430-5030a4274b60\" (UID: \"20f4989c-4fe8-49d2-9430-5030a4274b60\") "
Nov 04 01:03:22.670152 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:22.670059    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/20f4989c-4fe8-49d2-9430-5030a4274b60-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/20f4989c-4fe8-49d2-9430-5030a4274b60-installation-pull-secrets\") pod \"20f4989c-4fe8-49d2-9430-5030a4274b60\" (UID: \"20f4989c-4fe8-49d2-9430-5030a4274b60\") "
Nov 04 01:03:22.670361 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:22.670301    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20f4989c-4fe8-49d2-9430-5030a4274b60-registry-certificates" pod "20f4989c-4fe8-49d2-9430-5030a4274b60" (UID: "20f4989c-4fe8-49d2-9430-5030a4274b60"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:03:22.670870 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:22.670786    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20f4989c-4fe8-49d2-9430-5030a4274b60-trusted-ca" pod "20f4989c-4fe8-49d2-9430-5030a4274b60" (UID: "20f4989c-4fe8-49d2-9430-5030a4274b60"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:03:22.672575 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:22.672532    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20f4989c-4fe8-49d2-9430-5030a4274b60-image-registry-private-configuration" pod "20f4989c-4fe8-49d2-9430-5030a4274b60" (UID: "20f4989c-4fe8-49d2-9430-5030a4274b60"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:03:22.672692 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:22.672605    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20f4989c-4fe8-49d2-9430-5030a4274b60-kube-api-access-rct8p" pod "20f4989c-4fe8-49d2-9430-5030a4274b60" (UID: "20f4989c-4fe8-49d2-9430-5030a4274b60"). InnerVolumeSpecName "kube-api-access-rct8p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:03:22.672692 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:22.672679    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20f4989c-4fe8-49d2-9430-5030a4274b60-installation-pull-secrets" pod "20f4989c-4fe8-49d2-9430-5030a4274b60" (UID: "20f4989c-4fe8-49d2-9430-5030a4274b60"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:03:22.672807 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:22.672711    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20f4989c-4fe8-49d2-9430-5030a4274b60-registry-tls" pod "20f4989c-4fe8-49d2-9430-5030a4274b60" (UID: "20f4989c-4fe8-49d2-9430-5030a4274b60"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:03:22.672807 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:22.672721    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20f4989c-4fe8-49d2-9430-5030a4274b60-bound-sa-token" pod "20f4989c-4fe8-49d2-9430-5030a4274b60" (UID: "20f4989c-4fe8-49d2-9430-5030a4274b60"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:03:22.677186 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:22.677170    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20f4989c-4fe8-49d2-9430-5030a4274b60-ca-trust-extracted" pod "20f4989c-4fe8-49d2-9430-5030a4274b60" (UID: "20f4989c-4fe8-49d2-9430-5030a4274b60"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:03:22.771212 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:22.771175    5068 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/20f4989c-4fe8-49d2-9430-5030a4274b60-registry-certificates\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:03:22.771212 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:22.771208    5068 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/20f4989c-4fe8-49d2-9430-5030a4274b60-ca-trust-extracted\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:03:22.771212 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:22.771220    5068 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/20f4989c-4fe8-49d2-9430-5030a4274b60-bound-sa-token\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:03:22.771371 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:22.771229    5068 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/20f4989c-4fe8-49d2-9430-5030a4274b60-registry-tls\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:03:22.771371 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:22.771238    5068 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/20f4989c-4fe8-49d2-9430-5030a4274b60-trusted-ca\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:03:22.771371 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:22.771248    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rct8p\" (UniqueName: \"kubernetes.io/projected/20f4989c-4fe8-49d2-9430-5030a4274b60-kube-api-access-rct8p\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:03:22.771371 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:22.771257    5068 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/20f4989c-4fe8-49d2-9430-5030a4274b60-installation-pull-secrets\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:03:22.771371 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:22.771267    5068 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/20f4989c-4fe8-49d2-9430-5030a4274b60-image-registry-private-configuration\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:03:23.385225 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:23.385192    5068 generic.go:356] "Generic (PLEG): container finished" podID="20f4989c-4fe8-49d2-9430-5030a4274b60" containerID="77af54067f39ec3bd0ed2726ca2ece863a1aac3d452d739a881f4137b4e19809" exitCode=0
Nov 04 01:03:23.385636 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:23.385238    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6574d8498f-74vmj"
Nov 04 01:03:23.385636 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:23.385279    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6574d8498f-74vmj" event={"ID":"20f4989c-4fe8-49d2-9430-5030a4274b60","Type":"ContainerDied","Data":"77af54067f39ec3bd0ed2726ca2ece863a1aac3d452d739a881f4137b4e19809"}
Nov 04 01:03:23.385636 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:23.385326    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6574d8498f-74vmj" event={"ID":"20f4989c-4fe8-49d2-9430-5030a4274b60","Type":"ContainerDied","Data":"5d8bd94e84de17d7d076b48b189f850d6f0bf0468bb23780f77eb1f4534d57f7"}
Nov 04 01:03:23.385636 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:23.385345    5068 scope.go:122] "RemoveContainer" containerID="77af54067f39ec3bd0ed2726ca2ece863a1aac3d452d739a881f4137b4e19809"
Nov 04 01:03:23.394183 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:23.394169    5068 scope.go:122] "RemoveContainer" containerID="77af54067f39ec3bd0ed2726ca2ece863a1aac3d452d739a881f4137b4e19809"
Nov 04 01:03:23.394424 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:03:23.394410    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77af54067f39ec3bd0ed2726ca2ece863a1aac3d452d739a881f4137b4e19809\": container with ID starting with 77af54067f39ec3bd0ed2726ca2ece863a1aac3d452d739a881f4137b4e19809 not found: ID does not exist" containerID="77af54067f39ec3bd0ed2726ca2ece863a1aac3d452d739a881f4137b4e19809"
Nov 04 01:03:23.394471 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:23.394433    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77af54067f39ec3bd0ed2726ca2ece863a1aac3d452d739a881f4137b4e19809"} err="failed to get container status \"77af54067f39ec3bd0ed2726ca2ece863a1aac3d452d739a881f4137b4e19809\": rpc error: code = NotFound desc = could not find container \"77af54067f39ec3bd0ed2726ca2ece863a1aac3d452d739a881f4137b4e19809\": container with ID starting with 77af54067f39ec3bd0ed2726ca2ece863a1aac3d452d739a881f4137b4e19809 not found: ID does not exist"
Nov 04 01:03:23.405892 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:23.405865    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6574d8498f-74vmj"]
Nov 04 01:03:23.409909 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:23.409889    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6574d8498f-74vmj"]
Nov 04 01:03:24.110369 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:24.110331    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="20f4989c-4fe8-49d2-9430-5030a4274b60" path="/var/lib/kubelet/pods/20f4989c-4fe8-49d2-9430-5030a4274b60/volumes"
Nov 04 01:03:43.449125 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:03:43.449091    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:03:44.262592 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:44.262530    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5c5894ddd-j4phf" podUID="0eedbf45-4e7d-47b7-806f-a6abf9aa311f" containerName="registry" containerID="cri-o://241362af369d65ca558f8de6078270ebcc97edecba2d327df927474c79d903b2" gracePeriod=30
Nov 04 01:03:44.465868 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:44.465838    5068 generic.go:356] "Generic (PLEG): container finished" podID="0eedbf45-4e7d-47b7-806f-a6abf9aa311f" containerID="241362af369d65ca558f8de6078270ebcc97edecba2d327df927474c79d903b2" exitCode=0
Nov 04 01:03:44.466285 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:44.465916    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5c5894ddd-j4phf" event={"ID":"0eedbf45-4e7d-47b7-806f-a6abf9aa311f","Type":"ContainerDied","Data":"241362af369d65ca558f8de6078270ebcc97edecba2d327df927474c79d903b2"}
Nov 04 01:03:44.513974 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:44.513923    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5c5894ddd-j4phf"
Nov 04 01:03:44.640043 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:44.640012    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-ca-trust-extracted\") pod \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\" (UID: \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\") "
Nov 04 01:03:44.640203 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:44.640056    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-kube-api-access-ncqm5\" (UniqueName: \"kubernetes.io/projected/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-kube-api-access-ncqm5\") pod \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\" (UID: \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\") "
Nov 04 01:03:44.640203 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:44.640109    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-registry-certificates\") pod \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\" (UID: \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\") "
Nov 04 01:03:44.640203 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:44.640138    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-bound-sa-token\") pod \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\" (UID: \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\") "
Nov 04 01:03:44.640203 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:44.640155    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-registry-tls\" (UniqueName: \"kubernetes.io/projected/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-registry-tls\") pod \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\" (UID: \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\") "
Nov 04 01:03:44.640203 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:44.640177    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-installation-pull-secrets\") pod \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\" (UID: \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\") "
Nov 04 01:03:44.640471 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:44.640225    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-image-registry-private-configuration\") pod \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\" (UID: \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\") "
Nov 04 01:03:44.640471 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:44.640257    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-trusted-ca\") pod \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\" (UID: \"0eedbf45-4e7d-47b7-806f-a6abf9aa311f\") "
Nov 04 01:03:44.640605 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:44.640569    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-registry-certificates" pod "0eedbf45-4e7d-47b7-806f-a6abf9aa311f" (UID: "0eedbf45-4e7d-47b7-806f-a6abf9aa311f"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:03:44.641149 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:44.641118    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-trusted-ca" pod "0eedbf45-4e7d-47b7-806f-a6abf9aa311f" (UID: "0eedbf45-4e7d-47b7-806f-a6abf9aa311f"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:03:44.642689 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:44.642647    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-bound-sa-token" pod "0eedbf45-4e7d-47b7-806f-a6abf9aa311f" (UID: "0eedbf45-4e7d-47b7-806f-a6abf9aa311f"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:03:44.642885 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:44.642856    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-registry-tls" pod "0eedbf45-4e7d-47b7-806f-a6abf9aa311f" (UID: "0eedbf45-4e7d-47b7-806f-a6abf9aa311f"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:03:44.642992 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:44.642919    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-image-registry-private-configuration" pod "0eedbf45-4e7d-47b7-806f-a6abf9aa311f" (UID: "0eedbf45-4e7d-47b7-806f-a6abf9aa311f"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:03:44.643054 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:44.643012    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-kube-api-access-ncqm5" pod "0eedbf45-4e7d-47b7-806f-a6abf9aa311f" (UID: "0eedbf45-4e7d-47b7-806f-a6abf9aa311f"). InnerVolumeSpecName "kube-api-access-ncqm5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:03:44.643326 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:44.643300    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-installation-pull-secrets" pod "0eedbf45-4e7d-47b7-806f-a6abf9aa311f" (UID: "0eedbf45-4e7d-47b7-806f-a6abf9aa311f"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:03:44.647266 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:44.647206    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-ca-trust-extracted" pod "0eedbf45-4e7d-47b7-806f-a6abf9aa311f" (UID: "0eedbf45-4e7d-47b7-806f-a6abf9aa311f"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:03:44.740939 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:44.740915    5068 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-registry-certificates\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:03:44.740939 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:44.740936    5068 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-bound-sa-token\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:03:44.741068 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:44.740946    5068 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-registry-tls\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:03:44.741068 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:44.740955    5068 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-installation-pull-secrets\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:03:44.741068 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:44.740964    5068 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-image-registry-private-configuration\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:03:44.741068 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:44.740975    5068 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-trusted-ca\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:03:44.741068 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:44.740984    5068 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-ca-trust-extracted\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:03:44.741068 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:44.740992    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ncqm5\" (UniqueName: \"kubernetes.io/projected/0eedbf45-4e7d-47b7-806f-a6abf9aa311f-kube-api-access-ncqm5\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:03:45.471072 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:45.471044    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5c5894ddd-j4phf"
Nov 04 01:03:45.471072 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:45.471055    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5c5894ddd-j4phf" event={"ID":"0eedbf45-4e7d-47b7-806f-a6abf9aa311f","Type":"ContainerDied","Data":"731654e624407a53caa45304528a500db108bdd6f60024ca9993da7c2ee7a31b"}
Nov 04 01:03:45.471684 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:45.471109    5068 scope.go:122] "RemoveContainer" containerID="241362af369d65ca558f8de6078270ebcc97edecba2d327df927474c79d903b2"
Nov 04 01:03:45.492165 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:45.492140    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5c5894ddd-j4phf"]
Nov 04 01:03:45.497579 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:45.497554    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5c5894ddd-j4phf"]
Nov 04 01:03:46.111058 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:46.111027    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0eedbf45-4e7d-47b7-806f-a6abf9aa311f" path="/var/lib/kubelet/pods/0eedbf45-4e7d-47b7-806f-a6abf9aa311f/volumes"
Nov 04 01:03:51.498537 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:51.498502    5068 generic.go:356] "Generic (PLEG): container finished" podID="3d2d0e05-a773-4442-ab38-d460ad32f51e" containerID="d7fd030cf5489a0ae815519cb9b3727d097138490b776df059c60af9b48cf191" exitCode=0
Nov 04 01:03:51.499032 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:51.498578    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-58wxt" event={"ID":"3d2d0e05-a773-4442-ab38-d460ad32f51e","Type":"ContainerDied","Data":"d7fd030cf5489a0ae815519cb9b3727d097138490b776df059c60af9b48cf191"}
Nov 04 01:03:51.499032 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:51.498981    5068 scope.go:122] "RemoveContainer" containerID="d7fd030cf5489a0ae815519cb9b3727d097138490b776df059c60af9b48cf191"
Nov 04 01:03:51.500099 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:51.500080    5068 generic.go:356] "Generic (PLEG): container finished" podID="de97ada4-9cae-4775-a8f8-014004e17d25" containerID="cbd04d364f48b61094def5501e13336aa314a5067464fd1e0cd9862d9890aea9" exitCode=0
Nov 04 01:03:51.500178 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:51.500117    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-sbjxr" event={"ID":"de97ada4-9cae-4775-a8f8-014004e17d25","Type":"ContainerDied","Data":"cbd04d364f48b61094def5501e13336aa314a5067464fd1e0cd9862d9890aea9"}
Nov 04 01:03:51.500406 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:51.500370    5068 scope.go:122] "RemoveContainer" containerID="cbd04d364f48b61094def5501e13336aa314a5067464fd1e0cd9862d9890aea9"
Nov 04 01:03:52.124625 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:52.124580    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-t6tpg-6r4ws"]
Nov 04 01:03:52.124833 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:52.124779    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-insights/periodic-gathering-t6tpg-6r4ws" podUID="e2b1aee4-49c5-47a0-a79f-a3fe032df2d7" containerName="insights-gathering" containerID="cri-o://a0fce730bf11eaa5b9e2bb472b0945a0aa56467f69bb1c716e870daa069b6abb" gracePeriod=30
Nov 04 01:03:52.216661 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:52.216629    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-ntz4t-kjj6c"]
Nov 04 01:03:52.216943 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:52.216928    5068 cpu_manager.go:414] "containerMap: removing container" podUID="20f4989c-4fe8-49d2-9430-5030a4274b60" containerName="registry"
Nov 04 01:03:52.217039 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:52.216945    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="20f4989c-4fe8-49d2-9430-5030a4274b60" containerName="registry"
Nov 04 01:03:52.217039 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:52.216963    5068 cpu_manager.go:414] "containerMap: removing container" podUID="0eedbf45-4e7d-47b7-806f-a6abf9aa311f" containerName="registry"
Nov 04 01:03:52.217039 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:52.216971    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0eedbf45-4e7d-47b7-806f-a6abf9aa311f" containerName="registry"
Nov 04 01:03:52.217039 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:52.217033    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="20f4989c-4fe8-49d2-9430-5030a4274b60" containerName="registry"
Nov 04 01:03:52.217231 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:52.217046    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="0eedbf45-4e7d-47b7-806f-a6abf9aa311f" containerName="registry"
Nov 04 01:03:52.220150 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:52.220130    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-ntz4t-kjj6c"
Nov 04 01:03:52.229349 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:52.229322    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-ntz4t-kjj6c"]
Nov 04 01:03:52.379027 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:52.378973    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-t6tpg-6r4ws"
Nov 04 01:03:52.395745 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:52.395715    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/ccdd8744-3f01-4e10-84fe-cd936e6132fa-archives-path\") pod \"periodic-gathering-ntz4t-kjj6c\" (UID: \"ccdd8744-3f01-4e10-84fe-cd936e6132fa\") " pod="openshift-insights/periodic-gathering-ntz4t-kjj6c"
Nov 04 01:03:52.395854 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:52.395773    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65967\" (UniqueName: \"kubernetes.io/projected/ccdd8744-3f01-4e10-84fe-cd936e6132fa-kube-api-access-65967\") pod \"periodic-gathering-ntz4t-kjj6c\" (UID: \"ccdd8744-3f01-4e10-84fe-cd936e6132fa\") " pod="openshift-insights/periodic-gathering-ntz4t-kjj6c"
Nov 04 01:03:52.395948 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:52.395929    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ccdd8744-3f01-4e10-84fe-cd936e6132fa-trusted-ca-bundle\") pod \"periodic-gathering-ntz4t-kjj6c\" (UID: \"ccdd8744-3f01-4e10-84fe-cd936e6132fa\") " pod="openshift-insights/periodic-gathering-ntz4t-kjj6c"
Nov 04 01:03:52.396009 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:52.395993    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ccdd8744-3f01-4e10-84fe-cd936e6132fa-service-ca-bundle\") pod \"periodic-gathering-ntz4t-kjj6c\" (UID: \"ccdd8744-3f01-4e10-84fe-cd936e6132fa\") " pod="openshift-insights/periodic-gathering-ntz4t-kjj6c"
Nov 04 01:03:52.496712 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:52.496686    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e2b1aee4-49c5-47a0-a79f-a3fe032df2d7-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/e2b1aee4-49c5-47a0-a79f-a3fe032df2d7-archives-path\") pod \"e2b1aee4-49c5-47a0-a79f-a3fe032df2d7\" (UID: \"e2b1aee4-49c5-47a0-a79f-a3fe032df2d7\") "
Nov 04 01:03:52.496824 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:52.496717    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e2b1aee4-49c5-47a0-a79f-a3fe032df2d7-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2b1aee4-49c5-47a0-a79f-a3fe032df2d7-service-ca-bundle\") pod \"e2b1aee4-49c5-47a0-a79f-a3fe032df2d7\" (UID: \"e2b1aee4-49c5-47a0-a79f-a3fe032df2d7\") "
Nov 04 01:03:52.496824 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:52.496743    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e2b1aee4-49c5-47a0-a79f-a3fe032df2d7-kube-api-access-6b7vh\" (UniqueName: \"kubernetes.io/projected/e2b1aee4-49c5-47a0-a79f-a3fe032df2d7-kube-api-access-6b7vh\") pod \"e2b1aee4-49c5-47a0-a79f-a3fe032df2d7\" (UID: \"e2b1aee4-49c5-47a0-a79f-a3fe032df2d7\") "
Nov 04 01:03:52.496907 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:52.496834    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e2b1aee4-49c5-47a0-a79f-a3fe032df2d7-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2b1aee4-49c5-47a0-a79f-a3fe032df2d7-trusted-ca-bundle\") pod \"e2b1aee4-49c5-47a0-a79f-a3fe032df2d7\" (UID: \"e2b1aee4-49c5-47a0-a79f-a3fe032df2d7\") "
Nov 04 01:03:52.496948 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:52.496932    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ccdd8744-3f01-4e10-84fe-cd936e6132fa-service-ca-bundle\") pod \"periodic-gathering-ntz4t-kjj6c\" (UID: \"ccdd8744-3f01-4e10-84fe-cd936e6132fa\") " pod="openshift-insights/periodic-gathering-ntz4t-kjj6c"
Nov 04 01:03:52.496991 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:52.496980    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/ccdd8744-3f01-4e10-84fe-cd936e6132fa-archives-path\") pod \"periodic-gathering-ntz4t-kjj6c\" (UID: \"ccdd8744-3f01-4e10-84fe-cd936e6132fa\") " pod="openshift-insights/periodic-gathering-ntz4t-kjj6c"
Nov 04 01:03:52.497105 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:52.497021    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2b1aee4-49c5-47a0-a79f-a3fe032df2d7-service-ca-bundle" pod "e2b1aee4-49c5-47a0-a79f-a3fe032df2d7" (UID: "e2b1aee4-49c5-47a0-a79f-a3fe032df2d7"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:03:52.497190 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:52.497033    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65967\" (UniqueName: \"kubernetes.io/projected/ccdd8744-3f01-4e10-84fe-cd936e6132fa-kube-api-access-65967\") pod \"periodic-gathering-ntz4t-kjj6c\" (UID: \"ccdd8744-3f01-4e10-84fe-cd936e6132fa\") " pod="openshift-insights/periodic-gathering-ntz4t-kjj6c"
Nov 04 01:03:52.497242 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:52.497186    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ccdd8744-3f01-4e10-84fe-cd936e6132fa-trusted-ca-bundle\") pod \"periodic-gathering-ntz4t-kjj6c\" (UID: \"ccdd8744-3f01-4e10-84fe-cd936e6132fa\") " pod="openshift-insights/periodic-gathering-ntz4t-kjj6c"
Nov 04 01:03:52.497242 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:52.497015    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2b1aee4-49c5-47a0-a79f-a3fe032df2d7-archives-path" pod "e2b1aee4-49c5-47a0-a79f-a3fe032df2d7" (UID: "e2b1aee4-49c5-47a0-a79f-a3fe032df2d7"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:03:52.497242 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:52.497218    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2b1aee4-49c5-47a0-a79f-a3fe032df2d7-trusted-ca-bundle" pod "e2b1aee4-49c5-47a0-a79f-a3fe032df2d7" (UID: "e2b1aee4-49c5-47a0-a79f-a3fe032df2d7"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:03:52.497407 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:52.497300    5068 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2b1aee4-49c5-47a0-a79f-a3fe032df2d7-service-ca-bundle\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:03:52.497486 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:52.497466    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/ccdd8744-3f01-4e10-84fe-cd936e6132fa-archives-path\") pod \"periodic-gathering-ntz4t-kjj6c\" (UID: \"ccdd8744-3f01-4e10-84fe-cd936e6132fa\") " pod="openshift-insights/periodic-gathering-ntz4t-kjj6c"
Nov 04 01:03:52.497760 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:52.497738    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ccdd8744-3f01-4e10-84fe-cd936e6132fa-service-ca-bundle\") pod \"periodic-gathering-ntz4t-kjj6c\" (UID: \"ccdd8744-3f01-4e10-84fe-cd936e6132fa\") " pod="openshift-insights/periodic-gathering-ntz4t-kjj6c"
Nov 04 01:03:52.498017 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:52.498000    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ccdd8744-3f01-4e10-84fe-cd936e6132fa-trusted-ca-bundle\") pod \"periodic-gathering-ntz4t-kjj6c\" (UID: \"ccdd8744-3f01-4e10-84fe-cd936e6132fa\") " pod="openshift-insights/periodic-gathering-ntz4t-kjj6c"
Nov 04 01:03:52.499269 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:52.499247    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2b1aee4-49c5-47a0-a79f-a3fe032df2d7-kube-api-access-6b7vh" pod "e2b1aee4-49c5-47a0-a79f-a3fe032df2d7" (UID: "e2b1aee4-49c5-47a0-a79f-a3fe032df2d7"). InnerVolumeSpecName "kube-api-access-6b7vh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:03:52.505061 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:52.505037    5068 generic.go:356] "Generic (PLEG): container finished" podID="e2b1aee4-49c5-47a0-a79f-a3fe032df2d7" containerID="a0fce730bf11eaa5b9e2bb472b0945a0aa56467f69bb1c716e870daa069b6abb" exitCode=2
Nov 04 01:03:52.505146 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:52.505091    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-t6tpg-6r4ws"
Nov 04 01:03:52.505146 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:52.505111    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-t6tpg-6r4ws" event={"ID":"e2b1aee4-49c5-47a0-a79f-a3fe032df2d7","Type":"ContainerDied","Data":"a0fce730bf11eaa5b9e2bb472b0945a0aa56467f69bb1c716e870daa069b6abb"}
Nov 04 01:03:52.505238 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:52.505169    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-t6tpg-6r4ws" event={"ID":"e2b1aee4-49c5-47a0-a79f-a3fe032df2d7","Type":"ContainerDied","Data":"0acd953dd0d10f6aab6269a568a4bfe682d862b978047b926a7a8e3fd42e2bc6"}
Nov 04 01:03:52.505238 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:52.505196    5068 scope.go:122] "RemoveContainer" containerID="a0fce730bf11eaa5b9e2bb472b0945a0aa56467f69bb1c716e870daa069b6abb"
Nov 04 01:03:52.507220 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:52.507191    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-sbjxr" event={"ID":"de97ada4-9cae-4775-a8f8-014004e17d25","Type":"ContainerStarted","Data":"1e5be14122cfad56c810b9884f0f3b85b25c537d378704a11a15c53a197940f1"}
Nov 04 01:03:52.509400 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:52.509375    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-58wxt" event={"ID":"3d2d0e05-a773-4442-ab38-d460ad32f51e","Type":"ContainerStarted","Data":"e08c1dfc3a2212de6f7bebe6e7b94daf631fae8425842d22b168ae355ef7370b"}
Nov 04 01:03:52.509752 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:52.509734    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-65967\" (UniqueName: \"kubernetes.io/projected/ccdd8744-3f01-4e10-84fe-cd936e6132fa-kube-api-access-65967\") pod \"periodic-gathering-ntz4t-kjj6c\" (UID: \"ccdd8744-3f01-4e10-84fe-cd936e6132fa\") " pod="openshift-insights/periodic-gathering-ntz4t-kjj6c"
Nov 04 01:03:52.519366 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:52.519346    5068 scope.go:122] "RemoveContainer" containerID="a0fce730bf11eaa5b9e2bb472b0945a0aa56467f69bb1c716e870daa069b6abb"
Nov 04 01:03:52.519630 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:03:52.519598    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0fce730bf11eaa5b9e2bb472b0945a0aa56467f69bb1c716e870daa069b6abb\": container with ID starting with a0fce730bf11eaa5b9e2bb472b0945a0aa56467f69bb1c716e870daa069b6abb not found: ID does not exist" containerID="a0fce730bf11eaa5b9e2bb472b0945a0aa56467f69bb1c716e870daa069b6abb"
Nov 04 01:03:52.519706 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:52.519638    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0fce730bf11eaa5b9e2bb472b0945a0aa56467f69bb1c716e870daa069b6abb"} err="failed to get container status \"a0fce730bf11eaa5b9e2bb472b0945a0aa56467f69bb1c716e870daa069b6abb\": rpc error: code = NotFound desc = could not find container \"a0fce730bf11eaa5b9e2bb472b0945a0aa56467f69bb1c716e870daa069b6abb\": container with ID starting with a0fce730bf11eaa5b9e2bb472b0945a0aa56467f69bb1c716e870daa069b6abb not found: ID does not exist"
Nov 04 01:03:52.531278 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:52.531258    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-ntz4t-kjj6c"
Nov 04 01:03:52.560371 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:52.560344    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-t6tpg-6r4ws"]
Nov 04 01:03:52.569002 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:52.568972    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-insights/periodic-gathering-t6tpg-6r4ws"]
Nov 04 01:03:52.598586 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:52.598385    5068 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2b1aee4-49c5-47a0-a79f-a3fe032df2d7-trusted-ca-bundle\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:03:52.598586 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:52.598413    5068 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/e2b1aee4-49c5-47a0-a79f-a3fe032df2d7-archives-path\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:03:52.598586 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:52.598429    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6b7vh\" (UniqueName: \"kubernetes.io/projected/e2b1aee4-49c5-47a0-a79f-a3fe032df2d7-kube-api-access-6b7vh\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:03:52.806537 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:52.806392    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-ntz4t-kjj6c"]
Nov 04 01:03:52.808972 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:03:52.808946    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccdd8744_3f01_4e10_84fe_cd936e6132fa.slice/crio-e941b99112abe9b74e53ea98d6f14390b201466aea036ecb8178de61c69064c3 WatchSource:0}: Error finding container e941b99112abe9b74e53ea98d6f14390b201466aea036ecb8178de61c69064c3: Status 404 returned error can't find the container with id e941b99112abe9b74e53ea98d6f14390b201466aea036ecb8178de61c69064c3
Nov 04 01:03:53.515716 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:53.515670    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-ntz4t-kjj6c" event={"ID":"ccdd8744-3f01-4e10-84fe-cd936e6132fa","Type":"ContainerStarted","Data":"80f2fe2872a4b0826542b565a791ab333c315312b75f24a0f8c8d851e2e100dd"}
Nov 04 01:03:53.516206 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:53.515779    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-ntz4t-kjj6c" event={"ID":"ccdd8744-3f01-4e10-84fe-cd936e6132fa","Type":"ContainerStarted","Data":"e941b99112abe9b74e53ea98d6f14390b201466aea036ecb8178de61c69064c3"}
Nov 04 01:03:53.533181 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:53.533131    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-ntz4t-kjj6c" podStartSLOduration=1.5331150660000001 podStartE2EDuration="1.533115066s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:03:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:03:53.53230794 +0000 UTC m=+73.578868748" watchObservedRunningTime="2026-11-04 01:03:53.533115066 +0000 UTC m=+73.579675859"
Nov 04 01:03:54.110787 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:03:54.110756    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e2b1aee4-49c5-47a0-a79f-a3fe032df2d7" path="/var/lib/kubelet/pods/e2b1aee4-49c5-47a0-a79f-a3fe032df2d7/volumes"
Nov 04 01:04:13.448422 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:04:13.448382    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:04:43.448043 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:04:43.448000    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:05:13.448914 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:05:13.448869    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:05:15.831400 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:05:15.831367    5068 generic.go:356] "Generic (PLEG): container finished" podID="ccdd8744-3f01-4e10-84fe-cd936e6132fa" containerID="80f2fe2872a4b0826542b565a791ab333c315312b75f24a0f8c8d851e2e100dd" exitCode=0
Nov 04 01:05:15.831909 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:05:15.831437    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-ntz4t-kjj6c" event={"ID":"ccdd8744-3f01-4e10-84fe-cd936e6132fa","Type":"ContainerDied","Data":"80f2fe2872a4b0826542b565a791ab333c315312b75f24a0f8c8d851e2e100dd"}
Nov 04 01:05:16.959014 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:05:16.958991    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-ntz4t-kjj6c"
Nov 04 01:05:17.120066 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:05:17.119974    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ccdd8744-3f01-4e10-84fe-cd936e6132fa-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ccdd8744-3f01-4e10-84fe-cd936e6132fa-service-ca-bundle\") pod \"ccdd8744-3f01-4e10-84fe-cd936e6132fa\" (UID: \"ccdd8744-3f01-4e10-84fe-cd936e6132fa\") "
Nov 04 01:05:17.120066 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:05:17.120032    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ccdd8744-3f01-4e10-84fe-cd936e6132fa-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/ccdd8744-3f01-4e10-84fe-cd936e6132fa-archives-path\") pod \"ccdd8744-3f01-4e10-84fe-cd936e6132fa\" (UID: \"ccdd8744-3f01-4e10-84fe-cd936e6132fa\") "
Nov 04 01:05:17.120066 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:05:17.120067    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ccdd8744-3f01-4e10-84fe-cd936e6132fa-kube-api-access-65967\" (UniqueName: \"kubernetes.io/projected/ccdd8744-3f01-4e10-84fe-cd936e6132fa-kube-api-access-65967\") pod \"ccdd8744-3f01-4e10-84fe-cd936e6132fa\" (UID: \"ccdd8744-3f01-4e10-84fe-cd936e6132fa\") "
Nov 04 01:05:17.120311 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:05:17.120252    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ccdd8744-3f01-4e10-84fe-cd936e6132fa-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ccdd8744-3f01-4e10-84fe-cd936e6132fa-trusted-ca-bundle\") pod \"ccdd8744-3f01-4e10-84fe-cd936e6132fa\" (UID: \"ccdd8744-3f01-4e10-84fe-cd936e6132fa\") "
Nov 04 01:05:17.120562 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:05:17.120422    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ccdd8744-3f01-4e10-84fe-cd936e6132fa-service-ca-bundle" pod "ccdd8744-3f01-4e10-84fe-cd936e6132fa" (UID: "ccdd8744-3f01-4e10-84fe-cd936e6132fa"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:05:17.120562 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:05:17.120421    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ccdd8744-3f01-4e10-84fe-cd936e6132fa-archives-path" pod "ccdd8744-3f01-4e10-84fe-cd936e6132fa" (UID: "ccdd8744-3f01-4e10-84fe-cd936e6132fa"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:05:17.120562 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:05:17.120532    5068 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ccdd8744-3f01-4e10-84fe-cd936e6132fa-service-ca-bundle\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:05:17.120562 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:05:17.120554    5068 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/ccdd8744-3f01-4e10-84fe-cd936e6132fa-archives-path\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:05:17.120792 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:05:17.120704    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ccdd8744-3f01-4e10-84fe-cd936e6132fa-trusted-ca-bundle" pod "ccdd8744-3f01-4e10-84fe-cd936e6132fa" (UID: "ccdd8744-3f01-4e10-84fe-cd936e6132fa"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:05:17.122168 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:05:17.122148    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ccdd8744-3f01-4e10-84fe-cd936e6132fa-kube-api-access-65967" pod "ccdd8744-3f01-4e10-84fe-cd936e6132fa" (UID: "ccdd8744-3f01-4e10-84fe-cd936e6132fa"). InnerVolumeSpecName "kube-api-access-65967". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:05:17.220952 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:05:17.220915    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-65967\" (UniqueName: \"kubernetes.io/projected/ccdd8744-3f01-4e10-84fe-cd936e6132fa-kube-api-access-65967\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:05:17.220952 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:05:17.220948    5068 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ccdd8744-3f01-4e10-84fe-cd936e6132fa-trusted-ca-bundle\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:05:17.844621 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:05:17.844591    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-ntz4t-kjj6c"
Nov 04 01:05:17.844794 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:05:17.844586    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-ntz4t-kjj6c" event={"ID":"ccdd8744-3f01-4e10-84fe-cd936e6132fa","Type":"ContainerDied","Data":"e941b99112abe9b74e53ea98d6f14390b201466aea036ecb8178de61c69064c3"}
Nov 04 01:05:17.844794 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:05:17.844729    5068 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e941b99112abe9b74e53ea98d6f14390b201466aea036ecb8178de61c69064c3"
Nov 04 01:05:43.448513 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:05:43.448469    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:06:13.447474 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:06:13.447434    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:06:27.986930 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:27.986862    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71l74sh"]
Nov 04 01:06:27.987373 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:27.987165    5068 cpu_manager.go:414] "containerMap: removing container" podUID="e2b1aee4-49c5-47a0-a79f-a3fe032df2d7" containerName="insights-gathering"
Nov 04 01:06:27.987373 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:27.987177    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e2b1aee4-49c5-47a0-a79f-a3fe032df2d7" containerName="insights-gathering"
Nov 04 01:06:27.987373 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:27.987192    5068 cpu_manager.go:414] "containerMap: removing container" podUID="ccdd8744-3f01-4e10-84fe-cd936e6132fa" containerName="insights-gathering"
Nov 04 01:06:27.987373 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:27.987200    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ccdd8744-3f01-4e10-84fe-cd936e6132fa" containerName="insights-gathering"
Nov 04 01:06:27.987373 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:27.987258    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="e2b1aee4-49c5-47a0-a79f-a3fe032df2d7" containerName="insights-gathering"
Nov 04 01:06:27.987373 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:27.987267    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="ccdd8744-3f01-4e10-84fe-cd936e6132fa" containerName="insights-gathering"
Nov 04 01:06:27.990266 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:27.990250    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71l74sh"
Nov 04 01:06:27.993779 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:27.993750    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Nov 04 01:06:27.994948 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:27.994929    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Nov 04 01:06:27.995057 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:27.994962    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e644421527c169d876fdb59235c927986d6bd2-9d2815fc-dockercfg-js65q\""
Nov 04 01:06:27.999140 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:27.999118    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71l74sh"]
Nov 04 01:06:28.020624 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:28.020584    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9306d043-aed8-4f70-9a20-b85f83ca9840-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71l74sh\" (UID: \"9306d043-aed8-4f70-9a20-b85f83ca9840\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71l74sh"
Nov 04 01:06:28.020732 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:28.020629    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9306d043-aed8-4f70-9a20-b85f83ca9840-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71l74sh\" (UID: \"9306d043-aed8-4f70-9a20-b85f83ca9840\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71l74sh"
Nov 04 01:06:28.020732 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:28.020688    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xssnd\" (UniqueName: \"kubernetes.io/projected/9306d043-aed8-4f70-9a20-b85f83ca9840-kube-api-access-xssnd\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71l74sh\" (UID: \"9306d043-aed8-4f70-9a20-b85f83ca9840\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71l74sh"
Nov 04 01:06:28.121485 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:28.121464    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9306d043-aed8-4f70-9a20-b85f83ca9840-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71l74sh\" (UID: \"9306d043-aed8-4f70-9a20-b85f83ca9840\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71l74sh"
Nov 04 01:06:28.121632 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:28.121489    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9306d043-aed8-4f70-9a20-b85f83ca9840-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71l74sh\" (UID: \"9306d043-aed8-4f70-9a20-b85f83ca9840\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71l74sh"
Nov 04 01:06:28.121632 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:28.121506    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xssnd\" (UniqueName: \"kubernetes.io/projected/9306d043-aed8-4f70-9a20-b85f83ca9840-kube-api-access-xssnd\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71l74sh\" (UID: \"9306d043-aed8-4f70-9a20-b85f83ca9840\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71l74sh"
Nov 04 01:06:28.121839 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:28.121818    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9306d043-aed8-4f70-9a20-b85f83ca9840-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71l74sh\" (UID: \"9306d043-aed8-4f70-9a20-b85f83ca9840\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71l74sh"
Nov 04 01:06:28.121904 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:28.121854    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9306d043-aed8-4f70-9a20-b85f83ca9840-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71l74sh\" (UID: \"9306d043-aed8-4f70-9a20-b85f83ca9840\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71l74sh"
Nov 04 01:06:28.134375 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:28.134353    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xssnd\" (UniqueName: \"kubernetes.io/projected/9306d043-aed8-4f70-9a20-b85f83ca9840-kube-api-access-xssnd\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71l74sh\" (UID: \"9306d043-aed8-4f70-9a20-b85f83ca9840\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71l74sh"
Nov 04 01:06:28.306187 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:28.306161    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71l74sh"
Nov 04 01:06:28.576128 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:28.576052    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71l74sh"]
Nov 04 01:06:28.578791 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:06:28.578763    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9306d043_aed8_4f70_9a20_b85f83ca9840.slice/crio-7449f9840a2197805094efd6563bf0161bb038642c8812dd31d36f7c523b7848 WatchSource:0}: Error finding container 7449f9840a2197805094efd6563bf0161bb038642c8812dd31d36f7c523b7848: Status 404 returned error can't find the container with id 7449f9840a2197805094efd6563bf0161bb038642c8812dd31d36f7c523b7848
Nov 04 01:06:29.036779 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.036743    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpsjq4"]
Nov 04 01:06:29.041271 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.041242    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpsjq4"
Nov 04 01:06:29.045777 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.045755    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e3d011e69d9573cabb8c359e5ac709eb5ac240-be78d160-dockercfg-dlfxx\""
Nov 04 01:06:29.051760 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.051729    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpsjq4"]
Nov 04 01:06:29.121274 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.121223    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71l74sh" event={"ID":"9306d043-aed8-4f70-9a20-b85f83ca9840","Type":"ContainerStarted","Data":"7449f9840a2197805094efd6563bf0161bb038642c8812dd31d36f7c523b7848"}
Nov 04 01:06:29.127433 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.127407    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wk7ld"]
Nov 04 01:06:29.129462 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.129434    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqljb\" (UniqueName: \"kubernetes.io/projected/f4a221a4-bf98-45a5-a473-1bf2d0188dae-kube-api-access-tqljb\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpsjq4\" (UID: \"f4a221a4-bf98-45a5-a473-1bf2d0188dae\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpsjq4"
Nov 04 01:06:29.129572 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.129474    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f4a221a4-bf98-45a5-a473-1bf2d0188dae-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpsjq4\" (UID: \"f4a221a4-bf98-45a5-a473-1bf2d0188dae\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpsjq4"
Nov 04 01:06:29.129572 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.129502    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f4a221a4-bf98-45a5-a473-1bf2d0188dae-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpsjq4\" (UID: \"f4a221a4-bf98-45a5-a473-1bf2d0188dae\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpsjq4"
Nov 04 01:06:29.130758 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.130740    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wk7ld"
Nov 04 01:06:29.136116 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.136098    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3-b83c8cba-dockercfg-jm2vq\""
Nov 04 01:06:29.140913 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.140892    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wk7ld"]
Nov 04 01:06:29.226731 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.226700    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amv297"]
Nov 04 01:06:29.230084 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.230065    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amv297"
Nov 04 01:06:29.230764 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.230548    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqljb\" (UniqueName: \"kubernetes.io/projected/f4a221a4-bf98-45a5-a473-1bf2d0188dae-kube-api-access-tqljb\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpsjq4\" (UID: \"f4a221a4-bf98-45a5-a473-1bf2d0188dae\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpsjq4"
Nov 04 01:06:29.230764 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.230591    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f4a221a4-bf98-45a5-a473-1bf2d0188dae-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpsjq4\" (UID: \"f4a221a4-bf98-45a5-a473-1bf2d0188dae\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpsjq4"
Nov 04 01:06:29.230764 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.230658    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f4a221a4-bf98-45a5-a473-1bf2d0188dae-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpsjq4\" (UID: \"f4a221a4-bf98-45a5-a473-1bf2d0188dae\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpsjq4"
Nov 04 01:06:29.230764 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.230702    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qb8d2\" (UniqueName: \"kubernetes.io/projected/f0b5aa83-17d7-416f-9634-6544c03fb3db-kube-api-access-qb8d2\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wk7ld\" (UID: \"f0b5aa83-17d7-416f-9634-6544c03fb3db\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wk7ld"
Nov 04 01:06:29.230764 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.230738    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f0b5aa83-17d7-416f-9634-6544c03fb3db-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wk7ld\" (UID: \"f0b5aa83-17d7-416f-9634-6544c03fb3db\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wk7ld"
Nov 04 01:06:29.231100 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.230915    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f0b5aa83-17d7-416f-9634-6544c03fb3db-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wk7ld\" (UID: \"f0b5aa83-17d7-416f-9634-6544c03fb3db\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wk7ld"
Nov 04 01:06:29.231100 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.231026    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f4a221a4-bf98-45a5-a473-1bf2d0188dae-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpsjq4\" (UID: \"f4a221a4-bf98-45a5-a473-1bf2d0188dae\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpsjq4"
Nov 04 01:06:29.231100 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.231077    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f4a221a4-bf98-45a5-a473-1bf2d0188dae-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpsjq4\" (UID: \"f4a221a4-bf98-45a5-a473-1bf2d0188dae\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpsjq4"
Nov 04 01:06:29.234200 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.234181    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"6541d93aaee335d6227a8778ab4e840bbe7f52-f98fbcf8-dockercfg-ljs9d\""
Nov 04 01:06:29.242862 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.239970    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amv297"]
Nov 04 01:06:29.253314 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.253280    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqljb\" (UniqueName: \"kubernetes.io/projected/f4a221a4-bf98-45a5-a473-1bf2d0188dae-kube-api-access-tqljb\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpsjq4\" (UID: \"f4a221a4-bf98-45a5-a473-1bf2d0188dae\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpsjq4"
Nov 04 01:06:29.329898 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.329838    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72t5xb"]
Nov 04 01:06:29.331912 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.331886    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f0b5aa83-17d7-416f-9634-6544c03fb3db-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wk7ld\" (UID: \"f0b5aa83-17d7-416f-9634-6544c03fb3db\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wk7ld"
Nov 04 01:06:29.332032 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.331924    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3e911a11-0ed3-4ce9-a336-507851b62942-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amv297\" (UID: \"3e911a11-0ed3-4ce9-a336-507851b62942\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amv297"
Nov 04 01:06:29.332032 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.331958    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrp6v\" (UniqueName: \"kubernetes.io/projected/3e911a11-0ed3-4ce9-a336-507851b62942-kube-api-access-xrp6v\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amv297\" (UID: \"3e911a11-0ed3-4ce9-a336-507851b62942\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amv297"
Nov 04 01:06:29.332152 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.332030    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3e911a11-0ed3-4ce9-a336-507851b62942-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amv297\" (UID: \"3e911a11-0ed3-4ce9-a336-507851b62942\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amv297"
Nov 04 01:06:29.332152 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.332060    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qb8d2\" (UniqueName: \"kubernetes.io/projected/f0b5aa83-17d7-416f-9634-6544c03fb3db-kube-api-access-qb8d2\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wk7ld\" (UID: \"f0b5aa83-17d7-416f-9634-6544c03fb3db\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wk7ld"
Nov 04 01:06:29.332152 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.332081    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f0b5aa83-17d7-416f-9634-6544c03fb3db-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wk7ld\" (UID: \"f0b5aa83-17d7-416f-9634-6544c03fb3db\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wk7ld"
Nov 04 01:06:29.332332 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.332313    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f0b5aa83-17d7-416f-9634-6544c03fb3db-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wk7ld\" (UID: \"f0b5aa83-17d7-416f-9634-6544c03fb3db\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wk7ld"
Nov 04 01:06:29.332401 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.332382    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f0b5aa83-17d7-416f-9634-6544c03fb3db-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wk7ld\" (UID: \"f0b5aa83-17d7-416f-9634-6544c03fb3db\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wk7ld"
Nov 04 01:06:29.333333 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.333319    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72t5xb"
Nov 04 01:06:29.336813 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.336789    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"da112bd609bff87f7bacd2ba941225d7cff15c-bd73b2fc-dockercfg-srszl\""
Nov 04 01:06:29.343723 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.343704    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72t5xb"]
Nov 04 01:06:29.351279 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.351251    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qb8d2\" (UniqueName: \"kubernetes.io/projected/f0b5aa83-17d7-416f-9634-6544c03fb3db-kube-api-access-qb8d2\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wk7ld\" (UID: \"f0b5aa83-17d7-416f-9634-6544c03fb3db\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wk7ld"
Nov 04 01:06:29.359966 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.359943    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpsjq4"
Nov 04 01:06:29.433148 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.433117    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2422e7ec-790e-4d3c-bd2f-beed4726a21a-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72t5xb\" (UID: \"2422e7ec-790e-4d3c-bd2f-beed4726a21a\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72t5xb"
Nov 04 01:06:29.433290 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.433227    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrp6v\" (UniqueName: \"kubernetes.io/projected/3e911a11-0ed3-4ce9-a336-507851b62942-kube-api-access-xrp6v\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amv297\" (UID: \"3e911a11-0ed3-4ce9-a336-507851b62942\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amv297"
Nov 04 01:06:29.433290 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.433259    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2422e7ec-790e-4d3c-bd2f-beed4726a21a-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72t5xb\" (UID: \"2422e7ec-790e-4d3c-bd2f-beed4726a21a\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72t5xb"
Nov 04 01:06:29.433402 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.433375    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3e911a11-0ed3-4ce9-a336-507851b62942-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amv297\" (UID: \"3e911a11-0ed3-4ce9-a336-507851b62942\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amv297"
Nov 04 01:06:29.433462 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.433440    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndtbr\" (UniqueName: \"kubernetes.io/projected/2422e7ec-790e-4d3c-bd2f-beed4726a21a-kube-api-access-ndtbr\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72t5xb\" (UID: \"2422e7ec-790e-4d3c-bd2f-beed4726a21a\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72t5xb"
Nov 04 01:06:29.433540 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.433524    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3e911a11-0ed3-4ce9-a336-507851b62942-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amv297\" (UID: \"3e911a11-0ed3-4ce9-a336-507851b62942\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amv297"
Nov 04 01:06:29.433896 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.433875    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3e911a11-0ed3-4ce9-a336-507851b62942-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amv297\" (UID: \"3e911a11-0ed3-4ce9-a336-507851b62942\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amv297"
Nov 04 01:06:29.434355 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.434334    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3e911a11-0ed3-4ce9-a336-507851b62942-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amv297\" (UID: \"3e911a11-0ed3-4ce9-a336-507851b62942\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amv297"
Nov 04 01:06:29.447195 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.447154    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrp6v\" (UniqueName: \"kubernetes.io/projected/3e911a11-0ed3-4ce9-a336-507851b62942-kube-api-access-xrp6v\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amv297\" (UID: \"3e911a11-0ed3-4ce9-a336-507851b62942\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amv297"
Nov 04 01:06:29.447821 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.447800    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wk7ld"
Nov 04 01:06:29.534329 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.534293    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2422e7ec-790e-4d3c-bd2f-beed4726a21a-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72t5xb\" (UID: \"2422e7ec-790e-4d3c-bd2f-beed4726a21a\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72t5xb"
Nov 04 01:06:29.534478 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.534344    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2422e7ec-790e-4d3c-bd2f-beed4726a21a-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72t5xb\" (UID: \"2422e7ec-790e-4d3c-bd2f-beed4726a21a\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72t5xb"
Nov 04 01:06:29.534478 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.534423    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndtbr\" (UniqueName: \"kubernetes.io/projected/2422e7ec-790e-4d3c-bd2f-beed4726a21a-kube-api-access-ndtbr\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72t5xb\" (UID: \"2422e7ec-790e-4d3c-bd2f-beed4726a21a\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72t5xb"
Nov 04 01:06:29.534749 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.534724    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2422e7ec-790e-4d3c-bd2f-beed4726a21a-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72t5xb\" (UID: \"2422e7ec-790e-4d3c-bd2f-beed4726a21a\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72t5xb"
Nov 04 01:06:29.534841 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.534737    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2422e7ec-790e-4d3c-bd2f-beed4726a21a-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72t5xb\" (UID: \"2422e7ec-790e-4d3c-bd2f-beed4726a21a\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72t5xb"
Nov 04 01:06:29.547238 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.547209    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndtbr\" (UniqueName: \"kubernetes.io/projected/2422e7ec-790e-4d3c-bd2f-beed4726a21a-kube-api-access-ndtbr\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72t5xb\" (UID: \"2422e7ec-790e-4d3c-bd2f-beed4726a21a\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72t5xb"
Nov 04 01:06:29.553660 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.553641    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amv297"
Nov 04 01:06:29.641014 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.640981    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpsjq4"]
Nov 04 01:06:29.644174 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:06:29.644148    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4a221a4_bf98_45a5_a473_1bf2d0188dae.slice/crio-6edc380ee3225d4acb5545ffc9c608126b829dc549bf441ad01c3df63ada20e1 WatchSource:0}: Error finding container 6edc380ee3225d4acb5545ffc9c608126b829dc549bf441ad01c3df63ada20e1: Status 404 returned error can't find the container with id 6edc380ee3225d4acb5545ffc9c608126b829dc549bf441ad01c3df63ada20e1
Nov 04 01:06:29.664177 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.664153    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72t5xb"
Nov 04 01:06:29.734288 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.734263    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wk7ld"]
Nov 04 01:06:29.737076 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:06:29.737045    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0b5aa83_17d7_416f_9634_6544c03fb3db.slice/crio-9af67d435788a92474e29ce76fbfa5c391b7f61eca94d628ff21b8faaeb86ff4 WatchSource:0}: Error finding container 9af67d435788a92474e29ce76fbfa5c391b7f61eca94d628ff21b8faaeb86ff4: Status 404 returned error can't find the container with id 9af67d435788a92474e29ce76fbfa5c391b7f61eca94d628ff21b8faaeb86ff4
Nov 04 01:06:29.828862 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.828721    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amv297"]
Nov 04 01:06:29.831179 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:06:29.831153    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e911a11_0ed3_4ce9_a336_507851b62942.slice/crio-09748929e1905da72b5cd8ca5b1cfebddd3994e02c6e83aa3bde8fbd42a2a4af WatchSource:0}: Error finding container 09748929e1905da72b5cd8ca5b1cfebddd3994e02c6e83aa3bde8fbd42a2a4af: Status 404 returned error can't find the container with id 09748929e1905da72b5cd8ca5b1cfebddd3994e02c6e83aa3bde8fbd42a2a4af
Nov 04 01:06:29.942114 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:29.942056    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72t5xb"]
Nov 04 01:06:29.946080 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:06:29.946054    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2422e7ec_790e_4d3c_bd2f_beed4726a21a.slice/crio-dc04b1525161bc92c9a7b02f61e6352a8addae3f39e6e81c49e108ec63d38835 WatchSource:0}: Error finding container dc04b1525161bc92c9a7b02f61e6352a8addae3f39e6e81c49e108ec63d38835: Status 404 returned error can't find the container with id dc04b1525161bc92c9a7b02f61e6352a8addae3f39e6e81c49e108ec63d38835
Nov 04 01:06:30.127297 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:30.127261    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpsjq4" event={"ID":"f4a221a4-bf98-45a5-a473-1bf2d0188dae","Type":"ContainerStarted","Data":"6edc380ee3225d4acb5545ffc9c608126b829dc549bf441ad01c3df63ada20e1"}
Nov 04 01:06:30.128516 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:30.128495    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wk7ld" event={"ID":"f0b5aa83-17d7-416f-9634-6544c03fb3db","Type":"ContainerStarted","Data":"9af67d435788a92474e29ce76fbfa5c391b7f61eca94d628ff21b8faaeb86ff4"}
Nov 04 01:06:30.129665 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:30.129646    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amv297" event={"ID":"3e911a11-0ed3-4ce9-a336-507851b62942","Type":"ContainerStarted","Data":"09748929e1905da72b5cd8ca5b1cfebddd3994e02c6e83aa3bde8fbd42a2a4af"}
Nov 04 01:06:30.130801 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:30.130770    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72t5xb" event={"ID":"2422e7ec-790e-4d3c-bd2f-beed4726a21a","Type":"ContainerStarted","Data":"dc04b1525161bc92c9a7b02f61e6352a8addae3f39e6e81c49e108ec63d38835"}
Nov 04 01:06:30.312147 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:30.312112    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-df8wd"]
Nov 04 01:06:30.315760 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:30.315735    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-df8wd"
Nov 04 01:06:30.325952 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:30.325910    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Nov 04 01:06:30.325952 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:30.325948    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-4zhmf\""
Nov 04 01:06:30.334754 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:30.334710    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-df8wd"]
Nov 04 01:06:30.443566 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:30.443533    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70049bd7-ed93-4151-a7de-1938bbe0590d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-df8wd\" (UID: \"70049bd7-ed93-4151-a7de-1938bbe0590d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-df8wd"
Nov 04 01:06:30.443769 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:30.443581    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8v9hj\" (UniqueName: \"kubernetes.io/projected/70049bd7-ed93-4151-a7de-1938bbe0590d-kube-api-access-8v9hj\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-df8wd\" (UID: \"70049bd7-ed93-4151-a7de-1938bbe0590d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-df8wd"
Nov 04 01:06:30.443769 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:30.443639    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70049bd7-ed93-4151-a7de-1938bbe0590d-workload-certs\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-df8wd\" (UID: \"70049bd7-ed93-4151-a7de-1938bbe0590d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-df8wd"
Nov 04 01:06:30.443769 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:30.443675    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70049bd7-ed93-4151-a7de-1938bbe0590d-istio-token\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-df8wd\" (UID: \"70049bd7-ed93-4151-a7de-1938bbe0590d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-df8wd"
Nov 04 01:06:30.443769 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:30.443715    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70049bd7-ed93-4151-a7de-1938bbe0590d-istio-envoy\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-df8wd\" (UID: \"70049bd7-ed93-4151-a7de-1938bbe0590d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-df8wd"
Nov 04 01:06:30.443769 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:30.443753    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70049bd7-ed93-4151-a7de-1938bbe0590d-credential-socket\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-df8wd\" (UID: \"70049bd7-ed93-4151-a7de-1938bbe0590d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-df8wd"
Nov 04 01:06:30.444046 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:30.443804    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70049bd7-ed93-4151-a7de-1938bbe0590d-workload-socket\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-df8wd\" (UID: \"70049bd7-ed93-4151-a7de-1938bbe0590d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-df8wd"
Nov 04 01:06:30.444046 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:30.443831    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70049bd7-ed93-4151-a7de-1938bbe0590d-istio-data\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-df8wd\" (UID: \"70049bd7-ed93-4151-a7de-1938bbe0590d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-df8wd"
Nov 04 01:06:30.444046 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:30.443890    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70049bd7-ed93-4151-a7de-1938bbe0590d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-df8wd\" (UID: \"70049bd7-ed93-4151-a7de-1938bbe0590d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-df8wd"
Nov 04 01:06:30.545183 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:30.545137    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70049bd7-ed93-4151-a7de-1938bbe0590d-workload-socket\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-df8wd\" (UID: \"70049bd7-ed93-4151-a7de-1938bbe0590d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-df8wd"
Nov 04 01:06:30.545357 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:30.545192    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70049bd7-ed93-4151-a7de-1938bbe0590d-istio-data\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-df8wd\" (UID: \"70049bd7-ed93-4151-a7de-1938bbe0590d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-df8wd"
Nov 04 01:06:30.545357 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:30.545244    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70049bd7-ed93-4151-a7de-1938bbe0590d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-df8wd\" (UID: \"70049bd7-ed93-4151-a7de-1938bbe0590d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-df8wd"
Nov 04 01:06:30.545357 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:30.545284    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70049bd7-ed93-4151-a7de-1938bbe0590d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-df8wd\" (UID: \"70049bd7-ed93-4151-a7de-1938bbe0590d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-df8wd"
Nov 04 01:06:30.545357 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:30.545316    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8v9hj\" (UniqueName: \"kubernetes.io/projected/70049bd7-ed93-4151-a7de-1938bbe0590d-kube-api-access-8v9hj\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-df8wd\" (UID: \"70049bd7-ed93-4151-a7de-1938bbe0590d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-df8wd"
Nov 04 01:06:30.545357 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:30.545353    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70049bd7-ed93-4151-a7de-1938bbe0590d-workload-certs\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-df8wd\" (UID: \"70049bd7-ed93-4151-a7de-1938bbe0590d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-df8wd"
Nov 04 01:06:30.545641 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:30.545379    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70049bd7-ed93-4151-a7de-1938bbe0590d-istio-token\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-df8wd\" (UID: \"70049bd7-ed93-4151-a7de-1938bbe0590d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-df8wd"
Nov 04 01:06:30.545641 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:30.545418    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70049bd7-ed93-4151-a7de-1938bbe0590d-istio-envoy\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-df8wd\" (UID: \"70049bd7-ed93-4151-a7de-1938bbe0590d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-df8wd"
Nov 04 01:06:30.545641 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:30.545451    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70049bd7-ed93-4151-a7de-1938bbe0590d-credential-socket\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-df8wd\" (UID: \"70049bd7-ed93-4151-a7de-1938bbe0590d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-df8wd"
Nov 04 01:06:30.545909 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:30.545888    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70049bd7-ed93-4151-a7de-1938bbe0590d-credential-socket\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-df8wd\" (UID: \"70049bd7-ed93-4151-a7de-1938bbe0590d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-df8wd"
Nov 04 01:06:30.546153 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:30.546134    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70049bd7-ed93-4151-a7de-1938bbe0590d-workload-socket\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-df8wd\" (UID: \"70049bd7-ed93-4151-a7de-1938bbe0590d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-df8wd"
Nov 04 01:06:30.546390 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:30.546366    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70049bd7-ed93-4151-a7de-1938bbe0590d-istio-data\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-df8wd\" (UID: \"70049bd7-ed93-4151-a7de-1938bbe0590d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-df8wd"
Nov 04 01:06:30.547000 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:30.546974    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70049bd7-ed93-4151-a7de-1938bbe0590d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-df8wd\" (UID: \"70049bd7-ed93-4151-a7de-1938bbe0590d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-df8wd"
Nov 04 01:06:30.547435 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:30.547412    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70049bd7-ed93-4151-a7de-1938bbe0590d-workload-certs\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-df8wd\" (UID: \"70049bd7-ed93-4151-a7de-1938bbe0590d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-df8wd"
Nov 04 01:06:30.560392 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:30.560367    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8v9hj\" (UniqueName: \"kubernetes.io/projected/70049bd7-ed93-4151-a7de-1938bbe0590d-kube-api-access-8v9hj\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-df8wd\" (UID: \"70049bd7-ed93-4151-a7de-1938bbe0590d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-df8wd"
Nov 04 01:06:30.560392 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:30.560384    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70049bd7-ed93-4151-a7de-1938bbe0590d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-df8wd\" (UID: \"70049bd7-ed93-4151-a7de-1938bbe0590d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-df8wd"
Nov 04 01:06:30.562873 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:30.562818    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70049bd7-ed93-4151-a7de-1938bbe0590d-istio-envoy\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-df8wd\" (UID: \"70049bd7-ed93-4151-a7de-1938bbe0590d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-df8wd"
Nov 04 01:06:30.563294 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:30.563250    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70049bd7-ed93-4151-a7de-1938bbe0590d-istio-token\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-df8wd\" (UID: \"70049bd7-ed93-4151-a7de-1938bbe0590d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-df8wd"
Nov 04 01:06:30.633916 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:30.633888    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-df8wd"
Nov 04 01:06:30.780643 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:30.780577    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cs77c"]
Nov 04 01:06:30.785851 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:30.785828    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cs77c"
Nov 04 01:06:30.790548 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:30.790525    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c4959aeb960c8e20c88ac2de924a35b8221494-145dcd5a-dockercfg-klq4w\""
Nov 04 01:06:30.798674 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:30.798649    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cs77c"]
Nov 04 01:06:30.849672 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:30.849588    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcz7b\" (UniqueName: \"kubernetes.io/projected/972ec12d-42d6-41c1-812b-b555973f82f9-kube-api-access-bcz7b\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cs77c\" (UID: \"972ec12d-42d6-41c1-812b-b555973f82f9\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cs77c"
Nov 04 01:06:30.849672 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:30.849663    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/972ec12d-42d6-41c1-812b-b555973f82f9-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cs77c\" (UID: \"972ec12d-42d6-41c1-812b-b555973f82f9\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cs77c"
Nov 04 01:06:30.849882 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:30.849747    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/972ec12d-42d6-41c1-812b-b555973f82f9-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cs77c\" (UID: \"972ec12d-42d6-41c1-812b-b555973f82f9\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cs77c"
Nov 04 01:06:30.950709 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:30.950674    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcz7b\" (UniqueName: \"kubernetes.io/projected/972ec12d-42d6-41c1-812b-b555973f82f9-kube-api-access-bcz7b\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cs77c\" (UID: \"972ec12d-42d6-41c1-812b-b555973f82f9\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cs77c"
Nov 04 01:06:30.950885 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:30.950727    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/972ec12d-42d6-41c1-812b-b555973f82f9-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cs77c\" (UID: \"972ec12d-42d6-41c1-812b-b555973f82f9\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cs77c"
Nov 04 01:06:30.950885 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:30.950786    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/972ec12d-42d6-41c1-812b-b555973f82f9-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cs77c\" (UID: \"972ec12d-42d6-41c1-812b-b555973f82f9\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cs77c"
Nov 04 01:06:30.951155 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:30.951131    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/972ec12d-42d6-41c1-812b-b555973f82f9-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cs77c\" (UID: \"972ec12d-42d6-41c1-812b-b555973f82f9\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cs77c"
Nov 04 01:06:30.951229 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:30.951187    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/972ec12d-42d6-41c1-812b-b555973f82f9-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cs77c\" (UID: \"972ec12d-42d6-41c1-812b-b555973f82f9\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cs77c"
Nov 04 01:06:30.968198 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:30.968171    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-df8wd"]
Nov 04 01:06:30.969104 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:30.969081    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcz7b\" (UniqueName: \"kubernetes.io/projected/972ec12d-42d6-41c1-812b-b555973f82f9-kube-api-access-bcz7b\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cs77c\" (UID: \"972ec12d-42d6-41c1-812b-b555973f82f9\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cs77c"
Nov 04 01:06:30.976344 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:06:30.976318    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70049bd7_ed93_4151_a7de_1938bbe0590d.slice/crio-c1350885e3fa354075d3921c22567a7157335752ee322deffd3f3dcf10053d26 WatchSource:0}: Error finding container c1350885e3fa354075d3921c22567a7157335752ee322deffd3f3dcf10053d26: Status 404 returned error can't find the container with id c1350885e3fa354075d3921c22567a7157335752ee322deffd3f3dcf10053d26
Nov 04 01:06:31.063503 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:31.063468    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651crzlbh"]
Nov 04 01:06:31.068908 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:31.068796    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651crzlbh"
Nov 04 01:06:31.078902 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:31.078803    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651crzlbh"]
Nov 04 01:06:31.108360 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:31.108121    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cs77c"
Nov 04 01:06:31.148254 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:31.148177    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-df8wd" event={"ID":"70049bd7-ed93-4151-a7de-1938bbe0590d","Type":"ContainerStarted","Data":"c1350885e3fa354075d3921c22567a7157335752ee322deffd3f3dcf10053d26"}
Nov 04 01:06:31.152699 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:31.152629    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5cffdbec-e9d2-4b78-a9f2-ba87e4015584-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651crzlbh\" (UID: \"5cffdbec-e9d2-4b78-a9f2-ba87e4015584\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651crzlbh"
Nov 04 01:06:31.152817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:31.152719    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtswd\" (UniqueName: \"kubernetes.io/projected/5cffdbec-e9d2-4b78-a9f2-ba87e4015584-kube-api-access-gtswd\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651crzlbh\" (UID: \"5cffdbec-e9d2-4b78-a9f2-ba87e4015584\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651crzlbh"
Nov 04 01:06:31.152817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:31.152768    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5cffdbec-e9d2-4b78-a9f2-ba87e4015584-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651crzlbh\" (UID: \"5cffdbec-e9d2-4b78-a9f2-ba87e4015584\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651crzlbh"
Nov 04 01:06:31.153486 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:31.153277    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c7537007944577f64df34960ace3c69136c416-139ec370-dockercfg-cm5pv\""
Nov 04 01:06:31.254855 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:31.253663    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtswd\" (UniqueName: \"kubernetes.io/projected/5cffdbec-e9d2-4b78-a9f2-ba87e4015584-kube-api-access-gtswd\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651crzlbh\" (UID: \"5cffdbec-e9d2-4b78-a9f2-ba87e4015584\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651crzlbh"
Nov 04 01:06:31.254855 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:31.253723    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5cffdbec-e9d2-4b78-a9f2-ba87e4015584-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651crzlbh\" (UID: \"5cffdbec-e9d2-4b78-a9f2-ba87e4015584\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651crzlbh"
Nov 04 01:06:31.254855 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:31.253805    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5cffdbec-e9d2-4b78-a9f2-ba87e4015584-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651crzlbh\" (UID: \"5cffdbec-e9d2-4b78-a9f2-ba87e4015584\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651crzlbh"
Nov 04 01:06:31.254855 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:31.254213    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5cffdbec-e9d2-4b78-a9f2-ba87e4015584-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651crzlbh\" (UID: \"5cffdbec-e9d2-4b78-a9f2-ba87e4015584\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651crzlbh"
Nov 04 01:06:31.254855 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:31.254793    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5cffdbec-e9d2-4b78-a9f2-ba87e4015584-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651crzlbh\" (UID: \"5cffdbec-e9d2-4b78-a9f2-ba87e4015584\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651crzlbh"
Nov 04 01:06:31.268107 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:31.267526    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtswd\" (UniqueName: \"kubernetes.io/projected/5cffdbec-e9d2-4b78-a9f2-ba87e4015584-kube-api-access-gtswd\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651crzlbh\" (UID: \"5cffdbec-e9d2-4b78-a9f2-ba87e4015584\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651crzlbh"
Nov 04 01:06:31.395101 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:31.395024    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651crzlbh"
Nov 04 01:06:31.432647 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:31.432596    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cs77c"]
Nov 04 01:06:31.447524 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:06:31.447485    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod972ec12d_42d6_41c1_812b_b555973f82f9.slice/crio-1813181356ca0865cd2171d94e27cbf475eaedd75ab0258730908805432f817a WatchSource:0}: Error finding container 1813181356ca0865cd2171d94e27cbf475eaedd75ab0258730908805432f817a: Status 404 returned error can't find the container with id 1813181356ca0865cd2171d94e27cbf475eaedd75ab0258730908805432f817a
Nov 04 01:06:31.710464 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:31.710402    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651crzlbh"]
Nov 04 01:06:31.713972 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:06:31.713943    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cffdbec_e9d2_4b78_a9f2_ba87e4015584.slice/crio-aabc7dcafbb383b1160aead95d3e67ab067f10e99d7864f42d9581259f2ad398 WatchSource:0}: Error finding container aabc7dcafbb383b1160aead95d3e67ab067f10e99d7864f42d9581259f2ad398: Status 404 returned error can't find the container with id aabc7dcafbb383b1160aead95d3e67ab067f10e99d7864f42d9581259f2ad398
Nov 04 01:06:32.157007 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:32.156928    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651crzlbh" event={"ID":"5cffdbec-e9d2-4b78-a9f2-ba87e4015584","Type":"ContainerStarted","Data":"aabc7dcafbb383b1160aead95d3e67ab067f10e99d7864f42d9581259f2ad398"}
Nov 04 01:06:32.159567 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:32.159482    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cs77c" event={"ID":"972ec12d-42d6-41c1-812b-b555973f82f9","Type":"ContainerStarted","Data":"1813181356ca0865cd2171d94e27cbf475eaedd75ab0258730908805432f817a"}
Nov 04 01:06:38.676058 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:38.676021    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:06:38.676482 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:38.676113    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:06:38.676482 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:38.676157    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:06:39.224496 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:39.223539    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-df8wd" event={"ID":"70049bd7-ed93-4151-a7de-1938bbe0590d","Type":"ContainerStarted","Data":"49f2830e424fbd578cd6a6821e25c04bc26d6bc61f80cfd780aed1c237641c93"}
Nov 04 01:06:39.636849 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:39.634936    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-df8wd"
Nov 04 01:06:39.641494 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:39.641237    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-df8wd"
Nov 04 01:06:39.667381 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:39.666468    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-df8wd" podStartSLOduration=1.972711593 podStartE2EDuration="9.666451778s" totalImagesPullingTime="7.693740185s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:06:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:06:39.25075006 +0000 UTC m=+239.297310849" watchObservedRunningTime="2026-11-04 01:06:39.666451778 +0000 UTC m=+239.713012568"
Nov 04 01:06:40.239465 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:40.239397    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-df8wd"
Nov 04 01:06:40.240537 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:40.240309    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-df8wd"
Nov 04 01:06:43.268048 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:43.265490    5068 generic.go:356] "Generic (PLEG): container finished" podID="9306d043-aed8-4f70-9a20-b85f83ca9840" containerID="fc34f97758b7c2b16f5fbaf92aee5e8c2d6b7d0b5166d67e1b891fd2dbe12704" exitCode=0
Nov 04 01:06:43.268048 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:43.265622    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71l74sh" event={"ID":"9306d043-aed8-4f70-9a20-b85f83ca9840","Type":"ContainerDied","Data":"fc34f97758b7c2b16f5fbaf92aee5e8c2d6b7d0b5166d67e1b891fd2dbe12704"}
Nov 04 01:06:43.466010 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:06:43.465967    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:06:44.277622 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:44.277584    5068 generic.go:356] "Generic (PLEG): container finished" podID="f4a221a4-bf98-45a5-a473-1bf2d0188dae" containerID="51e6abb407f32de93c7c3b4b5383d44670d12577790568ee7a78900f7d2b7c78" exitCode=0
Nov 04 01:06:44.278033 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:44.277654    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpsjq4" event={"ID":"f4a221a4-bf98-45a5-a473-1bf2d0188dae","Type":"ContainerDied","Data":"51e6abb407f32de93c7c3b4b5383d44670d12577790568ee7a78900f7d2b7c78"}
Nov 04 01:06:45.284805 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:45.284758    5068 generic.go:356] "Generic (PLEG): container finished" podID="9306d043-aed8-4f70-9a20-b85f83ca9840" containerID="13ce94f2629dff44d8a2283b3aaba8698375c4f353b951cb44e81557d478726e" exitCode=0
Nov 04 01:06:45.285272 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:45.284851    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71l74sh" event={"ID":"9306d043-aed8-4f70-9a20-b85f83ca9840","Type":"ContainerDied","Data":"13ce94f2629dff44d8a2283b3aaba8698375c4f353b951cb44e81557d478726e"}
Nov 04 01:06:45.294628 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:45.291058    5068 generic.go:356] "Generic (PLEG): container finished" podID="f0b5aa83-17d7-416f-9634-6544c03fb3db" containerID="aa66418f85a6ad742802ce4d52c74e0c63d6962c3b217111af5705bba1089eec" exitCode=0
Nov 04 01:06:45.294628 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:45.291248    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wk7ld" event={"ID":"f0b5aa83-17d7-416f-9634-6544c03fb3db","Type":"ContainerDied","Data":"aa66418f85a6ad742802ce4d52c74e0c63d6962c3b217111af5705bba1089eec"}
Nov 04 01:06:45.297303 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:45.297279    5068 generic.go:356] "Generic (PLEG): container finished" podID="2422e7ec-790e-4d3c-bd2f-beed4726a21a" containerID="3b1689d905a8823678a47d2c74c0bfc252824ea449a3b63953b278c6fe05e414" exitCode=0
Nov 04 01:06:45.297410 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:45.297341    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72t5xb" event={"ID":"2422e7ec-790e-4d3c-bd2f-beed4726a21a","Type":"ContainerDied","Data":"3b1689d905a8823678a47d2c74c0bfc252824ea449a3b63953b278c6fe05e414"}
Nov 04 01:06:46.304984 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:46.304957    5068 generic.go:356] "Generic (PLEG): container finished" podID="972ec12d-42d6-41c1-812b-b555973f82f9" containerID="e3819119c282d73ca1297958739343779791668b16ddb34605df5a17f9219640" exitCode=0
Nov 04 01:06:46.305294 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:46.305072    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cs77c" event={"ID":"972ec12d-42d6-41c1-812b-b555973f82f9","Type":"ContainerDied","Data":"e3819119c282d73ca1297958739343779791668b16ddb34605df5a17f9219640"}
Nov 04 01:06:46.309537 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:46.309507    5068 generic.go:356] "Generic (PLEG): container finished" podID="3e911a11-0ed3-4ce9-a336-507851b62942" containerID="305ef7d47418b7b35819519119c8a84be125b25ab0c38deb4e84af025b6a779f" exitCode=0
Nov 04 01:06:46.309676 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:46.309656    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amv297" event={"ID":"3e911a11-0ed3-4ce9-a336-507851b62942","Type":"ContainerDied","Data":"305ef7d47418b7b35819519119c8a84be125b25ab0c38deb4e84af025b6a779f"}
Nov 04 01:06:46.313343 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:46.313310    5068 generic.go:356] "Generic (PLEG): container finished" podID="5cffdbec-e9d2-4b78-a9f2-ba87e4015584" containerID="50059816263f2864c0d93c99c64837da299cbe615e042b2eeec02bbbc24e9f8c" exitCode=0
Nov 04 01:06:46.313444 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:46.313349    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651crzlbh" event={"ID":"5cffdbec-e9d2-4b78-a9f2-ba87e4015584","Type":"ContainerDied","Data":"50059816263f2864c0d93c99c64837da299cbe615e042b2eeec02bbbc24e9f8c"}
Nov 04 01:06:47.323705 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:47.323661    5068 generic.go:356] "Generic (PLEG): container finished" podID="f0b5aa83-17d7-416f-9634-6544c03fb3db" containerID="85e175c4eed1c5ee276ba8771c121fa85e2a8f16162a74a2e789e8594fc8d56c" exitCode=0
Nov 04 01:06:47.324168 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:47.323860    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wk7ld" event={"ID":"f0b5aa83-17d7-416f-9634-6544c03fb3db","Type":"ContainerDied","Data":"85e175c4eed1c5ee276ba8771c121fa85e2a8f16162a74a2e789e8594fc8d56c"}
Nov 04 01:06:47.327250 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:47.327226    5068 generic.go:356] "Generic (PLEG): container finished" podID="2422e7ec-790e-4d3c-bd2f-beed4726a21a" containerID="b84a83d9c10a4b402462400a1e68e6b644e4596ae0e7d81cfa39913bb63eef25" exitCode=0
Nov 04 01:06:47.327410 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:47.327261    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72t5xb" event={"ID":"2422e7ec-790e-4d3c-bd2f-beed4726a21a","Type":"ContainerDied","Data":"b84a83d9c10a4b402462400a1e68e6b644e4596ae0e7d81cfa39913bb63eef25"}
Nov 04 01:06:48.336857 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:48.336817    5068 generic.go:356] "Generic (PLEG): container finished" podID="3e911a11-0ed3-4ce9-a336-507851b62942" containerID="86a06ea5e2045e70eb57cb9cb4aab99dd9628b0b70c0d9036fc6f98a4c6b6666" exitCode=0
Nov 04 01:06:48.337317 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:48.336866    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amv297" event={"ID":"3e911a11-0ed3-4ce9-a336-507851b62942","Type":"ContainerDied","Data":"86a06ea5e2045e70eb57cb9cb4aab99dd9628b0b70c0d9036fc6f98a4c6b6666"}
Nov 04 01:06:50.353377 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:50.353293    5068 generic.go:356] "Generic (PLEG): container finished" podID="5cffdbec-e9d2-4b78-a9f2-ba87e4015584" containerID="75198232da7ebed15d4fea3b79e0d140b727dcb15c3388f5367807250ccddf7c" exitCode=0
Nov 04 01:06:50.353829 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:50.353410    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651crzlbh" event={"ID":"5cffdbec-e9d2-4b78-a9f2-ba87e4015584","Type":"ContainerDied","Data":"75198232da7ebed15d4fea3b79e0d140b727dcb15c3388f5367807250ccddf7c"}
Nov 04 01:06:50.360409 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:50.360383    5068 generic.go:356] "Generic (PLEG): container finished" podID="972ec12d-42d6-41c1-812b-b555973f82f9" containerID="d4d192199fabdc111699fe6354ccb21bdd49983e5467e4c94ac2c2fe3920c64f" exitCode=0
Nov 04 01:06:50.360516 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:50.360434    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cs77c" event={"ID":"972ec12d-42d6-41c1-812b-b555973f82f9","Type":"ContainerDied","Data":"d4d192199fabdc111699fe6354ccb21bdd49983e5467e4c94ac2c2fe3920c64f"}
Nov 04 01:06:58.435438 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:58.435401    5068 generic.go:356] "Generic (PLEG): container finished" podID="9306d043-aed8-4f70-9a20-b85f83ca9840" containerID="c8919ad0b1b8810d4f695bd17b2038266e60cc80e0c869bb835f7ff75503d729" exitCode=0
Nov 04 01:06:58.435979 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:58.435453    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71l74sh" event={"ID":"9306d043-aed8-4f70-9a20-b85f83ca9840","Type":"ContainerDied","Data":"c8919ad0b1b8810d4f695bd17b2038266e60cc80e0c869bb835f7ff75503d729"}
Nov 04 01:06:59.443843 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:59.443766    5068 generic.go:356] "Generic (PLEG): container finished" podID="5cffdbec-e9d2-4b78-a9f2-ba87e4015584" containerID="0d4adfae793ad13a1bcd27bcccd10d19576074e848a933536dfed47086e5a908" exitCode=0
Nov 04 01:06:59.443843 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:59.443805    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651crzlbh" event={"ID":"5cffdbec-e9d2-4b78-a9f2-ba87e4015584","Type":"ContainerDied","Data":"0d4adfae793ad13a1bcd27bcccd10d19576074e848a933536dfed47086e5a908"}
Nov 04 01:06:59.446729 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:59.446703    5068 generic.go:356] "Generic (PLEG): container finished" podID="972ec12d-42d6-41c1-812b-b555973f82f9" containerID="43e86825f4232949844bdb8d02cc6efa5daa571b79e6fc88f8426cf49a681f11" exitCode=0
Nov 04 01:06:59.446884 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:59.446793    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cs77c" event={"ID":"972ec12d-42d6-41c1-812b-b555973f82f9","Type":"ContainerDied","Data":"43e86825f4232949844bdb8d02cc6efa5daa571b79e6fc88f8426cf49a681f11"}
Nov 04 01:06:59.593871 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:59.593840    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71l74sh"
Nov 04 01:06:59.746536 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:59.746508    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9306d043-aed8-4f70-9a20-b85f83ca9840-kube-api-access-xssnd\" (UniqueName: \"kubernetes.io/projected/9306d043-aed8-4f70-9a20-b85f83ca9840-kube-api-access-xssnd\") pod \"9306d043-aed8-4f70-9a20-b85f83ca9840\" (UID: \"9306d043-aed8-4f70-9a20-b85f83ca9840\") "
Nov 04 01:06:59.746707 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:59.746597    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9306d043-aed8-4f70-9a20-b85f83ca9840-bundle\" (UniqueName: \"kubernetes.io/empty-dir/9306d043-aed8-4f70-9a20-b85f83ca9840-bundle\") pod \"9306d043-aed8-4f70-9a20-b85f83ca9840\" (UID: \"9306d043-aed8-4f70-9a20-b85f83ca9840\") "
Nov 04 01:06:59.746832 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:59.746803    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9306d043-aed8-4f70-9a20-b85f83ca9840-util\" (UniqueName: \"kubernetes.io/empty-dir/9306d043-aed8-4f70-9a20-b85f83ca9840-util\") pod \"9306d043-aed8-4f70-9a20-b85f83ca9840\" (UID: \"9306d043-aed8-4f70-9a20-b85f83ca9840\") "
Nov 04 01:06:59.747335 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:59.747299    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9306d043-aed8-4f70-9a20-b85f83ca9840-bundle" pod "9306d043-aed8-4f70-9a20-b85f83ca9840" (UID: "9306d043-aed8-4f70-9a20-b85f83ca9840"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:06:59.748948 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:59.748920    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9306d043-aed8-4f70-9a20-b85f83ca9840-kube-api-access-xssnd" pod "9306d043-aed8-4f70-9a20-b85f83ca9840" (UID: "9306d043-aed8-4f70-9a20-b85f83ca9840"). InnerVolumeSpecName "kube-api-access-xssnd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:06:59.753091 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:59.753064    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9306d043-aed8-4f70-9a20-b85f83ca9840-util" pod "9306d043-aed8-4f70-9a20-b85f83ca9840" (UID: "9306d043-aed8-4f70-9a20-b85f83ca9840"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:06:59.848258 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:59.848220    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xssnd\" (UniqueName: \"kubernetes.io/projected/9306d043-aed8-4f70-9a20-b85f83ca9840-kube-api-access-xssnd\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:06:59.848258 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:59.848257    5068 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9306d043-aed8-4f70-9a20-b85f83ca9840-bundle\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:06:59.848456 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:06:59.848271    5068 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9306d043-aed8-4f70-9a20-b85f83ca9840-util\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:07:00.453880 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:00.453796    5068 generic.go:356] "Generic (PLEG): container finished" podID="2422e7ec-790e-4d3c-bd2f-beed4726a21a" containerID="baeffccce0c98523d20070919a7d6b5b8399c16397dccf9cc2ffa1e6946f6da7" exitCode=0
Nov 04 01:07:00.454268 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:00.453875    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72t5xb" event={"ID":"2422e7ec-790e-4d3c-bd2f-beed4726a21a","Type":"ContainerDied","Data":"baeffccce0c98523d20070919a7d6b5b8399c16397dccf9cc2ffa1e6946f6da7"}
Nov 04 01:07:00.455588 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:00.455562    5068 generic.go:356] "Generic (PLEG): container finished" podID="f4a221a4-bf98-45a5-a473-1bf2d0188dae" containerID="f7cf52aba4d5a7db5163ebb7de9a35afea58ec8fca94e4fb39d631f6da43b709" exitCode=0
Nov 04 01:07:00.455727 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:00.455647    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpsjq4" event={"ID":"f4a221a4-bf98-45a5-a473-1bf2d0188dae","Type":"ContainerDied","Data":"f7cf52aba4d5a7db5163ebb7de9a35afea58ec8fca94e4fb39d631f6da43b709"}
Nov 04 01:07:00.457750 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:00.457730    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71l74sh"
Nov 04 01:07:00.457857 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:00.457728    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71l74sh" event={"ID":"9306d043-aed8-4f70-9a20-b85f83ca9840","Type":"ContainerDied","Data":"7449f9840a2197805094efd6563bf0161bb038642c8812dd31d36f7c523b7848"}
Nov 04 01:07:00.457857 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:00.457820    5068 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7449f9840a2197805094efd6563bf0161bb038642c8812dd31d36f7c523b7848"
Nov 04 01:07:00.460262 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:00.460238    5068 generic.go:356] "Generic (PLEG): container finished" podID="f0b5aa83-17d7-416f-9634-6544c03fb3db" containerID="6d6b2d046c277b733c83251f1713624190a76448a154fd969c67398d9972d8b6" exitCode=0
Nov 04 01:07:00.460351 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:00.460329    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wk7ld" event={"ID":"f0b5aa83-17d7-416f-9634-6544c03fb3db","Type":"ContainerDied","Data":"6d6b2d046c277b733c83251f1713624190a76448a154fd969c67398d9972d8b6"}
Nov 04 01:07:00.462643 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:00.462606    5068 generic.go:356] "Generic (PLEG): container finished" podID="3e911a11-0ed3-4ce9-a336-507851b62942" containerID="c67c8033b6d70acbfa01bbc149b021b310a35e0ee95ed4b8d5c368b2dd47ce8c" exitCode=0
Nov 04 01:07:00.462744 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:00.462694    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amv297" event={"ID":"3e911a11-0ed3-4ce9-a336-507851b62942","Type":"ContainerDied","Data":"c67c8033b6d70acbfa01bbc149b021b310a35e0ee95ed4b8d5c368b2dd47ce8c"}
Nov 04 01:07:00.592962 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:00.592938    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651crzlbh"
Nov 04 01:07:00.677579 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:00.677553    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cs77c"
Nov 04 01:07:00.755024 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:00.754955    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5cffdbec-e9d2-4b78-a9f2-ba87e4015584-bundle\" (UniqueName: \"kubernetes.io/empty-dir/5cffdbec-e9d2-4b78-a9f2-ba87e4015584-bundle\") pod \"5cffdbec-e9d2-4b78-a9f2-ba87e4015584\" (UID: \"5cffdbec-e9d2-4b78-a9f2-ba87e4015584\") "
Nov 04 01:07:00.755150 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:00.755025    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5cffdbec-e9d2-4b78-a9f2-ba87e4015584-kube-api-access-gtswd\" (UniqueName: \"kubernetes.io/projected/5cffdbec-e9d2-4b78-a9f2-ba87e4015584-kube-api-access-gtswd\") pod \"5cffdbec-e9d2-4b78-a9f2-ba87e4015584\" (UID: \"5cffdbec-e9d2-4b78-a9f2-ba87e4015584\") "
Nov 04 01:07:00.755150 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:00.755103    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5cffdbec-e9d2-4b78-a9f2-ba87e4015584-util\" (UniqueName: \"kubernetes.io/empty-dir/5cffdbec-e9d2-4b78-a9f2-ba87e4015584-util\") pod \"5cffdbec-e9d2-4b78-a9f2-ba87e4015584\" (UID: \"5cffdbec-e9d2-4b78-a9f2-ba87e4015584\") "
Nov 04 01:07:00.755656 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:00.755626    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cffdbec-e9d2-4b78-a9f2-ba87e4015584-bundle" pod "5cffdbec-e9d2-4b78-a9f2-ba87e4015584" (UID: "5cffdbec-e9d2-4b78-a9f2-ba87e4015584"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:07:00.757086 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:00.757063    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5cffdbec-e9d2-4b78-a9f2-ba87e4015584-kube-api-access-gtswd" pod "5cffdbec-e9d2-4b78-a9f2-ba87e4015584" (UID: "5cffdbec-e9d2-4b78-a9f2-ba87e4015584"). InnerVolumeSpecName "kube-api-access-gtswd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:07:00.758972 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:00.758951    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cffdbec-e9d2-4b78-a9f2-ba87e4015584-util" pod "5cffdbec-e9d2-4b78-a9f2-ba87e4015584" (UID: "5cffdbec-e9d2-4b78-a9f2-ba87e4015584"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:07:00.856220 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:00.856190    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/972ec12d-42d6-41c1-812b-b555973f82f9-kube-api-access-bcz7b\" (UniqueName: \"kubernetes.io/projected/972ec12d-42d6-41c1-812b-b555973f82f9-kube-api-access-bcz7b\") pod \"972ec12d-42d6-41c1-812b-b555973f82f9\" (UID: \"972ec12d-42d6-41c1-812b-b555973f82f9\") "
Nov 04 01:07:00.856220 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:00.856220    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/972ec12d-42d6-41c1-812b-b555973f82f9-util\" (UniqueName: \"kubernetes.io/empty-dir/972ec12d-42d6-41c1-812b-b555973f82f9-util\") pod \"972ec12d-42d6-41c1-812b-b555973f82f9\" (UID: \"972ec12d-42d6-41c1-812b-b555973f82f9\") "
Nov 04 01:07:00.856376 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:00.856307    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/972ec12d-42d6-41c1-812b-b555973f82f9-bundle\" (UniqueName: \"kubernetes.io/empty-dir/972ec12d-42d6-41c1-812b-b555973f82f9-bundle\") pod \"972ec12d-42d6-41c1-812b-b555973f82f9\" (UID: \"972ec12d-42d6-41c1-812b-b555973f82f9\") "
Nov 04 01:07:00.856557 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:00.856538    5068 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5cffdbec-e9d2-4b78-a9f2-ba87e4015584-util\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:07:00.856629 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:00.856560    5068 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5cffdbec-e9d2-4b78-a9f2-ba87e4015584-bundle\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:07:00.856629 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:00.856574    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gtswd\" (UniqueName: \"kubernetes.io/projected/5cffdbec-e9d2-4b78-a9f2-ba87e4015584-kube-api-access-gtswd\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:07:00.856753 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:00.856711    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/972ec12d-42d6-41c1-812b-b555973f82f9-bundle" pod "972ec12d-42d6-41c1-812b-b555973f82f9" (UID: "972ec12d-42d6-41c1-812b-b555973f82f9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:07:00.858145 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:00.858127    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/972ec12d-42d6-41c1-812b-b555973f82f9-kube-api-access-bcz7b" pod "972ec12d-42d6-41c1-812b-b555973f82f9" (UID: "972ec12d-42d6-41c1-812b-b555973f82f9"). InnerVolumeSpecName "kube-api-access-bcz7b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:07:00.862394 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:00.862370    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/972ec12d-42d6-41c1-812b-b555973f82f9-util" pod "972ec12d-42d6-41c1-812b-b555973f82f9" (UID: "972ec12d-42d6-41c1-812b-b555973f82f9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:07:00.957230 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:00.957202    5068 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/972ec12d-42d6-41c1-812b-b555973f82f9-bundle\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:07:00.957230 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:00.957227    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bcz7b\" (UniqueName: \"kubernetes.io/projected/972ec12d-42d6-41c1-812b-b555973f82f9-kube-api-access-bcz7b\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:07:00.957367 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:00.957237    5068 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/972ec12d-42d6-41c1-812b-b555973f82f9-util\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:07:01.469068 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:01.469036    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651crzlbh"
Nov 04 01:07:01.469068 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:01.469044    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651crzlbh" event={"ID":"5cffdbec-e9d2-4b78-a9f2-ba87e4015584","Type":"ContainerDied","Data":"aabc7dcafbb383b1160aead95d3e67ab067f10e99d7864f42d9581259f2ad398"}
Nov 04 01:07:01.469535 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:01.469089    5068 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aabc7dcafbb383b1160aead95d3e67ab067f10e99d7864f42d9581259f2ad398"
Nov 04 01:07:01.470999 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:01.470976    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cs77c"
Nov 04 01:07:01.471132 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:01.471001    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644cs77c" event={"ID":"972ec12d-42d6-41c1-812b-b555973f82f9","Type":"ContainerDied","Data":"1813181356ca0865cd2171d94e27cbf475eaedd75ab0258730908805432f817a"}
Nov 04 01:07:01.471132 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:01.471043    5068 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1813181356ca0865cd2171d94e27cbf475eaedd75ab0258730908805432f817a"
Nov 04 01:07:01.473250 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:01.473222    5068 generic.go:356] "Generic (PLEG): container finished" podID="f4a221a4-bf98-45a5-a473-1bf2d0188dae" containerID="f09683b4e2434ec88b1c9e2b00ed0e23c635739e8fb55ea0c1d79379af52bf6a" exitCode=0
Nov 04 01:07:01.473348 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:01.473284    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpsjq4" event={"ID":"f4a221a4-bf98-45a5-a473-1bf2d0188dae","Type":"ContainerDied","Data":"f09683b4e2434ec88b1c9e2b00ed0e23c635739e8fb55ea0c1d79379af52bf6a"}
Nov 04 01:07:01.635712 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:01.635682    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wk7ld"
Nov 04 01:07:01.653833 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:01.653811    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amv297"
Nov 04 01:07:01.668993 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:01.668970    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72t5xb"
Nov 04 01:07:01.762729 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:01.762675    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3e911a11-0ed3-4ce9-a336-507851b62942-bundle\" (UniqueName: \"kubernetes.io/empty-dir/3e911a11-0ed3-4ce9-a336-507851b62942-bundle\") pod \"3e911a11-0ed3-4ce9-a336-507851b62942\" (UID: \"3e911a11-0ed3-4ce9-a336-507851b62942\") "
Nov 04 01:07:01.762900 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:01.762781    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3e911a11-0ed3-4ce9-a336-507851b62942-util\" (UniqueName: \"kubernetes.io/empty-dir/3e911a11-0ed3-4ce9-a336-507851b62942-util\") pod \"3e911a11-0ed3-4ce9-a336-507851b62942\" (UID: \"3e911a11-0ed3-4ce9-a336-507851b62942\") "
Nov 04 01:07:01.762900 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:01.762814    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f0b5aa83-17d7-416f-9634-6544c03fb3db-util\" (UniqueName: \"kubernetes.io/empty-dir/f0b5aa83-17d7-416f-9634-6544c03fb3db-util\") pod \"f0b5aa83-17d7-416f-9634-6544c03fb3db\" (UID: \"f0b5aa83-17d7-416f-9634-6544c03fb3db\") "
Nov 04 01:07:01.762900 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:01.762856    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f0b5aa83-17d7-416f-9634-6544c03fb3db-kube-api-access-qb8d2\" (UniqueName: \"kubernetes.io/projected/f0b5aa83-17d7-416f-9634-6544c03fb3db-kube-api-access-qb8d2\") pod \"f0b5aa83-17d7-416f-9634-6544c03fb3db\" (UID: \"f0b5aa83-17d7-416f-9634-6544c03fb3db\") "
Nov 04 01:07:01.763065 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:01.762922    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f0b5aa83-17d7-416f-9634-6544c03fb3db-bundle\" (UniqueName: \"kubernetes.io/empty-dir/f0b5aa83-17d7-416f-9634-6544c03fb3db-bundle\") pod \"f0b5aa83-17d7-416f-9634-6544c03fb3db\" (UID: \"f0b5aa83-17d7-416f-9634-6544c03fb3db\") "
Nov 04 01:07:01.763065 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:01.762941    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3e911a11-0ed3-4ce9-a336-507851b62942-kube-api-access-xrp6v\" (UniqueName: \"kubernetes.io/projected/3e911a11-0ed3-4ce9-a336-507851b62942-kube-api-access-xrp6v\") pod \"3e911a11-0ed3-4ce9-a336-507851b62942\" (UID: \"3e911a11-0ed3-4ce9-a336-507851b62942\") "
Nov 04 01:07:01.763463 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:01.763429    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e911a11-0ed3-4ce9-a336-507851b62942-bundle" pod "3e911a11-0ed3-4ce9-a336-507851b62942" (UID: "3e911a11-0ed3-4ce9-a336-507851b62942"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:07:01.763602 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:01.763438    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0b5aa83-17d7-416f-9634-6544c03fb3db-bundle" pod "f0b5aa83-17d7-416f-9634-6544c03fb3db" (UID: "f0b5aa83-17d7-416f-9634-6544c03fb3db"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:07:01.765074 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:01.765049    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e911a11-0ed3-4ce9-a336-507851b62942-kube-api-access-xrp6v" pod "3e911a11-0ed3-4ce9-a336-507851b62942" (UID: "3e911a11-0ed3-4ce9-a336-507851b62942"). InnerVolumeSpecName "kube-api-access-xrp6v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:07:01.765168 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:01.765079    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0b5aa83-17d7-416f-9634-6544c03fb3db-kube-api-access-qb8d2" pod "f0b5aa83-17d7-416f-9634-6544c03fb3db" (UID: "f0b5aa83-17d7-416f-9634-6544c03fb3db"). InnerVolumeSpecName "kube-api-access-qb8d2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:07:01.771256 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:01.771230    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0b5aa83-17d7-416f-9634-6544c03fb3db-util" pod "f0b5aa83-17d7-416f-9634-6544c03fb3db" (UID: "f0b5aa83-17d7-416f-9634-6544c03fb3db"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:07:01.771812 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:01.771782    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e911a11-0ed3-4ce9-a336-507851b62942-util" pod "3e911a11-0ed3-4ce9-a336-507851b62942" (UID: "3e911a11-0ed3-4ce9-a336-507851b62942"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:07:01.864078 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:01.864057    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2422e7ec-790e-4d3c-bd2f-beed4726a21a-util\" (UniqueName: \"kubernetes.io/empty-dir/2422e7ec-790e-4d3c-bd2f-beed4726a21a-util\") pod \"2422e7ec-790e-4d3c-bd2f-beed4726a21a\" (UID: \"2422e7ec-790e-4d3c-bd2f-beed4726a21a\") "
Nov 04 01:07:01.864163 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:01.864089    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2422e7ec-790e-4d3c-bd2f-beed4726a21a-bundle\" (UniqueName: \"kubernetes.io/empty-dir/2422e7ec-790e-4d3c-bd2f-beed4726a21a-bundle\") pod \"2422e7ec-790e-4d3c-bd2f-beed4726a21a\" (UID: \"2422e7ec-790e-4d3c-bd2f-beed4726a21a\") "
Nov 04 01:07:01.864163 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:01.864113    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2422e7ec-790e-4d3c-bd2f-beed4726a21a-kube-api-access-ndtbr\" (UniqueName: \"kubernetes.io/projected/2422e7ec-790e-4d3c-bd2f-beed4726a21a-kube-api-access-ndtbr\") pod \"2422e7ec-790e-4d3c-bd2f-beed4726a21a\" (UID: \"2422e7ec-790e-4d3c-bd2f-beed4726a21a\") "
Nov 04 01:07:01.864304 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:01.864286    5068 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3e911a11-0ed3-4ce9-a336-507851b62942-bundle\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:07:01.864371 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:01.864310    5068 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3e911a11-0ed3-4ce9-a336-507851b62942-util\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:07:01.864371 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:01.864325    5068 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f0b5aa83-17d7-416f-9634-6544c03fb3db-util\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:07:01.864371 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:01.864339    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qb8d2\" (UniqueName: \"kubernetes.io/projected/f0b5aa83-17d7-416f-9634-6544c03fb3db-kube-api-access-qb8d2\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:07:01.864371 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:01.864354    5068 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f0b5aa83-17d7-416f-9634-6544c03fb3db-bundle\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:07:01.864371 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:01.864366    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xrp6v\" (UniqueName: \"kubernetes.io/projected/3e911a11-0ed3-4ce9-a336-507851b62942-kube-api-access-xrp6v\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:07:01.864566 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:01.864532    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2422e7ec-790e-4d3c-bd2f-beed4726a21a-bundle" pod "2422e7ec-790e-4d3c-bd2f-beed4726a21a" (UID: "2422e7ec-790e-4d3c-bd2f-beed4726a21a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:07:01.866144 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:01.866124    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2422e7ec-790e-4d3c-bd2f-beed4726a21a-kube-api-access-ndtbr" pod "2422e7ec-790e-4d3c-bd2f-beed4726a21a" (UID: "2422e7ec-790e-4d3c-bd2f-beed4726a21a"). InnerVolumeSpecName "kube-api-access-ndtbr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:07:01.869688 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:01.869664    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2422e7ec-790e-4d3c-bd2f-beed4726a21a-util" pod "2422e7ec-790e-4d3c-bd2f-beed4726a21a" (UID: "2422e7ec-790e-4d3c-bd2f-beed4726a21a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:07:01.965669 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:01.965644    5068 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2422e7ec-790e-4d3c-bd2f-beed4726a21a-util\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:07:01.965669 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:01.965666    5068 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2422e7ec-790e-4d3c-bd2f-beed4726a21a-bundle\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:07:01.965791 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:01.965677    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ndtbr\" (UniqueName: \"kubernetes.io/projected/2422e7ec-790e-4d3c-bd2f-beed4726a21a-kube-api-access-ndtbr\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:07:02.479593 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:02.479512    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72t5xb" event={"ID":"2422e7ec-790e-4d3c-bd2f-beed4726a21a","Type":"ContainerDied","Data":"dc04b1525161bc92c9a7b02f61e6352a8addae3f39e6e81c49e108ec63d38835"}
Nov 04 01:07:02.479593 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:02.479567    5068 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dc04b1525161bc92c9a7b02f61e6352a8addae3f39e6e81c49e108ec63d38835"
Nov 04 01:07:02.479593 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:02.479526    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72t5xb"
Nov 04 01:07:02.482126 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:02.481687    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wk7ld"
Nov 04 01:07:02.482126 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:02.481694    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wk7ld" event={"ID":"f0b5aa83-17d7-416f-9634-6544c03fb3db","Type":"ContainerDied","Data":"9af67d435788a92474e29ce76fbfa5c391b7f61eca94d628ff21b8faaeb86ff4"}
Nov 04 01:07:02.482126 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:02.481734    5068 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9af67d435788a92474e29ce76fbfa5c391b7f61eca94d628ff21b8faaeb86ff4"
Nov 04 01:07:02.483677 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:02.483637    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amv297" event={"ID":"3e911a11-0ed3-4ce9-a336-507851b62942","Type":"ContainerDied","Data":"09748929e1905da72b5cd8ca5b1cfebddd3994e02c6e83aa3bde8fbd42a2a4af"}
Nov 04 01:07:02.483819 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:02.483690    5068 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="09748929e1905da72b5cd8ca5b1cfebddd3994e02c6e83aa3bde8fbd42a2a4af"
Nov 04 01:07:02.483819 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:02.483705    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amv297"
Nov 04 01:07:02.609535 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:02.609513    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpsjq4"
Nov 04 01:07:02.772721 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:02.772693    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f4a221a4-bf98-45a5-a473-1bf2d0188dae-util\" (UniqueName: \"kubernetes.io/empty-dir/f4a221a4-bf98-45a5-a473-1bf2d0188dae-util\") pod \"f4a221a4-bf98-45a5-a473-1bf2d0188dae\" (UID: \"f4a221a4-bf98-45a5-a473-1bf2d0188dae\") "
Nov 04 01:07:02.772885 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:02.772744    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f4a221a4-bf98-45a5-a473-1bf2d0188dae-bundle\" (UniqueName: \"kubernetes.io/empty-dir/f4a221a4-bf98-45a5-a473-1bf2d0188dae-bundle\") pod \"f4a221a4-bf98-45a5-a473-1bf2d0188dae\" (UID: \"f4a221a4-bf98-45a5-a473-1bf2d0188dae\") "
Nov 04 01:07:02.772885 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:02.772805    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f4a221a4-bf98-45a5-a473-1bf2d0188dae-kube-api-access-tqljb\" (UniqueName: \"kubernetes.io/projected/f4a221a4-bf98-45a5-a473-1bf2d0188dae-kube-api-access-tqljb\") pod \"f4a221a4-bf98-45a5-a473-1bf2d0188dae\" (UID: \"f4a221a4-bf98-45a5-a473-1bf2d0188dae\") "
Nov 04 01:07:02.773291 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:02.773260    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4a221a4-bf98-45a5-a473-1bf2d0188dae-bundle" pod "f4a221a4-bf98-45a5-a473-1bf2d0188dae" (UID: "f4a221a4-bf98-45a5-a473-1bf2d0188dae"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:07:02.775009 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:02.774988    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4a221a4-bf98-45a5-a473-1bf2d0188dae-kube-api-access-tqljb" pod "f4a221a4-bf98-45a5-a473-1bf2d0188dae" (UID: "f4a221a4-bf98-45a5-a473-1bf2d0188dae"). InnerVolumeSpecName "kube-api-access-tqljb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:07:02.777928 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:02.777907    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4a221a4-bf98-45a5-a473-1bf2d0188dae-util" pod "f4a221a4-bf98-45a5-a473-1bf2d0188dae" (UID: "f4a221a4-bf98-45a5-a473-1bf2d0188dae"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:07:02.873482 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:02.873457    5068 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f4a221a4-bf98-45a5-a473-1bf2d0188dae-bundle\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:07:02.873482 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:02.873480    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tqljb\" (UniqueName: \"kubernetes.io/projected/f4a221a4-bf98-45a5-a473-1bf2d0188dae-kube-api-access-tqljb\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:07:02.873642 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:02.873489    5068 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f4a221a4-bf98-45a5-a473-1bf2d0188dae-util\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:07:03.490744 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:03.490711    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpsjq4" event={"ID":"f4a221a4-bf98-45a5-a473-1bf2d0188dae","Type":"ContainerDied","Data":"6edc380ee3225d4acb5545ffc9c608126b829dc549bf441ad01c3df63ada20e1"}
Nov 04 01:07:03.491158 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:03.490747    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpsjq4"
Nov 04 01:07:03.491158 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:03.490756    5068 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6edc380ee3225d4acb5545ffc9c608126b829dc549bf441ad01c3df63ada20e1"
Nov 04 01:07:06.296881 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.296845    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-8ff7fd4ff-65jvg"]
Nov 04 01:07:06.297356 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.297143    5068 cpu_manager.go:414] "containerMap: removing container" podUID="3e911a11-0ed3-4ce9-a336-507851b62942" containerName="extract"
Nov 04 01:07:06.297356 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.297152    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3e911a11-0ed3-4ce9-a336-507851b62942" containerName="extract"
Nov 04 01:07:06.297356 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.297162    5068 cpu_manager.go:414] "containerMap: removing container" podUID="f4a221a4-bf98-45a5-a473-1bf2d0188dae" containerName="extract"
Nov 04 01:07:06.297356 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.297167    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f4a221a4-bf98-45a5-a473-1bf2d0188dae" containerName="extract"
Nov 04 01:07:06.297356 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.297174    5068 cpu_manager.go:414] "containerMap: removing container" podUID="972ec12d-42d6-41c1-812b-b555973f82f9" containerName="pull"
Nov 04 01:07:06.297356 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.297179    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="972ec12d-42d6-41c1-812b-b555973f82f9" containerName="pull"
Nov 04 01:07:06.297356 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.297189    5068 cpu_manager.go:414] "containerMap: removing container" podUID="9306d043-aed8-4f70-9a20-b85f83ca9840" containerName="pull"
Nov 04 01:07:06.297356 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.297193    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9306d043-aed8-4f70-9a20-b85f83ca9840" containerName="pull"
Nov 04 01:07:06.297356 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.297201    5068 cpu_manager.go:414] "containerMap: removing container" podUID="2422e7ec-790e-4d3c-bd2f-beed4726a21a" containerName="pull"
Nov 04 01:07:06.297356 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.297205    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2422e7ec-790e-4d3c-bd2f-beed4726a21a" containerName="pull"
Nov 04 01:07:06.297356 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.297212    5068 cpu_manager.go:414] "containerMap: removing container" podUID="9306d043-aed8-4f70-9a20-b85f83ca9840" containerName="extract"
Nov 04 01:07:06.297356 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.297216    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9306d043-aed8-4f70-9a20-b85f83ca9840" containerName="extract"
Nov 04 01:07:06.297356 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.297228    5068 cpu_manager.go:414] "containerMap: removing container" podUID="3e911a11-0ed3-4ce9-a336-507851b62942" containerName="util"
Nov 04 01:07:06.297356 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.297232    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3e911a11-0ed3-4ce9-a336-507851b62942" containerName="util"
Nov 04 01:07:06.297356 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.297238    5068 cpu_manager.go:414] "containerMap: removing container" podUID="5cffdbec-e9d2-4b78-a9f2-ba87e4015584" containerName="util"
Nov 04 01:07:06.297356 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.297244    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5cffdbec-e9d2-4b78-a9f2-ba87e4015584" containerName="util"
Nov 04 01:07:06.297356 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.297251    5068 cpu_manager.go:414] "containerMap: removing container" podUID="5cffdbec-e9d2-4b78-a9f2-ba87e4015584" containerName="pull"
Nov 04 01:07:06.297356 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.297256    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5cffdbec-e9d2-4b78-a9f2-ba87e4015584" containerName="pull"
Nov 04 01:07:06.297356 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.297264    5068 cpu_manager.go:414] "containerMap: removing container" podUID="f4a221a4-bf98-45a5-a473-1bf2d0188dae" containerName="pull"
Nov 04 01:07:06.297356 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.297268    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f4a221a4-bf98-45a5-a473-1bf2d0188dae" containerName="pull"
Nov 04 01:07:06.297356 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.297274    5068 cpu_manager.go:414] "containerMap: removing container" podUID="2422e7ec-790e-4d3c-bd2f-beed4726a21a" containerName="extract"
Nov 04 01:07:06.297356 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.297279    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2422e7ec-790e-4d3c-bd2f-beed4726a21a" containerName="extract"
Nov 04 01:07:06.297356 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.297285    5068 cpu_manager.go:414] "containerMap: removing container" podUID="3e911a11-0ed3-4ce9-a336-507851b62942" containerName="pull"
Nov 04 01:07:06.297356 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.297290    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3e911a11-0ed3-4ce9-a336-507851b62942" containerName="pull"
Nov 04 01:07:06.297356 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.297297    5068 cpu_manager.go:414] "containerMap: removing container" podUID="f0b5aa83-17d7-416f-9634-6544c03fb3db" containerName="util"
Nov 04 01:07:06.297356 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.297301    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f0b5aa83-17d7-416f-9634-6544c03fb3db" containerName="util"
Nov 04 01:07:06.297356 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.297308    5068 cpu_manager.go:414] "containerMap: removing container" podUID="9306d043-aed8-4f70-9a20-b85f83ca9840" containerName="util"
Nov 04 01:07:06.297356 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.297312    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9306d043-aed8-4f70-9a20-b85f83ca9840" containerName="util"
Nov 04 01:07:06.297356 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.297317    5068 cpu_manager.go:414] "containerMap: removing container" podUID="972ec12d-42d6-41c1-812b-b555973f82f9" containerName="util"
Nov 04 01:07:06.297356 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.297321    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="972ec12d-42d6-41c1-812b-b555973f82f9" containerName="util"
Nov 04 01:07:06.297356 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.297327    5068 cpu_manager.go:414] "containerMap: removing container" podUID="f0b5aa83-17d7-416f-9634-6544c03fb3db" containerName="pull"
Nov 04 01:07:06.297356 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.297331    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f0b5aa83-17d7-416f-9634-6544c03fb3db" containerName="pull"
Nov 04 01:07:06.297356 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.297336    5068 cpu_manager.go:414] "containerMap: removing container" podUID="f0b5aa83-17d7-416f-9634-6544c03fb3db" containerName="extract"
Nov 04 01:07:06.297356 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.297340    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f0b5aa83-17d7-416f-9634-6544c03fb3db" containerName="extract"
Nov 04 01:07:06.297356 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.297347    5068 cpu_manager.go:414] "containerMap: removing container" podUID="972ec12d-42d6-41c1-812b-b555973f82f9" containerName="extract"
Nov 04 01:07:06.297356 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.297351    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="972ec12d-42d6-41c1-812b-b555973f82f9" containerName="extract"
Nov 04 01:07:06.297356 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.297360    5068 cpu_manager.go:414] "containerMap: removing container" podUID="f4a221a4-bf98-45a5-a473-1bf2d0188dae" containerName="util"
Nov 04 01:07:06.297356 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.297364    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f4a221a4-bf98-45a5-a473-1bf2d0188dae" containerName="util"
Nov 04 01:07:06.297356 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.297371    5068 cpu_manager.go:414] "containerMap: removing container" podUID="2422e7ec-790e-4d3c-bd2f-beed4726a21a" containerName="util"
Nov 04 01:07:06.297356 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.297375    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2422e7ec-790e-4d3c-bd2f-beed4726a21a" containerName="util"
Nov 04 01:07:06.298651 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.297380    5068 cpu_manager.go:414] "containerMap: removing container" podUID="5cffdbec-e9d2-4b78-a9f2-ba87e4015584" containerName="extract"
Nov 04 01:07:06.298651 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.297384    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5cffdbec-e9d2-4b78-a9f2-ba87e4015584" containerName="extract"
Nov 04 01:07:06.298651 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.297431    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="5cffdbec-e9d2-4b78-a9f2-ba87e4015584" containerName="extract"
Nov 04 01:07:06.298651 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.297438    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="3e911a11-0ed3-4ce9-a336-507851b62942" containerName="extract"
Nov 04 01:07:06.298651 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.297443    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="f4a221a4-bf98-45a5-a473-1bf2d0188dae" containerName="extract"
Nov 04 01:07:06.298651 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.297450    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="9306d043-aed8-4f70-9a20-b85f83ca9840" containerName="extract"
Nov 04 01:07:06.298651 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.297457    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="2422e7ec-790e-4d3c-bd2f-beed4726a21a" containerName="extract"
Nov 04 01:07:06.298651 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.297463    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="f0b5aa83-17d7-416f-9634-6544c03fb3db" containerName="extract"
Nov 04 01:07:06.298651 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.297468    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="972ec12d-42d6-41c1-812b-b555973f82f9" containerName="extract"
Nov 04 01:07:06.325532 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.325509    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8ff7fd4ff-65jvg"
Nov 04 01:07:06.341330 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.341304    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Nov 04 01:07:06.341465 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.341397    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Nov 04 01:07:06.342431 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.342138    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Nov 04 01:07:06.342431 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.342261    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Nov 04 01:07:06.342749 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.342728    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-ptvlp\""
Nov 04 01:07:06.343183 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.343164    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-8ff7fd4ff-65jvg"]
Nov 04 01:07:06.343537 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.343520    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Nov 04 01:07:06.401168 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.401139    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/781d0e29-3d65-4a97-ac09-aa8f3973e32a-metrics-cert\") pod \"opendatahub-operator-controller-manager-8ff7fd4ff-65jvg\" (UID: \"781d0e29-3d65-4a97-ac09-aa8f3973e32a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8ff7fd4ff-65jvg"
Nov 04 01:07:06.401300 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.401189    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/781d0e29-3d65-4a97-ac09-aa8f3973e32a-tmp\") pod \"opendatahub-operator-controller-manager-8ff7fd4ff-65jvg\" (UID: \"781d0e29-3d65-4a97-ac09-aa8f3973e32a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8ff7fd4ff-65jvg"
Nov 04 01:07:06.401300 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.401280    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/781d0e29-3d65-4a97-ac09-aa8f3973e32a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8ff7fd4ff-65jvg\" (UID: \"781d0e29-3d65-4a97-ac09-aa8f3973e32a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8ff7fd4ff-65jvg"
Nov 04 01:07:06.401416 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.401333    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cn2s\" (UniqueName: \"kubernetes.io/projected/781d0e29-3d65-4a97-ac09-aa8f3973e32a-kube-api-access-4cn2s\") pod \"opendatahub-operator-controller-manager-8ff7fd4ff-65jvg\" (UID: \"781d0e29-3d65-4a97-ac09-aa8f3973e32a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8ff7fd4ff-65jvg"
Nov 04 01:07:06.401416 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.401375    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/781d0e29-3d65-4a97-ac09-aa8f3973e32a-manifests\") pod \"opendatahub-operator-controller-manager-8ff7fd4ff-65jvg\" (UID: \"781d0e29-3d65-4a97-ac09-aa8f3973e32a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8ff7fd4ff-65jvg"
Nov 04 01:07:06.401489 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.401440    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/781d0e29-3d65-4a97-ac09-aa8f3973e32a-webhook-cert\") pod \"opendatahub-operator-controller-manager-8ff7fd4ff-65jvg\" (UID: \"781d0e29-3d65-4a97-ac09-aa8f3973e32a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8ff7fd4ff-65jvg"
Nov 04 01:07:06.502318 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.502290    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/781d0e29-3d65-4a97-ac09-aa8f3973e32a-metrics-cert\") pod \"opendatahub-operator-controller-manager-8ff7fd4ff-65jvg\" (UID: \"781d0e29-3d65-4a97-ac09-aa8f3973e32a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8ff7fd4ff-65jvg"
Nov 04 01:07:06.502476 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.502334    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/781d0e29-3d65-4a97-ac09-aa8f3973e32a-tmp\") pod \"opendatahub-operator-controller-manager-8ff7fd4ff-65jvg\" (UID: \"781d0e29-3d65-4a97-ac09-aa8f3973e32a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8ff7fd4ff-65jvg"
Nov 04 01:07:06.502476 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.502355    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/781d0e29-3d65-4a97-ac09-aa8f3973e32a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8ff7fd4ff-65jvg\" (UID: \"781d0e29-3d65-4a97-ac09-aa8f3973e32a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8ff7fd4ff-65jvg"
Nov 04 01:07:06.502567 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.502487    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cn2s\" (UniqueName: \"kubernetes.io/projected/781d0e29-3d65-4a97-ac09-aa8f3973e32a-kube-api-access-4cn2s\") pod \"opendatahub-operator-controller-manager-8ff7fd4ff-65jvg\" (UID: \"781d0e29-3d65-4a97-ac09-aa8f3973e32a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8ff7fd4ff-65jvg"
Nov 04 01:07:06.502567 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.502538    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/781d0e29-3d65-4a97-ac09-aa8f3973e32a-manifests\") pod \"opendatahub-operator-controller-manager-8ff7fd4ff-65jvg\" (UID: \"781d0e29-3d65-4a97-ac09-aa8f3973e32a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8ff7fd4ff-65jvg"
Nov 04 01:07:06.502690 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.502587    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/781d0e29-3d65-4a97-ac09-aa8f3973e32a-webhook-cert\") pod \"opendatahub-operator-controller-manager-8ff7fd4ff-65jvg\" (UID: \"781d0e29-3d65-4a97-ac09-aa8f3973e32a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8ff7fd4ff-65jvg"
Nov 04 01:07:06.503050 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.503021    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/781d0e29-3d65-4a97-ac09-aa8f3973e32a-manifests\") pod \"opendatahub-operator-controller-manager-8ff7fd4ff-65jvg\" (UID: \"781d0e29-3d65-4a97-ac09-aa8f3973e32a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8ff7fd4ff-65jvg"
Nov 04 01:07:06.505152 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.505126    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/781d0e29-3d65-4a97-ac09-aa8f3973e32a-tmp\") pod \"opendatahub-operator-controller-manager-8ff7fd4ff-65jvg\" (UID: \"781d0e29-3d65-4a97-ac09-aa8f3973e32a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8ff7fd4ff-65jvg"
Nov 04 01:07:06.505485 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.505466    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/781d0e29-3d65-4a97-ac09-aa8f3973e32a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8ff7fd4ff-65jvg\" (UID: \"781d0e29-3d65-4a97-ac09-aa8f3973e32a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8ff7fd4ff-65jvg"
Nov 04 01:07:06.505545 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.505524    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/781d0e29-3d65-4a97-ac09-aa8f3973e32a-webhook-cert\") pod \"opendatahub-operator-controller-manager-8ff7fd4ff-65jvg\" (UID: \"781d0e29-3d65-4a97-ac09-aa8f3973e32a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8ff7fd4ff-65jvg"
Nov 04 01:07:06.505582 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.505524    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/781d0e29-3d65-4a97-ac09-aa8f3973e32a-metrics-cert\") pod \"opendatahub-operator-controller-manager-8ff7fd4ff-65jvg\" (UID: \"781d0e29-3d65-4a97-ac09-aa8f3973e32a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8ff7fd4ff-65jvg"
Nov 04 01:07:06.515032 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.515007    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cn2s\" (UniqueName: \"kubernetes.io/projected/781d0e29-3d65-4a97-ac09-aa8f3973e32a-kube-api-access-4cn2s\") pod \"opendatahub-operator-controller-manager-8ff7fd4ff-65jvg\" (UID: \"781d0e29-3d65-4a97-ac09-aa8f3973e32a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8ff7fd4ff-65jvg"
Nov 04 01:07:06.678578 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:06.678496    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8ff7fd4ff-65jvg"
Nov 04 01:07:07.002421 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:07.002398    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-8ff7fd4ff-65jvg"]
Nov 04 01:07:07.004831 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:07:07.004803    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod781d0e29_3d65_4a97_ac09_aa8f3973e32a.slice/crio-0995db12cb0b3bf04566249924e1d8497b1604e90cf599a1c579428b123fb69b WatchSource:0}: Error finding container 0995db12cb0b3bf04566249924e1d8497b1604e90cf599a1c579428b123fb69b: Status 404 returned error can't find the container with id 0995db12cb0b3bf04566249924e1d8497b1604e90cf599a1c579428b123fb69b
Nov 04 01:07:07.511138 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:07.511081    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8ff7fd4ff-65jvg" event={"ID":"781d0e29-3d65-4a97-ac09-aa8f3973e32a","Type":"ContainerStarted","Data":"0995db12cb0b3bf04566249924e1d8497b1604e90cf599a1c579428b123fb69b"}
Nov 04 01:07:10.528837 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:10.528803    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8ff7fd4ff-65jvg" event={"ID":"781d0e29-3d65-4a97-ac09-aa8f3973e32a","Type":"ContainerStarted","Data":"8533201df68bb51623f43b24bee8869211b8aee53a03ac5085bde7d548f47c2e"}
Nov 04 01:07:11.534589 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:11.534552    5068 generic.go:356] "Generic (PLEG): container finished" podID="781d0e29-3d65-4a97-ac09-aa8f3973e32a" containerID="8533201df68bb51623f43b24bee8869211b8aee53a03ac5085bde7d548f47c2e" exitCode=0
Nov 04 01:07:11.535108 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:11.534597    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8ff7fd4ff-65jvg" event={"ID":"781d0e29-3d65-4a97-ac09-aa8f3973e32a","Type":"ContainerDied","Data":"8533201df68bb51623f43b24bee8869211b8aee53a03ac5085bde7d548f47c2e"}
Nov 04 01:07:12.540681 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:12.540638    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8ff7fd4ff-65jvg" event={"ID":"781d0e29-3d65-4a97-ac09-aa8f3973e32a","Type":"ContainerStarted","Data":"e63612e47758794e7d5393bf618fe0b79bf120b477b75efadffd15179bdfc92d"}
Nov 04 01:07:12.541066 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:12.540810    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8ff7fd4ff-65jvg"
Nov 04 01:07:12.588082 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:12.588036    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8ff7fd4ff-65jvg" podStartSLOduration=2.494161041 podStartE2EDuration="6.588019723s" totalImagesPullingTime="3.681841452s" totalInitContainerRuntime="412.01723ms" isStatefulPod=false podCreationTimestamp="2026-11-04 01:07:06 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:07:12.585211594 +0000 UTC m=+272.631772393" watchObservedRunningTime="2026-11-04 01:07:12.588019723 +0000 UTC m=+272.634580546"
Nov 04 01:07:13.449314 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:07:13.449275    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:07:17.006954 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:17.006916    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-qrdbb"]
Nov 04 01:07:17.013382 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:17.013362    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-qrdbb"
Nov 04 01:07:17.018181 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:17.018157    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Nov 04 01:07:17.018329 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:17.018233    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Nov 04 01:07:17.018329 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:17.018303    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-pzzkf\""
Nov 04 01:07:17.032653 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:17.032628    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-qrdbb"]
Nov 04 01:07:17.093129 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:17.093092    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpq6h\" (UniqueName: \"kubernetes.io/projected/e78aa3b2-cf59-4c9a-bbe7-a06d6f40fd60-kube-api-access-rpq6h\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-qrdbb\" (UID: \"e78aa3b2-cf59-4c9a-bbe7-a06d6f40fd60\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-qrdbb"
Nov 04 01:07:17.093129 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:17.093135    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e78aa3b2-cf59-4c9a-bbe7-a06d6f40fd60-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-qrdbb\" (UID: \"e78aa3b2-cf59-4c9a-bbe7-a06d6f40fd60\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-qrdbb"
Nov 04 01:07:17.194730 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:17.194687    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e78aa3b2-cf59-4c9a-bbe7-a06d6f40fd60-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-qrdbb\" (UID: \"e78aa3b2-cf59-4c9a-bbe7-a06d6f40fd60\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-qrdbb"
Nov 04 01:07:17.194915 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:17.194873    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpq6h\" (UniqueName: \"kubernetes.io/projected/e78aa3b2-cf59-4c9a-bbe7-a06d6f40fd60-kube-api-access-rpq6h\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-qrdbb\" (UID: \"e78aa3b2-cf59-4c9a-bbe7-a06d6f40fd60\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-qrdbb"
Nov 04 01:07:17.195079 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:17.195051    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e78aa3b2-cf59-4c9a-bbe7-a06d6f40fd60-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-qrdbb\" (UID: \"e78aa3b2-cf59-4c9a-bbe7-a06d6f40fd60\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-qrdbb"
Nov 04 01:07:17.207840 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:17.207809    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpq6h\" (UniqueName: \"kubernetes.io/projected/e78aa3b2-cf59-4c9a-bbe7-a06d6f40fd60-kube-api-access-rpq6h\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-qrdbb\" (UID: \"e78aa3b2-cf59-4c9a-bbe7-a06d6f40fd60\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-qrdbb"
Nov 04 01:07:17.324846 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:17.324803    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-qrdbb"
Nov 04 01:07:17.604355 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:17.604326    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-qrdbb"]
Nov 04 01:07:17.606579 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:07:17.606550    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode78aa3b2_cf59_4c9a_bbe7_a06d6f40fd60.slice/crio-6827822f3c45427d12b28c1f9e28315ad1e96fff81974562d771c2004613ec64 WatchSource:0}: Error finding container 6827822f3c45427d12b28c1f9e28315ad1e96fff81974562d771c2004613ec64: Status 404 returned error can't find the container with id 6827822f3c45427d12b28c1f9e28315ad1e96fff81974562d771c2004613ec64
Nov 04 01:07:18.567719 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:18.567674    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-qrdbb" event={"ID":"e78aa3b2-cf59-4c9a-bbe7-a06d6f40fd60","Type":"ContainerStarted","Data":"6827822f3c45427d12b28c1f9e28315ad1e96fff81974562d771c2004613ec64"}
Nov 04 01:07:20.578562 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:20.578516    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-qrdbb" event={"ID":"e78aa3b2-cf59-4c9a-bbe7-a06d6f40fd60","Type":"ContainerStarted","Data":"5bcfbfab19fe215d02ead6fc912bc4126d63032ae07de8a4c8440434b35366de"}
Nov 04 01:07:20.606305 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:20.606251    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-qrdbb" podStartSLOduration=2.1752657 podStartE2EDuration="4.606237224s" totalImagesPullingTime="2.430971524s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:07:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:07:20.604457534 +0000 UTC m=+280.651018326" watchObservedRunningTime="2026-11-04 01:07:20.606237224 +0000 UTC m=+280.652798014"
Nov 04 01:07:23.547679 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:23.547650    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8ff7fd4ff-65jvg"
Nov 04 01:07:24.123849 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:24.123819    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-92fv8"]
Nov 04 01:07:24.127496 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:24.127476    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-92fv8"
Nov 04 01:07:24.135439 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:24.135414    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Nov 04 01:07:24.135566 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:24.135531    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Nov 04 01:07:24.136631 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:24.136590    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-rmxhl\""
Nov 04 01:07:24.137410 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:24.137388    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-92fv8"]
Nov 04 01:07:24.257674 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:24.257635    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfb9s\" (UniqueName: \"kubernetes.io/projected/1000d7e4-0479-4d79-80a5-ab4b441406dd-kube-api-access-gfb9s\") pod \"cert-manager-cainjector-5c57bbd4fc-92fv8\" (UID: \"1000d7e4-0479-4d79-80a5-ab4b441406dd\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-92fv8"
Nov 04 01:07:24.257823 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:24.257728    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1000d7e4-0479-4d79-80a5-ab4b441406dd-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-92fv8\" (UID: \"1000d7e4-0479-4d79-80a5-ab4b441406dd\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-92fv8"
Nov 04 01:07:24.358995 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:24.358955    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1000d7e4-0479-4d79-80a5-ab4b441406dd-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-92fv8\" (UID: \"1000d7e4-0479-4d79-80a5-ab4b441406dd\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-92fv8"
Nov 04 01:07:24.359165 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:24.359026    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfb9s\" (UniqueName: \"kubernetes.io/projected/1000d7e4-0479-4d79-80a5-ab4b441406dd-kube-api-access-gfb9s\") pod \"cert-manager-cainjector-5c57bbd4fc-92fv8\" (UID: \"1000d7e4-0479-4d79-80a5-ab4b441406dd\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-92fv8"
Nov 04 01:07:24.371280 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:24.371251    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1000d7e4-0479-4d79-80a5-ab4b441406dd-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-92fv8\" (UID: \"1000d7e4-0479-4d79-80a5-ab4b441406dd\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-92fv8"
Nov 04 01:07:24.371446 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:24.371427    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfb9s\" (UniqueName: \"kubernetes.io/projected/1000d7e4-0479-4d79-80a5-ab4b441406dd-kube-api-access-gfb9s\") pod \"cert-manager-cainjector-5c57bbd4fc-92fv8\" (UID: \"1000d7e4-0479-4d79-80a5-ab4b441406dd\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-92fv8"
Nov 04 01:07:24.450133 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:24.450046    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-92fv8"
Nov 04 01:07:24.760583 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:24.760558    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-92fv8"]
Nov 04 01:07:24.762685 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:07:24.762657    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1000d7e4_0479_4d79_80a5_ab4b441406dd.slice/crio-9c92093dcbe26c80742f0c67c422388cbb7fad914df4a184bd9b917594976608 WatchSource:0}: Error finding container 9c92093dcbe26c80742f0c67c422388cbb7fad914df4a184bd9b917594976608: Status 404 returned error can't find the container with id 9c92093dcbe26c80742f0c67c422388cbb7fad914df4a184bd9b917594976608
Nov 04 01:07:25.602499 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:25.602459    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-92fv8" event={"ID":"1000d7e4-0479-4d79-80a5-ab4b441406dd","Type":"ContainerStarted","Data":"9c92093dcbe26c80742f0c67c422388cbb7fad914df4a184bd9b917594976608"}
Nov 04 01:07:26.313906 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:26.313870    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5778bb4449-vcbhc"]
Nov 04 01:07:26.318822 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:26.318797    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-vcbhc"
Nov 04 01:07:26.326353 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:26.325936    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-vjr9m\""
Nov 04 01:07:26.326353 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:26.325998    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Nov 04 01:07:26.326353 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:26.326024    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Nov 04 01:07:26.328322 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:26.328300    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5778bb4449-vcbhc"]
Nov 04 01:07:26.376129 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:26.376093    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dt5cx\" (UniqueName: \"kubernetes.io/projected/65e1cd08-d934-467d-a5cb-3fcbdff09f95-kube-api-access-dt5cx\") pod \"limitador-operator-controller-manager-5778bb4449-vcbhc\" (UID: \"65e1cd08-d934-467d-a5cb-3fcbdff09f95\") " pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-vcbhc"
Nov 04 01:07:26.477163 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:26.477122    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dt5cx\" (UniqueName: \"kubernetes.io/projected/65e1cd08-d934-467d-a5cb-3fcbdff09f95-kube-api-access-dt5cx\") pod \"limitador-operator-controller-manager-5778bb4449-vcbhc\" (UID: \"65e1cd08-d934-467d-a5cb-3fcbdff09f95\") " pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-vcbhc"
Nov 04 01:07:26.492022 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:26.491992    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dt5cx\" (UniqueName: \"kubernetes.io/projected/65e1cd08-d934-467d-a5cb-3fcbdff09f95-kube-api-access-dt5cx\") pod \"limitador-operator-controller-manager-5778bb4449-vcbhc\" (UID: \"65e1cd08-d934-467d-a5cb-3fcbdff09f95\") " pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-vcbhc"
Nov 04 01:07:26.633305 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:26.633205    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-vcbhc"
Nov 04 01:07:26.911290 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:26.911228    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5778bb4449-vcbhc"]
Nov 04 01:07:26.913962 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:07:26.913931    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65e1cd08_d934_467d_a5cb_3fcbdff09f95.slice/crio-e80dd2399e85ea1425baf5a10f1a3aa640b5851490edf655a5b5c6fe7699c7b1 WatchSource:0}: Error finding container e80dd2399e85ea1425baf5a10f1a3aa640b5851490edf655a5b5c6fe7699c7b1: Status 404 returned error can't find the container with id e80dd2399e85ea1425baf5a10f1a3aa640b5851490edf655a5b5c6fe7699c7b1
Nov 04 01:07:27.614214 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:27.614177    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-vcbhc" event={"ID":"65e1cd08-d934-467d-a5cb-3fcbdff09f95","Type":"ContainerStarted","Data":"e80dd2399e85ea1425baf5a10f1a3aa640b5851490edf655a5b5c6fe7699c7b1"}
Nov 04 01:07:31.636736 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:31.636696    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-vcbhc" event={"ID":"65e1cd08-d934-467d-a5cb-3fcbdff09f95","Type":"ContainerStarted","Data":"8ad9cd2048568ae7b02d301b59bb541e9695e69ac24858f0162d8b6fd0e84908"}
Nov 04 01:07:31.637118 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:31.636775    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-vcbhc"
Nov 04 01:07:31.659578 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:31.659533    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-vcbhc" podStartSLOduration=1.927214311 podStartE2EDuration="5.659518786s" totalImagesPullingTime="3.732304475s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:07:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:07:31.657602676 +0000 UTC m=+291.704163466" watchObservedRunningTime="2026-11-04 01:07:31.659518786 +0000 UTC m=+291.706079576"
Nov 04 01:07:33.814183 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:33.814153    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-2l2q9"]
Nov 04 01:07:33.817764 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:33.817746    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-2l2q9"
Nov 04 01:07:33.824939 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:33.824918    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-ss6z5\""
Nov 04 01:07:33.833079 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:33.833060    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-2l2q9"]
Nov 04 01:07:33.946968 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:33.946943    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kx9ww\" (UniqueName: \"kubernetes.io/projected/090cca38-c8e7-4a6f-96a7-e569fae6df5a-kube-api-access-kx9ww\") pod \"kuadrant-operator-controller-manager-678d87766f-2l2q9\" (UID: \"090cca38-c8e7-4a6f-96a7-e569fae6df5a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-2l2q9"
Nov 04 01:07:33.947094 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:33.946990    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/090cca38-c8e7-4a6f-96a7-e569fae6df5a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-2l2q9\" (UID: \"090cca38-c8e7-4a6f-96a7-e569fae6df5a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-2l2q9"
Nov 04 01:07:34.048013 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:34.047982    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kx9ww\" (UniqueName: \"kubernetes.io/projected/090cca38-c8e7-4a6f-96a7-e569fae6df5a-kube-api-access-kx9ww\") pod \"kuadrant-operator-controller-manager-678d87766f-2l2q9\" (UID: \"090cca38-c8e7-4a6f-96a7-e569fae6df5a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-2l2q9"
Nov 04 01:07:34.048143 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:34.048038    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/090cca38-c8e7-4a6f-96a7-e569fae6df5a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-2l2q9\" (UID: \"090cca38-c8e7-4a6f-96a7-e569fae6df5a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-2l2q9"
Nov 04 01:07:34.048405 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:34.048383    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/090cca38-c8e7-4a6f-96a7-e569fae6df5a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-2l2q9\" (UID: \"090cca38-c8e7-4a6f-96a7-e569fae6df5a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-2l2q9"
Nov 04 01:07:34.061745 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:34.061724    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kx9ww\" (UniqueName: \"kubernetes.io/projected/090cca38-c8e7-4a6f-96a7-e569fae6df5a-kube-api-access-kx9ww\") pod \"kuadrant-operator-controller-manager-678d87766f-2l2q9\" (UID: \"090cca38-c8e7-4a6f-96a7-e569fae6df5a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-2l2q9"
Nov 04 01:07:34.130252 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:34.130183    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-2l2q9"
Nov 04 01:07:34.411635 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:34.411594    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-2l2q9"]
Nov 04 01:07:34.413859 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:07:34.413831    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod090cca38_c8e7_4a6f_96a7_e569fae6df5a.slice/crio-63d1392501efe5797ca95c232ebace6cc8d34f846f9090d61544beb0c84835f8 WatchSource:0}: Error finding container 63d1392501efe5797ca95c232ebace6cc8d34f846f9090d61544beb0c84835f8: Status 404 returned error can't find the container with id 63d1392501efe5797ca95c232ebace6cc8d34f846f9090d61544beb0c84835f8
Nov 04 01:07:34.651724 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:34.651684    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-2l2q9" event={"ID":"090cca38-c8e7-4a6f-96a7-e569fae6df5a","Type":"ContainerStarted","Data":"63d1392501efe5797ca95c232ebace6cc8d34f846f9090d61544beb0c84835f8"}
Nov 04 01:07:39.489221 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:39.489188    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c"]
Nov 04 01:07:39.498081 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:39.497339    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c"
Nov 04 01:07:39.507493 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:39.507448    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-84xlp\""
Nov 04 01:07:39.511251 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:39.511180    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c"]
Nov 04 01:07:39.600341 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:39.600303    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96d34b7a-d945-4c03-b858-88ef5c11fe03-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c\" (UID: \"96d34b7a-d945-4c03-b858-88ef5c11fe03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c"
Nov 04 01:07:39.600498 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:39.600362    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96d34b7a-d945-4c03-b858-88ef5c11fe03-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c\" (UID: \"96d34b7a-d945-4c03-b858-88ef5c11fe03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c"
Nov 04 01:07:39.600498 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:39.600387    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96d34b7a-d945-4c03-b858-88ef5c11fe03-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c\" (UID: \"96d34b7a-d945-4c03-b858-88ef5c11fe03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c"
Nov 04 01:07:39.600498 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:39.600411    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96d34b7a-d945-4c03-b858-88ef5c11fe03-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c\" (UID: \"96d34b7a-d945-4c03-b858-88ef5c11fe03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c"
Nov 04 01:07:39.600498 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:39.600430    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96d34b7a-d945-4c03-b858-88ef5c11fe03-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c\" (UID: \"96d34b7a-d945-4c03-b858-88ef5c11fe03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c"
Nov 04 01:07:39.600498 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:39.600459    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96d34b7a-d945-4c03-b858-88ef5c11fe03-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c\" (UID: \"96d34b7a-d945-4c03-b858-88ef5c11fe03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c"
Nov 04 01:07:39.600498 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:39.600476    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96d34b7a-d945-4c03-b858-88ef5c11fe03-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c\" (UID: \"96d34b7a-d945-4c03-b858-88ef5c11fe03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c"
Nov 04 01:07:39.600782 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:39.600508    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96d34b7a-d945-4c03-b858-88ef5c11fe03-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c\" (UID: \"96d34b7a-d945-4c03-b858-88ef5c11fe03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c"
Nov 04 01:07:39.600782 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:39.600551    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5db9\" (UniqueName: \"kubernetes.io/projected/96d34b7a-d945-4c03-b858-88ef5c11fe03-kube-api-access-j5db9\") pod \"data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c\" (UID: \"96d34b7a-d945-4c03-b858-88ef5c11fe03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c"
Nov 04 01:07:39.701228 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:39.701186    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5db9\" (UniqueName: \"kubernetes.io/projected/96d34b7a-d945-4c03-b858-88ef5c11fe03-kube-api-access-j5db9\") pod \"data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c\" (UID: \"96d34b7a-d945-4c03-b858-88ef5c11fe03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c"
Nov 04 01:07:39.701448 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:39.701270    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96d34b7a-d945-4c03-b858-88ef5c11fe03-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c\" (UID: \"96d34b7a-d945-4c03-b858-88ef5c11fe03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c"
Nov 04 01:07:39.701448 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:39.701316    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96d34b7a-d945-4c03-b858-88ef5c11fe03-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c\" (UID: \"96d34b7a-d945-4c03-b858-88ef5c11fe03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c"
Nov 04 01:07:39.701448 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:39.701341    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96d34b7a-d945-4c03-b858-88ef5c11fe03-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c\" (UID: \"96d34b7a-d945-4c03-b858-88ef5c11fe03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c"
Nov 04 01:07:39.701448 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:39.701367    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96d34b7a-d945-4c03-b858-88ef5c11fe03-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c\" (UID: \"96d34b7a-d945-4c03-b858-88ef5c11fe03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c"
Nov 04 01:07:39.701448 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:39.701388    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96d34b7a-d945-4c03-b858-88ef5c11fe03-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c\" (UID: \"96d34b7a-d945-4c03-b858-88ef5c11fe03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c"
Nov 04 01:07:39.701448 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:39.701429    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96d34b7a-d945-4c03-b858-88ef5c11fe03-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c\" (UID: \"96d34b7a-d945-4c03-b858-88ef5c11fe03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c"
Nov 04 01:07:39.701811 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:39.701451    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96d34b7a-d945-4c03-b858-88ef5c11fe03-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c\" (UID: \"96d34b7a-d945-4c03-b858-88ef5c11fe03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c"
Nov 04 01:07:39.701811 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:39.701483    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96d34b7a-d945-4c03-b858-88ef5c11fe03-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c\" (UID: \"96d34b7a-d945-4c03-b858-88ef5c11fe03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c"
Nov 04 01:07:39.701923 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:39.701870    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96d34b7a-d945-4c03-b858-88ef5c11fe03-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c\" (UID: \"96d34b7a-d945-4c03-b858-88ef5c11fe03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c"
Nov 04 01:07:39.701923 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:39.701900    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96d34b7a-d945-4c03-b858-88ef5c11fe03-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c\" (UID: \"96d34b7a-d945-4c03-b858-88ef5c11fe03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c"
Nov 04 01:07:39.702026 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:39.701941    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96d34b7a-d945-4c03-b858-88ef5c11fe03-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c\" (UID: \"96d34b7a-d945-4c03-b858-88ef5c11fe03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c"
Nov 04 01:07:39.702180 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:39.702151    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96d34b7a-d945-4c03-b858-88ef5c11fe03-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c\" (UID: \"96d34b7a-d945-4c03-b858-88ef5c11fe03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c"
Nov 04 01:07:39.702598 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:39.702573    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96d34b7a-d945-4c03-b858-88ef5c11fe03-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c\" (UID: \"96d34b7a-d945-4c03-b858-88ef5c11fe03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c"
Nov 04 01:07:39.713945 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:39.713885    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96d34b7a-d945-4c03-b858-88ef5c11fe03-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c\" (UID: \"96d34b7a-d945-4c03-b858-88ef5c11fe03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c"
Nov 04 01:07:39.714267 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:39.714243    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96d34b7a-d945-4c03-b858-88ef5c11fe03-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c\" (UID: \"96d34b7a-d945-4c03-b858-88ef5c11fe03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c"
Nov 04 01:07:39.719878 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:39.719852    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5db9\" (UniqueName: \"kubernetes.io/projected/96d34b7a-d945-4c03-b858-88ef5c11fe03-kube-api-access-j5db9\") pod \"data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c\" (UID: \"96d34b7a-d945-4c03-b858-88ef5c11fe03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c"
Nov 04 01:07:39.720236 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:39.720218    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96d34b7a-d945-4c03-b858-88ef5c11fe03-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c\" (UID: \"96d34b7a-d945-4c03-b858-88ef5c11fe03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c"
Nov 04 01:07:39.819136 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:39.819055    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c"
Nov 04 01:07:40.137912 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:40.137833    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tr8b2_21d1d23b-7b6e-4ab5-9593-cf7784fffea2/console-operator/2.log"
Nov 04 01:07:40.138680 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:40.138658    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tr8b2_21d1d23b-7b6e-4ab5-9593-cf7784fffea2/console-operator/2.log"
Nov 04 01:07:40.147590 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:40.147565    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c"]
Nov 04 01:07:40.150725 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:07:40.150701    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96d34b7a_d945_4c03_b858_88ef5c11fe03.slice/crio-c57d5dda7e3b2f58e161be6075d742eeed3fe19621c31e78ad408038ab6f16d6 WatchSource:0}: Error finding container c57d5dda7e3b2f58e161be6075d742eeed3fe19621c31e78ad408038ab6f16d6: Status 404 returned error can't find the container with id c57d5dda7e3b2f58e161be6075d742eeed3fe19621c31e78ad408038ab6f16d6
Nov 04 01:07:40.153188 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:40.153155    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:07:40.153289 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:40.153241    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:07:40.153338 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:40.153286    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:07:40.690460 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:40.690366    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-92fv8" event={"ID":"1000d7e4-0479-4d79-80a5-ab4b441406dd","Type":"ContainerStarted","Data":"8eb244b7907ab79c2641acf0071960c70e7344cef09fc9b339c59e1ce3610404"}
Nov 04 01:07:40.692826 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:40.692657    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c" event={"ID":"96d34b7a-d945-4c03-b858-88ef5c11fe03","Type":"ContainerStarted","Data":"52496bcf84fe331996789d309e7f9235fdacb8eaae66a1714b18dd862d0bb309"}
Nov 04 01:07:40.693453 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:40.692721    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c" event={"ID":"96d34b7a-d945-4c03-b858-88ef5c11fe03","Type":"ContainerStarted","Data":"c57d5dda7e3b2f58e161be6075d742eeed3fe19621c31e78ad408038ab6f16d6"}
Nov 04 01:07:40.710089 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:40.710046    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-92fv8" podStartSLOduration=1.061835111 podStartE2EDuration="16.710029779s" totalImagesPullingTime="15.648194668s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:07:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:07:40.708180325 +0000 UTC m=+300.754741127" watchObservedRunningTime="2026-11-04 01:07:40.710029779 +0000 UTC m=+300.756590569"
Nov 04 01:07:40.743287 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:40.743231    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c" podStartSLOduration=1.743211827 podStartE2EDuration="1.743211827s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:07:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:07:40.741066177 +0000 UTC m=+300.787626969" watchObservedRunningTime="2026-11-04 01:07:40.743211827 +0000 UTC m=+300.789772619"
Nov 04 01:07:40.819946 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:40.819916    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c"
Nov 04 01:07:41.528774 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:41.528741    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-q9rwc"]
Nov 04 01:07:41.532653 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:41.532600    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-q9rwc"
Nov 04 01:07:41.543583 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:41.543545    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Nov 04 01:07:41.543767 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:41.543662    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Nov 04 01:07:41.543767 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:41.543712    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-module-controller-manager-dockercfg-zt9wk\""
Nov 04 01:07:41.549356 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:41.549304    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"odh-trusted-ca-bundle\""
Nov 04 01:07:41.551904 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:41.551882    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-q9rwc"]
Nov 04 01:07:41.621773 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:41.621743    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/578bfcab-9f61-467e-89b2-911084fbe80f-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-q9rwc\" (UID: \"578bfcab-9f61-467e-89b2-911084fbe80f\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-q9rwc"
Nov 04 01:07:41.621966 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:41.621807    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/578bfcab-9f61-467e-89b2-911084fbe80f-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-q9rwc\" (UID: \"578bfcab-9f61-467e-89b2-911084fbe80f\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-q9rwc"
Nov 04 01:07:41.621966 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:41.621919    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/578bfcab-9f61-467e-89b2-911084fbe80f-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-q9rwc\" (UID: \"578bfcab-9f61-467e-89b2-911084fbe80f\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-q9rwc"
Nov 04 01:07:41.622059 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:41.621972    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rl64x\" (UniqueName: \"kubernetes.io/projected/578bfcab-9f61-467e-89b2-911084fbe80f-kube-api-access-rl64x\") pod \"kserve-module-controller-manager-7566cf5fc8-q9rwc\" (UID: \"578bfcab-9f61-467e-89b2-911084fbe80f\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-q9rwc"
Nov 04 01:07:41.699212 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:41.699171    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-2l2q9" event={"ID":"090cca38-c8e7-4a6f-96a7-e569fae6df5a","Type":"ContainerStarted","Data":"bb76d8f5b420fb00d8906b457d0326b1d999669c3787ff99d2ef0962576c2862"}
Nov 04 01:07:41.699726 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:41.699330    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-2l2q9"
Nov 04 01:07:41.721649 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:41.721583    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-2l2q9" podStartSLOduration=1.989773094 podStartE2EDuration="8.721570056s" totalImagesPullingTime="6.731796962s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:07:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:07:41.720652923 +0000 UTC m=+301.767213712" watchObservedRunningTime="2026-11-04 01:07:41.721570056 +0000 UTC m=+301.768130842"
Nov 04 01:07:41.723190 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:41.723169    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/578bfcab-9f61-467e-89b2-911084fbe80f-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-q9rwc\" (UID: \"578bfcab-9f61-467e-89b2-911084fbe80f\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-q9rwc"
Nov 04 01:07:41.723337 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:41.723226    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rl64x\" (UniqueName: \"kubernetes.io/projected/578bfcab-9f61-467e-89b2-911084fbe80f-kube-api-access-rl64x\") pod \"kserve-module-controller-manager-7566cf5fc8-q9rwc\" (UID: \"578bfcab-9f61-467e-89b2-911084fbe80f\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-q9rwc"
Nov 04 01:07:41.723337 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:41.723258    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/578bfcab-9f61-467e-89b2-911084fbe80f-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-q9rwc\" (UID: \"578bfcab-9f61-467e-89b2-911084fbe80f\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-q9rwc"
Nov 04 01:07:41.723452 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:41.723350    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/578bfcab-9f61-467e-89b2-911084fbe80f-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-q9rwc\" (UID: \"578bfcab-9f61-467e-89b2-911084fbe80f\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-q9rwc"
Nov 04 01:07:41.723575 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:41.723555    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/578bfcab-9f61-467e-89b2-911084fbe80f-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-q9rwc\" (UID: \"578bfcab-9f61-467e-89b2-911084fbe80f\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-q9rwc"
Nov 04 01:07:41.723673 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:41.723645    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/578bfcab-9f61-467e-89b2-911084fbe80f-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-q9rwc\" (UID: \"578bfcab-9f61-467e-89b2-911084fbe80f\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-q9rwc"
Nov 04 01:07:41.724337 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:41.724315    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/578bfcab-9f61-467e-89b2-911084fbe80f-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-q9rwc\" (UID: \"578bfcab-9f61-467e-89b2-911084fbe80f\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-q9rwc"
Nov 04 01:07:41.743058 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:41.743031    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rl64x\" (UniqueName: \"kubernetes.io/projected/578bfcab-9f61-467e-89b2-911084fbe80f-kube-api-access-rl64x\") pod \"kserve-module-controller-manager-7566cf5fc8-q9rwc\" (UID: \"578bfcab-9f61-467e-89b2-911084fbe80f\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-q9rwc"
Nov 04 01:07:41.824944 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:41.824859    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c"
Nov 04 01:07:41.848214 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:41.848186    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-q9rwc"
Nov 04 01:07:42.126840 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:42.126812    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-q9rwc"]
Nov 04 01:07:42.128885 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:07:42.128858    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod578bfcab_9f61_467e_89b2_911084fbe80f.slice/crio-23135d20216521d10dc9921c4feeb1e526db2d5220b4dc42c165dae1b278a883 WatchSource:0}: Error finding container 23135d20216521d10dc9921c4feeb1e526db2d5220b4dc42c165dae1b278a883: Status 404 returned error can't find the container with id 23135d20216521d10dc9921c4feeb1e526db2d5220b4dc42c165dae1b278a883
Nov 04 01:07:42.130984 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:42.130961    5068 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 01:07:42.394754 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:42.394679    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-597b95cb84-jpvwv"]
Nov 04 01:07:42.399271 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:42.399251    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-jpvwv"
Nov 04 01:07:42.407452 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:42.407428    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-jpvwv"]
Nov 04 01:07:42.442941 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:42.442914    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-2rm45\""
Nov 04 01:07:42.531040 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:42.531006    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smnxf\" (UniqueName: \"kubernetes.io/projected/af72862e-4091-4b85-bed2-09b551618e35-kube-api-access-smnxf\") pod \"cert-manager-597b95cb84-jpvwv\" (UID: \"af72862e-4091-4b85-bed2-09b551618e35\") " pod="cert-manager/cert-manager-597b95cb84-jpvwv"
Nov 04 01:07:42.531232 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:42.531147    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/af72862e-4091-4b85-bed2-09b551618e35-bound-sa-token\") pod \"cert-manager-597b95cb84-jpvwv\" (UID: \"af72862e-4091-4b85-bed2-09b551618e35\") " pod="cert-manager/cert-manager-597b95cb84-jpvwv"
Nov 04 01:07:42.632192 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:42.632154    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/af72862e-4091-4b85-bed2-09b551618e35-bound-sa-token\") pod \"cert-manager-597b95cb84-jpvwv\" (UID: \"af72862e-4091-4b85-bed2-09b551618e35\") " pod="cert-manager/cert-manager-597b95cb84-jpvwv"
Nov 04 01:07:42.632504 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:42.632210    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smnxf\" (UniqueName: \"kubernetes.io/projected/af72862e-4091-4b85-bed2-09b551618e35-kube-api-access-smnxf\") pod \"cert-manager-597b95cb84-jpvwv\" (UID: \"af72862e-4091-4b85-bed2-09b551618e35\") " pod="cert-manager/cert-manager-597b95cb84-jpvwv"
Nov 04 01:07:42.642304 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:42.642280    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-vcbhc"
Nov 04 01:07:42.647538 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:42.647479    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/af72862e-4091-4b85-bed2-09b551618e35-bound-sa-token\") pod \"cert-manager-597b95cb84-jpvwv\" (UID: \"af72862e-4091-4b85-bed2-09b551618e35\") " pod="cert-manager/cert-manager-597b95cb84-jpvwv"
Nov 04 01:07:42.647663 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:42.647649    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-smnxf\" (UniqueName: \"kubernetes.io/projected/af72862e-4091-4b85-bed2-09b551618e35-kube-api-access-smnxf\") pod \"cert-manager-597b95cb84-jpvwv\" (UID: \"af72862e-4091-4b85-bed2-09b551618e35\") " pod="cert-manager/cert-manager-597b95cb84-jpvwv"
Nov 04 01:07:42.705993 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:42.705956    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-q9rwc" event={"ID":"578bfcab-9f61-467e-89b2-911084fbe80f","Type":"ContainerStarted","Data":"23135d20216521d10dc9921c4feeb1e526db2d5220b4dc42c165dae1b278a883"}
Nov 04 01:07:42.706461 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:42.706348    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c"
Nov 04 01:07:42.707227 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:42.707202    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c"
Nov 04 01:07:42.712279 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:42.712260    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-jpvwv"
Nov 04 01:07:42.730080 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:42.730055    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:07:42.733662 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:42.733643    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7c9b87d574-gqfds"
Nov 04 01:07:42.744786 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:42.744724    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-czjxd\""
Nov 04 01:07:42.755774 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:42.754423    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:07:42.780186 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:42.780158    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:07:42.807198 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:42.807163    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Nov 04 01:07:42.833809 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:42.833772    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fd577f45-13c7-4247-9621-256b73bcf243-config-file\") pod \"limitador-limitador-7c9b87d574-gqfds\" (UID: \"fd577f45-13c7-4247-9621-256b73bcf243\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-gqfds"
Nov 04 01:07:42.833961 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:42.833884    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kz8fm\" (UniqueName: \"kubernetes.io/projected/fd577f45-13c7-4247-9621-256b73bcf243-kube-api-access-kz8fm\") pod \"limitador-limitador-7c9b87d574-gqfds\" (UID: \"fd577f45-13c7-4247-9621-256b73bcf243\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-gqfds"
Nov 04 01:07:42.935437 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:42.935358    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fd577f45-13c7-4247-9621-256b73bcf243-config-file\") pod \"limitador-limitador-7c9b87d574-gqfds\" (UID: \"fd577f45-13c7-4247-9621-256b73bcf243\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-gqfds"
Nov 04 01:07:42.935437 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:42.935426    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kz8fm\" (UniqueName: \"kubernetes.io/projected/fd577f45-13c7-4247-9621-256b73bcf243-kube-api-access-kz8fm\") pod \"limitador-limitador-7c9b87d574-gqfds\" (UID: \"fd577f45-13c7-4247-9621-256b73bcf243\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-gqfds"
Nov 04 01:07:42.936096 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:42.936073    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fd577f45-13c7-4247-9621-256b73bcf243-config-file\") pod \"limitador-limitador-7c9b87d574-gqfds\" (UID: \"fd577f45-13c7-4247-9621-256b73bcf243\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-gqfds"
Nov 04 01:07:42.978024 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:42.977989    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kz8fm\" (UniqueName: \"kubernetes.io/projected/fd577f45-13c7-4247-9621-256b73bcf243-kube-api-access-kz8fm\") pod \"limitador-limitador-7c9b87d574-gqfds\" (UID: \"fd577f45-13c7-4247-9621-256b73bcf243\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-gqfds"
Nov 04 01:07:43.005347 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:43.005310    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-jpvwv"]
Nov 04 01:07:43.010546 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:07:43.010516    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaf72862e_4091_4b85_bed2_09b551618e35.slice/crio-882fcebdf5fc47f83d5f771eb2441ce80dd2636667fe3ebcdda61de878809a71 WatchSource:0}: Error finding container 882fcebdf5fc47f83d5f771eb2441ce80dd2636667fe3ebcdda61de878809a71: Status 404 returned error can't find the container with id 882fcebdf5fc47f83d5f771eb2441ce80dd2636667fe3ebcdda61de878809a71
Nov 04 01:07:43.064896 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:43.064867    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7c9b87d574-gqfds"
Nov 04 01:07:43.383958 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:43.383918    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:07:43.386296 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:07:43.386257    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd577f45_13c7_4247_9621_256b73bcf243.slice/crio-bf12d30f9a1d9c2d6eb966dfe6963f32a54305d1ea873bf0aab57d22a41d328c WatchSource:0}: Error finding container bf12d30f9a1d9c2d6eb966dfe6963f32a54305d1ea873bf0aab57d22a41d328c: Status 404 returned error can't find the container with id bf12d30f9a1d9c2d6eb966dfe6963f32a54305d1ea873bf0aab57d22a41d328c
Nov 04 01:07:43.458954 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:07:43.458900    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:07:43.714742 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:43.714650    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-jpvwv" event={"ID":"af72862e-4091-4b85-bed2-09b551618e35","Type":"ContainerStarted","Data":"d083d61a2008a388396799d86f6909838447954a9c570cc03134ce4662bfc009"}
Nov 04 01:07:43.714742 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:43.714710    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-jpvwv" event={"ID":"af72862e-4091-4b85-bed2-09b551618e35","Type":"ContainerStarted","Data":"882fcebdf5fc47f83d5f771eb2441ce80dd2636667fe3ebcdda61de878809a71"}
Nov 04 01:07:43.716051 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:43.716026    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7c9b87d574-gqfds" event={"ID":"fd577f45-13c7-4247-9621-256b73bcf243","Type":"ContainerStarted","Data":"bf12d30f9a1d9c2d6eb966dfe6963f32a54305d1ea873bf0aab57d22a41d328c"}
Nov 04 01:07:43.739843 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:43.739791    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-597b95cb84-jpvwv" podStartSLOduration=1.739776359 podStartE2EDuration="1.739776359s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:07:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:07:43.73740899 +0000 UTC m=+303.783969781" watchObservedRunningTime="2026-11-04 01:07:43.739776359 +0000 UTC m=+303.786337149"
Nov 04 01:07:45.729743 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:45.729702    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-q9rwc" event={"ID":"578bfcab-9f61-467e-89b2-911084fbe80f","Type":"ContainerStarted","Data":"51b56da2a5b86b7c65c053e84788bae11008aa63d4a45f137339ccb63333511d"}
Nov 04 01:07:45.730174 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:45.729804    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-q9rwc"
Nov 04 01:07:45.731221 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:45.731200    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7c9b87d574-gqfds" event={"ID":"fd577f45-13c7-4247-9621-256b73bcf243","Type":"ContainerStarted","Data":"259f44afa61bc878ac9d2134ee55b145b0e30711d08b0c2b6a093c0de3a7daff"}
Nov 04 01:07:45.731290 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:45.731247    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-7c9b87d574-gqfds"
Nov 04 01:07:45.753022 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:45.752972    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-q9rwc" podStartSLOduration=1.99381794 podStartE2EDuration="4.75296042s" totalImagesPullingTime="2.75914248s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:07:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:07:45.751370207 +0000 UTC m=+305.797930998" watchObservedRunningTime="2026-11-04 01:07:45.75296042 +0000 UTC m=+305.799521209"
Nov 04 01:07:45.776973 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:45.776928    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-7c9b87d574-gqfds" podStartSLOduration=2.275964631 podStartE2EDuration="3.77691626s" totalImagesPullingTime="1.500951629s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:07:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:07:45.775031647 +0000 UTC m=+305.821592441" watchObservedRunningTime="2026-11-04 01:07:45.77691626 +0000 UTC m=+305.823477050"
Nov 04 01:07:52.708298 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:52.708216    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-2l2q9"
Nov 04 01:07:56.737825 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:07:56.737792    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-7c9b87d574-gqfds"
Nov 04 01:08:06.738184 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:08:06.738154    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-q9rwc"
Nov 04 01:08:13.448863 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:08:13.448823    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:08:35.314266 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:08:35.314235    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-bzqjr"]
Nov 04 01:08:35.318576 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:08:35.318553    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-bzqjr"
Nov 04 01:08:35.331303 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:08:35.331278    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-np2f2\""
Nov 04 01:08:35.331594 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:08:35.331506    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-bzqjr"]
Nov 04 01:08:35.331835 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:08:35.331550    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Nov 04 01:08:35.371109 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:08:35.371086    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b898224c-e77b-4a10-aa3a-56eaed0da2c5-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-bzqjr\" (UID: \"b898224c-e77b-4a10-aa3a-56eaed0da2c5\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-bzqjr"
Nov 04 01:08:35.371223 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:08:35.371149    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zq7g8\" (UniqueName: \"kubernetes.io/projected/b898224c-e77b-4a10-aa3a-56eaed0da2c5-kube-api-access-zq7g8\") pod \"llmisvc-controller-manager-869d5dcc5d-bzqjr\" (UID: \"b898224c-e77b-4a10-aa3a-56eaed0da2c5\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-bzqjr"
Nov 04 01:08:35.472587 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:08:35.472554    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b898224c-e77b-4a10-aa3a-56eaed0da2c5-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-bzqjr\" (UID: \"b898224c-e77b-4a10-aa3a-56eaed0da2c5\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-bzqjr"
Nov 04 01:08:35.472746 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:08:35.472651    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zq7g8\" (UniqueName: \"kubernetes.io/projected/b898224c-e77b-4a10-aa3a-56eaed0da2c5-kube-api-access-zq7g8\") pod \"llmisvc-controller-manager-869d5dcc5d-bzqjr\" (UID: \"b898224c-e77b-4a10-aa3a-56eaed0da2c5\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-bzqjr"
Nov 04 01:08:35.475055 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:08:35.475030    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b898224c-e77b-4a10-aa3a-56eaed0da2c5-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-bzqjr\" (UID: \"b898224c-e77b-4a10-aa3a-56eaed0da2c5\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-bzqjr"
Nov 04 01:08:35.485090 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:08:35.485065    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zq7g8\" (UniqueName: \"kubernetes.io/projected/b898224c-e77b-4a10-aa3a-56eaed0da2c5-kube-api-access-zq7g8\") pod \"llmisvc-controller-manager-869d5dcc5d-bzqjr\" (UID: \"b898224c-e77b-4a10-aa3a-56eaed0da2c5\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-bzqjr"
Nov 04 01:08:35.632228 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:08:35.632170    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-bzqjr"
Nov 04 01:08:35.904222 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:08:35.904168    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-bzqjr"]
Nov 04 01:08:35.906308 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:08:35.906276    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb898224c_e77b_4a10_aa3a_56eaed0da2c5.slice/crio-4ada9f351fc80a7412eebee8cc69923cc4fea286d5685e5a5475930455ffeb1b WatchSource:0}: Error finding container 4ada9f351fc80a7412eebee8cc69923cc4fea286d5685e5a5475930455ffeb1b: Status 404 returned error can't find the container with id 4ada9f351fc80a7412eebee8cc69923cc4fea286d5685e5a5475930455ffeb1b
Nov 04 01:08:35.982271 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:08:35.982240    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-bzqjr" event={"ID":"b898224c-e77b-4a10-aa3a-56eaed0da2c5","Type":"ContainerStarted","Data":"4ada9f351fc80a7412eebee8cc69923cc4fea286d5685e5a5475930455ffeb1b"}
Nov 04 01:08:39.000939 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:08:39.000896    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-bzqjr" event={"ID":"b898224c-e77b-4a10-aa3a-56eaed0da2c5","Type":"ContainerStarted","Data":"2daa492332bd36fd47ca75a71ee617e56bf225208265b574f3d848f51d8acb46"}
Nov 04 01:08:39.001455 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:08:39.001035    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-bzqjr"
Nov 04 01:08:39.022750 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:08:39.022702    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-bzqjr" podStartSLOduration=1.7711288760000001 podStartE2EDuration="4.0226877s" totalImagesPullingTime="2.251558824s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:08:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:08:39.021088084 +0000 UTC m=+359.067648900" watchObservedRunningTime="2026-11-04 01:08:39.0226877 +0000 UTC m=+359.069248555"
Nov 04 01:08:43.449587 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:08:43.449544    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:09:03.312519 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:03.312483    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:09:10.011759 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:10.011725    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-bzqjr"
Nov 04 01:09:11.280774 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:11.280729    5068 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-848f6bf8bc-df8wd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:09:11.281150 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:11.280792    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-df8wd" podUID="70049bd7-ed93-4151-a7de-1938bbe0590d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:09:13.206914 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:13.206881    5068 generic.go:356] "Generic (PLEG): container finished" podID="70049bd7-ed93-4151-a7de-1938bbe0590d" containerID="49f2830e424fbd578cd6a6821e25c04bc26d6bc61f80cfd780aed1c237641c93" exitCode=0
Nov 04 01:09:13.207390 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:13.206954    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-df8wd" event={"ID":"70049bd7-ed93-4151-a7de-1938bbe0590d","Type":"ContainerDied","Data":"49f2830e424fbd578cd6a6821e25c04bc26d6bc61f80cfd780aed1c237641c93"}
Nov 04 01:09:13.207390 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:13.207321    5068 scope.go:122] "RemoveContainer" containerID="49f2830e424fbd578cd6a6821e25c04bc26d6bc61f80cfd780aed1c237641c93"
Nov 04 01:09:13.208604 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:13.208574    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:09:13.208723 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:13.208665    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:09:13.208723 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:13.208701    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:09:13.456182 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:09:13.456137    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:09:13.626468 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:13.626432    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-668dc6579-zt6jf"]
Nov 04 01:09:13.630164 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:13.630142    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-668dc6579-zt6jf"
Nov 04 01:09:13.643632 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:13.639793    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-668dc6579-zt6jf"]
Nov 04 01:09:13.663635 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:13.662461    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Nov 04 01:09:13.663635 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:13.662471    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-rbkg2\""
Nov 04 01:09:13.663635 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:13.662789    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Nov 04 01:09:13.694172 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:13.694143    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 01:09:13.694369 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:13.694224    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 01:09:13.720669 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:13.720634    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efb6ce93-18f8-44a6-9df0-90b96469447f-maas-api-tls\") pod \"maas-api-668dc6579-zt6jf\" (UID: \"efb6ce93-18f8-44a6-9df0-90b96469447f\") " pod="odh-ai-gateway-infra/maas-api-668dc6579-zt6jf"
Nov 04 01:09:13.720804 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:13.720674    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efb6ce93-18f8-44a6-9df0-90b96469447f-metrics-certs\") pod \"maas-api-668dc6579-zt6jf\" (UID: \"efb6ce93-18f8-44a6-9df0-90b96469447f\") " pod="odh-ai-gateway-infra/maas-api-668dc6579-zt6jf"
Nov 04 01:09:13.720804 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:13.720702    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efb6ce93-18f8-44a6-9df0-90b96469447f-openshift-service-ca\") pod \"maas-api-668dc6579-zt6jf\" (UID: \"efb6ce93-18f8-44a6-9df0-90b96469447f\") " pod="odh-ai-gateway-infra/maas-api-668dc6579-zt6jf"
Nov 04 01:09:13.720804 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:13.720782    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zx79b\" (UniqueName: \"kubernetes.io/projected/efb6ce93-18f8-44a6-9df0-90b96469447f-kube-api-access-zx79b\") pod \"maas-api-668dc6579-zt6jf\" (UID: \"efb6ce93-18f8-44a6-9df0-90b96469447f\") " pod="odh-ai-gateway-infra/maas-api-668dc6579-zt6jf"
Nov 04 01:09:13.821791 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:13.821756    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zx79b\" (UniqueName: \"kubernetes.io/projected/efb6ce93-18f8-44a6-9df0-90b96469447f-kube-api-access-zx79b\") pod \"maas-api-668dc6579-zt6jf\" (UID: \"efb6ce93-18f8-44a6-9df0-90b96469447f\") " pod="odh-ai-gateway-infra/maas-api-668dc6579-zt6jf"
Nov 04 01:09:13.821995 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:13.821845    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efb6ce93-18f8-44a6-9df0-90b96469447f-maas-api-tls\") pod \"maas-api-668dc6579-zt6jf\" (UID: \"efb6ce93-18f8-44a6-9df0-90b96469447f\") " pod="odh-ai-gateway-infra/maas-api-668dc6579-zt6jf"
Nov 04 01:09:13.821995 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:13.821893    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efb6ce93-18f8-44a6-9df0-90b96469447f-metrics-certs\") pod \"maas-api-668dc6579-zt6jf\" (UID: \"efb6ce93-18f8-44a6-9df0-90b96469447f\") " pod="odh-ai-gateway-infra/maas-api-668dc6579-zt6jf"
Nov 04 01:09:13.821995 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:13.821939    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efb6ce93-18f8-44a6-9df0-90b96469447f-openshift-service-ca\") pod \"maas-api-668dc6579-zt6jf\" (UID: \"efb6ce93-18f8-44a6-9df0-90b96469447f\") " pod="odh-ai-gateway-infra/maas-api-668dc6579-zt6jf"
Nov 04 01:09:13.822588 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:13.822561    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efb6ce93-18f8-44a6-9df0-90b96469447f-openshift-service-ca\") pod \"maas-api-668dc6579-zt6jf\" (UID: \"efb6ce93-18f8-44a6-9df0-90b96469447f\") " pod="odh-ai-gateway-infra/maas-api-668dc6579-zt6jf"
Nov 04 01:09:13.824266 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:13.824246    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efb6ce93-18f8-44a6-9df0-90b96469447f-metrics-certs\") pod \"maas-api-668dc6579-zt6jf\" (UID: \"efb6ce93-18f8-44a6-9df0-90b96469447f\") " pod="odh-ai-gateway-infra/maas-api-668dc6579-zt6jf"
Nov 04 01:09:13.824357 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:13.824283    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efb6ce93-18f8-44a6-9df0-90b96469447f-maas-api-tls\") pod \"maas-api-668dc6579-zt6jf\" (UID: \"efb6ce93-18f8-44a6-9df0-90b96469447f\") " pod="odh-ai-gateway-infra/maas-api-668dc6579-zt6jf"
Nov 04 01:09:13.833728 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:13.833708    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zx79b\" (UniqueName: \"kubernetes.io/projected/efb6ce93-18f8-44a6-9df0-90b96469447f-kube-api-access-zx79b\") pod \"maas-api-668dc6579-zt6jf\" (UID: \"efb6ce93-18f8-44a6-9df0-90b96469447f\") " pod="odh-ai-gateway-infra/maas-api-668dc6579-zt6jf"
Nov 04 01:09:13.967856 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:13.967766    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-668dc6579-zt6jf"
Nov 04 01:09:14.213619 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:14.213569    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-df8wd" event={"ID":"70049bd7-ed93-4151-a7de-1938bbe0590d","Type":"ContainerStarted","Data":"0392fe44c965240524ec43812538046427d333a22ea3df83cadf82e6062b890f"}
Nov 04 01:09:14.273941 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:14.273908    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-668dc6579-zt6jf"]
Nov 04 01:09:14.274573 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:09:14.274547    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefb6ce93_18f8_44a6_9df0_90b96469447f.slice/crio-de7a3173f23325cab060c3267c38666b2f97951bfef1b27bbe3eb48af8b077ad WatchSource:0}: Error finding container de7a3173f23325cab060c3267c38666b2f97951bfef1b27bbe3eb48af8b077ad: Status 404 returned error can't find the container with id de7a3173f23325cab060c3267c38666b2f97951bfef1b27bbe3eb48af8b077ad
Nov 04 01:09:14.634732 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:14.634636    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-df8wd"
Nov 04 01:09:14.634901 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:14.634798    5068 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-848f6bf8bc-df8wd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" start-of-body=
Nov 04 01:09:14.634901 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:14.634861    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-df8wd" podUID="70049bd7-ed93-4151-a7de-1938bbe0590d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused"
Nov 04 01:09:15.222058 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:15.222011    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-668dc6579-zt6jf" event={"ID":"efb6ce93-18f8-44a6-9df0-90b96469447f","Type":"ContainerStarted","Data":"de7a3173f23325cab060c3267c38666b2f97951bfef1b27bbe3eb48af8b077ad"}
Nov 04 01:09:15.634569 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:15.634533    5068 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-848f6bf8bc-df8wd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" start-of-body=
Nov 04 01:09:15.634768 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:15.634636    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-df8wd" podUID="70049bd7-ed93-4151-a7de-1938bbe0590d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused"
Nov 04 01:09:16.634857 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:16.634815    5068 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-848f6bf8bc-df8wd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" start-of-body=
Nov 04 01:09:16.635272 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:16.634883    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-df8wd" podUID="70049bd7-ed93-4151-a7de-1938bbe0590d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused"
Nov 04 01:09:17.634705 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:17.634672    5068 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-848f6bf8bc-df8wd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" start-of-body=
Nov 04 01:09:17.634864 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:17.634727    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-df8wd" podUID="70049bd7-ed93-4151-a7de-1938bbe0590d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused"
Nov 04 01:09:18.256646 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:18.255978    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-668dc6579-zt6jf" event={"ID":"efb6ce93-18f8-44a6-9df0-90b96469447f","Type":"ContainerStarted","Data":"0554c45190fc62ff1515ceb96ec8d46313cb7f84e173a731cdf34ea9154abc85"}
Nov 04 01:09:18.257159 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:18.256957    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-668dc6579-zt6jf"
Nov 04 01:09:18.280831 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:18.280775    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-668dc6579-zt6jf" podStartSLOduration=1.8627406039999999 podStartE2EDuration="5.280761916s" totalImagesPullingTime="3.418021312s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:09:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:09:18.278998242 +0000 UTC m=+398.325559032" watchObservedRunningTime="2026-11-04 01:09:18.280761916 +0000 UTC m=+398.327322706"
Nov 04 01:09:18.635416 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:18.635379    5068 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-848f6bf8bc-df8wd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" start-of-body=
Nov 04 01:09:18.635588 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:18.635436    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-df8wd" podUID="70049bd7-ed93-4151-a7de-1938bbe0590d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused"
Nov 04 01:09:19.638192 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:19.638116    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-df8wd"
Nov 04 01:09:19.638598 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:19.638379    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-df8wd"
Nov 04 01:09:19.639217 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:19.639194    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-df8wd"
Nov 04 01:09:25.270935 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:25.270907    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-668dc6579-zt6jf"
Nov 04 01:09:36.911997 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:36.911959    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-ntcsr"]
Nov 04 01:09:36.918364 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:36.918340    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-ntcsr"
Nov 04 01:09:36.925053 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:36.925023    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-ntcsr"]
Nov 04 01:09:36.949950 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:36.949915    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1d70f32b-d928-4c1f-a5c3-fd4a55455107-cert\") pod \"llmisvc-controller-manager-6466c554d8-ntcsr\" (UID: \"1d70f32b-d928-4c1f-a5c3-fd4a55455107\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-ntcsr"
Nov 04 01:09:36.950107 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:36.950021    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7789\" (UniqueName: \"kubernetes.io/projected/1d70f32b-d928-4c1f-a5c3-fd4a55455107-kube-api-access-x7789\") pod \"llmisvc-controller-manager-6466c554d8-ntcsr\" (UID: \"1d70f32b-d928-4c1f-a5c3-fd4a55455107\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-ntcsr"
Nov 04 01:09:37.050566 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:37.050525    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1d70f32b-d928-4c1f-a5c3-fd4a55455107-cert\") pod \"llmisvc-controller-manager-6466c554d8-ntcsr\" (UID: \"1d70f32b-d928-4c1f-a5c3-fd4a55455107\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-ntcsr"
Nov 04 01:09:37.050764 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:37.050599    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7789\" (UniqueName: \"kubernetes.io/projected/1d70f32b-d928-4c1f-a5c3-fd4a55455107-kube-api-access-x7789\") pod \"llmisvc-controller-manager-6466c554d8-ntcsr\" (UID: \"1d70f32b-d928-4c1f-a5c3-fd4a55455107\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-ntcsr"
Nov 04 01:09:37.052970 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:37.052951    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1d70f32b-d928-4c1f-a5c3-fd4a55455107-cert\") pod \"llmisvc-controller-manager-6466c554d8-ntcsr\" (UID: \"1d70f32b-d928-4c1f-a5c3-fd4a55455107\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-ntcsr"
Nov 04 01:09:37.063059 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:37.063036    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7789\" (UniqueName: \"kubernetes.io/projected/1d70f32b-d928-4c1f-a5c3-fd4a55455107-kube-api-access-x7789\") pod \"llmisvc-controller-manager-6466c554d8-ntcsr\" (UID: \"1d70f32b-d928-4c1f-a5c3-fd4a55455107\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-ntcsr"
Nov 04 01:09:37.234292 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:37.234210    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-ntcsr"
Nov 04 01:09:37.513979 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:37.513950    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-ntcsr"]
Nov 04 01:09:37.515459 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:09:37.515438    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d70f32b_d928_4c1f_a5c3_fd4a55455107.slice/crio-93ea3ab2926758f20fe5f6b2d36290f00bd43fbde44f07708c2492c3458fe8ab WatchSource:0}: Error finding container 93ea3ab2926758f20fe5f6b2d36290f00bd43fbde44f07708c2492c3458fe8ab: Status 404 returned error can't find the container with id 93ea3ab2926758f20fe5f6b2d36290f00bd43fbde44f07708c2492c3458fe8ab
Nov 04 01:09:38.357452 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:38.357350    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-ntcsr" event={"ID":"1d70f32b-d928-4c1f-a5c3-fd4a55455107","Type":"ContainerStarted","Data":"b49a2acdb8f3b29357f12adab5b16fdf5c11c698209e7674b8775357c4d3e0a0"}
Nov 04 01:09:38.357452 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:38.357419    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-ntcsr" event={"ID":"1d70f32b-d928-4c1f-a5c3-fd4a55455107","Type":"ContainerStarted","Data":"93ea3ab2926758f20fe5f6b2d36290f00bd43fbde44f07708c2492c3458fe8ab"}
Nov 04 01:09:38.358044 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:38.357468    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-ntcsr"
Nov 04 01:09:38.378355 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:09:38.378292    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-6466c554d8-ntcsr" podStartSLOduration=1.789226422 podStartE2EDuration="2.378272693s" totalImagesPullingTime="589.046271ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:09:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:09:38.375555139 +0000 UTC m=+418.422115931" watchObservedRunningTime="2026-11-04 01:09:38.378272693 +0000 UTC m=+418.424833487"
Nov 04 01:09:43.449083 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:09:43.449044    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:10:09.364555 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:10:09.364523    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-ntcsr"
Nov 04 01:10:09.413605 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:10:09.413575    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-bzqjr"]
Nov 04 01:10:09.413794 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:10:09.413772    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-bzqjr" podUID="b898224c-e77b-4a10-aa3a-56eaed0da2c5" containerName="manager" containerID="cri-o://2daa492332bd36fd47ca75a71ee617e56bf225208265b574f3d848f51d8acb46" gracePeriod=30
Nov 04 01:10:09.658672 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:10:09.658648    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-bzqjr"
Nov 04 01:10:09.830709 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:10:09.830679    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b898224c-e77b-4a10-aa3a-56eaed0da2c5-cert\" (UniqueName: \"kubernetes.io/secret/b898224c-e77b-4a10-aa3a-56eaed0da2c5-cert\") pod \"b898224c-e77b-4a10-aa3a-56eaed0da2c5\" (UID: \"b898224c-e77b-4a10-aa3a-56eaed0da2c5\") "
Nov 04 01:10:09.830853 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:10:09.830793    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b898224c-e77b-4a10-aa3a-56eaed0da2c5-kube-api-access-zq7g8\" (UniqueName: \"kubernetes.io/projected/b898224c-e77b-4a10-aa3a-56eaed0da2c5-kube-api-access-zq7g8\") pod \"b898224c-e77b-4a10-aa3a-56eaed0da2c5\" (UID: \"b898224c-e77b-4a10-aa3a-56eaed0da2c5\") "
Nov 04 01:10:09.832892 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:10:09.832859    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b898224c-e77b-4a10-aa3a-56eaed0da2c5-cert" pod "b898224c-e77b-4a10-aa3a-56eaed0da2c5" (UID: "b898224c-e77b-4a10-aa3a-56eaed0da2c5"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:10:09.832999 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:10:09.832909    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b898224c-e77b-4a10-aa3a-56eaed0da2c5-kube-api-access-zq7g8" pod "b898224c-e77b-4a10-aa3a-56eaed0da2c5" (UID: "b898224c-e77b-4a10-aa3a-56eaed0da2c5"). InnerVolumeSpecName "kube-api-access-zq7g8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:10:09.932038 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:10:09.931973    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zq7g8\" (UniqueName: \"kubernetes.io/projected/b898224c-e77b-4a10-aa3a-56eaed0da2c5-kube-api-access-zq7g8\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:10:09.932038 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:10:09.931999    5068 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b898224c-e77b-4a10-aa3a-56eaed0da2c5-cert\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:10:10.518023 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:10:10.517982    5068 generic.go:356] "Generic (PLEG): container finished" podID="b898224c-e77b-4a10-aa3a-56eaed0da2c5" containerID="2daa492332bd36fd47ca75a71ee617e56bf225208265b574f3d848f51d8acb46" exitCode=0
Nov 04 01:10:10.518426 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:10:10.518048    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-bzqjr"
Nov 04 01:10:10.518426 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:10:10.518049    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-bzqjr" event={"ID":"b898224c-e77b-4a10-aa3a-56eaed0da2c5","Type":"ContainerDied","Data":"2daa492332bd36fd47ca75a71ee617e56bf225208265b574f3d848f51d8acb46"}
Nov 04 01:10:10.518426 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:10:10.518103    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-bzqjr" event={"ID":"b898224c-e77b-4a10-aa3a-56eaed0da2c5","Type":"ContainerDied","Data":"4ada9f351fc80a7412eebee8cc69923cc4fea286d5685e5a5475930455ffeb1b"}
Nov 04 01:10:10.518426 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:10:10.518133    5068 scope.go:122] "RemoveContainer" containerID="2daa492332bd36fd47ca75a71ee617e56bf225208265b574f3d848f51d8acb46"
Nov 04 01:10:10.527807 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:10:10.527788    5068 scope.go:122] "RemoveContainer" containerID="2daa492332bd36fd47ca75a71ee617e56bf225208265b574f3d848f51d8acb46"
Nov 04 01:10:10.528061 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:10:10.528039    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2daa492332bd36fd47ca75a71ee617e56bf225208265b574f3d848f51d8acb46\": container with ID starting with 2daa492332bd36fd47ca75a71ee617e56bf225208265b574f3d848f51d8acb46 not found: ID does not exist" containerID="2daa492332bd36fd47ca75a71ee617e56bf225208265b574f3d848f51d8acb46"
Nov 04 01:10:10.528123 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:10:10.528074    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2daa492332bd36fd47ca75a71ee617e56bf225208265b574f3d848f51d8acb46"} err="failed to get container status \"2daa492332bd36fd47ca75a71ee617e56bf225208265b574f3d848f51d8acb46\": rpc error: code = NotFound desc = could not find container \"2daa492332bd36fd47ca75a71ee617e56bf225208265b574f3d848f51d8acb46\": container with ID starting with 2daa492332bd36fd47ca75a71ee617e56bf225208265b574f3d848f51d8acb46 not found: ID does not exist"
Nov 04 01:10:10.539167 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:10:10.539141    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-bzqjr"]
Nov 04 01:10:10.550291 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:10:10.550267    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-bzqjr"]
Nov 04 01:10:12.111288 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:10:12.111260    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b898224c-e77b-4a10-aa3a-56eaed0da2c5" path="/var/lib/kubelet/pods/b898224c-e77b-4a10-aa3a-56eaed0da2c5/volumes"
Nov 04 01:10:13.449334 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:10:13.449297    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:10:27.479554 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:10:27.479517    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:10:29.615107 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:10:29.615073    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:10:30.923118 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:10:30.923079    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:10:33.349039 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:10:33.349005    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:10:36.613922 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:10:36.613886    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:10:40.732562 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:10:40.732528    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:10:43.457993 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:10:43.457948    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:11:13.448802 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:11:13.448763    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:11:19.861205 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:19.861166    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:11:30.700055 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.700018    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-mdxsf"]
Nov 04 01:11:30.700419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.700397    5068 cpu_manager.go:414] "containerMap: removing container" podUID="b898224c-e77b-4a10-aa3a-56eaed0da2c5" containerName="manager"
Nov 04 01:11:30.700419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.700408    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b898224c-e77b-4a10-aa3a-56eaed0da2c5" containerName="manager"
Nov 04 01:11:30.700489 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.700476    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="b898224c-e77b-4a10-aa3a-56eaed0da2c5" containerName="manager"
Nov 04 01:11:30.703772 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.703733    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-mdxsf"
Nov 04 01:11:30.717699 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.717674    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-mdxsf"]
Nov 04 01:11:30.733773 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.733747    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-5g7t8"]
Nov 04 01:11:30.737356 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.737333    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-5g7t8"
Nov 04 01:11:30.750728 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.750707    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-5g7t8"]
Nov 04 01:11:30.830115 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.830089    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8fea1dc2-7dc8-44b2-ae40-e84791006ba0-istio-envoy\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-mdxsf\" (UID: \"8fea1dc2-7dc8-44b2-ae40-e84791006ba0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-mdxsf"
Nov 04 01:11:30.830262 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.830119    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8fea1dc2-7dc8-44b2-ae40-e84791006ba0-istio-podinfo\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-mdxsf\" (UID: \"8fea1dc2-7dc8-44b2-ae40-e84791006ba0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-mdxsf"
Nov 04 01:11:30.830262 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.830137    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8fea1dc2-7dc8-44b2-ae40-e84791006ba0-istio-token\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-mdxsf\" (UID: \"8fea1dc2-7dc8-44b2-ae40-e84791006ba0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-mdxsf"
Nov 04 01:11:30.830262 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.830153    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8fea1dc2-7dc8-44b2-ae40-e84791006ba0-istio-data\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-mdxsf\" (UID: \"8fea1dc2-7dc8-44b2-ae40-e84791006ba0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-mdxsf"
Nov 04 01:11:30.830262 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.830227    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c-workload-socket\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-5g7t8\" (UID: \"5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-5g7t8"
Nov 04 01:11:30.830262 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.830259    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c-credential-socket\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-5g7t8\" (UID: \"5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-5g7t8"
Nov 04 01:11:30.830482 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.830327    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-5g7t8\" (UID: \"5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-5g7t8"
Nov 04 01:11:30.830482 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.830350    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fea1dc2-7dc8-44b2-ae40-e84791006ba0-credential-socket\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-mdxsf\" (UID: \"8fea1dc2-7dc8-44b2-ae40-e84791006ba0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-mdxsf"
Nov 04 01:11:30.830482 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.830366    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8fea1dc2-7dc8-44b2-ae40-e84791006ba0-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-mdxsf\" (UID: \"8fea1dc2-7dc8-44b2-ae40-e84791006ba0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-mdxsf"
Nov 04 01:11:30.830482 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.830385    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nz9fm\" (UniqueName: \"kubernetes.io/projected/5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c-kube-api-access-nz9fm\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-5g7t8\" (UID: \"5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-5g7t8"
Nov 04 01:11:30.830482 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.830441    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8fea1dc2-7dc8-44b2-ae40-e84791006ba0-workload-certs\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-mdxsf\" (UID: \"8fea1dc2-7dc8-44b2-ae40-e84791006ba0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-mdxsf"
Nov 04 01:11:30.830787 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.830511    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c-workload-certs\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-5g7t8\" (UID: \"5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-5g7t8"
Nov 04 01:11:30.830787 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.830544    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c-istio-token\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-5g7t8\" (UID: \"5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-5g7t8"
Nov 04 01:11:30.830787 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.830573    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c-istio-data\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-5g7t8\" (UID: \"5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-5g7t8"
Nov 04 01:11:30.830787 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.830637    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcmx8\" (UniqueName: \"kubernetes.io/projected/8fea1dc2-7dc8-44b2-ae40-e84791006ba0-kube-api-access-wcmx8\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-mdxsf\" (UID: \"8fea1dc2-7dc8-44b2-ae40-e84791006ba0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-mdxsf"
Nov 04 01:11:30.830787 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.830656    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c-istio-envoy\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-5g7t8\" (UID: \"5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-5g7t8"
Nov 04 01:11:30.830787 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.830673    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-5g7t8\" (UID: \"5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-5g7t8"
Nov 04 01:11:30.830787 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.830707    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fea1dc2-7dc8-44b2-ae40-e84791006ba0-workload-socket\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-mdxsf\" (UID: \"8fea1dc2-7dc8-44b2-ae40-e84791006ba0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-mdxsf"
Nov 04 01:11:30.931457 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.931427    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c-credential-socket\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-5g7t8\" (UID: \"5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-5g7t8"
Nov 04 01:11:30.931595 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.931479    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-5g7t8\" (UID: \"5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-5g7t8"
Nov 04 01:11:30.931595 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.931508    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fea1dc2-7dc8-44b2-ae40-e84791006ba0-credential-socket\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-mdxsf\" (UID: \"8fea1dc2-7dc8-44b2-ae40-e84791006ba0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-mdxsf"
Nov 04 01:11:30.931595 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.931531    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8fea1dc2-7dc8-44b2-ae40-e84791006ba0-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-mdxsf\" (UID: \"8fea1dc2-7dc8-44b2-ae40-e84791006ba0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-mdxsf"
Nov 04 01:11:30.931595 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.931564    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nz9fm\" (UniqueName: \"kubernetes.io/projected/5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c-kube-api-access-nz9fm\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-5g7t8\" (UID: \"5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-5g7t8"
Nov 04 01:11:30.931882 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.931599    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8fea1dc2-7dc8-44b2-ae40-e84791006ba0-workload-certs\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-mdxsf\" (UID: \"8fea1dc2-7dc8-44b2-ae40-e84791006ba0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-mdxsf"
Nov 04 01:11:30.931882 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.931658    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c-workload-certs\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-5g7t8\" (UID: \"5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-5g7t8"
Nov 04 01:11:30.931882 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.931692    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c-istio-token\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-5g7t8\" (UID: \"5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-5g7t8"
Nov 04 01:11:30.931882 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.931727    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c-istio-data\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-5g7t8\" (UID: \"5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-5g7t8"
Nov 04 01:11:30.931882 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.931773    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcmx8\" (UniqueName: \"kubernetes.io/projected/8fea1dc2-7dc8-44b2-ae40-e84791006ba0-kube-api-access-wcmx8\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-mdxsf\" (UID: \"8fea1dc2-7dc8-44b2-ae40-e84791006ba0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-mdxsf"
Nov 04 01:11:30.931882 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.931804    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c-istio-envoy\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-5g7t8\" (UID: \"5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-5g7t8"
Nov 04 01:11:30.931882 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.931834    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-5g7t8\" (UID: \"5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-5g7t8"
Nov 04 01:11:30.931882 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.931859    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fea1dc2-7dc8-44b2-ae40-e84791006ba0-workload-socket\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-mdxsf\" (UID: \"8fea1dc2-7dc8-44b2-ae40-e84791006ba0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-mdxsf"
Nov 04 01:11:30.932275 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.931919    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8fea1dc2-7dc8-44b2-ae40-e84791006ba0-istio-envoy\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-mdxsf\" (UID: \"8fea1dc2-7dc8-44b2-ae40-e84791006ba0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-mdxsf"
Nov 04 01:11:30.932275 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.931955    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8fea1dc2-7dc8-44b2-ae40-e84791006ba0-istio-podinfo\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-mdxsf\" (UID: \"8fea1dc2-7dc8-44b2-ae40-e84791006ba0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-mdxsf"
Nov 04 01:11:30.932275 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.931966    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fea1dc2-7dc8-44b2-ae40-e84791006ba0-credential-socket\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-mdxsf\" (UID: \"8fea1dc2-7dc8-44b2-ae40-e84791006ba0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-mdxsf"
Nov 04 01:11:30.932275 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.931981    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8fea1dc2-7dc8-44b2-ae40-e84791006ba0-istio-token\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-mdxsf\" (UID: \"8fea1dc2-7dc8-44b2-ae40-e84791006ba0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-mdxsf"
Nov 04 01:11:30.932275 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.932011    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8fea1dc2-7dc8-44b2-ae40-e84791006ba0-workload-certs\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-mdxsf\" (UID: \"8fea1dc2-7dc8-44b2-ae40-e84791006ba0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-mdxsf"
Nov 04 01:11:30.932275 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.931916    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c-credential-socket\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-5g7t8\" (UID: \"5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-5g7t8"
Nov 04 01:11:30.932275 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.932033    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8fea1dc2-7dc8-44b2-ae40-e84791006ba0-istio-data\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-mdxsf\" (UID: \"8fea1dc2-7dc8-44b2-ae40-e84791006ba0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-mdxsf"
Nov 04 01:11:30.932275 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.932014    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c-workload-certs\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-5g7t8\" (UID: \"5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-5g7t8"
Nov 04 01:11:30.932275 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.932087    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c-workload-socket\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-5g7t8\" (UID: \"5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-5g7t8"
Nov 04 01:11:30.932776 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.932284    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fea1dc2-7dc8-44b2-ae40-e84791006ba0-workload-socket\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-mdxsf\" (UID: \"8fea1dc2-7dc8-44b2-ae40-e84791006ba0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-mdxsf"
Nov 04 01:11:30.932776 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.932305    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8fea1dc2-7dc8-44b2-ae40-e84791006ba0-istio-data\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-mdxsf\" (UID: \"8fea1dc2-7dc8-44b2-ae40-e84791006ba0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-mdxsf"
Nov 04 01:11:30.932776 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.932388    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c-workload-socket\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-5g7t8\" (UID: \"5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-5g7t8"
Nov 04 01:11:30.932776 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.932480    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c-istio-data\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-5g7t8\" (UID: \"5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-5g7t8"
Nov 04 01:11:30.932995 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.932909    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-5g7t8\" (UID: \"5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-5g7t8"
Nov 04 01:11:30.932995 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.932946    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8fea1dc2-7dc8-44b2-ae40-e84791006ba0-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-mdxsf\" (UID: \"8fea1dc2-7dc8-44b2-ae40-e84791006ba0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-mdxsf"
Nov 04 01:11:30.934284 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.934258    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-5g7t8\" (UID: \"5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-5g7t8"
Nov 04 01:11:30.934726 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.934704    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8fea1dc2-7dc8-44b2-ae40-e84791006ba0-istio-envoy\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-mdxsf\" (UID: \"8fea1dc2-7dc8-44b2-ae40-e84791006ba0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-mdxsf"
Nov 04 01:11:30.934812 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.934778    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c-istio-envoy\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-5g7t8\" (UID: \"5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-5g7t8"
Nov 04 01:11:30.934987 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.934972    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8fea1dc2-7dc8-44b2-ae40-e84791006ba0-istio-podinfo\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-mdxsf\" (UID: \"8fea1dc2-7dc8-44b2-ae40-e84791006ba0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-mdxsf"
Nov 04 01:11:30.944695 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.944670    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c-istio-token\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-5g7t8\" (UID: \"5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-5g7t8"
Nov 04 01:11:30.944798 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.944774    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8fea1dc2-7dc8-44b2-ae40-e84791006ba0-istio-token\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-mdxsf\" (UID: \"8fea1dc2-7dc8-44b2-ae40-e84791006ba0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-mdxsf"
Nov 04 01:11:30.944935 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.944916    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nz9fm\" (UniqueName: \"kubernetes.io/projected/5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c-kube-api-access-nz9fm\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-5g7t8\" (UID: \"5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-5g7t8"
Nov 04 01:11:30.945061 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:30.945042    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcmx8\" (UniqueName: \"kubernetes.io/projected/8fea1dc2-7dc8-44b2-ae40-e84791006ba0-kube-api-access-wcmx8\") pod \"maas-default-gateway-openshift-default-848f6bf8bc-mdxsf\" (UID: \"8fea1dc2-7dc8-44b2-ae40-e84791006ba0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-mdxsf"
Nov 04 01:11:31.016941 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:31.016919    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-mdxsf"
Nov 04 01:11:31.051003 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:31.050976    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-5g7t8"
Nov 04 01:11:31.295527 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:31.295500    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-mdxsf"]
Nov 04 01:11:31.296987 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:11:31.296950    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fea1dc2_7dc8_44b2_ae40_e84791006ba0.slice/crio-9d63e57de95a125dbc64cc181c2dc322332748c88569bf2fdfcced960ff291d1 WatchSource:0}: Error finding container 9d63e57de95a125dbc64cc181c2dc322332748c88569bf2fdfcced960ff291d1: Status 404 returned error can't find the container with id 9d63e57de95a125dbc64cc181c2dc322332748c88569bf2fdfcced960ff291d1
Nov 04 01:11:31.299217 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:31.299185    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:11:31.299304 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:31.299248    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:11:31.299304 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:31.299276    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:11:31.337948 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:31.337927    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-5g7t8"]
Nov 04 01:11:31.347639 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:11:31.347587    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cc6cc9a_2ce1_4505_bbd8_5eba3a93237c.slice/crio-953180a24f19b424f3a6be7a57b169157273faffda9b6dac6b89a242b23ae632 WatchSource:0}: Error finding container 953180a24f19b424f3a6be7a57b169157273faffda9b6dac6b89a242b23ae632: Status 404 returned error can't find the container with id 953180a24f19b424f3a6be7a57b169157273faffda9b6dac6b89a242b23ae632
Nov 04 01:11:31.349871 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:31.349817    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:11:31.349941 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:31.349899    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:11:31.349941 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:31.349934    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:11:31.996909 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:31.996872    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-5g7t8" event={"ID":"5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c","Type":"ContainerStarted","Data":"44ff7a68081b763409599762098ab520e7c37b77422d30e7152f32a67a7193cc"}
Nov 04 01:11:31.997354 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:31.997227    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-5g7t8" event={"ID":"5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c","Type":"ContainerStarted","Data":"953180a24f19b424f3a6be7a57b169157273faffda9b6dac6b89a242b23ae632"}
Nov 04 01:11:32.006331 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:32.006301    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-mdxsf" event={"ID":"8fea1dc2-7dc8-44b2-ae40-e84791006ba0","Type":"ContainerStarted","Data":"c10b9a0aeb9a25d6db1c9a58466b1942ba3478a72ccb4836dfb60db59a5807ba"}
Nov 04 01:11:32.006468 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:32.006383    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-mdxsf" event={"ID":"8fea1dc2-7dc8-44b2-ae40-e84791006ba0","Type":"ContainerStarted","Data":"9d63e57de95a125dbc64cc181c2dc322332748c88569bf2fdfcced960ff291d1"}
Nov 04 01:11:32.017795 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:32.017777    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-mdxsf"
Nov 04 01:11:32.019256 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:32.019228    5068 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-848f6bf8bc-mdxsf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Nov 04 01:11:32.019346 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:32.019272    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-mdxsf" podUID="8fea1dc2-7dc8-44b2-ae40-e84791006ba0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Nov 04 01:11:32.024806 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:32.024770    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-5g7t8" podStartSLOduration=2.024759601 podStartE2EDuration="2.024759601s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:11:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:11:32.023331414 +0000 UTC m=+532.069892203" watchObservedRunningTime="2026-11-04 01:11:32.024759601 +0000 UTC m=+532.071320391"
Nov 04 01:11:32.056390 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:32.056352    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-5g7t8"
Nov 04 01:11:32.060933 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:32.060895    5068 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-848f6bf8bc-5g7t8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Nov 04 01:11:32.061411 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:32.061283    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-5g7t8" podUID="5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Nov 04 01:11:32.091303 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:32.091243    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-mdxsf" podStartSLOduration=2.091223095 podStartE2EDuration="2.091223095s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:11:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:11:32.062103731 +0000 UTC m=+532.108664520" watchObservedRunningTime="2026-11-04 01:11:32.091223095 +0000 UTC m=+532.137783883"
Nov 04 01:11:33.018199 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:33.018165    5068 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-848f6bf8bc-mdxsf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Nov 04 01:11:33.018561 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:33.018208    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-mdxsf" podUID="8fea1dc2-7dc8-44b2-ae40-e84791006ba0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Nov 04 01:11:33.059631 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:33.059113    5068 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-848f6bf8bc-5g7t8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Nov 04 01:11:33.059631 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:33.059185    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-5g7t8" podUID="5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Nov 04 01:11:34.017760 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:34.017731    5068 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-848f6bf8bc-mdxsf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Nov 04 01:11:34.017974 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:34.017956    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-mdxsf" podUID="8fea1dc2-7dc8-44b2-ae40-e84791006ba0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Nov 04 01:11:34.056199 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:34.054962    5068 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-848f6bf8bc-5g7t8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Nov 04 01:11:34.056199 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:34.055018    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-5g7t8" podUID="5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Nov 04 01:11:35.018263 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:35.018225    5068 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-848f6bf8bc-mdxsf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Nov 04 01:11:35.018406 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:35.018294    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-mdxsf" podUID="8fea1dc2-7dc8-44b2-ae40-e84791006ba0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Nov 04 01:11:35.058742 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:35.058706    5068 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-848f6bf8bc-5g7t8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Nov 04 01:11:35.059283 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:35.059255    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-5g7t8" podUID="5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Nov 04 01:11:36.018164 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:36.018129    5068 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-848f6bf8bc-mdxsf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Nov 04 01:11:36.018356 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:36.018186    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-mdxsf" podUID="8fea1dc2-7dc8-44b2-ae40-e84791006ba0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Nov 04 01:11:36.061632 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:36.054981    5068 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-848f6bf8bc-5g7t8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Nov 04 01:11:36.061632 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:36.055040    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-5g7t8" podUID="5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Nov 04 01:11:37.017515 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:37.017479    5068 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-848f6bf8bc-mdxsf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Nov 04 01:11:37.017722 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:37.017533    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-mdxsf" podUID="8fea1dc2-7dc8-44b2-ae40-e84791006ba0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Nov 04 01:11:37.058633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:37.054984    5068 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-848f6bf8bc-5g7t8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Nov 04 01:11:37.058633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:37.055044    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-5g7t8" podUID="5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Nov 04 01:11:38.017247 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:38.017207    5068 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-848f6bf8bc-mdxsf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Nov 04 01:11:38.017764 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:38.017260    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-mdxsf" podUID="8fea1dc2-7dc8-44b2-ae40-e84791006ba0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Nov 04 01:11:38.051563 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:38.051523    5068 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-848f6bf8bc-5g7t8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Nov 04 01:11:38.051732 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:38.051586    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-5g7t8" podUID="5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Nov 04 01:11:39.817502 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:39.816253    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-mdxsf"
Nov 04 01:11:39.817502 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:39.817133    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-mdxsf"
Nov 04 01:11:39.818067 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:39.817760    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-mdxsf"
Nov 04 01:11:39.875766 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:39.875733    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-5g7t8"
Nov 04 01:11:39.877319 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:39.877189    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-5g7t8"
Nov 04 01:11:39.880685 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:11:39.880657    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-5g7t8"
Nov 04 01:11:43.448663 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:11:43.448619    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:12:13.448620 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:12:13.448570    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:12:36.545594 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:36.545554    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-cffa80-564fd64f66-qhhzq"]
Nov 04 01:12:36.549341 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:36.549321    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-cffa80-564fd64f66-qhhzq"
Nov 04 01:12:36.564418 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:36.564394    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-cffa80-564fd64f66-qhhzq"]
Nov 04 01:12:36.570600 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:36.570573    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w2-cffa80\""
Nov 04 01:12:36.571918 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:36.571900    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-cffa80-9b645ff4c-2pj8n"]
Nov 04 01:12:36.575383 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:36.575367    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-cffa80-9b645ff4c-2pj8n"
Nov 04 01:12:36.588578 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:36.588557    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w2-cffa80-dockercfg-2ghnn\""
Nov 04 01:12:36.591199 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:36.591175    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-cffa80-9b645ff4c-2pj8n"]
Nov 04 01:12:36.626436 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:36.626410    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftcfz\" (UniqueName: \"kubernetes.io/projected/1b4a1b43-a286-436d-bafe-a10d9a57a4d2-kube-api-access-ftcfz\") pod \"payload-processing-e2e-worker-w2-cffa80-564fd64f66-qhhzq\" (UID: \"1b4a1b43-a286-436d-bafe-a10d9a57a4d2\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-cffa80-564fd64f66-qhhzq"
Nov 04 01:12:36.626547 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:36.626457    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b4a1b43-a286-436d-bafe-a10d9a57a4d2-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-cffa80-564fd64f66-qhhzq\" (UID: \"1b4a1b43-a286-436d-bafe-a10d9a57a4d2\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-cffa80-564fd64f66-qhhzq"
Nov 04 01:12:36.727312 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:36.727277    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftcfz\" (UniqueName: \"kubernetes.io/projected/1b4a1b43-a286-436d-bafe-a10d9a57a4d2-kube-api-access-ftcfz\") pod \"payload-processing-e2e-worker-w2-cffa80-564fd64f66-qhhzq\" (UID: \"1b4a1b43-a286-436d-bafe-a10d9a57a4d2\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-cffa80-564fd64f66-qhhzq"
Nov 04 01:12:36.727481 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:36.727330    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/841e9fef-46d6-4625-9b72-14b4e0eeb7dd-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-cffa80-9b645ff4c-2pj8n\" (UID: \"841e9fef-46d6-4625-9b72-14b4e0eeb7dd\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-cffa80-9b645ff4c-2pj8n"
Nov 04 01:12:36.727481 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:36.727441    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cq8n5\" (UniqueName: \"kubernetes.io/projected/841e9fef-46d6-4625-9b72-14b4e0eeb7dd-kube-api-access-cq8n5\") pod \"payload-pre-processing-e2e-worker-w2-cffa80-9b645ff4c-2pj8n\" (UID: \"841e9fef-46d6-4625-9b72-14b4e0eeb7dd\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-cffa80-9b645ff4c-2pj8n"
Nov 04 01:12:36.727594 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:36.727538    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b4a1b43-a286-436d-bafe-a10d9a57a4d2-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-cffa80-564fd64f66-qhhzq\" (UID: \"1b4a1b43-a286-436d-bafe-a10d9a57a4d2\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-cffa80-564fd64f66-qhhzq"
Nov 04 01:12:36.728270 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:36.728250    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b4a1b43-a286-436d-bafe-a10d9a57a4d2-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-cffa80-564fd64f66-qhhzq\" (UID: \"1b4a1b43-a286-436d-bafe-a10d9a57a4d2\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-cffa80-564fd64f66-qhhzq"
Nov 04 01:12:36.739576 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:36.739552    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftcfz\" (UniqueName: \"kubernetes.io/projected/1b4a1b43-a286-436d-bafe-a10d9a57a4d2-kube-api-access-ftcfz\") pod \"payload-processing-e2e-worker-w2-cffa80-564fd64f66-qhhzq\" (UID: \"1b4a1b43-a286-436d-bafe-a10d9a57a4d2\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-cffa80-564fd64f66-qhhzq"
Nov 04 01:12:36.828802 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:36.828731    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/841e9fef-46d6-4625-9b72-14b4e0eeb7dd-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-cffa80-9b645ff4c-2pj8n\" (UID: \"841e9fef-46d6-4625-9b72-14b4e0eeb7dd\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-cffa80-9b645ff4c-2pj8n"
Nov 04 01:12:36.828802 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:36.828760    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cq8n5\" (UniqueName: \"kubernetes.io/projected/841e9fef-46d6-4625-9b72-14b4e0eeb7dd-kube-api-access-cq8n5\") pod \"payload-pre-processing-e2e-worker-w2-cffa80-9b645ff4c-2pj8n\" (UID: \"841e9fef-46d6-4625-9b72-14b4e0eeb7dd\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-cffa80-9b645ff4c-2pj8n"
Nov 04 01:12:36.829469 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:36.829443    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/841e9fef-46d6-4625-9b72-14b4e0eeb7dd-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-cffa80-9b645ff4c-2pj8n\" (UID: \"841e9fef-46d6-4625-9b72-14b4e0eeb7dd\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-cffa80-9b645ff4c-2pj8n"
Nov 04 01:12:36.844441 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:36.844413    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cq8n5\" (UniqueName: \"kubernetes.io/projected/841e9fef-46d6-4625-9b72-14b4e0eeb7dd-kube-api-access-cq8n5\") pod \"payload-pre-processing-e2e-worker-w2-cffa80-9b645ff4c-2pj8n\" (UID: \"841e9fef-46d6-4625-9b72-14b4e0eeb7dd\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-cffa80-9b645ff4c-2pj8n"
Nov 04 01:12:36.862368 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:36.862343    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-cffa80-564fd64f66-qhhzq"
Nov 04 01:12:36.888625 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:36.888592    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-cffa80-9b645ff4c-2pj8n"
Nov 04 01:12:37.043246 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:37.043213    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-cffa80-8dcbb5c89-jx699"]
Nov 04 01:12:37.047457 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:37.047431    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-cffa80-8dcbb5c89-jx699"
Nov 04 01:12:37.051369 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:37.051346    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w2-cffa80-serving-cert\""
Nov 04 01:12:37.058575 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:37.058432    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-cffa80-8dcbb5c89-jx699"]
Nov 04 01:12:37.130899 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:37.130829    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4f8ccdf-b225-453b-87a9-9c6212d77802-maas-api-tls\") pod \"maas-api-e2e-worker-w2-cffa80-8dcbb5c89-jx699\" (UID: \"c4f8ccdf-b225-453b-87a9-9c6212d77802\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-cffa80-8dcbb5c89-jx699"
Nov 04 01:12:37.131027 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:37.130923    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4f8ccdf-b225-453b-87a9-9c6212d77802-metrics-certs\") pod \"maas-api-e2e-worker-w2-cffa80-8dcbb5c89-jx699\" (UID: \"c4f8ccdf-b225-453b-87a9-9c6212d77802\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-cffa80-8dcbb5c89-jx699"
Nov 04 01:12:37.131027 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:37.130959    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9csg\" (UniqueName: \"kubernetes.io/projected/c4f8ccdf-b225-453b-87a9-9c6212d77802-kube-api-access-d9csg\") pod \"maas-api-e2e-worker-w2-cffa80-8dcbb5c89-jx699\" (UID: \"c4f8ccdf-b225-453b-87a9-9c6212d77802\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-cffa80-8dcbb5c89-jx699"
Nov 04 01:12:37.131120 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:37.131097    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4f8ccdf-b225-453b-87a9-9c6212d77802-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-cffa80-8dcbb5c89-jx699\" (UID: \"c4f8ccdf-b225-453b-87a9-9c6212d77802\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-cffa80-8dcbb5c89-jx699"
Nov 04 01:12:37.147806 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:37.147780    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-cffa80-564fd64f66-qhhzq"]
Nov 04 01:12:37.151041 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:12:37.151012    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b4a1b43_a286_436d_bafe_a10d9a57a4d2.slice/crio-039ffb7fc2d5f3101f7045667017b696fc55e5adfe632451f197d15c4976c542 WatchSource:0}: Error finding container 039ffb7fc2d5f3101f7045667017b696fc55e5adfe632451f197d15c4976c542: Status 404 returned error can't find the container with id 039ffb7fc2d5f3101f7045667017b696fc55e5adfe632451f197d15c4976c542
Nov 04 01:12:37.180894 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:37.180874    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-cffa80-9b645ff4c-2pj8n"]
Nov 04 01:12:37.182601 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:12:37.182578    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod841e9fef_46d6_4625_9b72_14b4e0eeb7dd.slice/crio-a8db9090b9e68247abcb72e39c6839e64feb8d5a6948623f5194ba2dd1c8a2be WatchSource:0}: Error finding container a8db9090b9e68247abcb72e39c6839e64feb8d5a6948623f5194ba2dd1c8a2be: Status 404 returned error can't find the container with id a8db9090b9e68247abcb72e39c6839e64feb8d5a6948623f5194ba2dd1c8a2be
Nov 04 01:12:37.232417 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:37.232390    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4f8ccdf-b225-453b-87a9-9c6212d77802-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-cffa80-8dcbb5c89-jx699\" (UID: \"c4f8ccdf-b225-453b-87a9-9c6212d77802\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-cffa80-8dcbb5c89-jx699"
Nov 04 01:12:37.232571 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:37.232447    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4f8ccdf-b225-453b-87a9-9c6212d77802-maas-api-tls\") pod \"maas-api-e2e-worker-w2-cffa80-8dcbb5c89-jx699\" (UID: \"c4f8ccdf-b225-453b-87a9-9c6212d77802\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-cffa80-8dcbb5c89-jx699"
Nov 04 01:12:37.232571 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:37.232474    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4f8ccdf-b225-453b-87a9-9c6212d77802-metrics-certs\") pod \"maas-api-e2e-worker-w2-cffa80-8dcbb5c89-jx699\" (UID: \"c4f8ccdf-b225-453b-87a9-9c6212d77802\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-cffa80-8dcbb5c89-jx699"
Nov 04 01:12:37.232571 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:37.232492    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9csg\" (UniqueName: \"kubernetes.io/projected/c4f8ccdf-b225-453b-87a9-9c6212d77802-kube-api-access-d9csg\") pod \"maas-api-e2e-worker-w2-cffa80-8dcbb5c89-jx699\" (UID: \"c4f8ccdf-b225-453b-87a9-9c6212d77802\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-cffa80-8dcbb5c89-jx699"
Nov 04 01:12:37.233130 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:37.233107    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4f8ccdf-b225-453b-87a9-9c6212d77802-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-cffa80-8dcbb5c89-jx699\" (UID: \"c4f8ccdf-b225-453b-87a9-9c6212d77802\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-cffa80-8dcbb5c89-jx699"
Nov 04 01:12:37.234851 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:37.234827    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4f8ccdf-b225-453b-87a9-9c6212d77802-maas-api-tls\") pod \"maas-api-e2e-worker-w2-cffa80-8dcbb5c89-jx699\" (UID: \"c4f8ccdf-b225-453b-87a9-9c6212d77802\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-cffa80-8dcbb5c89-jx699"
Nov 04 01:12:37.234944 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:37.234892    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4f8ccdf-b225-453b-87a9-9c6212d77802-metrics-certs\") pod \"maas-api-e2e-worker-w2-cffa80-8dcbb5c89-jx699\" (UID: \"c4f8ccdf-b225-453b-87a9-9c6212d77802\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-cffa80-8dcbb5c89-jx699"
Nov 04 01:12:37.244346 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:37.244324    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9csg\" (UniqueName: \"kubernetes.io/projected/c4f8ccdf-b225-453b-87a9-9c6212d77802-kube-api-access-d9csg\") pod \"maas-api-e2e-worker-w2-cffa80-8dcbb5c89-jx699\" (UID: \"c4f8ccdf-b225-453b-87a9-9c6212d77802\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-cffa80-8dcbb5c89-jx699"
Nov 04 01:12:37.359332 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:37.359301    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-cffa80-9b645ff4c-2pj8n" event={"ID":"841e9fef-46d6-4625-9b72-14b4e0eeb7dd","Type":"ContainerStarted","Data":"a8db9090b9e68247abcb72e39c6839e64feb8d5a6948623f5194ba2dd1c8a2be"}
Nov 04 01:12:37.360767 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:37.360736    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-cffa80-564fd64f66-qhhzq" event={"ID":"1b4a1b43-a286-436d-bafe-a10d9a57a4d2","Type":"ContainerStarted","Data":"039ffb7fc2d5f3101f7045667017b696fc55e5adfe632451f197d15c4976c542"}
Nov 04 01:12:37.366060 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:37.366041    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-cffa80-8dcbb5c89-jx699"
Nov 04 01:12:37.678926 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:37.678890    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-cffa80-8dcbb5c89-jx699"]
Nov 04 01:12:37.681842 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:12:37.681817    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4f8ccdf_b225_453b_87a9_9c6212d77802.slice/crio-72e4981da1ff1806edbe90aacb9eacaecc0f0b7eede600dfd7ec83577d0df71e WatchSource:0}: Error finding container 72e4981da1ff1806edbe90aacb9eacaecc0f0b7eede600dfd7ec83577d0df71e: Status 404 returned error can't find the container with id 72e4981da1ff1806edbe90aacb9eacaecc0f0b7eede600dfd7ec83577d0df71e
Nov 04 01:12:38.369420 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:38.369385    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-cffa80-8dcbb5c89-jx699" event={"ID":"c4f8ccdf-b225-453b-87a9-9c6212d77802","Type":"ContainerStarted","Data":"72e4981da1ff1806edbe90aacb9eacaecc0f0b7eede600dfd7ec83577d0df71e"}
Nov 04 01:12:38.777336 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:38.777289    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-6dac99-6d46fbff94-fcfwv"]
Nov 04 01:12:38.782215 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:38.782187    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-6dac99-6d46fbff94-fcfwv"
Nov 04 01:12:38.789868 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:38.788793    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-6dac99-6d46fbff94-fcfwv"]
Nov 04 01:12:38.792473 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:38.791252    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-6dac99\""
Nov 04 01:12:38.792473 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:38.791460    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-6dac99-dockercfg-hkz26\""
Nov 04 01:12:38.954079 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:38.954042    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98e65cde-ca44-4ad9-9a57-ce81c0369a0f-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-6dac99-6d46fbff94-fcfwv\" (UID: \"98e65cde-ca44-4ad9-9a57-ce81c0369a0f\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-6dac99-6d46fbff94-fcfwv"
Nov 04 01:12:38.954246 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:38.954096    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7b7zh\" (UniqueName: \"kubernetes.io/projected/98e65cde-ca44-4ad9-9a57-ce81c0369a0f-kube-api-access-7b7zh\") pod \"payload-processing-e2e-worker-w0-6dac99-6d46fbff94-fcfwv\" (UID: \"98e65cde-ca44-4ad9-9a57-ce81c0369a0f\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-6dac99-6d46fbff94-fcfwv"
Nov 04 01:12:39.055690 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:39.055583    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98e65cde-ca44-4ad9-9a57-ce81c0369a0f-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-6dac99-6d46fbff94-fcfwv\" (UID: \"98e65cde-ca44-4ad9-9a57-ce81c0369a0f\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-6dac99-6d46fbff94-fcfwv"
Nov 04 01:12:39.055690 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:39.055671    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7b7zh\" (UniqueName: \"kubernetes.io/projected/98e65cde-ca44-4ad9-9a57-ce81c0369a0f-kube-api-access-7b7zh\") pod \"payload-processing-e2e-worker-w0-6dac99-6d46fbff94-fcfwv\" (UID: \"98e65cde-ca44-4ad9-9a57-ce81c0369a0f\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-6dac99-6d46fbff94-fcfwv"
Nov 04 01:12:39.056341 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:39.056320    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98e65cde-ca44-4ad9-9a57-ce81c0369a0f-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-6dac99-6d46fbff94-fcfwv\" (UID: \"98e65cde-ca44-4ad9-9a57-ce81c0369a0f\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-6dac99-6d46fbff94-fcfwv"
Nov 04 01:12:39.076575 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:39.076544    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7b7zh\" (UniqueName: \"kubernetes.io/projected/98e65cde-ca44-4ad9-9a57-ce81c0369a0f-kube-api-access-7b7zh\") pod \"payload-processing-e2e-worker-w0-6dac99-6d46fbff94-fcfwv\" (UID: \"98e65cde-ca44-4ad9-9a57-ce81c0369a0f\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-6dac99-6d46fbff94-fcfwv"
Nov 04 01:12:39.102938 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:39.102288    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-6dac99-6d46fbff94-fcfwv"
Nov 04 01:12:39.119754 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:39.119727    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:12:39.377917 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:39.377836    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-cffa80-8dcbb5c89-jx699" event={"ID":"c4f8ccdf-b225-453b-87a9-9c6212d77802","Type":"ContainerStarted","Data":"3f715d388c4658cde8f49e563a68f41146c83409b5beefaa31b184d321ee5611"}
Nov 04 01:12:39.378150 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:39.378126    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-cffa80-8dcbb5c89-jx699"
Nov 04 01:12:39.402160 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:39.402133    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-6dac99-6d46fbff94-fcfwv"]
Nov 04 01:12:39.403042 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:39.402989    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-cffa80-8dcbb5c89-jx699" podStartSLOduration=1.608667259 podStartE2EDuration="2.402972401s" totalImagesPullingTime="794.305142ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:12:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:12:39.400738955 +0000 UTC m=+599.447299747" watchObservedRunningTime="2026-11-04 01:12:39.402972401 +0000 UTC m=+599.449533193"
Nov 04 01:12:39.403875 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:12:39.403849    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98e65cde_ca44_4ad9_9a57_ce81c0369a0f.slice/crio-37a4b9244fa6db19d9f73cdd829f052f3ec264918e26c3a72fc41b4bee4db92c WatchSource:0}: Error finding container 37a4b9244fa6db19d9f73cdd829f052f3ec264918e26c3a72fc41b4bee4db92c: Status 404 returned error can't find the container with id 37a4b9244fa6db19d9f73cdd829f052f3ec264918e26c3a72fc41b4bee4db92c
Nov 04 01:12:40.101670 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:40.101604    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-ba844c-7bc4dfcfd-nwk5b"]
Nov 04 01:12:40.113296 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:40.113269    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-ba844c-7bc4dfcfd-nwk5b"
Nov 04 01:12:40.123941 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:40.123643    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-ba844c\""
Nov 04 01:12:40.124260 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:40.124234    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-ba844c-dockercfg-pllvv\""
Nov 04 01:12:40.126467 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:40.126448    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-ba844c-7bc4dfcfd-nwk5b"]
Nov 04 01:12:40.231288 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:40.231260    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tr8b2_21d1d23b-7b6e-4ab5-9593-cf7784fffea2/console-operator/2.log"
Nov 04 01:12:40.235491 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:40.235463    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tr8b2_21d1d23b-7b6e-4ab5-9593-cf7784fffea2/console-operator/2.log"
Nov 04 01:12:40.273714 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:40.273681    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxj5h\" (UniqueName: \"kubernetes.io/projected/e0946b4c-7a9f-4044-8f44-53484315882d-kube-api-access-cxj5h\") pod \"payload-processing-e2e-worker-w4-ba844c-7bc4dfcfd-nwk5b\" (UID: \"e0946b4c-7a9f-4044-8f44-53484315882d\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-ba844c-7bc4dfcfd-nwk5b"
Nov 04 01:12:40.273877 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:40.273839    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0946b4c-7a9f-4044-8f44-53484315882d-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-ba844c-7bc4dfcfd-nwk5b\" (UID: \"e0946b4c-7a9f-4044-8f44-53484315882d\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-ba844c-7bc4dfcfd-nwk5b"
Nov 04 01:12:40.374742 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:40.374657    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxj5h\" (UniqueName: \"kubernetes.io/projected/e0946b4c-7a9f-4044-8f44-53484315882d-kube-api-access-cxj5h\") pod \"payload-processing-e2e-worker-w4-ba844c-7bc4dfcfd-nwk5b\" (UID: \"e0946b4c-7a9f-4044-8f44-53484315882d\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-ba844c-7bc4dfcfd-nwk5b"
Nov 04 01:12:40.374905 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:40.374793    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0946b4c-7a9f-4044-8f44-53484315882d-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-ba844c-7bc4dfcfd-nwk5b\" (UID: \"e0946b4c-7a9f-4044-8f44-53484315882d\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-ba844c-7bc4dfcfd-nwk5b"
Nov 04 01:12:40.375834 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:40.375808    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0946b4c-7a9f-4044-8f44-53484315882d-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-ba844c-7bc4dfcfd-nwk5b\" (UID: \"e0946b4c-7a9f-4044-8f44-53484315882d\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-ba844c-7bc4dfcfd-nwk5b"
Nov 04 01:12:40.387783 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:40.387699    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxj5h\" (UniqueName: \"kubernetes.io/projected/e0946b4c-7a9f-4044-8f44-53484315882d-kube-api-access-cxj5h\") pod \"payload-processing-e2e-worker-w4-ba844c-7bc4dfcfd-nwk5b\" (UID: \"e0946b4c-7a9f-4044-8f44-53484315882d\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-ba844c-7bc4dfcfd-nwk5b"
Nov 04 01:12:40.388177 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:40.388149    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-6dac99-6d46fbff94-fcfwv" event={"ID":"98e65cde-ca44-4ad9-9a57-ce81c0369a0f","Type":"ContainerStarted","Data":"37a4b9244fa6db19d9f73cdd829f052f3ec264918e26c3a72fc41b4bee4db92c"}
Nov 04 01:12:40.436279 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:40.436255    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-ba844c-7bc4dfcfd-nwk5b"
Nov 04 01:12:40.517235 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:40.517197    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-a5115a-8754c4f46-g677p"]
Nov 04 01:12:40.525123 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:40.525101    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-a5115a-8754c4f46-g677p"
Nov 04 01:12:40.539668 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:40.537058    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w3-a5115a-serving-cert\""
Nov 04 01:12:40.542518 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:40.541380    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-a5115a-8754c4f46-g677p"]
Nov 04 01:12:40.678262 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:40.678195    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ccb4aaa-fcb8-4797-8295-9e8521de235b-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-a5115a-8754c4f46-g677p\" (UID: \"2ccb4aaa-fcb8-4797-8295-9e8521de235b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-a5115a-8754c4f46-g677p"
Nov 04 01:12:40.678387 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:40.678262    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ccb4aaa-fcb8-4797-8295-9e8521de235b-maas-api-tls\") pod \"maas-api-e2e-worker-w3-a5115a-8754c4f46-g677p\" (UID: \"2ccb4aaa-fcb8-4797-8295-9e8521de235b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-a5115a-8754c4f46-g677p"
Nov 04 01:12:40.678387 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:40.678290    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbgjn\" (UniqueName: \"kubernetes.io/projected/2ccb4aaa-fcb8-4797-8295-9e8521de235b-kube-api-access-wbgjn\") pod \"maas-api-e2e-worker-w3-a5115a-8754c4f46-g677p\" (UID: \"2ccb4aaa-fcb8-4797-8295-9e8521de235b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-a5115a-8754c4f46-g677p"
Nov 04 01:12:40.678387 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:40.678335    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2ccb4aaa-fcb8-4797-8295-9e8521de235b-metrics-certs\") pod \"maas-api-e2e-worker-w3-a5115a-8754c4f46-g677p\" (UID: \"2ccb4aaa-fcb8-4797-8295-9e8521de235b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-a5115a-8754c4f46-g677p"
Nov 04 01:12:40.741101 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:40.741076    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-ba844c-7bc4dfcfd-nwk5b"]
Nov 04 01:12:40.779284 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:40.779246    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ccb4aaa-fcb8-4797-8295-9e8521de235b-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-a5115a-8754c4f46-g677p\" (UID: \"2ccb4aaa-fcb8-4797-8295-9e8521de235b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-a5115a-8754c4f46-g677p"
Nov 04 01:12:40.779404 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:40.779324    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ccb4aaa-fcb8-4797-8295-9e8521de235b-maas-api-tls\") pod \"maas-api-e2e-worker-w3-a5115a-8754c4f46-g677p\" (UID: \"2ccb4aaa-fcb8-4797-8295-9e8521de235b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-a5115a-8754c4f46-g677p"
Nov 04 01:12:40.779404 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:40.779352    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbgjn\" (UniqueName: \"kubernetes.io/projected/2ccb4aaa-fcb8-4797-8295-9e8521de235b-kube-api-access-wbgjn\") pod \"maas-api-e2e-worker-w3-a5115a-8754c4f46-g677p\" (UID: \"2ccb4aaa-fcb8-4797-8295-9e8521de235b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-a5115a-8754c4f46-g677p"
Nov 04 01:12:40.779404 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:40.779398    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2ccb4aaa-fcb8-4797-8295-9e8521de235b-metrics-certs\") pod \"maas-api-e2e-worker-w3-a5115a-8754c4f46-g677p\" (UID: \"2ccb4aaa-fcb8-4797-8295-9e8521de235b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-a5115a-8754c4f46-g677p"
Nov 04 01:12:40.781080 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:40.781051    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ccb4aaa-fcb8-4797-8295-9e8521de235b-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-a5115a-8754c4f46-g677p\" (UID: \"2ccb4aaa-fcb8-4797-8295-9e8521de235b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-a5115a-8754c4f46-g677p"
Nov 04 01:12:40.783599 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:40.783573    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2ccb4aaa-fcb8-4797-8295-9e8521de235b-metrics-certs\") pod \"maas-api-e2e-worker-w3-a5115a-8754c4f46-g677p\" (UID: \"2ccb4aaa-fcb8-4797-8295-9e8521de235b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-a5115a-8754c4f46-g677p"
Nov 04 01:12:40.786047 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:40.786027    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ccb4aaa-fcb8-4797-8295-9e8521de235b-maas-api-tls\") pod \"maas-api-e2e-worker-w3-a5115a-8754c4f46-g677p\" (UID: \"2ccb4aaa-fcb8-4797-8295-9e8521de235b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-a5115a-8754c4f46-g677p"
Nov 04 01:12:40.795104 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:40.795083    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbgjn\" (UniqueName: \"kubernetes.io/projected/2ccb4aaa-fcb8-4797-8295-9e8521de235b-kube-api-access-wbgjn\") pod \"maas-api-e2e-worker-w3-a5115a-8754c4f46-g677p\" (UID: \"2ccb4aaa-fcb8-4797-8295-9e8521de235b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-a5115a-8754c4f46-g677p"
Nov 04 01:12:40.856172 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:40.856144    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-a5115a-8754c4f46-g677p"
Nov 04 01:12:41.157595 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:41.157566    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-a5115a-8754c4f46-g677p"]
Nov 04 01:12:41.159990 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:12:41.159960    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ccb4aaa_fcb8_4797_8295_9e8521de235b.slice/crio-533a36ee81bde5d3a6726c54b3c69ce6c93b7a6fec9b8e6a98d478e8d3f91e5b WatchSource:0}: Error finding container 533a36ee81bde5d3a6726c54b3c69ce6c93b7a6fec9b8e6a98d478e8d3f91e5b: Status 404 returned error can't find the container with id 533a36ee81bde5d3a6726c54b3c69ce6c93b7a6fec9b8e6a98d478e8d3f91e5b
Nov 04 01:12:41.397221 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:41.397187    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-a5115a-8754c4f46-g677p" event={"ID":"2ccb4aaa-fcb8-4797-8295-9e8521de235b","Type":"ContainerStarted","Data":"533a36ee81bde5d3a6726c54b3c69ce6c93b7a6fec9b8e6a98d478e8d3f91e5b"}
Nov 04 01:12:41.399521 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:41.399492    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-cffa80-9b645ff4c-2pj8n" event={"ID":"841e9fef-46d6-4625-9b72-14b4e0eeb7dd","Type":"ContainerStarted","Data":"586e192813d95194bcadb83bf3a3cd34721c99f089734ca29eff252dbb718dab"}
Nov 04 01:12:41.399690 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:41.399635    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-cffa80-9b645ff4c-2pj8n"
Nov 04 01:12:41.401454 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:41.401427    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-cffa80-564fd64f66-qhhzq" event={"ID":"1b4a1b43-a286-436d-bafe-a10d9a57a4d2","Type":"ContainerStarted","Data":"3abb2e93264e4e70655b9bbadb6fae705629758c380b56c46258f3da781ab15d"}
Nov 04 01:12:41.401575 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:41.401484    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w2-cffa80-564fd64f66-qhhzq"
Nov 04 01:12:41.403323 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:41.403299    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-ba844c-7bc4dfcfd-nwk5b" event={"ID":"e0946b4c-7a9f-4044-8f44-53484315882d","Type":"ContainerStarted","Data":"b0d96574a62da39c743f1fec23623671952015364b05212fbe6fe334b0aa634c"}
Nov 04 01:12:41.403435 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:41.403349    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-ba844c-7bc4dfcfd-nwk5b" event={"ID":"e0946b4c-7a9f-4044-8f44-53484315882d","Type":"ContainerStarted","Data":"da4afd994971d474b1fa4e462ca9fdc3102938f6d2d64ec8e683069a7b917947"}
Nov 04 01:12:41.403767 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:41.403741    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-ba844c-7bc4dfcfd-nwk5b"
Nov 04 01:12:41.419556 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:41.419477    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-cffa80-9b645ff4c-2pj8n" podStartSLOduration=1.475102916 podStartE2EDuration="5.419461446s" totalImagesPullingTime="3.94435853s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:12:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:12:41.418164582 +0000 UTC m=+601.464725362" watchObservedRunningTime="2026-11-04 01:12:41.419461446 +0000 UTC m=+601.466022239"
Nov 04 01:12:41.436726 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:41.436500    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w2-cffa80-564fd64f66-qhhzq" podStartSLOduration=1.975651814 podStartE2EDuration="5.436488002s" totalImagesPullingTime="3.460836188s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:12:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:12:41.436050257 +0000 UTC m=+601.482611062" watchObservedRunningTime="2026-11-04 01:12:41.436488002 +0000 UTC m=+601.483048793"
Nov 04 01:12:41.455245 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:41.455195    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w4-ba844c-7bc4dfcfd-nwk5b" podStartSLOduration=1.4551798329999999 podStartE2EDuration="1.455179833s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:12:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:12:41.454067912 +0000 UTC m=+601.500628726" watchObservedRunningTime="2026-11-04 01:12:41.455179833 +0000 UTC m=+601.501740624"
Nov 04 01:12:42.411890 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:42.411852    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-6dac99-6d46fbff94-fcfwv" event={"ID":"98e65cde-ca44-4ad9-9a57-ce81c0369a0f","Type":"ContainerStarted","Data":"307a28b2c7e0cbeb01d080fae154cff7f63bb3598b2f756d178747115c5b319e"}
Nov 04 01:12:42.412341 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:42.411947    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-6dac99-6d46fbff94-fcfwv"
Nov 04 01:12:42.413717 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:42.413688    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-a5115a-8754c4f46-g677p" event={"ID":"2ccb4aaa-fcb8-4797-8295-9e8521de235b","Type":"ContainerStarted","Data":"bfde4d33a227dce129a9bb2a790e1597fc69017131f4ec9dc5e342a600ad2a96"}
Nov 04 01:12:42.413829 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:42.413757    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-a5115a-8754c4f46-g677p"
Nov 04 01:12:42.431582 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:42.431534    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-6dac99-6d46fbff94-fcfwv" podStartSLOduration=2.211589402 podStartE2EDuration="4.43151903s" totalImagesPullingTime="2.219929628s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:12:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:12:42.43026825 +0000 UTC m=+602.476829042" watchObservedRunningTime="2026-11-04 01:12:42.43151903 +0000 UTC m=+602.478079820"
Nov 04 01:12:43.442243 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:43.442182    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-a5115a-8754c4f46-g677p" podStartSLOduration=2.701093571 podStartE2EDuration="3.442163014s" totalImagesPullingTime="741.069443ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:12:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:12:42.452706484 +0000 UTC m=+602.499267275" watchObservedRunningTime="2026-11-04 01:12:43.442163014 +0000 UTC m=+603.488723806"
Nov 04 01:12:43.442712 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:43.442477    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-875528-74f64c5467-26b7b"]
Nov 04 01:12:43.447038 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:43.447017    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-875528-74f64c5467-26b7b"
Nov 04 01:12:43.451748 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:12:43.451720    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:12:43.455971 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:43.455950    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w5-875528-serving-cert\""
Nov 04 01:12:43.461015 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:43.460990    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-875528-74f64c5467-26b7b"]
Nov 04 01:12:43.607278 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:43.607249    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb1ea6b5-c573-4bfe-8339-caf72acbfc36-maas-api-tls\") pod \"maas-api-e2e-worker-w5-875528-74f64c5467-26b7b\" (UID: \"bb1ea6b5-c573-4bfe-8339-caf72acbfc36\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-875528-74f64c5467-26b7b"
Nov 04 01:12:43.607432 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:43.607287    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb1ea6b5-c573-4bfe-8339-caf72acbfc36-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-875528-74f64c5467-26b7b\" (UID: \"bb1ea6b5-c573-4bfe-8339-caf72acbfc36\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-875528-74f64c5467-26b7b"
Nov 04 01:12:43.607432 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:43.607312    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb1ea6b5-c573-4bfe-8339-caf72acbfc36-metrics-certs\") pod \"maas-api-e2e-worker-w5-875528-74f64c5467-26b7b\" (UID: \"bb1ea6b5-c573-4bfe-8339-caf72acbfc36\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-875528-74f64c5467-26b7b"
Nov 04 01:12:43.607539 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:43.607465    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tb6kq\" (UniqueName: \"kubernetes.io/projected/bb1ea6b5-c573-4bfe-8339-caf72acbfc36-kube-api-access-tb6kq\") pod \"maas-api-e2e-worker-w5-875528-74f64c5467-26b7b\" (UID: \"bb1ea6b5-c573-4bfe-8339-caf72acbfc36\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-875528-74f64c5467-26b7b"
Nov 04 01:12:43.708420 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:43.708313    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb1ea6b5-c573-4bfe-8339-caf72acbfc36-maas-api-tls\") pod \"maas-api-e2e-worker-w5-875528-74f64c5467-26b7b\" (UID: \"bb1ea6b5-c573-4bfe-8339-caf72acbfc36\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-875528-74f64c5467-26b7b"
Nov 04 01:12:43.708420 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:43.708375    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb1ea6b5-c573-4bfe-8339-caf72acbfc36-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-875528-74f64c5467-26b7b\" (UID: \"bb1ea6b5-c573-4bfe-8339-caf72acbfc36\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-875528-74f64c5467-26b7b"
Nov 04 01:12:43.708656 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:43.708420    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb1ea6b5-c573-4bfe-8339-caf72acbfc36-metrics-certs\") pod \"maas-api-e2e-worker-w5-875528-74f64c5467-26b7b\" (UID: \"bb1ea6b5-c573-4bfe-8339-caf72acbfc36\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-875528-74f64c5467-26b7b"
Nov 04 01:12:43.708656 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:43.708481    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tb6kq\" (UniqueName: \"kubernetes.io/projected/bb1ea6b5-c573-4bfe-8339-caf72acbfc36-kube-api-access-tb6kq\") pod \"maas-api-e2e-worker-w5-875528-74f64c5467-26b7b\" (UID: \"bb1ea6b5-c573-4bfe-8339-caf72acbfc36\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-875528-74f64c5467-26b7b"
Nov 04 01:12:43.709237 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:43.709208    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb1ea6b5-c573-4bfe-8339-caf72acbfc36-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-875528-74f64c5467-26b7b\" (UID: \"bb1ea6b5-c573-4bfe-8339-caf72acbfc36\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-875528-74f64c5467-26b7b"
Nov 04 01:12:43.711255 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:43.711227    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb1ea6b5-c573-4bfe-8339-caf72acbfc36-metrics-certs\") pod \"maas-api-e2e-worker-w5-875528-74f64c5467-26b7b\" (UID: \"bb1ea6b5-c573-4bfe-8339-caf72acbfc36\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-875528-74f64c5467-26b7b"
Nov 04 01:12:43.711449 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:43.711423    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb1ea6b5-c573-4bfe-8339-caf72acbfc36-maas-api-tls\") pod \"maas-api-e2e-worker-w5-875528-74f64c5467-26b7b\" (UID: \"bb1ea6b5-c573-4bfe-8339-caf72acbfc36\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-875528-74f64c5467-26b7b"
Nov 04 01:12:43.721047 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:43.721019    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tb6kq\" (UniqueName: \"kubernetes.io/projected/bb1ea6b5-c573-4bfe-8339-caf72acbfc36-kube-api-access-tb6kq\") pod \"maas-api-e2e-worker-w5-875528-74f64c5467-26b7b\" (UID: \"bb1ea6b5-c573-4bfe-8339-caf72acbfc36\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-875528-74f64c5467-26b7b"
Nov 04 01:12:43.763763 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:43.763732    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-875528-74f64c5467-26b7b"
Nov 04 01:12:44.049604 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:44.049570    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-875528-74f64c5467-26b7b"]
Nov 04 01:12:44.051940 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:12:44.051914    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb1ea6b5_c573_4bfe_8339_caf72acbfc36.slice/crio-0927c3cf607baee4352f528a970b13ae70d54676155defbdf8789f95e28f2079 WatchSource:0}: Error finding container 0927c3cf607baee4352f528a970b13ae70d54676155defbdf8789f95e28f2079: Status 404 returned error can't find the container with id 0927c3cf607baee4352f528a970b13ae70d54676155defbdf8789f95e28f2079
Nov 04 01:12:44.053440 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:44.053422    5068 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 01:12:44.427631 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:44.427507    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-875528-74f64c5467-26b7b" event={"ID":"bb1ea6b5-c573-4bfe-8339-caf72acbfc36","Type":"ContainerStarted","Data":"0927c3cf607baee4352f528a970b13ae70d54676155defbdf8789f95e28f2079"}
Nov 04 01:12:45.395732 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:45.395692    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-cffa80-8dcbb5c89-jx699"
Nov 04 01:12:45.442680 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:45.442645    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-875528-74f64c5467-26b7b" event={"ID":"bb1ea6b5-c573-4bfe-8339-caf72acbfc36","Type":"ContainerStarted","Data":"4b869b3553258c78432ea706d0f1043ad490d1dc2d9a0d0562dcd012671a5c58"}
Nov 04 01:12:45.442900 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:45.442877    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-875528-74f64c5467-26b7b"
Nov 04 01:12:45.464964 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:45.464903    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-875528-74f64c5467-26b7b" podStartSLOduration=1.975908612 podStartE2EDuration="2.464885254s" totalImagesPullingTime="488.976642ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:12:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:12:45.464818418 +0000 UTC m=+605.511379209" watchObservedRunningTime="2026-11-04 01:12:45.464885254 +0000 UTC m=+605.511446046"
Nov 04 01:12:45.732276 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:45.732192    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-1036e8-555f8d74d7-shvjl"]
Nov 04 01:12:45.736109 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:45.736088    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-1036e8-555f8d74d7-shvjl"
Nov 04 01:12:45.741713 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:45.741690    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-1036e8-555f8d74d7-shvjl"]
Nov 04 01:12:45.746907 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:45.746886    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w1-1036e8-serving-cert\""
Nov 04 01:12:45.880678 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:45.880647    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:12:45.931604 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:45.931562    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36b71cc4-21b7-42e0-86d7-1f25e215d068-maas-api-tls\") pod \"maas-api-e2e-worker-w1-1036e8-555f8d74d7-shvjl\" (UID: \"36b71cc4-21b7-42e0-86d7-1f25e215d068\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-1036e8-555f8d74d7-shvjl"
Nov 04 01:12:45.931604 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:45.931596    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36b71cc4-21b7-42e0-86d7-1f25e215d068-metrics-certs\") pod \"maas-api-e2e-worker-w1-1036e8-555f8d74d7-shvjl\" (UID: \"36b71cc4-21b7-42e0-86d7-1f25e215d068\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-1036e8-555f8d74d7-shvjl"
Nov 04 01:12:45.931867 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:45.931744    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ss9hr\" (UniqueName: \"kubernetes.io/projected/36b71cc4-21b7-42e0-86d7-1f25e215d068-kube-api-access-ss9hr\") pod \"maas-api-e2e-worker-w1-1036e8-555f8d74d7-shvjl\" (UID: \"36b71cc4-21b7-42e0-86d7-1f25e215d068\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-1036e8-555f8d74d7-shvjl"
Nov 04 01:12:45.931867 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:45.931786    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36b71cc4-21b7-42e0-86d7-1f25e215d068-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-1036e8-555f8d74d7-shvjl\" (UID: \"36b71cc4-21b7-42e0-86d7-1f25e215d068\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-1036e8-555f8d74d7-shvjl"
Nov 04 01:12:46.032541 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:46.032503    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36b71cc4-21b7-42e0-86d7-1f25e215d068-maas-api-tls\") pod \"maas-api-e2e-worker-w1-1036e8-555f8d74d7-shvjl\" (UID: \"36b71cc4-21b7-42e0-86d7-1f25e215d068\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-1036e8-555f8d74d7-shvjl"
Nov 04 01:12:46.032541 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:46.032541    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36b71cc4-21b7-42e0-86d7-1f25e215d068-metrics-certs\") pod \"maas-api-e2e-worker-w1-1036e8-555f8d74d7-shvjl\" (UID: \"36b71cc4-21b7-42e0-86d7-1f25e215d068\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-1036e8-555f8d74d7-shvjl"
Nov 04 01:12:46.032810 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:46.032646    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ss9hr\" (UniqueName: \"kubernetes.io/projected/36b71cc4-21b7-42e0-86d7-1f25e215d068-kube-api-access-ss9hr\") pod \"maas-api-e2e-worker-w1-1036e8-555f8d74d7-shvjl\" (UID: \"36b71cc4-21b7-42e0-86d7-1f25e215d068\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-1036e8-555f8d74d7-shvjl"
Nov 04 01:12:46.032810 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:46.032685    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36b71cc4-21b7-42e0-86d7-1f25e215d068-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-1036e8-555f8d74d7-shvjl\" (UID: \"36b71cc4-21b7-42e0-86d7-1f25e215d068\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-1036e8-555f8d74d7-shvjl"
Nov 04 01:12:46.033501 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:46.033470    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36b71cc4-21b7-42e0-86d7-1f25e215d068-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-1036e8-555f8d74d7-shvjl\" (UID: \"36b71cc4-21b7-42e0-86d7-1f25e215d068\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-1036e8-555f8d74d7-shvjl"
Nov 04 01:12:46.035192 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:46.035171    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36b71cc4-21b7-42e0-86d7-1f25e215d068-maas-api-tls\") pod \"maas-api-e2e-worker-w1-1036e8-555f8d74d7-shvjl\" (UID: \"36b71cc4-21b7-42e0-86d7-1f25e215d068\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-1036e8-555f8d74d7-shvjl"
Nov 04 01:12:46.035302 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:46.035221    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36b71cc4-21b7-42e0-86d7-1f25e215d068-metrics-certs\") pod \"maas-api-e2e-worker-w1-1036e8-555f8d74d7-shvjl\" (UID: \"36b71cc4-21b7-42e0-86d7-1f25e215d068\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-1036e8-555f8d74d7-shvjl"
Nov 04 01:12:46.052142 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:46.052114    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ss9hr\" (UniqueName: \"kubernetes.io/projected/36b71cc4-21b7-42e0-86d7-1f25e215d068-kube-api-access-ss9hr\") pod \"maas-api-e2e-worker-w1-1036e8-555f8d74d7-shvjl\" (UID: \"36b71cc4-21b7-42e0-86d7-1f25e215d068\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-1036e8-555f8d74d7-shvjl"
Nov 04 01:12:46.061567 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:46.061543    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-1036e8-555f8d74d7-shvjl"
Nov 04 01:12:46.350642 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:46.350017    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-1036e8-555f8d74d7-shvjl"]
Nov 04 01:12:46.352694 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:12:46.352647    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36b71cc4_21b7_42e0_86d7_1f25e215d068.slice/crio-f38eab67db1396f3c1761e657b5934242866a1e4dd7116269cc9fda8c771de43 WatchSource:0}: Error finding container f38eab67db1396f3c1761e657b5934242866a1e4dd7116269cc9fda8c771de43: Status 404 returned error can't find the container with id f38eab67db1396f3c1761e657b5934242866a1e4dd7116269cc9fda8c771de43
Nov 04 01:12:46.454658 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:46.454558    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-1036e8-555f8d74d7-shvjl" event={"ID":"36b71cc4-21b7-42e0-86d7-1f25e215d068","Type":"ContainerStarted","Data":"f38eab67db1396f3c1761e657b5934242866a1e4dd7116269cc9fda8c771de43"}
Nov 04 01:12:47.461247 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:47.461209    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-1036e8-555f8d74d7-shvjl" event={"ID":"36b71cc4-21b7-42e0-86d7-1f25e215d068","Type":"ContainerStarted","Data":"7f42411b12a36dcfd60e2c5916d1bb33a425798248f35c29895d1b11e7d072f0"}
Nov 04 01:12:47.461674 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:47.461278    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-1036e8-555f8d74d7-shvjl"
Nov 04 01:12:47.482439 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:47.482383    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-1036e8-555f8d74d7-shvjl" podStartSLOduration=1.907657154 podStartE2EDuration="2.482370236s" totalImagesPullingTime="574.713082ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:12:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:12:47.481074558 +0000 UTC m=+607.527635354" watchObservedRunningTime="2026-11-04 01:12:47.482370236 +0000 UTC m=+607.528931026"
Nov 04 01:12:48.424224 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:48.424195    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-a5115a-8754c4f46-g677p"
Nov 04 01:12:51.461017 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:51.460989    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-875528-74f64c5467-26b7b"
Nov 04 01:12:52.417742 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:52.417710    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w2-cffa80-564fd64f66-qhhzq"
Nov 04 01:12:52.418040 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:52.418018    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-cffa80-9b645ff4c-2pj8n"
Nov 04 01:12:52.418166 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:52.418075    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-ba844c-7bc4dfcfd-nwk5b"
Nov 04 01:12:53.423707 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:53.423671    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-6dac99-6d46fbff94-fcfwv"
Nov 04 01:12:53.478457 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:53.478425    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-1036e8-555f8d74d7-shvjl"
Nov 04 01:12:54.318823 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:54.318786    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:12:58.730475 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:58.730434    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n"]
Nov 04 01:12:58.734742 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:58.734724    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n"
Nov 04 01:12:58.743126 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:58.743094    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-9902c3\"/\"simulator-w6-9902c3-kserve-self-signed-certs\""
Nov 04 01:12:58.743416 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:58.743396    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-9902c3\"/\"openshift-service-ca.crt\""
Nov 04 01:12:58.743969 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:58.743953    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-9902c3\"/\"kube-root-ca.crt\""
Nov 04 01:12:58.746189 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:58.746133    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-9902c3\"/\"default-dockercfg-prz8w\""
Nov 04 01:12:58.748278 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:58.748252    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n"]
Nov 04 01:12:58.753803 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:58.753776    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3223811-a857-4279-a671-189e65383481-model-cache\") pod \"simulator-w6-9902c3-kserve-6bdb88785f-7tf6n\" (UID: \"b3223811-a857-4279-a671-189e65383481\") " pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n"
Nov 04 01:12:58.753915 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:58.753838    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3223811-a857-4279-a671-189e65383481-tls-certs\") pod \"simulator-w6-9902c3-kserve-6bdb88785f-7tf6n\" (UID: \"b3223811-a857-4279-a671-189e65383481\") " pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n"
Nov 04 01:12:58.753986 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:58.753933    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3223811-a857-4279-a671-189e65383481-home\") pod \"simulator-w6-9902c3-kserve-6bdb88785f-7tf6n\" (UID: \"b3223811-a857-4279-a671-189e65383481\") " pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n"
Nov 04 01:12:58.753986 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:58.753960    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3223811-a857-4279-a671-189e65383481-dshm\") pod \"simulator-w6-9902c3-kserve-6bdb88785f-7tf6n\" (UID: \"b3223811-a857-4279-a671-189e65383481\") " pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n"
Nov 04 01:12:58.754155 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:58.754130    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3223811-a857-4279-a671-189e65383481-tmp-dir\") pod \"simulator-w6-9902c3-kserve-6bdb88785f-7tf6n\" (UID: \"b3223811-a857-4279-a671-189e65383481\") " pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n"
Nov 04 01:12:58.754207 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:58.754183    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4m8n6\" (UniqueName: \"kubernetes.io/projected/b3223811-a857-4279-a671-189e65383481-kube-api-access-4m8n6\") pod \"simulator-w6-9902c3-kserve-6bdb88785f-7tf6n\" (UID: \"b3223811-a857-4279-a671-189e65383481\") " pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n"
Nov 04 01:12:58.855345 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:58.855306    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3223811-a857-4279-a671-189e65383481-model-cache\") pod \"simulator-w6-9902c3-kserve-6bdb88785f-7tf6n\" (UID: \"b3223811-a857-4279-a671-189e65383481\") " pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n"
Nov 04 01:12:58.855345 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:58.855361    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3223811-a857-4279-a671-189e65383481-tls-certs\") pod \"simulator-w6-9902c3-kserve-6bdb88785f-7tf6n\" (UID: \"b3223811-a857-4279-a671-189e65383481\") " pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n"
Nov 04 01:12:58.855670 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:58.855390    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3223811-a857-4279-a671-189e65383481-home\") pod \"simulator-w6-9902c3-kserve-6bdb88785f-7tf6n\" (UID: \"b3223811-a857-4279-a671-189e65383481\") " pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n"
Nov 04 01:12:58.855670 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:58.855463    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3223811-a857-4279-a671-189e65383481-dshm\") pod \"simulator-w6-9902c3-kserve-6bdb88785f-7tf6n\" (UID: \"b3223811-a857-4279-a671-189e65383481\") " pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n"
Nov 04 01:12:58.855670 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:58.855577    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3223811-a857-4279-a671-189e65383481-tmp-dir\") pod \"simulator-w6-9902c3-kserve-6bdb88785f-7tf6n\" (UID: \"b3223811-a857-4279-a671-189e65383481\") " pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n"
Nov 04 01:12:58.855846 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:58.855686    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4m8n6\" (UniqueName: \"kubernetes.io/projected/b3223811-a857-4279-a671-189e65383481-kube-api-access-4m8n6\") pod \"simulator-w6-9902c3-kserve-6bdb88785f-7tf6n\" (UID: \"b3223811-a857-4279-a671-189e65383481\") " pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n"
Nov 04 01:12:58.856024 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:58.855999    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3223811-a857-4279-a671-189e65383481-home\") pod \"simulator-w6-9902c3-kserve-6bdb88785f-7tf6n\" (UID: \"b3223811-a857-4279-a671-189e65383481\") " pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n"
Nov 04 01:12:58.856400 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:58.856377    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3223811-a857-4279-a671-189e65383481-model-cache\") pod \"simulator-w6-9902c3-kserve-6bdb88785f-7tf6n\" (UID: \"b3223811-a857-4279-a671-189e65383481\") " pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n"
Nov 04 01:12:58.857148 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:58.857123    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3223811-a857-4279-a671-189e65383481-tmp-dir\") pod \"simulator-w6-9902c3-kserve-6bdb88785f-7tf6n\" (UID: \"b3223811-a857-4279-a671-189e65383481\") " pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n"
Nov 04 01:12:58.858921 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:58.858895    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3223811-a857-4279-a671-189e65383481-dshm\") pod \"simulator-w6-9902c3-kserve-6bdb88785f-7tf6n\" (UID: \"b3223811-a857-4279-a671-189e65383481\") " pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n"
Nov 04 01:12:58.859757 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:58.859730    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3223811-a857-4279-a671-189e65383481-tls-certs\") pod \"simulator-w6-9902c3-kserve-6bdb88785f-7tf6n\" (UID: \"b3223811-a857-4279-a671-189e65383481\") " pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n"
Nov 04 01:12:58.869282 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:58.869251    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4m8n6\" (UniqueName: \"kubernetes.io/projected/b3223811-a857-4279-a671-189e65383481-kube-api-access-4m8n6\") pod \"simulator-w6-9902c3-kserve-6bdb88785f-7tf6n\" (UID: \"b3223811-a857-4279-a671-189e65383481\") " pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n"
Nov 04 01:12:59.052079 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:59.052045    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n"
Nov 04 01:12:59.336794 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:59.336765    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n"]
Nov 04 01:12:59.337371 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:12:59.337343    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb3223811_a857_4279_a671_189e65383481.slice/crio-e85236436c194bff4dd1ad9d0fdaac7c82750a5b5fd027a00244fc89cf093df9 WatchSource:0}: Error finding container e85236436c194bff4dd1ad9d0fdaac7c82750a5b5fd027a00244fc89cf093df9: Status 404 returned error can't find the container with id e85236436c194bff4dd1ad9d0fdaac7c82750a5b5fd027a00244fc89cf093df9
Nov 04 01:12:59.546593 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:12:59.546547    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n" event={"ID":"b3223811-a857-4279-a671-189e65383481","Type":"ContainerStarted","Data":"e85236436c194bff4dd1ad9d0fdaac7c82750a5b5fd027a00244fc89cf093df9"}
Nov 04 01:13:01.564124 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:01.564069    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n" event={"ID":"b3223811-a857-4279-a671-189e65383481","Type":"ContainerStarted","Data":"b0a18eed313cc80f07f5c24837dfced89714fe177b1cc708ed77b03144073ffc"}
Nov 04 01:13:01.586499 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:01.586439    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n" podStartSLOduration=2.209008608 podStartE2EDuration="3.586424507s" totalImagesPullingTime="1.377415899s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:12:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:13:01.585156593 +0000 UTC m=+621.631717385" watchObservedRunningTime="2026-11-04 01:13:01.586424507 +0000 UTC m=+621.632985296"
Nov 04 01:13:02.052865 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:02.052832    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n"
Nov 04 01:13:02.859236 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:02.859197    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:13:04.143432 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:04.143394    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9"]
Nov 04 01:13:04.147725 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:04.147695    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9"
Nov 04 01:13:04.157978 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:04.157951    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-ba844c\"/\"default-dockercfg-95xzg\""
Nov 04 01:13:04.158120 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:04.158056    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-ba844c\"/\"openshift-service-ca.crt\""
Nov 04 01:13:04.158120 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:04.158113    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-ba844c\"/\"kube-root-ca.crt\""
Nov 04 01:13:04.158228 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:04.158203    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-ba844c\"/\"simulator-w4-ba844c-kserve-self-signed-certs\""
Nov 04 01:13:04.162565 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:04.162539    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9"]
Nov 04 01:13:04.217362 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:04.217324    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/530fc578-6622-4c42-a384-02e73bf17b1f-tmp-dir\") pod \"simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9\" (UID: \"530fc578-6622-4c42-a384-02e73bf17b1f\") " pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9"
Nov 04 01:13:04.217362 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:04.217369    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/530fc578-6622-4c42-a384-02e73bf17b1f-model-cache\") pod \"simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9\" (UID: \"530fc578-6622-4c42-a384-02e73bf17b1f\") " pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9"
Nov 04 01:13:04.217644 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:04.217393    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/530fc578-6622-4c42-a384-02e73bf17b1f-dshm\") pod \"simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9\" (UID: \"530fc578-6622-4c42-a384-02e73bf17b1f\") " pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9"
Nov 04 01:13:04.217716 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:04.217675    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/530fc578-6622-4c42-a384-02e73bf17b1f-tls-certs\") pod \"simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9\" (UID: \"530fc578-6622-4c42-a384-02e73bf17b1f\") " pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9"
Nov 04 01:13:04.217873 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:04.217855    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xj55g\" (UniqueName: \"kubernetes.io/projected/530fc578-6622-4c42-a384-02e73bf17b1f-kube-api-access-xj55g\") pod \"simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9\" (UID: \"530fc578-6622-4c42-a384-02e73bf17b1f\") " pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9"
Nov 04 01:13:04.217935 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:04.217922    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/530fc578-6622-4c42-a384-02e73bf17b1f-home\") pod \"simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9\" (UID: \"530fc578-6622-4c42-a384-02e73bf17b1f\") " pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9"
Nov 04 01:13:04.319165 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:04.319126    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/530fc578-6622-4c42-a384-02e73bf17b1f-tmp-dir\") pod \"simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9\" (UID: \"530fc578-6622-4c42-a384-02e73bf17b1f\") " pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9"
Nov 04 01:13:04.319165 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:04.319174    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/530fc578-6622-4c42-a384-02e73bf17b1f-model-cache\") pod \"simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9\" (UID: \"530fc578-6622-4c42-a384-02e73bf17b1f\") " pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9"
Nov 04 01:13:04.319415 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:04.319197    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/530fc578-6622-4c42-a384-02e73bf17b1f-dshm\") pod \"simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9\" (UID: \"530fc578-6622-4c42-a384-02e73bf17b1f\") " pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9"
Nov 04 01:13:04.319415 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:04.319257    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/530fc578-6622-4c42-a384-02e73bf17b1f-tls-certs\") pod \"simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9\" (UID: \"530fc578-6622-4c42-a384-02e73bf17b1f\") " pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9"
Nov 04 01:13:04.319415 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:04.319375    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xj55g\" (UniqueName: \"kubernetes.io/projected/530fc578-6622-4c42-a384-02e73bf17b1f-kube-api-access-xj55g\") pod \"simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9\" (UID: \"530fc578-6622-4c42-a384-02e73bf17b1f\") " pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9"
Nov 04 01:13:04.319580 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:04.319440    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/530fc578-6622-4c42-a384-02e73bf17b1f-home\") pod \"simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9\" (UID: \"530fc578-6622-4c42-a384-02e73bf17b1f\") " pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9"
Nov 04 01:13:04.319674 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:04.319639    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/530fc578-6622-4c42-a384-02e73bf17b1f-model-cache\") pod \"simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9\" (UID: \"530fc578-6622-4c42-a384-02e73bf17b1f\") " pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9"
Nov 04 01:13:04.319759 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:04.319734    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/530fc578-6622-4c42-a384-02e73bf17b1f-tmp-dir\") pod \"simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9\" (UID: \"530fc578-6622-4c42-a384-02e73bf17b1f\") " pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9"
Nov 04 01:13:04.320015 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:04.319978    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/530fc578-6622-4c42-a384-02e73bf17b1f-home\") pod \"simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9\" (UID: \"530fc578-6622-4c42-a384-02e73bf17b1f\") " pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9"
Nov 04 01:13:04.322066 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:04.322043    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/530fc578-6622-4c42-a384-02e73bf17b1f-dshm\") pod \"simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9\" (UID: \"530fc578-6622-4c42-a384-02e73bf17b1f\") " pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9"
Nov 04 01:13:04.322600 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:04.322579    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/530fc578-6622-4c42-a384-02e73bf17b1f-tls-certs\") pod \"simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9\" (UID: \"530fc578-6622-4c42-a384-02e73bf17b1f\") " pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9"
Nov 04 01:13:04.337369 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:04.337322    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xj55g\" (UniqueName: \"kubernetes.io/projected/530fc578-6622-4c42-a384-02e73bf17b1f-kube-api-access-xj55g\") pod \"simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9\" (UID: \"530fc578-6622-4c42-a384-02e73bf17b1f\") " pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9"
Nov 04 01:13:04.465674 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:04.464881    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9"
Nov 04 01:13:04.760786 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:04.760756    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9"]
Nov 04 01:13:04.762113 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:13:04.762086    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod530fc578_6622_4c42_a384_02e73bf17b1f.slice/crio-eb559794dfad2b60135913f8a294b65938ec4c631df795e1cbbf7f67da4a244c WatchSource:0}: Error finding container eb559794dfad2b60135913f8a294b65938ec4c631df795e1cbbf7f67da4a244c: Status 404 returned error can't find the container with id eb559794dfad2b60135913f8a294b65938ec4c631df795e1cbbf7f67da4a244c
Nov 04 01:13:05.596337 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:05.596302    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9" event={"ID":"530fc578-6622-4c42-a384-02e73bf17b1f","Type":"ContainerStarted","Data":"6260d6e9440205c0e82ed5b4f6239f47fb7f94a97253f8f2f22703596d1129cd"}
Nov 04 01:13:05.596772 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:05.596347    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9" event={"ID":"530fc578-6622-4c42-a384-02e73bf17b1f","Type":"ContainerStarted","Data":"eb559794dfad2b60135913f8a294b65938ec4c631df795e1cbbf7f67da4a244c"}
Nov 04 01:13:05.619012 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:05.618951    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9" podStartSLOduration=1.618931523 podStartE2EDuration="1.618931523s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:13:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:13:05.61533005 +0000 UTC m=+625.661890845" watchObservedRunningTime="2026-11-04 01:13:05.618931523 +0000 UTC m=+625.665492318"
Nov 04 01:13:06.465473 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:06.465437    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9"
Nov 04 01:13:09.052680 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:09.052641    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n"
Nov 04 01:13:09.068877 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:09.068843    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n"
Nov 04 01:13:09.631537 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:09.631504    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n"
Nov 04 01:13:10.528815 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:10.528778    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5"]
Nov 04 01:13:10.534646 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:10.534601    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5"
Nov 04 01:13:10.539960 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:10.539873    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-a5115a\"/\"default-dockercfg-458wm\""
Nov 04 01:13:10.540182 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:10.539937    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-a5115a\"/\"kube-root-ca.crt\""
Nov 04 01:13:10.540182 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:10.540022    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-a5115a\"/\"openshift-service-ca.crt\""
Nov 04 01:13:10.540359 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:10.540075    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-a5115a\"/\"simulator-w3-a5115a-kserve-self-signed-certs\""
Nov 04 01:13:10.550246 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:10.549578    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5"]
Nov 04 01:13:10.594509 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:10.594465    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpnkh\" (UniqueName: \"kubernetes.io/projected/60f5c87a-48f8-4ad9-847b-cff571cc9340-kube-api-access-jpnkh\") pod \"simulator-w3-a5115a-kserve-569c5c47bd-wrkv5\" (UID: \"60f5c87a-48f8-4ad9-847b-cff571cc9340\") " pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5"
Nov 04 01:13:10.594753 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:10.594629    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60f5c87a-48f8-4ad9-847b-cff571cc9340-home\") pod \"simulator-w3-a5115a-kserve-569c5c47bd-wrkv5\" (UID: \"60f5c87a-48f8-4ad9-847b-cff571cc9340\") " pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5"
Nov 04 01:13:10.594753 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:10.594742    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60f5c87a-48f8-4ad9-847b-cff571cc9340-dshm\") pod \"simulator-w3-a5115a-kserve-569c5c47bd-wrkv5\" (UID: \"60f5c87a-48f8-4ad9-847b-cff571cc9340\") " pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5"
Nov 04 01:13:10.594895 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:10.594785    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60f5c87a-48f8-4ad9-847b-cff571cc9340-model-cache\") pod \"simulator-w3-a5115a-kserve-569c5c47bd-wrkv5\" (UID: \"60f5c87a-48f8-4ad9-847b-cff571cc9340\") " pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5"
Nov 04 01:13:10.594967 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:10.594883    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60f5c87a-48f8-4ad9-847b-cff571cc9340-tmp-dir\") pod \"simulator-w3-a5115a-kserve-569c5c47bd-wrkv5\" (UID: \"60f5c87a-48f8-4ad9-847b-cff571cc9340\") " pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5"
Nov 04 01:13:10.594967 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:10.594940    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60f5c87a-48f8-4ad9-847b-cff571cc9340-tls-certs\") pod \"simulator-w3-a5115a-kserve-569c5c47bd-wrkv5\" (UID: \"60f5c87a-48f8-4ad9-847b-cff571cc9340\") " pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5"
Nov 04 01:13:10.695718 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:10.695670    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60f5c87a-48f8-4ad9-847b-cff571cc9340-tmp-dir\") pod \"simulator-w3-a5115a-kserve-569c5c47bd-wrkv5\" (UID: \"60f5c87a-48f8-4ad9-847b-cff571cc9340\") " pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5"
Nov 04 01:13:10.695911 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:10.695741    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60f5c87a-48f8-4ad9-847b-cff571cc9340-tls-certs\") pod \"simulator-w3-a5115a-kserve-569c5c47bd-wrkv5\" (UID: \"60f5c87a-48f8-4ad9-847b-cff571cc9340\") " pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5"
Nov 04 01:13:10.695911 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:10.695794    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpnkh\" (UniqueName: \"kubernetes.io/projected/60f5c87a-48f8-4ad9-847b-cff571cc9340-kube-api-access-jpnkh\") pod \"simulator-w3-a5115a-kserve-569c5c47bd-wrkv5\" (UID: \"60f5c87a-48f8-4ad9-847b-cff571cc9340\") " pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5"
Nov 04 01:13:10.695911 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:10.695855    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60f5c87a-48f8-4ad9-847b-cff571cc9340-home\") pod \"simulator-w3-a5115a-kserve-569c5c47bd-wrkv5\" (UID: \"60f5c87a-48f8-4ad9-847b-cff571cc9340\") " pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5"
Nov 04 01:13:10.696024 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:10.695927    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60f5c87a-48f8-4ad9-847b-cff571cc9340-dshm\") pod \"simulator-w3-a5115a-kserve-569c5c47bd-wrkv5\" (UID: \"60f5c87a-48f8-4ad9-847b-cff571cc9340\") " pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5"
Nov 04 01:13:10.696024 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:10.695953    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60f5c87a-48f8-4ad9-847b-cff571cc9340-model-cache\") pod \"simulator-w3-a5115a-kserve-569c5c47bd-wrkv5\" (UID: \"60f5c87a-48f8-4ad9-847b-cff571cc9340\") " pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5"
Nov 04 01:13:10.696208 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:10.696178    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60f5c87a-48f8-4ad9-847b-cff571cc9340-tmp-dir\") pod \"simulator-w3-a5115a-kserve-569c5c47bd-wrkv5\" (UID: \"60f5c87a-48f8-4ad9-847b-cff571cc9340\") " pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5"
Nov 04 01:13:10.696379 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:10.696258    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60f5c87a-48f8-4ad9-847b-cff571cc9340-model-cache\") pod \"simulator-w3-a5115a-kserve-569c5c47bd-wrkv5\" (UID: \"60f5c87a-48f8-4ad9-847b-cff571cc9340\") " pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5"
Nov 04 01:13:10.696534 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:10.696513    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60f5c87a-48f8-4ad9-847b-cff571cc9340-home\") pod \"simulator-w3-a5115a-kserve-569c5c47bd-wrkv5\" (UID: \"60f5c87a-48f8-4ad9-847b-cff571cc9340\") " pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5"
Nov 04 01:13:10.698826 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:10.698804    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60f5c87a-48f8-4ad9-847b-cff571cc9340-dshm\") pod \"simulator-w3-a5115a-kserve-569c5c47bd-wrkv5\" (UID: \"60f5c87a-48f8-4ad9-847b-cff571cc9340\") " pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5"
Nov 04 01:13:10.700070 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:10.700047    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60f5c87a-48f8-4ad9-847b-cff571cc9340-tls-certs\") pod \"simulator-w3-a5115a-kserve-569c5c47bd-wrkv5\" (UID: \"60f5c87a-48f8-4ad9-847b-cff571cc9340\") " pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5"
Nov 04 01:13:10.711196 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:10.711168    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpnkh\" (UniqueName: \"kubernetes.io/projected/60f5c87a-48f8-4ad9-847b-cff571cc9340-kube-api-access-jpnkh\") pod \"simulator-w3-a5115a-kserve-569c5c47bd-wrkv5\" (UID: \"60f5c87a-48f8-4ad9-847b-cff571cc9340\") " pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5"
Nov 04 01:13:10.857960 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:10.857851    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5"
Nov 04 01:13:11.145945 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:11.145856    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5"]
Nov 04 01:13:11.149519 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:13:11.149471    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod60f5c87a_48f8_4ad9_847b_cff571cc9340.slice/crio-b84fa3d3cedca73535ffa0f430e2408672672778410408a592b1f8d180c85df5 WatchSource:0}: Error finding container b84fa3d3cedca73535ffa0f430e2408672672778410408a592b1f8d180c85df5: Status 404 returned error can't find the container with id b84fa3d3cedca73535ffa0f430e2408672672778410408a592b1f8d180c85df5
Nov 04 01:13:11.634525 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:11.634490    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5" event={"ID":"60f5c87a-48f8-4ad9-847b-cff571cc9340","Type":"ContainerStarted","Data":"d0e8cee1f4946b2b00040a0af013b5fc0187f86d45cb571c1d2e02b00da992f6"}
Nov 04 01:13:11.634967 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:11.634533    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5" event={"ID":"60f5c87a-48f8-4ad9-847b-cff571cc9340","Type":"ContainerStarted","Data":"b84fa3d3cedca73535ffa0f430e2408672672778410408a592b1f8d180c85df5"}
Nov 04 01:13:11.654778 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:11.654717    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5" podStartSLOduration=1.654696202 podStartE2EDuration="1.654696202s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:13:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:13:11.654097882 +0000 UTC m=+631.700658676" watchObservedRunningTime="2026-11-04 01:13:11.654696202 +0000 UTC m=+631.701256992"
Nov 04 01:13:11.858301 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:11.858265    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5"
Nov 04 01:13:13.463744 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:13:13.463690    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:13:14.077485 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:14.077440    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:13:14.466051 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:14.465949    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9"
Nov 04 01:13:14.484668 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:14.484641    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9"
Nov 04 01:13:14.667892 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:14.667861    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9"
Nov 04 01:13:20.857976 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:20.857936    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5"
Nov 04 01:13:20.871939 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:20.871909    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5"
Nov 04 01:13:21.710184 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:21.710154    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5"
Nov 04 01:13:26.076174 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:26.076136    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx"]
Nov 04 01:13:26.081457 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:26.081429    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx"
Nov 04 01:13:26.087547 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:26.087517    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx"]
Nov 04 01:13:26.090142 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:26.090115    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-9902c3\"/\"premium-w6-9902c3-kserve-self-signed-certs\""
Nov 04 01:13:26.258026 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:26.257980    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qc267\" (UniqueName: \"kubernetes.io/projected/46d4ec7b-558d-4fb5-8011-177760a3291c-kube-api-access-qc267\") pod \"premium-w6-9902c3-kserve-6c485f796b-p7cpx\" (UID: \"46d4ec7b-558d-4fb5-8011-177760a3291c\") " pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx"
Nov 04 01:13:26.258251 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:26.258055    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/46d4ec7b-558d-4fb5-8011-177760a3291c-home\") pod \"premium-w6-9902c3-kserve-6c485f796b-p7cpx\" (UID: \"46d4ec7b-558d-4fb5-8011-177760a3291c\") " pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx"
Nov 04 01:13:26.258251 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:26.258103    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/46d4ec7b-558d-4fb5-8011-177760a3291c-dshm\") pod \"premium-w6-9902c3-kserve-6c485f796b-p7cpx\" (UID: \"46d4ec7b-558d-4fb5-8011-177760a3291c\") " pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx"
Nov 04 01:13:26.258251 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:26.258131    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46d4ec7b-558d-4fb5-8011-177760a3291c-tmp-dir\") pod \"premium-w6-9902c3-kserve-6c485f796b-p7cpx\" (UID: \"46d4ec7b-558d-4fb5-8011-177760a3291c\") " pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx"
Nov 04 01:13:26.258251 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:26.258204    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46d4ec7b-558d-4fb5-8011-177760a3291c-tls-certs\") pod \"premium-w6-9902c3-kserve-6c485f796b-p7cpx\" (UID: \"46d4ec7b-558d-4fb5-8011-177760a3291c\") " pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx"
Nov 04 01:13:26.258251 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:26.258246    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46d4ec7b-558d-4fb5-8011-177760a3291c-model-cache\") pod \"premium-w6-9902c3-kserve-6c485f796b-p7cpx\" (UID: \"46d4ec7b-558d-4fb5-8011-177760a3291c\") " pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx"
Nov 04 01:13:26.359181 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:26.359092    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qc267\" (UniqueName: \"kubernetes.io/projected/46d4ec7b-558d-4fb5-8011-177760a3291c-kube-api-access-qc267\") pod \"premium-w6-9902c3-kserve-6c485f796b-p7cpx\" (UID: \"46d4ec7b-558d-4fb5-8011-177760a3291c\") " pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx"
Nov 04 01:13:26.359181 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:26.359145    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/46d4ec7b-558d-4fb5-8011-177760a3291c-home\") pod \"premium-w6-9902c3-kserve-6c485f796b-p7cpx\" (UID: \"46d4ec7b-558d-4fb5-8011-177760a3291c\") " pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx"
Nov 04 01:13:26.359181 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:26.359169    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/46d4ec7b-558d-4fb5-8011-177760a3291c-dshm\") pod \"premium-w6-9902c3-kserve-6c485f796b-p7cpx\" (UID: \"46d4ec7b-558d-4fb5-8011-177760a3291c\") " pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx"
Nov 04 01:13:26.359493 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:26.359184    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46d4ec7b-558d-4fb5-8011-177760a3291c-tmp-dir\") pod \"premium-w6-9902c3-kserve-6c485f796b-p7cpx\" (UID: \"46d4ec7b-558d-4fb5-8011-177760a3291c\") " pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx"
Nov 04 01:13:26.359493 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:26.359219    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46d4ec7b-558d-4fb5-8011-177760a3291c-tls-certs\") pod \"premium-w6-9902c3-kserve-6c485f796b-p7cpx\" (UID: \"46d4ec7b-558d-4fb5-8011-177760a3291c\") " pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx"
Nov 04 01:13:26.359493 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:26.359242    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46d4ec7b-558d-4fb5-8011-177760a3291c-model-cache\") pod \"premium-w6-9902c3-kserve-6c485f796b-p7cpx\" (UID: \"46d4ec7b-558d-4fb5-8011-177760a3291c\") " pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx"
Nov 04 01:13:26.359723 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:26.359695    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46d4ec7b-558d-4fb5-8011-177760a3291c-tmp-dir\") pod \"premium-w6-9902c3-kserve-6c485f796b-p7cpx\" (UID: \"46d4ec7b-558d-4fb5-8011-177760a3291c\") " pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx"
Nov 04 01:13:26.359796 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:26.359726    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/46d4ec7b-558d-4fb5-8011-177760a3291c-home\") pod \"premium-w6-9902c3-kserve-6c485f796b-p7cpx\" (UID: \"46d4ec7b-558d-4fb5-8011-177760a3291c\") " pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx"
Nov 04 01:13:26.359942 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:26.359920    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46d4ec7b-558d-4fb5-8011-177760a3291c-model-cache\") pod \"premium-w6-9902c3-kserve-6c485f796b-p7cpx\" (UID: \"46d4ec7b-558d-4fb5-8011-177760a3291c\") " pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx"
Nov 04 01:13:26.361657 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:26.361629    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/46d4ec7b-558d-4fb5-8011-177760a3291c-dshm\") pod \"premium-w6-9902c3-kserve-6c485f796b-p7cpx\" (UID: \"46d4ec7b-558d-4fb5-8011-177760a3291c\") " pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx"
Nov 04 01:13:26.361925 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:26.361908    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46d4ec7b-558d-4fb5-8011-177760a3291c-tls-certs\") pod \"premium-w6-9902c3-kserve-6c485f796b-p7cpx\" (UID: \"46d4ec7b-558d-4fb5-8011-177760a3291c\") " pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx"
Nov 04 01:13:26.371758 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:26.371717    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qc267\" (UniqueName: \"kubernetes.io/projected/46d4ec7b-558d-4fb5-8011-177760a3291c-kube-api-access-qc267\") pod \"premium-w6-9902c3-kserve-6c485f796b-p7cpx\" (UID: \"46d4ec7b-558d-4fb5-8011-177760a3291c\") " pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx"
Nov 04 01:13:26.401370 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:26.401332    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx"
Nov 04 01:13:26.614723 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:26.614642    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:13:26.688031 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:26.687990    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx"]
Nov 04 01:13:26.691284 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:13:26.691255    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod46d4ec7b_558d_4fb5_8011_177760a3291c.slice/crio-0acb30231e0b0d146753d3b5f0326775a2cbf065de94e648c7121247308cf043 WatchSource:0}: Error finding container 0acb30231e0b0d146753d3b5f0326775a2cbf065de94e648c7121247308cf043: Status 404 returned error can't find the container with id 0acb30231e0b0d146753d3b5f0326775a2cbf065de94e648c7121247308cf043
Nov 04 01:13:26.731250 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:26.731220    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx" event={"ID":"46d4ec7b-558d-4fb5-8011-177760a3291c","Type":"ContainerStarted","Data":"0acb30231e0b0d146753d3b5f0326775a2cbf065de94e648c7121247308cf043"}
Nov 04 01:13:27.739115 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:27.739075    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx" event={"ID":"46d4ec7b-558d-4fb5-8011-177760a3291c","Type":"ContainerStarted","Data":"7b08752aa3a0af5715424d861acaf320decde640a2d39d3d63ab1725ee78098b"}
Nov 04 01:13:27.764416 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:27.764354    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx" podStartSLOduration=1.764333224 podStartE2EDuration="1.764333224s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:13:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:13:27.760843851 +0000 UTC m=+647.807404643" watchObservedRunningTime="2026-11-04 01:13:27.764333224 +0000 UTC m=+647.810894021"
Nov 04 01:13:28.402344 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:28.402312    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx"
Nov 04 01:13:32.869854 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:32.869816    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz"]
Nov 04 01:13:32.874850 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:32.874816    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz"
Nov 04 01:13:32.881366 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:32.881335    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-ba844c\"/\"premium-w4-ba844c-kserve-self-signed-certs\""
Nov 04 01:13:32.903032 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:32.895407    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz"]
Nov 04 01:13:32.931828 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:32.931783    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ccafa61-1c9b-4595-a511-3404d7479526-tls-certs\") pod \"premium-w4-ba844c-kserve-79b7f75f5d-r6hfz\" (UID: \"4ccafa61-1c9b-4595-a511-3404d7479526\") " pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz"
Nov 04 01:13:32.932049 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:32.931905    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ccafa61-1c9b-4595-a511-3404d7479526-home\") pod \"premium-w4-ba844c-kserve-79b7f75f5d-r6hfz\" (UID: \"4ccafa61-1c9b-4595-a511-3404d7479526\") " pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz"
Nov 04 01:13:32.932049 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:32.931982    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ccafa61-1c9b-4595-a511-3404d7479526-tmp-dir\") pod \"premium-w4-ba844c-kserve-79b7f75f5d-r6hfz\" (UID: \"4ccafa61-1c9b-4595-a511-3404d7479526\") " pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz"
Nov 04 01:13:32.932181 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:32.932157    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ccafa61-1c9b-4595-a511-3404d7479526-model-cache\") pod \"premium-w4-ba844c-kserve-79b7f75f5d-r6hfz\" (UID: \"4ccafa61-1c9b-4595-a511-3404d7479526\") " pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz"
Nov 04 01:13:32.932249 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:32.932231    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ccafa61-1c9b-4595-a511-3404d7479526-dshm\") pod \"premium-w4-ba844c-kserve-79b7f75f5d-r6hfz\" (UID: \"4ccafa61-1c9b-4595-a511-3404d7479526\") " pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz"
Nov 04 01:13:32.932330 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:32.932315    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62ghr\" (UniqueName: \"kubernetes.io/projected/4ccafa61-1c9b-4595-a511-3404d7479526-kube-api-access-62ghr\") pod \"premium-w4-ba844c-kserve-79b7f75f5d-r6hfz\" (UID: \"4ccafa61-1c9b-4595-a511-3404d7479526\") " pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz"
Nov 04 01:13:33.035957 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:33.035912    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ccafa61-1c9b-4595-a511-3404d7479526-model-cache\") pod \"premium-w4-ba844c-kserve-79b7f75f5d-r6hfz\" (UID: \"4ccafa61-1c9b-4595-a511-3404d7479526\") " pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz"
Nov 04 01:13:33.036212 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:33.035966    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ccafa61-1c9b-4595-a511-3404d7479526-dshm\") pod \"premium-w4-ba844c-kserve-79b7f75f5d-r6hfz\" (UID: \"4ccafa61-1c9b-4595-a511-3404d7479526\") " pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz"
Nov 04 01:13:33.036212 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:33.036015    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62ghr\" (UniqueName: \"kubernetes.io/projected/4ccafa61-1c9b-4595-a511-3404d7479526-kube-api-access-62ghr\") pod \"premium-w4-ba844c-kserve-79b7f75f5d-r6hfz\" (UID: \"4ccafa61-1c9b-4595-a511-3404d7479526\") " pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz"
Nov 04 01:13:33.036212 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:33.036062    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ccafa61-1c9b-4595-a511-3404d7479526-tls-certs\") pod \"premium-w4-ba844c-kserve-79b7f75f5d-r6hfz\" (UID: \"4ccafa61-1c9b-4595-a511-3404d7479526\") " pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz"
Nov 04 01:13:33.036554 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:33.036530    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ccafa61-1c9b-4595-a511-3404d7479526-model-cache\") pod \"premium-w4-ba844c-kserve-79b7f75f5d-r6hfz\" (UID: \"4ccafa61-1c9b-4595-a511-3404d7479526\") " pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz"
Nov 04 01:13:33.036677 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:33.036652    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ccafa61-1c9b-4595-a511-3404d7479526-home\") pod \"premium-w4-ba844c-kserve-79b7f75f5d-r6hfz\" (UID: \"4ccafa61-1c9b-4595-a511-3404d7479526\") " pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz"
Nov 04 01:13:33.036745 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:33.036733    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ccafa61-1c9b-4595-a511-3404d7479526-tmp-dir\") pod \"premium-w4-ba844c-kserve-79b7f75f5d-r6hfz\" (UID: \"4ccafa61-1c9b-4595-a511-3404d7479526\") " pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz"
Nov 04 01:13:33.036992 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:33.036973    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ccafa61-1c9b-4595-a511-3404d7479526-tmp-dir\") pod \"premium-w4-ba844c-kserve-79b7f75f5d-r6hfz\" (UID: \"4ccafa61-1c9b-4595-a511-3404d7479526\") " pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz"
Nov 04 01:13:33.038180 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:33.038151    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ccafa61-1c9b-4595-a511-3404d7479526-home\") pod \"premium-w4-ba844c-kserve-79b7f75f5d-r6hfz\" (UID: \"4ccafa61-1c9b-4595-a511-3404d7479526\") " pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz"
Nov 04 01:13:33.041721 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:33.039950    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ccafa61-1c9b-4595-a511-3404d7479526-dshm\") pod \"premium-w4-ba844c-kserve-79b7f75f5d-r6hfz\" (UID: \"4ccafa61-1c9b-4595-a511-3404d7479526\") " pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz"
Nov 04 01:13:33.041721 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:33.040587    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ccafa61-1c9b-4595-a511-3404d7479526-tls-certs\") pod \"premium-w4-ba844c-kserve-79b7f75f5d-r6hfz\" (UID: \"4ccafa61-1c9b-4595-a511-3404d7479526\") " pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz"
Nov 04 01:13:33.058529 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:33.058469    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-62ghr\" (UniqueName: \"kubernetes.io/projected/4ccafa61-1c9b-4595-a511-3404d7479526-kube-api-access-62ghr\") pod \"premium-w4-ba844c-kserve-79b7f75f5d-r6hfz\" (UID: \"4ccafa61-1c9b-4595-a511-3404d7479526\") " pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz"
Nov 04 01:13:33.204027 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:33.203943    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz"
Nov 04 01:13:33.498043 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:33.498012    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz"]
Nov 04 01:13:33.499915 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:13:33.499887    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4ccafa61_1c9b_4595_a511_3404d7479526.slice/crio-040690e7e691376e95f02cd647194b58ae610f2b4bb5a308a6939e3375feed0d WatchSource:0}: Error finding container 040690e7e691376e95f02cd647194b58ae610f2b4bb5a308a6939e3375feed0d: Status 404 returned error can't find the container with id 040690e7e691376e95f02cd647194b58ae610f2b4bb5a308a6939e3375feed0d
Nov 04 01:13:33.781814 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:33.781775    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz" event={"ID":"4ccafa61-1c9b-4595-a511-3404d7479526","Type":"ContainerStarted","Data":"a2ff4fb0c4fcceff17ddbfb82eeb035a68edab36cb506afd2a4836dffef76c75"}
Nov 04 01:13:33.781978 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:33.781830    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz" event={"ID":"4ccafa61-1c9b-4595-a511-3404d7479526","Type":"ContainerStarted","Data":"040690e7e691376e95f02cd647194b58ae610f2b4bb5a308a6939e3375feed0d"}
Nov 04 01:13:33.804465 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:33.804393    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz" podStartSLOduration=1.804378541 podStartE2EDuration="1.804378541s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:13:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:13:33.800505544 +0000 UTC m=+653.847066335" watchObservedRunningTime="2026-11-04 01:13:33.804378541 +0000 UTC m=+653.850939349"
Nov 04 01:13:34.204338 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:34.204248    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz"
Nov 04 01:13:36.402473 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:36.402440    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx"
Nov 04 01:13:36.421409 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:36.421385    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx"
Nov 04 01:13:36.812380 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:36.812344    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx"
Nov 04 01:13:40.537631 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:40.537589    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:13:43.204821 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:43.204785    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz"
Nov 04 01:13:43.218354 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:43.218324    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz"
Nov 04 01:13:43.451731 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:13:43.451683    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:13:43.861917 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:13:43.861883    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz"
Nov 04 01:14:01.377088 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:01.377054    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:14:13.450304 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:14:13.450268    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:14:16.632340 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:16.632302    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds"]
Nov 04 01:14:16.647803 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:16.647761    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds"
Nov 04 01:14:16.652668 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:16.652641    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds"]
Nov 04 01:14:16.689834 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:16.689800    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84f5c469-8fea-4b88-969b-d1eed184c268-istio-data\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds\" (UID: \"84f5c469-8fea-4b88-969b-d1eed184c268\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds"
Nov 04 01:14:16.689972 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:16.689895    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8dc6\" (UniqueName: \"kubernetes.io/projected/84f5c469-8fea-4b88-969b-d1eed184c268-kube-api-access-z8dc6\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds\" (UID: \"84f5c469-8fea-4b88-969b-d1eed184c268\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds"
Nov 04 01:14:16.689972 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:16.689942    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84f5c469-8fea-4b88-969b-d1eed184c268-istio-podinfo\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds\" (UID: \"84f5c469-8fea-4b88-969b-d1eed184c268\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds"
Nov 04 01:14:16.689972 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:16.689965    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84f5c469-8fea-4b88-969b-d1eed184c268-istiod-ca-cert\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds\" (UID: \"84f5c469-8fea-4b88-969b-d1eed184c268\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds"
Nov 04 01:14:16.690136 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:16.690101    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84f5c469-8fea-4b88-969b-d1eed184c268-istio-envoy\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds\" (UID: \"84f5c469-8fea-4b88-969b-d1eed184c268\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds"
Nov 04 01:14:16.690191 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:16.690154    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84f5c469-8fea-4b88-969b-d1eed184c268-istio-token\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds\" (UID: \"84f5c469-8fea-4b88-969b-d1eed184c268\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds"
Nov 04 01:14:16.690248 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:16.690222    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84f5c469-8fea-4b88-969b-d1eed184c268-credential-socket\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds\" (UID: \"84f5c469-8fea-4b88-969b-d1eed184c268\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds"
Nov 04 01:14:16.690285 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:16.690260    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84f5c469-8fea-4b88-969b-d1eed184c268-workload-socket\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds\" (UID: \"84f5c469-8fea-4b88-969b-d1eed184c268\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds"
Nov 04 01:14:16.690317 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:16.690306    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84f5c469-8fea-4b88-969b-d1eed184c268-workload-certs\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds\" (UID: \"84f5c469-8fea-4b88-969b-d1eed184c268\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds"
Nov 04 01:14:16.733333 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:16.733312    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-a5115a-openshift-default-dockercfg-fx5s6\""
Nov 04 01:14:16.791603 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:16.791572    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84f5c469-8fea-4b88-969b-d1eed184c268-istio-token\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds\" (UID: \"84f5c469-8fea-4b88-969b-d1eed184c268\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds"
Nov 04 01:14:16.791828 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:16.791638    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84f5c469-8fea-4b88-969b-d1eed184c268-credential-socket\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds\" (UID: \"84f5c469-8fea-4b88-969b-d1eed184c268\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds"
Nov 04 01:14:16.791828 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:16.791675    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84f5c469-8fea-4b88-969b-d1eed184c268-workload-socket\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds\" (UID: \"84f5c469-8fea-4b88-969b-d1eed184c268\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds"
Nov 04 01:14:16.791828 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:16.791765    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84f5c469-8fea-4b88-969b-d1eed184c268-workload-certs\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds\" (UID: \"84f5c469-8fea-4b88-969b-d1eed184c268\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds"
Nov 04 01:14:16.791828 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:16.791806    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84f5c469-8fea-4b88-969b-d1eed184c268-istio-data\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds\" (UID: \"84f5c469-8fea-4b88-969b-d1eed184c268\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds"
Nov 04 01:14:16.792035 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:16.791850    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8dc6\" (UniqueName: \"kubernetes.io/projected/84f5c469-8fea-4b88-969b-d1eed184c268-kube-api-access-z8dc6\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds\" (UID: \"84f5c469-8fea-4b88-969b-d1eed184c268\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds"
Nov 04 01:14:16.792035 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:16.791899    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84f5c469-8fea-4b88-969b-d1eed184c268-istio-podinfo\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds\" (UID: \"84f5c469-8fea-4b88-969b-d1eed184c268\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds"
Nov 04 01:14:16.792146 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:16.792034    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84f5c469-8fea-4b88-969b-d1eed184c268-istiod-ca-cert\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds\" (UID: \"84f5c469-8fea-4b88-969b-d1eed184c268\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds"
Nov 04 01:14:16.792146 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:16.792062    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84f5c469-8fea-4b88-969b-d1eed184c268-credential-socket\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds\" (UID: \"84f5c469-8fea-4b88-969b-d1eed184c268\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds"
Nov 04 01:14:16.792146 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:16.792097    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84f5c469-8fea-4b88-969b-d1eed184c268-istio-envoy\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds\" (UID: \"84f5c469-8fea-4b88-969b-d1eed184c268\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds"
Nov 04 01:14:16.792146 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:16.792110    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84f5c469-8fea-4b88-969b-d1eed184c268-workload-socket\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds\" (UID: \"84f5c469-8fea-4b88-969b-d1eed184c268\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds"
Nov 04 01:14:16.792313 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:16.792179    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84f5c469-8fea-4b88-969b-d1eed184c268-workload-certs\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds\" (UID: \"84f5c469-8fea-4b88-969b-d1eed184c268\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds"
Nov 04 01:14:16.792313 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:16.792270    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84f5c469-8fea-4b88-969b-d1eed184c268-istio-data\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds\" (UID: \"84f5c469-8fea-4b88-969b-d1eed184c268\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds"
Nov 04 01:14:16.792627 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:16.792594    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84f5c469-8fea-4b88-969b-d1eed184c268-istiod-ca-cert\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds\" (UID: \"84f5c469-8fea-4b88-969b-d1eed184c268\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds"
Nov 04 01:14:16.794329 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:16.794300    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84f5c469-8fea-4b88-969b-d1eed184c268-istio-envoy\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds\" (UID: \"84f5c469-8fea-4b88-969b-d1eed184c268\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds"
Nov 04 01:14:16.794451 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:16.794435    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84f5c469-8fea-4b88-969b-d1eed184c268-istio-podinfo\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds\" (UID: \"84f5c469-8fea-4b88-969b-d1eed184c268\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds"
Nov 04 01:14:16.803634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:16.803595    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84f5c469-8fea-4b88-969b-d1eed184c268-istio-token\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds\" (UID: \"84f5c469-8fea-4b88-969b-d1eed184c268\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds"
Nov 04 01:14:16.803857 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:16.803833    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8dc6\" (UniqueName: \"kubernetes.io/projected/84f5c469-8fea-4b88-969b-d1eed184c268-kube-api-access-z8dc6\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds\" (UID: \"84f5c469-8fea-4b88-969b-d1eed184c268\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds"
Nov 04 01:14:16.970122 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:16.970035    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds"
Nov 04 01:14:17.124796 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.124760    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr"]
Nov 04 01:14:17.128850 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.128833    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr"
Nov 04 01:14:17.135494 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.135463    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-1036e8-openshift-default-dockercfg-wbg6m\""
Nov 04 01:14:17.143932 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.143904    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr"]
Nov 04 01:14:17.195623 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.195578    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b21786c9-e7fa-43eb-bb36-f2633486b8b5-istio-envoy\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr\" (UID: \"b21786c9-e7fa-43eb-bb36-f2633486b8b5\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr"
Nov 04 01:14:17.195793 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.195686    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dnnrs\" (UniqueName: \"kubernetes.io/projected/b21786c9-e7fa-43eb-bb36-f2633486b8b5-kube-api-access-dnnrs\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr\" (UID: \"b21786c9-e7fa-43eb-bb36-f2633486b8b5\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr"
Nov 04 01:14:17.195793 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.195737    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b21786c9-e7fa-43eb-bb36-f2633486b8b5-credential-socket\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr\" (UID: \"b21786c9-e7fa-43eb-bb36-f2633486b8b5\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr"
Nov 04 01:14:17.195913 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.195875    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b21786c9-e7fa-43eb-bb36-f2633486b8b5-workload-certs\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr\" (UID: \"b21786c9-e7fa-43eb-bb36-f2633486b8b5\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr"
Nov 04 01:14:17.195981 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.195924    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b21786c9-e7fa-43eb-bb36-f2633486b8b5-workload-socket\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr\" (UID: \"b21786c9-e7fa-43eb-bb36-f2633486b8b5\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr"
Nov 04 01:14:17.195981 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.195962    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b21786c9-e7fa-43eb-bb36-f2633486b8b5-istio-token\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr\" (UID: \"b21786c9-e7fa-43eb-bb36-f2633486b8b5\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr"
Nov 04 01:14:17.196053 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.196030    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b21786c9-e7fa-43eb-bb36-f2633486b8b5-istio-podinfo\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr\" (UID: \"b21786c9-e7fa-43eb-bb36-f2633486b8b5\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr"
Nov 04 01:14:17.196088 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.196070    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b21786c9-e7fa-43eb-bb36-f2633486b8b5-istiod-ca-cert\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr\" (UID: \"b21786c9-e7fa-43eb-bb36-f2633486b8b5\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr"
Nov 04 01:14:17.196129 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.196101    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b21786c9-e7fa-43eb-bb36-f2633486b8b5-istio-data\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr\" (UID: \"b21786c9-e7fa-43eb-bb36-f2633486b8b5\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr"
Nov 04 01:14:17.262797 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.262768    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds"]
Nov 04 01:14:17.267936 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:14:17.267894    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84f5c469_8fea_4b88_969b_d1eed184c268.slice/crio-bfc6c1b78e3829b43e9eeb64f02f881242a0b8cfd6c3abe3a70a47acc5211bb0 WatchSource:0}: Error finding container bfc6c1b78e3829b43e9eeb64f02f881242a0b8cfd6c3abe3a70a47acc5211bb0: Status 404 returned error can't find the container with id bfc6c1b78e3829b43e9eeb64f02f881242a0b8cfd6c3abe3a70a47acc5211bb0
Nov 04 01:14:17.270783 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.270753    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:14:17.270888 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.270827    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:14:17.270950 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.270889    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:14:17.297376 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.297350    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b21786c9-e7fa-43eb-bb36-f2633486b8b5-istio-token\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr\" (UID: \"b21786c9-e7fa-43eb-bb36-f2633486b8b5\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr"
Nov 04 01:14:17.297465 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.297394    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b21786c9-e7fa-43eb-bb36-f2633486b8b5-istio-podinfo\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr\" (UID: \"b21786c9-e7fa-43eb-bb36-f2633486b8b5\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr"
Nov 04 01:14:17.297465 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.297429    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b21786c9-e7fa-43eb-bb36-f2633486b8b5-istiod-ca-cert\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr\" (UID: \"b21786c9-e7fa-43eb-bb36-f2633486b8b5\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr"
Nov 04 01:14:17.297465 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.297456    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b21786c9-e7fa-43eb-bb36-f2633486b8b5-istio-data\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr\" (UID: \"b21786c9-e7fa-43eb-bb36-f2633486b8b5\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr"
Nov 04 01:14:17.297592 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.297477    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b21786c9-e7fa-43eb-bb36-f2633486b8b5-istio-envoy\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr\" (UID: \"b21786c9-e7fa-43eb-bb36-f2633486b8b5\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr"
Nov 04 01:14:17.297592 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.297506    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnnrs\" (UniqueName: \"kubernetes.io/projected/b21786c9-e7fa-43eb-bb36-f2633486b8b5-kube-api-access-dnnrs\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr\" (UID: \"b21786c9-e7fa-43eb-bb36-f2633486b8b5\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr"
Nov 04 01:14:17.297592 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.297550    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b21786c9-e7fa-43eb-bb36-f2633486b8b5-credential-socket\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr\" (UID: \"b21786c9-e7fa-43eb-bb36-f2633486b8b5\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr"
Nov 04 01:14:17.297799 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.297596    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b21786c9-e7fa-43eb-bb36-f2633486b8b5-workload-certs\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr\" (UID: \"b21786c9-e7fa-43eb-bb36-f2633486b8b5\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr"
Nov 04 01:14:17.297799 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.297658    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b21786c9-e7fa-43eb-bb36-f2633486b8b5-workload-socket\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr\" (UID: \"b21786c9-e7fa-43eb-bb36-f2633486b8b5\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr"
Nov 04 01:14:17.297958 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.297938    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b21786c9-e7fa-43eb-bb36-f2633486b8b5-istio-data\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr\" (UID: \"b21786c9-e7fa-43eb-bb36-f2633486b8b5\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr"
Nov 04 01:14:17.298030 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.297972    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b21786c9-e7fa-43eb-bb36-f2633486b8b5-workload-socket\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr\" (UID: \"b21786c9-e7fa-43eb-bb36-f2633486b8b5\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr"
Nov 04 01:14:17.298098 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.298078    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b21786c9-e7fa-43eb-bb36-f2633486b8b5-credential-socket\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr\" (UID: \"b21786c9-e7fa-43eb-bb36-f2633486b8b5\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr"
Nov 04 01:14:17.298179 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.298162    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b21786c9-e7fa-43eb-bb36-f2633486b8b5-workload-certs\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr\" (UID: \"b21786c9-e7fa-43eb-bb36-f2633486b8b5\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr"
Nov 04 01:14:17.298226 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.298190    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b21786c9-e7fa-43eb-bb36-f2633486b8b5-istiod-ca-cert\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr\" (UID: \"b21786c9-e7fa-43eb-bb36-f2633486b8b5\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr"
Nov 04 01:14:17.299820 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.299799    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b21786c9-e7fa-43eb-bb36-f2633486b8b5-istio-envoy\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr\" (UID: \"b21786c9-e7fa-43eb-bb36-f2633486b8b5\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr"
Nov 04 01:14:17.299945 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.299928    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b21786c9-e7fa-43eb-bb36-f2633486b8b5-istio-podinfo\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr\" (UID: \"b21786c9-e7fa-43eb-bb36-f2633486b8b5\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr"
Nov 04 01:14:17.310781 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.310762    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b21786c9-e7fa-43eb-bb36-f2633486b8b5-istio-token\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr\" (UID: \"b21786c9-e7fa-43eb-bb36-f2633486b8b5\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr"
Nov 04 01:14:17.310859 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.310763    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnnrs\" (UniqueName: \"kubernetes.io/projected/b21786c9-e7fa-43eb-bb36-f2633486b8b5-kube-api-access-dnnrs\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr\" (UID: \"b21786c9-e7fa-43eb-bb36-f2633486b8b5\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr"
Nov 04 01:14:17.328879 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.328860    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h"]
Nov 04 01:14:17.334142 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.334126    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h"
Nov 04 01:14:17.338711 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.338679    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-ba844c-openshift-default-dockercfg-xd85j\""
Nov 04 01:14:17.348348 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.347156    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h"]
Nov 04 01:14:17.398682 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.398643    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-workload-certs\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h\" (UID: \"fc92d968-9cb0-46bf-bf06-7f75d63a12b2\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h"
Nov 04 01:14:17.398838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.398698    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-credential-socket\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h\" (UID: \"fc92d968-9cb0-46bf-bf06-7f75d63a12b2\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h"
Nov 04 01:14:17.398838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.398766    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-workload-socket\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h\" (UID: \"fc92d968-9cb0-46bf-bf06-7f75d63a12b2\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h"
Nov 04 01:14:17.398838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.398796    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-istio-token\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h\" (UID: \"fc92d968-9cb0-46bf-bf06-7f75d63a12b2\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h"
Nov 04 01:14:17.398838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.398817    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8r2gw\" (UniqueName: \"kubernetes.io/projected/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-kube-api-access-8r2gw\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h\" (UID: \"fc92d968-9cb0-46bf-bf06-7f75d63a12b2\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h"
Nov 04 01:14:17.399001 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.398878    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-istiod-ca-cert\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h\" (UID: \"fc92d968-9cb0-46bf-bf06-7f75d63a12b2\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h"
Nov 04 01:14:17.399001 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.398940    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-istio-envoy\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h\" (UID: \"fc92d968-9cb0-46bf-bf06-7f75d63a12b2\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h"
Nov 04 01:14:17.399001 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.398974    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-istio-podinfo\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h\" (UID: \"fc92d968-9cb0-46bf-bf06-7f75d63a12b2\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h"
Nov 04 01:14:17.399001 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.398991    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-istio-data\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h\" (UID: \"fc92d968-9cb0-46bf-bf06-7f75d63a12b2\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h"
Nov 04 01:14:17.453152 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.453118    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr"
Nov 04 01:14:17.500081 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.500050    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8r2gw\" (UniqueName: \"kubernetes.io/projected/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-kube-api-access-8r2gw\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h\" (UID: \"fc92d968-9cb0-46bf-bf06-7f75d63a12b2\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h"
Nov 04 01:14:17.500269 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.500253    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-istiod-ca-cert\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h\" (UID: \"fc92d968-9cb0-46bf-bf06-7f75d63a12b2\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h"
Nov 04 01:14:17.500352 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.500343    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-istio-envoy\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h\" (UID: \"fc92d968-9cb0-46bf-bf06-7f75d63a12b2\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h"
Nov 04 01:14:17.500436 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.500426    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-istio-podinfo\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h\" (UID: \"fc92d968-9cb0-46bf-bf06-7f75d63a12b2\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h"
Nov 04 01:14:17.500509 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.500499    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-istio-data\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h\" (UID: \"fc92d968-9cb0-46bf-bf06-7f75d63a12b2\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h"
Nov 04 01:14:17.500654 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.500643    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-workload-certs\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h\" (UID: \"fc92d968-9cb0-46bf-bf06-7f75d63a12b2\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h"
Nov 04 01:14:17.500739 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.500730    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-credential-socket\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h\" (UID: \"fc92d968-9cb0-46bf-bf06-7f75d63a12b2\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h"
Nov 04 01:14:17.500829 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.500821    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-workload-socket\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h\" (UID: \"fc92d968-9cb0-46bf-bf06-7f75d63a12b2\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h"
Nov 04 01:14:17.500901 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.500893    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-istio-token\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h\" (UID: \"fc92d968-9cb0-46bf-bf06-7f75d63a12b2\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h"
Nov 04 01:14:17.501193 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.501065    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-istio-data\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h\" (UID: \"fc92d968-9cb0-46bf-bf06-7f75d63a12b2\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h"
Nov 04 01:14:17.501193 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.501081    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-istiod-ca-cert\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h\" (UID: \"fc92d968-9cb0-46bf-bf06-7f75d63a12b2\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h"
Nov 04 01:14:17.501734 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.501325    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-credential-socket\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h\" (UID: \"fc92d968-9cb0-46bf-bf06-7f75d63a12b2\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h"
Nov 04 01:14:17.501734 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.501348    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-workload-certs\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h\" (UID: \"fc92d968-9cb0-46bf-bf06-7f75d63a12b2\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h"
Nov 04 01:14:17.501734 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.501444    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-workload-socket\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h\" (UID: \"fc92d968-9cb0-46bf-bf06-7f75d63a12b2\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h"
Nov 04 01:14:17.502872 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.502849    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-istio-envoy\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h\" (UID: \"fc92d968-9cb0-46bf-bf06-7f75d63a12b2\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h"
Nov 04 01:14:17.503042 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.503026    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-istio-podinfo\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h\" (UID: \"fc92d968-9cb0-46bf-bf06-7f75d63a12b2\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h"
Nov 04 01:14:17.519553 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.519523    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-istio-token\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h\" (UID: \"fc92d968-9cb0-46bf-bf06-7f75d63a12b2\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h"
Nov 04 01:14:17.522014 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.521973    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8r2gw\" (UniqueName: \"kubernetes.io/projected/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-kube-api-access-8r2gw\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h\" (UID: \"fc92d968-9cb0-46bf-bf06-7f75d63a12b2\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h"
Nov 04 01:14:17.652856 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.652600    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h"
Nov 04 01:14:17.761823 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.761406    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr"]
Nov 04 01:14:17.762853 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:14:17.762825    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb21786c9_e7fa_43eb_bb36_f2633486b8b5.slice/crio-b617e940f3bc13146ec95e4bb2f940dedc4d3979479aaef3d5fb4dd408ab6772 WatchSource:0}: Error finding container b617e940f3bc13146ec95e4bb2f940dedc4d3979479aaef3d5fb4dd408ab6772: Status 404 returned error can't find the container with id b617e940f3bc13146ec95e4bb2f940dedc4d3979479aaef3d5fb4dd408ab6772
Nov 04 01:14:17.766150 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.766108    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:14:17.766246 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.766202    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:14:17.766283 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.766243    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:14:17.961636 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.960668    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h"]
Nov 04 01:14:17.983399 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.983360    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:14:17.983523 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.983452    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:14:17.983523 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:17.983497    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:14:18.088241 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:18.088197    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h" event={"ID":"fc92d968-9cb0-46bf-bf06-7f75d63a12b2","Type":"ContainerStarted","Data":"a4972e35aa46350d470d8b5d7f56d7d517ea8bbce0cf5bedabe8cf70d411389a"}
Nov 04 01:14:18.090552 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:18.090091    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds" event={"ID":"84f5c469-8fea-4b88-969b-d1eed184c268","Type":"ContainerStarted","Data":"9feeca1a762b85e8f9c6a4a63effaf66094f94dbab3b637db2ce7bec889f8796"}
Nov 04 01:14:18.090552 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:18.090147    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds" event={"ID":"84f5c469-8fea-4b88-969b-d1eed184c268","Type":"ContainerStarted","Data":"bfc6c1b78e3829b43e9eeb64f02f881242a0b8cfd6c3abe3a70a47acc5211bb0"}
Nov 04 01:14:18.092465 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:18.091688    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr" event={"ID":"b21786c9-e7fa-43eb-bb36-f2633486b8b5","Type":"ContainerStarted","Data":"a8e4220acdff9eb9a1893f30eadaf820fbf95dccbf293d3b9d1ca0aaa746d4d1"}
Nov 04 01:14:18.092465 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:18.091790    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr" event={"ID":"b21786c9-e7fa-43eb-bb36-f2633486b8b5","Type":"ContainerStarted","Data":"b617e940f3bc13146ec95e4bb2f940dedc4d3979479aaef3d5fb4dd408ab6772"}
Nov 04 01:14:18.152633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:18.150463    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds" podStartSLOduration=2.150441534 podStartE2EDuration="2.150441534s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:14:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:14:18.115738776 +0000 UTC m=+698.162299567" watchObservedRunningTime="2026-11-04 01:14:18.150441534 +0000 UTC m=+698.197002317"
Nov 04 01:14:18.156629 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:18.153885    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr" podStartSLOduration=1.153865854 podStartE2EDuration="1.153865854s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:14:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:14:18.153368599 +0000 UTC m=+698.199929393" watchObservedRunningTime="2026-11-04 01:14:18.153865854 +0000 UTC m=+698.200426645"
Nov 04 01:14:18.458030 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:18.457942    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr"
Nov 04 01:14:18.469776 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:18.469738    5068 patch_prober.go:29] interesting pod/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Nov 04 01:14:18.470035 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:18.470010    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr" podUID="b21786c9-e7fa-43eb-bb36-f2633486b8b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Nov 04 01:14:18.985633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:18.984714    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds"
Nov 04 01:14:18.985633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:18.984846    5068 patch_prober.go:29] interesting pod/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Nov 04 01:14:18.985633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:18.984910    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds" podUID="84f5c469-8fea-4b88-969b-d1eed184c268" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Nov 04 01:14:19.111493 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:19.110935    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h" event={"ID":"fc92d968-9cb0-46bf-bf06-7f75d63a12b2","Type":"ContainerStarted","Data":"420c92dec053736258e6fe8120170bcf809dfbcb65a2881e6c1ab5abc537759a"}
Nov 04 01:14:19.145735 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:19.145663    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h" podStartSLOduration=2.145643147 podStartE2EDuration="2.145643147s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:14:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:14:19.137726904 +0000 UTC m=+699.184287709" watchObservedRunningTime="2026-11-04 01:14:19.145643147 +0000 UTC m=+699.192203942"
Nov 04 01:14:19.455510 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:19.455049    5068 patch_prober.go:29] interesting pod/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Nov 04 01:14:19.455510 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:19.455114    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr" podUID="b21786c9-e7fa-43eb-bb36-f2633486b8b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Nov 04 01:14:19.684772 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:19.684721    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h"
Nov 04 01:14:19.692644 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:19.691889    5068 patch_prober.go:29] interesting pod/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Nov 04 01:14:19.692644 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:19.691950    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h" podUID="fc92d968-9cb0-46bf-bf06-7f75d63a12b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Nov 04 01:14:19.974104 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:19.974053    5068 patch_prober.go:29] interesting pod/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Nov 04 01:14:19.974388 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:19.974361    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds" podUID="84f5c469-8fea-4b88-969b-d1eed184c268" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Nov 04 01:14:20.508634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:20.508472    5068 patch_prober.go:29] interesting pod/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Nov 04 01:14:20.508634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:20.508534    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr" podUID="b21786c9-e7fa-43eb-bb36-f2633486b8b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Nov 04 01:14:20.660633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:20.659976    5068 patch_prober.go:29] interesting pod/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Nov 04 01:14:20.660633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:20.660038    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h" podUID="fc92d968-9cb0-46bf-bf06-7f75d63a12b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Nov 04 01:14:20.984165 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:20.984122    5068 patch_prober.go:29] interesting pod/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Nov 04 01:14:20.984451 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:20.984418    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds" podUID="84f5c469-8fea-4b88-969b-d1eed184c268" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Nov 04 01:14:21.458634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:21.457014    5068 patch_prober.go:29] interesting pod/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Nov 04 01:14:21.458634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:21.457076    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr" podUID="b21786c9-e7fa-43eb-bb36-f2633486b8b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Nov 04 01:14:21.661663 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:21.656716    5068 patch_prober.go:29] interesting pod/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Nov 04 01:14:21.661663 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:21.656779    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h" podUID="fc92d968-9cb0-46bf-bf06-7f75d63a12b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Nov 04 01:14:21.973634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:21.973069    5068 patch_prober.go:29] interesting pod/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Nov 04 01:14:21.973634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:21.973148    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds" podUID="84f5c469-8fea-4b88-969b-d1eed184c268" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Nov 04 01:14:22.469460 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:22.469413    5068 patch_prober.go:29] interesting pod/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Nov 04 01:14:22.469788 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:22.469748    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr" podUID="b21786c9-e7fa-43eb-bb36-f2633486b8b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Nov 04 01:14:22.673434 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:22.673402    5068 patch_prober.go:29] interesting pod/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Nov 04 01:14:22.674078 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:22.674023    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h" podUID="fc92d968-9cb0-46bf-bf06-7f75d63a12b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Nov 04 01:14:22.973636 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:22.970963    5068 patch_prober.go:29] interesting pod/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Nov 04 01:14:22.973636 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:22.971029    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds" podUID="84f5c469-8fea-4b88-969b-d1eed184c268" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Nov 04 01:14:23.459636 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:23.458084    5068 patch_prober.go:29] interesting pod/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Nov 04 01:14:23.459636 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:23.458154    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr" podUID="b21786c9-e7fa-43eb-bb36-f2633486b8b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Nov 04 01:14:23.655634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:23.653958    5068 patch_prober.go:29] interesting pod/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Nov 04 01:14:23.655634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:23.654017    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h" podUID="fc92d968-9cb0-46bf-bf06-7f75d63a12b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Nov 04 01:14:23.972633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:23.971849    5068 patch_prober.go:29] interesting pod/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Nov 04 01:14:23.972633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:23.971910    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds" podUID="84f5c469-8fea-4b88-969b-d1eed184c268" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Nov 04 01:14:24.454696 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:24.454658    5068 patch_prober.go:29] interesting pod/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Nov 04 01:14:24.454984 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:24.454954    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr" podUID="b21786c9-e7fa-43eb-bb36-f2633486b8b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Nov 04 01:14:24.653560 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:24.653521    5068 patch_prober.go:29] interesting pod/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Nov 04 01:14:24.653798 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:24.653586    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h" podUID="fc92d968-9cb0-46bf-bf06-7f75d63a12b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Nov 04 01:14:24.978633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:24.978510    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds"
Nov 04 01:14:24.982633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:24.979906    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds"
Nov 04 01:14:24.982633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:24.980829    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds"
Nov 04 01:14:25.458866 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:25.458834    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr"
Nov 04 01:14:25.459226 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:25.459206    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr"
Nov 04 01:14:25.460030 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:25.460003    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr"
Nov 04 01:14:25.658375 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:25.658346    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h"
Nov 04 01:14:25.658733 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:25.658711    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h"
Nov 04 01:14:25.659399 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:25.659376    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h"
Nov 04 01:14:43.449105 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:14:43.449066    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:14:47.335942 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:14:47.335905    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:15:00.180164 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:00.180129    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895915-bgpl8"]
Nov 04 01:15:00.184795 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:00.184770    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895915-bgpl8"
Nov 04 01:15:00.193762 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:00.193390    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qdgjq\""
Nov 04 01:15:00.199334 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:00.199306    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895915-bgpl8"]
Nov 04 01:15:00.226383 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:00.226355    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4qq8\" (UniqueName: \"kubernetes.io/projected/6b6c8fa4-6394-4dd4-98e2-0441dfea042b-kube-api-access-f4qq8\") pod \"maas-api-key-cleanup-29895915-bgpl8\" (UID: \"6b6c8fa4-6394-4dd4-98e2-0441dfea042b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895915-bgpl8"
Nov 04 01:15:00.226561 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:00.226480    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b6c8fa4-6394-4dd4-98e2-0441dfea042b-maas-api-service-ca\") pod \"maas-api-key-cleanup-29895915-bgpl8\" (UID: \"6b6c8fa4-6394-4dd4-98e2-0441dfea042b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895915-bgpl8"
Nov 04 01:15:00.327983 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:00.327942    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4qq8\" (UniqueName: \"kubernetes.io/projected/6b6c8fa4-6394-4dd4-98e2-0441dfea042b-kube-api-access-f4qq8\") pod \"maas-api-key-cleanup-29895915-bgpl8\" (UID: \"6b6c8fa4-6394-4dd4-98e2-0441dfea042b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895915-bgpl8"
Nov 04 01:15:00.328175 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:00.328058    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b6c8fa4-6394-4dd4-98e2-0441dfea042b-maas-api-service-ca\") pod \"maas-api-key-cleanup-29895915-bgpl8\" (UID: \"6b6c8fa4-6394-4dd4-98e2-0441dfea042b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895915-bgpl8"
Nov 04 01:15:00.328969 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:00.328937    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b6c8fa4-6394-4dd4-98e2-0441dfea042b-maas-api-service-ca\") pod \"maas-api-key-cleanup-29895915-bgpl8\" (UID: \"6b6c8fa4-6394-4dd4-98e2-0441dfea042b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895915-bgpl8"
Nov 04 01:15:00.340834 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:00.340792    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4qq8\" (UniqueName: \"kubernetes.io/projected/6b6c8fa4-6394-4dd4-98e2-0441dfea042b-kube-api-access-f4qq8\") pod \"maas-api-key-cleanup-29895915-bgpl8\" (UID: \"6b6c8fa4-6394-4dd4-98e2-0441dfea042b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895915-bgpl8"
Nov 04 01:15:00.503388 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:00.503299    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895915-bgpl8"
Nov 04 01:15:00.805636 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:00.801959    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895915-bgpl8"]
Nov 04 01:15:01.617004 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:01.616962    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895915-bgpl8" event={"ID":"6b6c8fa4-6394-4dd4-98e2-0441dfea042b","Type":"ContainerStarted","Data":"17127a66d13c5d7227481d445c9d7e2c0a3603d7123d3e5effeb1ffc51e4fcbb"}
Nov 04 01:15:01.873191 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:01.873102    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p"]
Nov 04 01:15:01.879345 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:01.879316    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p"
Nov 04 01:15:01.885136 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:01.885116    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-9902c3-openshift-default-dockercfg-5blnq\""
Nov 04 01:15:01.901131 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:01.901105    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p"]
Nov 04 01:15:01.963893 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:01.963859    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa8debd2-b803-412c-9e4c-53917a748a59-istio-podinfo\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p\" (UID: \"fa8debd2-b803-412c-9e4c-53917a748a59\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p"
Nov 04 01:15:01.963893 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:01.963899    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa8debd2-b803-412c-9e4c-53917a748a59-istio-token\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p\" (UID: \"fa8debd2-b803-412c-9e4c-53917a748a59\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p"
Nov 04 01:15:01.964152 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:01.963936    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa8debd2-b803-412c-9e4c-53917a748a59-workload-socket\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p\" (UID: \"fa8debd2-b803-412c-9e4c-53917a748a59\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p"
Nov 04 01:15:01.964152 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:01.964050    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa8debd2-b803-412c-9e4c-53917a748a59-istiod-ca-cert\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p\" (UID: \"fa8debd2-b803-412c-9e4c-53917a748a59\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p"
Nov 04 01:15:01.964152 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:01.964083    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa8debd2-b803-412c-9e4c-53917a748a59-credential-socket\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p\" (UID: \"fa8debd2-b803-412c-9e4c-53917a748a59\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p"
Nov 04 01:15:01.964152 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:01.964143    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t45c2\" (UniqueName: \"kubernetes.io/projected/fa8debd2-b803-412c-9e4c-53917a748a59-kube-api-access-t45c2\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p\" (UID: \"fa8debd2-b803-412c-9e4c-53917a748a59\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p"
Nov 04 01:15:01.964386 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:01.964163    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa8debd2-b803-412c-9e4c-53917a748a59-workload-certs\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p\" (UID: \"fa8debd2-b803-412c-9e4c-53917a748a59\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p"
Nov 04 01:15:01.964386 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:01.964206    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa8debd2-b803-412c-9e4c-53917a748a59-istio-data\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p\" (UID: \"fa8debd2-b803-412c-9e4c-53917a748a59\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p"
Nov 04 01:15:01.964386 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:01.964258    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa8debd2-b803-412c-9e4c-53917a748a59-istio-envoy\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p\" (UID: \"fa8debd2-b803-412c-9e4c-53917a748a59\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p"
Nov 04 01:15:02.064983 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.064944    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa8debd2-b803-412c-9e4c-53917a748a59-credential-socket\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p\" (UID: \"fa8debd2-b803-412c-9e4c-53917a748a59\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p"
Nov 04 01:15:02.065285 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.065256    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t45c2\" (UniqueName: \"kubernetes.io/projected/fa8debd2-b803-412c-9e4c-53917a748a59-kube-api-access-t45c2\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p\" (UID: \"fa8debd2-b803-412c-9e4c-53917a748a59\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p"
Nov 04 01:15:02.065434 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.065415    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa8debd2-b803-412c-9e4c-53917a748a59-workload-certs\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p\" (UID: \"fa8debd2-b803-412c-9e4c-53917a748a59\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p"
Nov 04 01:15:02.065556 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.065540    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa8debd2-b803-412c-9e4c-53917a748a59-istio-data\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p\" (UID: \"fa8debd2-b803-412c-9e4c-53917a748a59\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p"
Nov 04 01:15:02.065760 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.065741    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa8debd2-b803-412c-9e4c-53917a748a59-istio-envoy\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p\" (UID: \"fa8debd2-b803-412c-9e4c-53917a748a59\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p"
Nov 04 01:15:02.066044 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.065934    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa8debd2-b803-412c-9e4c-53917a748a59-istio-podinfo\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p\" (UID: \"fa8debd2-b803-412c-9e4c-53917a748a59\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p"
Nov 04 01:15:02.066211 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.066191    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa8debd2-b803-412c-9e4c-53917a748a59-istio-token\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p\" (UID: \"fa8debd2-b803-412c-9e4c-53917a748a59\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p"
Nov 04 01:15:02.066382 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.066357    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa8debd2-b803-412c-9e4c-53917a748a59-workload-socket\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p\" (UID: \"fa8debd2-b803-412c-9e4c-53917a748a59\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p"
Nov 04 01:15:02.066536 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.066518    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa8debd2-b803-412c-9e4c-53917a748a59-istiod-ca-cert\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p\" (UID: \"fa8debd2-b803-412c-9e4c-53917a748a59\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p"
Nov 04 01:15:02.067504 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.067481    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa8debd2-b803-412c-9e4c-53917a748a59-istiod-ca-cert\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p\" (UID: \"fa8debd2-b803-412c-9e4c-53917a748a59\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p"
Nov 04 01:15:02.068021 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.067985    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa8debd2-b803-412c-9e4c-53917a748a59-credential-socket\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p\" (UID: \"fa8debd2-b803-412c-9e4c-53917a748a59\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p"
Nov 04 01:15:02.070366 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.069183    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa8debd2-b803-412c-9e4c-53917a748a59-workload-socket\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p\" (UID: \"fa8debd2-b803-412c-9e4c-53917a748a59\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p"
Nov 04 01:15:02.070508 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.069416    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa8debd2-b803-412c-9e4c-53917a748a59-workload-certs\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p\" (UID: \"fa8debd2-b803-412c-9e4c-53917a748a59\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p"
Nov 04 01:15:02.070645 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.069995    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa8debd2-b803-412c-9e4c-53917a748a59-istio-data\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p\" (UID: \"fa8debd2-b803-412c-9e4c-53917a748a59\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p"
Nov 04 01:15:02.085872 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.083381    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa8debd2-b803-412c-9e4c-53917a748a59-istio-podinfo\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p\" (UID: \"fa8debd2-b803-412c-9e4c-53917a748a59\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p"
Nov 04 01:15:02.101972 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.094136    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa8debd2-b803-412c-9e4c-53917a748a59-istio-token\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p\" (UID: \"fa8debd2-b803-412c-9e4c-53917a748a59\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p"
Nov 04 01:15:02.103820 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.103768    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa8debd2-b803-412c-9e4c-53917a748a59-istio-envoy\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p\" (UID: \"fa8debd2-b803-412c-9e4c-53917a748a59\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p"
Nov 04 01:15:02.117806 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.113563    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t45c2\" (UniqueName: \"kubernetes.io/projected/fa8debd2-b803-412c-9e4c-53917a748a59-kube-api-access-t45c2\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p\" (UID: \"fa8debd2-b803-412c-9e4c-53917a748a59\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p"
Nov 04 01:15:02.123696 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.123587    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8"]
Nov 04 01:15:02.127803 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.127784    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8"
Nov 04 01:15:02.135655 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.135633    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-cffa80-openshift-default-dockercfg-s6kbq\""
Nov 04 01:15:02.139513 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.139493    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8"]
Nov 04 01:15:02.215957 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.215919    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p"
Nov 04 01:15:02.302143 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.302093    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n"]
Nov 04 01:15:02.317634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.312046    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43eaae59-c37c-4553-a856-fc8aa9506c00-istio-data\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8\" (UID: \"43eaae59-c37c-4553-a856-fc8aa9506c00\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8"
Nov 04 01:15:02.317634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.312091    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43eaae59-c37c-4553-a856-fc8aa9506c00-istiod-ca-cert\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8\" (UID: \"43eaae59-c37c-4553-a856-fc8aa9506c00\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8"
Nov 04 01:15:02.317634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.312144    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43eaae59-c37c-4553-a856-fc8aa9506c00-istio-podinfo\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8\" (UID: \"43eaae59-c37c-4553-a856-fc8aa9506c00\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8"
Nov 04 01:15:02.317886 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.317786    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43eaae59-c37c-4553-a856-fc8aa9506c00-credential-socket\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8\" (UID: \"43eaae59-c37c-4553-a856-fc8aa9506c00\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8"
Nov 04 01:15:02.317886 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.317852    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43eaae59-c37c-4553-a856-fc8aa9506c00-workload-socket\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8\" (UID: \"43eaae59-c37c-4553-a856-fc8aa9506c00\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8"
Nov 04 01:15:02.317958 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.317905    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43eaae59-c37c-4553-a856-fc8aa9506c00-workload-certs\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8\" (UID: \"43eaae59-c37c-4553-a856-fc8aa9506c00\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8"
Nov 04 01:15:02.317993 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.317960    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jnvd\" (UniqueName: \"kubernetes.io/projected/43eaae59-c37c-4553-a856-fc8aa9506c00-kube-api-access-8jnvd\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8\" (UID: \"43eaae59-c37c-4553-a856-fc8aa9506c00\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8"
Nov 04 01:15:02.318027 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.318016    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43eaae59-c37c-4553-a856-fc8aa9506c00-istio-envoy\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8\" (UID: \"43eaae59-c37c-4553-a856-fc8aa9506c00\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8"
Nov 04 01:15:02.318058 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.318044    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43eaae59-c37c-4553-a856-fc8aa9506c00-istio-token\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8\" (UID: \"43eaae59-c37c-4553-a856-fc8aa9506c00\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8"
Nov 04 01:15:02.318249 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.318224    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n"
Nov 04 01:15:02.323041 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.323017    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-875528-openshift-default-dockercfg-9xdkz\""
Nov 04 01:15:02.328504 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.328275    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n"]
Nov 04 01:15:02.426691 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.426246    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43eaae59-c37c-4553-a856-fc8aa9506c00-credential-socket\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8\" (UID: \"43eaae59-c37c-4553-a856-fc8aa9506c00\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8"
Nov 04 01:15:02.426691 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.426298    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-workload-certs\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n\" (UID: \"d8725463-9c1f-4cee-b5ec-28fb3ca9f939\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n"
Nov 04 01:15:02.426691 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.426323    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-istio-token\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n\" (UID: \"d8725463-9c1f-4cee-b5ec-28fb3ca9f939\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n"
Nov 04 01:15:02.426691 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.426363    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9ssf\" (UniqueName: \"kubernetes.io/projected/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-kube-api-access-g9ssf\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n\" (UID: \"d8725463-9c1f-4cee-b5ec-28fb3ca9f939\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n"
Nov 04 01:15:02.426691 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.426395    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43eaae59-c37c-4553-a856-fc8aa9506c00-workload-socket\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8\" (UID: \"43eaae59-c37c-4553-a856-fc8aa9506c00\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8"
Nov 04 01:15:02.426691 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.426454    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43eaae59-c37c-4553-a856-fc8aa9506c00-workload-certs\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8\" (UID: \"43eaae59-c37c-4553-a856-fc8aa9506c00\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8"
Nov 04 01:15:02.426691 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.426497    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-istio-podinfo\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n\" (UID: \"d8725463-9c1f-4cee-b5ec-28fb3ca9f939\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n"
Nov 04 01:15:02.426691 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.426535    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jnvd\" (UniqueName: \"kubernetes.io/projected/43eaae59-c37c-4553-a856-fc8aa9506c00-kube-api-access-8jnvd\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8\" (UID: \"43eaae59-c37c-4553-a856-fc8aa9506c00\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8"
Nov 04 01:15:02.426691 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.426590    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-istio-data\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n\" (UID: \"d8725463-9c1f-4cee-b5ec-28fb3ca9f939\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n"
Nov 04 01:15:02.428280 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.427268    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43eaae59-c37c-4553-a856-fc8aa9506c00-istio-envoy\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8\" (UID: \"43eaae59-c37c-4553-a856-fc8aa9506c00\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8"
Nov 04 01:15:02.428280 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.427734    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43eaae59-c37c-4553-a856-fc8aa9506c00-credential-socket\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8\" (UID: \"43eaae59-c37c-4553-a856-fc8aa9506c00\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8"
Nov 04 01:15:02.428280 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.427996    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43eaae59-c37c-4553-a856-fc8aa9506c00-workload-socket\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8\" (UID: \"43eaae59-c37c-4553-a856-fc8aa9506c00\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8"
Nov 04 01:15:02.428280 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.428241    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43eaae59-c37c-4553-a856-fc8aa9506c00-workload-certs\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8\" (UID: \"43eaae59-c37c-4553-a856-fc8aa9506c00\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8"
Nov 04 01:15:02.428557 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.428279    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43eaae59-c37c-4553-a856-fc8aa9506c00-istio-token\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8\" (UID: \"43eaae59-c37c-4553-a856-fc8aa9506c00\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8"
Nov 04 01:15:02.428557 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.428538    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43eaae59-c37c-4553-a856-fc8aa9506c00-istio-data\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8\" (UID: \"43eaae59-c37c-4553-a856-fc8aa9506c00\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8"
Nov 04 01:15:02.428734 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.428568    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43eaae59-c37c-4553-a856-fc8aa9506c00-istiod-ca-cert\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8\" (UID: \"43eaae59-c37c-4553-a856-fc8aa9506c00\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8"
Nov 04 01:15:02.428734 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.428603    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-istiod-ca-cert\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n\" (UID: \"d8725463-9c1f-4cee-b5ec-28fb3ca9f939\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n"
Nov 04 01:15:02.428734 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.428654    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-istio-envoy\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n\" (UID: \"d8725463-9c1f-4cee-b5ec-28fb3ca9f939\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n"
Nov 04 01:15:02.428734 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.428712    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43eaae59-c37c-4553-a856-fc8aa9506c00-istio-podinfo\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8\" (UID: \"43eaae59-c37c-4553-a856-fc8aa9506c00\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8"
Nov 04 01:15:02.428938 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.428778    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-workload-socket\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n\" (UID: \"d8725463-9c1f-4cee-b5ec-28fb3ca9f939\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n"
Nov 04 01:15:02.428938 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.428806    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-credential-socket\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n\" (UID: \"d8725463-9c1f-4cee-b5ec-28fb3ca9f939\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n"
Nov 04 01:15:02.429867 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.429372    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43eaae59-c37c-4553-a856-fc8aa9506c00-istiod-ca-cert\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8\" (UID: \"43eaae59-c37c-4553-a856-fc8aa9506c00\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8"
Nov 04 01:15:02.430665 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.430140    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43eaae59-c37c-4553-a856-fc8aa9506c00-istio-data\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8\" (UID: \"43eaae59-c37c-4553-a856-fc8aa9506c00\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8"
Nov 04 01:15:02.433779 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.433027    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43eaae59-c37c-4553-a856-fc8aa9506c00-istio-envoy\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8\" (UID: \"43eaae59-c37c-4553-a856-fc8aa9506c00\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8"
Nov 04 01:15:02.435112 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.435001    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43eaae59-c37c-4553-a856-fc8aa9506c00-istio-podinfo\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8\" (UID: \"43eaae59-c37c-4553-a856-fc8aa9506c00\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8"
Nov 04 01:15:02.447691 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.443846    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43eaae59-c37c-4553-a856-fc8aa9506c00-istio-token\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8\" (UID: \"43eaae59-c37c-4553-a856-fc8aa9506c00\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8"
Nov 04 01:15:02.506633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.506359    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jnvd\" (UniqueName: \"kubernetes.io/projected/43eaae59-c37c-4553-a856-fc8aa9506c00-kube-api-access-8jnvd\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8\" (UID: \"43eaae59-c37c-4553-a856-fc8aa9506c00\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8"
Nov 04 01:15:02.516190 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.516155    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8"
Nov 04 01:15:02.529404 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.529372    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-istio-data\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n\" (UID: \"d8725463-9c1f-4cee-b5ec-28fb3ca9f939\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n"
Nov 04 01:15:02.529489 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.529450    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-istiod-ca-cert\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n\" (UID: \"d8725463-9c1f-4cee-b5ec-28fb3ca9f939\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n"
Nov 04 01:15:02.529489 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.529481    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-istio-envoy\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n\" (UID: \"d8725463-9c1f-4cee-b5ec-28fb3ca9f939\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n"
Nov 04 01:15:02.529564 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.529541    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-workload-socket\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n\" (UID: \"d8725463-9c1f-4cee-b5ec-28fb3ca9f939\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n"
Nov 04 01:15:02.529564 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.529556    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-credential-socket\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n\" (UID: \"d8725463-9c1f-4cee-b5ec-28fb3ca9f939\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n"
Nov 04 01:15:02.529652 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.529585    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-workload-certs\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n\" (UID: \"d8725463-9c1f-4cee-b5ec-28fb3ca9f939\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n"
Nov 04 01:15:02.529652 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.529603    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-istio-token\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n\" (UID: \"d8725463-9c1f-4cee-b5ec-28fb3ca9f939\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n"
Nov 04 01:15:02.529713 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.529653    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9ssf\" (UniqueName: \"kubernetes.io/projected/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-kube-api-access-g9ssf\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n\" (UID: \"d8725463-9c1f-4cee-b5ec-28fb3ca9f939\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n"
Nov 04 01:15:02.529756 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.529725    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-istio-podinfo\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n\" (UID: \"d8725463-9c1f-4cee-b5ec-28fb3ca9f939\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n"
Nov 04 01:15:02.529925 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.529907    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-workload-socket\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n\" (UID: \"d8725463-9c1f-4cee-b5ec-28fb3ca9f939\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n"
Nov 04 01:15:02.530093 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.530077    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-istio-data\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n\" (UID: \"d8725463-9c1f-4cee-b5ec-28fb3ca9f939\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n"
Nov 04 01:15:02.530496 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.530480    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-istiod-ca-cert\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n\" (UID: \"d8725463-9c1f-4cee-b5ec-28fb3ca9f939\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n"
Nov 04 01:15:02.530953 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.530929    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-workload-certs\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n\" (UID: \"d8725463-9c1f-4cee-b5ec-28fb3ca9f939\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n"
Nov 04 01:15:02.531164 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.531134    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-credential-socket\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n\" (UID: \"d8725463-9c1f-4cee-b5ec-28fb3ca9f939\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n"
Nov 04 01:15:02.532944 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.532855    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-istio-envoy\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n\" (UID: \"d8725463-9c1f-4cee-b5ec-28fb3ca9f939\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n"
Nov 04 01:15:02.534205 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.534184    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-istio-podinfo\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n\" (UID: \"d8725463-9c1f-4cee-b5ec-28fb3ca9f939\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n"
Nov 04 01:15:02.584633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.575779    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-istio-token\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n\" (UID: \"d8725463-9c1f-4cee-b5ec-28fb3ca9f939\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n"
Nov 04 01:15:02.584633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.579071    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9ssf\" (UniqueName: \"kubernetes.io/projected/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-kube-api-access-g9ssf\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n\" (UID: \"d8725463-9c1f-4cee-b5ec-28fb3ca9f939\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n"
Nov 04 01:15:02.635928 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.635898    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n"
Nov 04 01:15:02.737203 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.734350    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p"]
Nov 04 01:15:02.792001 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.776503    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:15:02.792001 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.776577    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:15:02.792001 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.776637    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:15:02.932828 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.932089    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8"]
Nov 04 01:15:02.933153 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:15:02.933084    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43eaae59_c37c_4553_a856_fc8aa9506c00.slice/crio-34666a567819f0bc165e1e61c58a88aee038ecc9d26c3d65e198a9f10196b6b4 WatchSource:0}: Error finding container 34666a567819f0bc165e1e61c58a88aee038ecc9d26c3d65e198a9f10196b6b4: Status 404 returned error can't find the container with id 34666a567819f0bc165e1e61c58a88aee038ecc9d26c3d65e198a9f10196b6b4
Nov 04 01:15:02.956827 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.938394    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:15:02.956827 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.938488    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:15:02.956827 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:02.938526    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:15:03.195645 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:03.191741    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n"]
Nov 04 01:15:03.209435 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:03.203853    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:15:03.209435 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:03.203931    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:15:03.209435 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:03.203975    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:15:03.650652 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:03.647806    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8" event={"ID":"43eaae59-c37c-4553-a856-fc8aa9506c00","Type":"ContainerStarted","Data":"845dd23dc57bc3672fcca20302134c048016c9ff6715f965bbdf3998c69c9764"}
Nov 04 01:15:03.650652 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:03.649207    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8" event={"ID":"43eaae59-c37c-4553-a856-fc8aa9506c00","Type":"ContainerStarted","Data":"34666a567819f0bc165e1e61c58a88aee038ecc9d26c3d65e198a9f10196b6b4"}
Nov 04 01:15:03.679099 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:03.679063    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p" event={"ID":"fa8debd2-b803-412c-9e4c-53917a748a59","Type":"ContainerStarted","Data":"17f6a25c0747c6f62e4c96a5682b89b9994e762152c2f2bad4e9f77a4b92b634"}
Nov 04 01:15:03.679365 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:03.679342    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p" event={"ID":"fa8debd2-b803-412c-9e4c-53917a748a59","Type":"ContainerStarted","Data":"9733fc232e7cf4befebbb534865e91f1934772ee9b9e6794c7affb8bec6213a7"}
Nov 04 01:15:03.688840 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:03.688807    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n" event={"ID":"d8725463-9c1f-4cee-b5ec-28fb3ca9f939","Type":"ContainerStarted","Data":"e23b484dd69e1e7c555dffaaf47a76ee4845e1a6536e714f6a576db96dc795c5"}
Nov 04 01:15:03.689589 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:03.689561    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n" event={"ID":"d8725463-9c1f-4cee-b5ec-28fb3ca9f939","Type":"ContainerStarted","Data":"c322f62513767f5c00d63808a84f068ed77464d630670da916bceddbc49bb003"}
Nov 04 01:15:03.699521 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:03.699473    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8" podStartSLOduration=1.6994555770000002 podStartE2EDuration="1.699455577s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:15:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:15:03.696896899 +0000 UTC m=+743.743457691" watchObservedRunningTime="2026-11-04 01:15:03.699455577 +0000 UTC m=+743.746016367"
Nov 04 01:15:03.817643 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:03.816350    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p" podStartSLOduration=2.816314378 podStartE2EDuration="2.816314378s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:15:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:15:03.760999779 +0000 UTC m=+743.807560570" watchObservedRunningTime="2026-11-04 01:15:03.816314378 +0000 UTC m=+743.862875170"
Nov 04 01:15:03.822630 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:03.820544    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n" podStartSLOduration=1.820527096 podStartE2EDuration="1.820527096s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:15:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:15:03.794466361 +0000 UTC m=+743.841027172" watchObservedRunningTime="2026-11-04 01:15:03.820527096 +0000 UTC m=+743.867087890"
Nov 04 01:15:04.217954 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:04.217921    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p"
Nov 04 01:15:04.219324 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:04.219297    5068 patch_prober.go:29] interesting pod/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Nov 04 01:15:04.219517 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:04.219492    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p" podUID="fa8debd2-b803-412c-9e4c-53917a748a59" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Nov 04 01:15:04.518977 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:04.518877    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8"
Nov 04 01:15:04.519692 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:04.519315    5068 patch_prober.go:29] interesting pod/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Nov 04 01:15:04.519692 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:04.519367    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8" podUID="43eaae59-c37c-4553-a856-fc8aa9506c00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Nov 04 01:15:04.636822 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:04.636769    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n"
Nov 04 01:15:04.638261 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:04.638230    5068 patch_prober.go:29] interesting pod/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Nov 04 01:15:04.638409 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:04.638274    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n" podUID="d8725463-9c1f-4cee-b5ec-28fb3ca9f939" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Nov 04 01:15:05.237644 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:05.224108    5068 patch_prober.go:29] interesting pod/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Nov 04 01:15:05.237644 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:05.224183    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p" podUID="fa8debd2-b803-412c-9e4c-53917a748a59" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Nov 04 01:15:05.517740 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:05.517709    5068 patch_prober.go:29] interesting pod/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Nov 04 01:15:05.517999 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:05.517970    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8" podUID="43eaae59-c37c-4553-a856-fc8aa9506c00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Nov 04 01:15:05.641168 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:05.640982    5068 patch_prober.go:29] interesting pod/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Nov 04 01:15:05.641168 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:05.641039    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n" podUID="d8725463-9c1f-4cee-b5ec-28fb3ca9f939" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Nov 04 01:15:06.223638 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:06.219537    5068 patch_prober.go:29] interesting pod/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Nov 04 01:15:06.223638 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:06.221846    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p" podUID="fa8debd2-b803-412c-9e4c-53917a748a59" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Nov 04 01:15:06.463634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:06.448404    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq"]
Nov 04 01:15:06.573466 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:06.573426    5068 patch_prober.go:29] interesting pod/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Nov 04 01:15:06.573681 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:06.573506    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8" podUID="43eaae59-c37c-4553-a856-fc8aa9506c00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Nov 04 01:15:06.662634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:06.647506    5068 patch_prober.go:29] interesting pod/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Nov 04 01:15:06.662634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:06.647560    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n" podUID="d8725463-9c1f-4cee-b5ec-28fb3ca9f939" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Nov 04 01:15:06.668370 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:06.668333    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq"]
Nov 04 01:15:06.668532 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:06.668458    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq"
Nov 04 01:15:06.681983 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:06.676271    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-9902c3\"/\"distinct-w6-9902c3-kserve-self-signed-certs\""
Nov 04 01:15:06.845711 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:06.826197    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b87aafac-9ab8-40e5-8d08-72eafc4187dd-home\") pod \"distinct-w6-9902c3-kserve-5c9894f7df-zg5mq\" (UID: \"b87aafac-9ab8-40e5-8d08-72eafc4187dd\") " pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq"
Nov 04 01:15:06.845982 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:06.845950    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpsrj\" (UniqueName: \"kubernetes.io/projected/b87aafac-9ab8-40e5-8d08-72eafc4187dd-kube-api-access-mpsrj\") pod \"distinct-w6-9902c3-kserve-5c9894f7df-zg5mq\" (UID: \"b87aafac-9ab8-40e5-8d08-72eafc4187dd\") " pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq"
Nov 04 01:15:06.846216 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:06.846198    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b87aafac-9ab8-40e5-8d08-72eafc4187dd-tmp-dir\") pod \"distinct-w6-9902c3-kserve-5c9894f7df-zg5mq\" (UID: \"b87aafac-9ab8-40e5-8d08-72eafc4187dd\") " pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq"
Nov 04 01:15:06.846369 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:06.846355    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b87aafac-9ab8-40e5-8d08-72eafc4187dd-model-cache\") pod \"distinct-w6-9902c3-kserve-5c9894f7df-zg5mq\" (UID: \"b87aafac-9ab8-40e5-8d08-72eafc4187dd\") " pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq"
Nov 04 01:15:06.846477 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:06.846462    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b87aafac-9ab8-40e5-8d08-72eafc4187dd-tls-certs\") pod \"distinct-w6-9902c3-kserve-5c9894f7df-zg5mq\" (UID: \"b87aafac-9ab8-40e5-8d08-72eafc4187dd\") " pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq"
Nov 04 01:15:06.846744 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:06.846721    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b87aafac-9ab8-40e5-8d08-72eafc4187dd-dshm\") pod \"distinct-w6-9902c3-kserve-5c9894f7df-zg5mq\" (UID: \"b87aafac-9ab8-40e5-8d08-72eafc4187dd\") " pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq"
Nov 04 01:15:06.955889 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:06.955050    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b87aafac-9ab8-40e5-8d08-72eafc4187dd-dshm\") pod \"distinct-w6-9902c3-kserve-5c9894f7df-zg5mq\" (UID: \"b87aafac-9ab8-40e5-8d08-72eafc4187dd\") " pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq"
Nov 04 01:15:06.955889 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:06.955123    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b87aafac-9ab8-40e5-8d08-72eafc4187dd-home\") pod \"distinct-w6-9902c3-kserve-5c9894f7df-zg5mq\" (UID: \"b87aafac-9ab8-40e5-8d08-72eafc4187dd\") " pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq"
Nov 04 01:15:06.955889 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:06.955155    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpsrj\" (UniqueName: \"kubernetes.io/projected/b87aafac-9ab8-40e5-8d08-72eafc4187dd-kube-api-access-mpsrj\") pod \"distinct-w6-9902c3-kserve-5c9894f7df-zg5mq\" (UID: \"b87aafac-9ab8-40e5-8d08-72eafc4187dd\") " pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq"
Nov 04 01:15:06.955889 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:06.955244    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b87aafac-9ab8-40e5-8d08-72eafc4187dd-tmp-dir\") pod \"distinct-w6-9902c3-kserve-5c9894f7df-zg5mq\" (UID: \"b87aafac-9ab8-40e5-8d08-72eafc4187dd\") " pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq"
Nov 04 01:15:06.955889 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:06.955300    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b87aafac-9ab8-40e5-8d08-72eafc4187dd-model-cache\") pod \"distinct-w6-9902c3-kserve-5c9894f7df-zg5mq\" (UID: \"b87aafac-9ab8-40e5-8d08-72eafc4187dd\") " pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq"
Nov 04 01:15:06.955889 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:06.955326    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b87aafac-9ab8-40e5-8d08-72eafc4187dd-tls-certs\") pod \"distinct-w6-9902c3-kserve-5c9894f7df-zg5mq\" (UID: \"b87aafac-9ab8-40e5-8d08-72eafc4187dd\") " pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq"
Nov 04 01:15:06.974638 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:06.973038    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b87aafac-9ab8-40e5-8d08-72eafc4187dd-tmp-dir\") pod \"distinct-w6-9902c3-kserve-5c9894f7df-zg5mq\" (UID: \"b87aafac-9ab8-40e5-8d08-72eafc4187dd\") " pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq"
Nov 04 01:15:06.974638 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:06.973302    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b87aafac-9ab8-40e5-8d08-72eafc4187dd-home\") pod \"distinct-w6-9902c3-kserve-5c9894f7df-zg5mq\" (UID: \"b87aafac-9ab8-40e5-8d08-72eafc4187dd\") " pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq"
Nov 04 01:15:06.974638 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:06.973535    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b87aafac-9ab8-40e5-8d08-72eafc4187dd-model-cache\") pod \"distinct-w6-9902c3-kserve-5c9894f7df-zg5mq\" (UID: \"b87aafac-9ab8-40e5-8d08-72eafc4187dd\") " pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq"
Nov 04 01:15:06.995488 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:06.995250    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b87aafac-9ab8-40e5-8d08-72eafc4187dd-tls-certs\") pod \"distinct-w6-9902c3-kserve-5c9894f7df-zg5mq\" (UID: \"b87aafac-9ab8-40e5-8d08-72eafc4187dd\") " pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq"
Nov 04 01:15:07.017646 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:07.005559    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpsrj\" (UniqueName: \"kubernetes.io/projected/b87aafac-9ab8-40e5-8d08-72eafc4187dd-kube-api-access-mpsrj\") pod \"distinct-w6-9902c3-kserve-5c9894f7df-zg5mq\" (UID: \"b87aafac-9ab8-40e5-8d08-72eafc4187dd\") " pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq"
Nov 04 01:15:07.034657 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:07.025909    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b87aafac-9ab8-40e5-8d08-72eafc4187dd-dshm\") pod \"distinct-w6-9902c3-kserve-5c9894f7df-zg5mq\" (UID: \"b87aafac-9ab8-40e5-8d08-72eafc4187dd\") " pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq"
Nov 04 01:15:07.081714 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:07.065814    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq"
Nov 04 01:15:07.235647 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:07.232372    5068 patch_prober.go:29] interesting pod/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Nov 04 01:15:07.235647 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:07.232431    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p" podUID="fa8debd2-b803-412c-9e4c-53917a748a59" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Nov 04 01:15:07.517454 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:07.517408    5068 patch_prober.go:29] interesting pod/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Nov 04 01:15:07.519083 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:07.518067    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8" podUID="43eaae59-c37c-4553-a856-fc8aa9506c00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Nov 04 01:15:07.647635 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:07.641930    5068 patch_prober.go:29] interesting pod/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Nov 04 01:15:07.647635 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:07.641991    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n" podUID="d8725463-9c1f-4cee-b5ec-28fb3ca9f939" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Nov 04 01:15:07.818721 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:07.802938    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq"]
Nov 04 01:15:07.830769 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:15:07.830732    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb87aafac_9ab8_40e5_8d08_72eafc4187dd.slice/crio-eb600abab916071dee0555f16c69cdfe4b7b87825df79f8cd412851f33cbc374 WatchSource:0}: Error finding container eb600abab916071dee0555f16c69cdfe4b7b87825df79f8cd412851f33cbc374: Status 404 returned error can't find the container with id eb600abab916071dee0555f16c69cdfe4b7b87825df79f8cd412851f33cbc374
Nov 04 01:15:08.225691 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:08.223041    5068 patch_prober.go:29] interesting pod/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Nov 04 01:15:08.225691 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:08.223101    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p" podUID="fa8debd2-b803-412c-9e4c-53917a748a59" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Nov 04 01:15:08.538643 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:08.533937    5068 patch_prober.go:29] interesting pod/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Nov 04 01:15:08.538643 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:08.533995    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8" podUID="43eaae59-c37c-4553-a856-fc8aa9506c00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Nov 04 01:15:08.641671 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:08.640650    5068 patch_prober.go:29] interesting pod/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Nov 04 01:15:08.641671 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:08.640711    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n" podUID="d8725463-9c1f-4cee-b5ec-28fb3ca9f939" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Nov 04 01:15:08.981541 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:08.979093    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq" event={"ID":"b87aafac-9ab8-40e5-8d08-72eafc4187dd","Type":"ContainerStarted","Data":"9c26cad512c9c16f8fe19b6bd028eeedd6857d76eafda5e5011abd2ccabf090d"}
Nov 04 01:15:08.981541 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:08.979165    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq" event={"ID":"b87aafac-9ab8-40e5-8d08-72eafc4187dd","Type":"ContainerStarted","Data":"eb600abab916071dee0555f16c69cdfe4b7b87825df79f8cd412851f33cbc374"}
Nov 04 01:15:09.025284 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:09.025224    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq" podStartSLOduration=3.025204982 podStartE2EDuration="3.025204982s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:15:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:15:09.019707311 +0000 UTC m=+749.066268091" watchObservedRunningTime="2026-11-04 01:15:09.025204982 +0000 UTC m=+749.071765774"
Nov 04 01:15:09.067768 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:09.067708    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq"
Nov 04 01:15:09.223138 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:09.223108    5068 patch_prober.go:29] interesting pod/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Nov 04 01:15:09.223395 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:09.223370    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p" podUID="fa8debd2-b803-412c-9e4c-53917a748a59" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Nov 04 01:15:09.532636 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:09.529182    5068 patch_prober.go:29] interesting pod/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Nov 04 01:15:09.532636 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:09.529258    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8" podUID="43eaae59-c37c-4553-a856-fc8aa9506c00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Nov 04 01:15:09.636511 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:09.636476    5068 patch_prober.go:29] interesting pod/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Nov 04 01:15:09.637099 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:09.637074    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n" podUID="d8725463-9c1f-4cee-b5ec-28fb3ca9f939" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Nov 04 01:15:10.226998 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:10.224422    5068 patch_prober.go:29] interesting pod/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Nov 04 01:15:10.226998 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:10.224480    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p" podUID="fa8debd2-b803-412c-9e4c-53917a748a59" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Nov 04 01:15:10.517347 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:10.517129    5068 patch_prober.go:29] interesting pod/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Nov 04 01:15:10.517347 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:10.517186    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8" podUID="43eaae59-c37c-4553-a856-fc8aa9506c00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Nov 04 01:15:10.675782 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:10.674242    5068 patch_prober.go:29] interesting pod/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Nov 04 01:15:10.675782 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:10.674305    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n" podUID="d8725463-9c1f-4cee-b5ec-28fb3ca9f939" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Nov 04 01:15:11.417380 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:11.416459    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p"
Nov 04 01:15:11.418197 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:11.418130    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p"
Nov 04 01:15:11.419384 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:11.419338    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p"
Nov 04 01:15:11.686535 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:11.686454    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8"
Nov 04 01:15:11.687928 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:11.687811    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8"
Nov 04 01:15:11.688868 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:11.688806    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8"
Nov 04 01:15:11.717437 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:11.714940    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n"
Nov 04 01:15:11.718738 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:11.717701    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n"
Nov 04 01:15:11.723450 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:11.723407    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n"
Nov 04 01:15:13.035816 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:13.035773    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895915-bgpl8" event={"ID":"6b6c8fa4-6394-4dd4-98e2-0441dfea042b","Type":"ContainerStarted","Data":"fa69cc44f07ef3315d7e000e6287282da2171989d3cd97840f4e2446df96aaff"}
Nov 04 01:15:13.054240 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:13.054182    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895915-bgpl8" podStartSLOduration=1.291886858 podStartE2EDuration="13.054169697s" totalImagesPullingTime="11.762282839s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:15:13.052716763 +0000 UTC m=+753.099277551" watchObservedRunningTime="2026-11-04 01:15:13.054169697 +0000 UTC m=+753.100730477"
Nov 04 01:15:13.146727 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:13.146655    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:15:13.448744 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:15:13.448705    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:15:14.046427 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:14.046391    5068 generic.go:356] "Generic (PLEG): container finished" podID="6b6c8fa4-6394-4dd4-98e2-0441dfea042b" containerID="fa69cc44f07ef3315d7e000e6287282da2171989d3cd97840f4e2446df96aaff" exitCode=60
Nov 04 01:15:14.046932 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:14.046469    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895915-bgpl8" event={"ID":"6b6c8fa4-6394-4dd4-98e2-0441dfea042b","Type":"ContainerDied","Data":"fa69cc44f07ef3315d7e000e6287282da2171989d3cd97840f4e2446df96aaff"}
Nov 04 01:15:14.046932 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:14.046747    5068 scope.go:122] "RemoveContainer" containerID="fa69cc44f07ef3315d7e000e6287282da2171989d3cd97840f4e2446df96aaff"
Nov 04 01:15:15.059445 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:15.059411    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895915-bgpl8" event={"ID":"6b6c8fa4-6394-4dd4-98e2-0441dfea042b","Type":"ContainerStarted","Data":"a7474e66f6c217139e7b6c5ceb8e579b0ca44957ed5fdbf35ee2f1a37e76e4c9"}
Nov 04 01:15:16.071798 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:16.071743    5068 generic.go:356] "Generic (PLEG): container finished" podID="6b6c8fa4-6394-4dd4-98e2-0441dfea042b" containerID="a7474e66f6c217139e7b6c5ceb8e579b0ca44957ed5fdbf35ee2f1a37e76e4c9" exitCode=60
Nov 04 01:15:16.072255 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:16.071804    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895915-bgpl8" event={"ID":"6b6c8fa4-6394-4dd4-98e2-0441dfea042b","Type":"ContainerDied","Data":"a7474e66f6c217139e7b6c5ceb8e579b0ca44957ed5fdbf35ee2f1a37e76e4c9"}
Nov 04 01:15:16.072255 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:16.071874    5068 scope.go:122] "RemoveContainer" containerID="fa69cc44f07ef3315d7e000e6287282da2171989d3cd97840f4e2446df96aaff"
Nov 04 01:15:16.072255 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:16.072134    5068 scope.go:122] "RemoveContainer" containerID="a7474e66f6c217139e7b6c5ceb8e579b0ca44957ed5fdbf35ee2f1a37e76e4c9"
Nov 04 01:15:16.072485 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:15:16.072449    5068 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29895915-bgpl8_odh-ai-gateway-infra(6b6c8fa4-6394-4dd4-98e2-0441dfea042b)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895915-bgpl8" podUID="6b6c8fa4-6394-4dd4-98e2-0441dfea042b"
Nov 04 01:15:17.067814 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:17.067779    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq"
Nov 04 01:15:17.085433 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:17.085405    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq"
Nov 04 01:15:17.090066 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:17.090043    5068 scope.go:122] "RemoveContainer" containerID="a7474e66f6c217139e7b6c5ceb8e579b0ca44957ed5fdbf35ee2f1a37e76e4c9"
Nov 04 01:15:17.090349 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:15:17.090325    5068 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29895915-bgpl8_odh-ai-gateway-infra(6b6c8fa4-6394-4dd4-98e2-0441dfea042b)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895915-bgpl8" podUID="6b6c8fa4-6394-4dd4-98e2-0441dfea042b"
Nov 04 01:15:17.101258 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:17.101238    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq"
Nov 04 01:15:28.105927 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:28.105858    5068 scope.go:122] "RemoveContainer" containerID="a7474e66f6c217139e7b6c5ceb8e579b0ca44957ed5fdbf35ee2f1a37e76e4c9"
Nov 04 01:15:29.176145 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:29.176103    5068 generic.go:356] "Generic (PLEG): container finished" podID="6b6c8fa4-6394-4dd4-98e2-0441dfea042b" containerID="c88d13560c5d07cf79c5137b00b20cc6f33c9e24ee3098678fa161e1aca0f68d" exitCode=60
Nov 04 01:15:29.176667 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:29.176329    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895915-bgpl8" event={"ID":"6b6c8fa4-6394-4dd4-98e2-0441dfea042b","Type":"ContainerDied","Data":"c88d13560c5d07cf79c5137b00b20cc6f33c9e24ee3098678fa161e1aca0f68d"}
Nov 04 01:15:29.176667 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:29.176387    5068 scope.go:122] "RemoveContainer" containerID="a7474e66f6c217139e7b6c5ceb8e579b0ca44957ed5fdbf35ee2f1a37e76e4c9"
Nov 04 01:15:29.176903 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:29.176881    5068 scope.go:122] "RemoveContainer" containerID="c88d13560c5d07cf79c5137b00b20cc6f33c9e24ee3098678fa161e1aca0f68d"
Nov 04 01:15:29.177238 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:15:29.177213    5068 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29895915-bgpl8_odh-ai-gateway-infra(6b6c8fa4-6394-4dd4-98e2-0441dfea042b)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895915-bgpl8" podUID="6b6c8fa4-6394-4dd4-98e2-0441dfea042b"
Nov 04 01:15:30.305722 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:30.305682    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895915-bgpl8"]
Nov 04 01:15:31.278520 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:31.278484    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895915-bgpl8" event={"ID":"6b6c8fa4-6394-4dd4-98e2-0441dfea042b","Type":"ContainerDied","Data":"17127a66d13c5d7227481d445c9d7e2c0a3603d7123d3e5effeb1ffc51e4fcbb"}
Nov 04 01:15:31.278784 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:31.278763    5068 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="17127a66d13c5d7227481d445c9d7e2c0a3603d7123d3e5effeb1ffc51e4fcbb"
Nov 04 01:15:31.282265 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:31.282241    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895915-bgpl8"
Nov 04 01:15:31.411703 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:31.411504    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6b6c8fa4-6394-4dd4-98e2-0441dfea042b-kube-api-access-f4qq8\" (UniqueName: \"kubernetes.io/projected/6b6c8fa4-6394-4dd4-98e2-0441dfea042b-kube-api-access-f4qq8\") pod \"6b6c8fa4-6394-4dd4-98e2-0441dfea042b\" (UID: \"6b6c8fa4-6394-4dd4-98e2-0441dfea042b\") "
Nov 04 01:15:31.413036 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:31.412285    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6b6c8fa4-6394-4dd4-98e2-0441dfea042b-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b6c8fa4-6394-4dd4-98e2-0441dfea042b-maas-api-service-ca\") pod \"6b6c8fa4-6394-4dd4-98e2-0441dfea042b\" (UID: \"6b6c8fa4-6394-4dd4-98e2-0441dfea042b\") "
Nov 04 01:15:31.413324 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:31.413284    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b6c8fa4-6394-4dd4-98e2-0441dfea042b-maas-api-service-ca" pod "6b6c8fa4-6394-4dd4-98e2-0441dfea042b" (UID: "6b6c8fa4-6394-4dd4-98e2-0441dfea042b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:15:31.414910 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:31.414874    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b6c8fa4-6394-4dd4-98e2-0441dfea042b-kube-api-access-f4qq8" pod "6b6c8fa4-6394-4dd4-98e2-0441dfea042b" (UID: "6b6c8fa4-6394-4dd4-98e2-0441dfea042b"). InnerVolumeSpecName "kube-api-access-f4qq8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:15:31.514124 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:31.514088    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f4qq8\" (UniqueName: \"kubernetes.io/projected/6b6c8fa4-6394-4dd4-98e2-0441dfea042b-kube-api-access-f4qq8\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:15:31.514654 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:31.514603    5068 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b6c8fa4-6394-4dd4-98e2-0441dfea042b-maas-api-service-ca\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:15:32.021873 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:32.021844    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz"]
Nov 04 01:15:32.022364 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:32.022351    5068 cpu_manager.go:414] "containerMap: removing container" podUID="6b6c8fa4-6394-4dd4-98e2-0441dfea042b" containerName="cleanup"
Nov 04 01:15:32.022412 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:32.022366    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6b6c8fa4-6394-4dd4-98e2-0441dfea042b" containerName="cleanup"
Nov 04 01:15:32.022412 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:32.022377    5068 cpu_manager.go:414] "containerMap: removing container" podUID="6b6c8fa4-6394-4dd4-98e2-0441dfea042b" containerName="cleanup"
Nov 04 01:15:32.022412 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:32.022381    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6b6c8fa4-6394-4dd4-98e2-0441dfea042b" containerName="cleanup"
Nov 04 01:15:32.022506 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:32.022482    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="6b6c8fa4-6394-4dd4-98e2-0441dfea042b" containerName="cleanup"
Nov 04 01:15:32.022506 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:32.022491    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="6b6c8fa4-6394-4dd4-98e2-0441dfea042b" containerName="cleanup"
Nov 04 01:15:32.022506 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:32.022497    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="6b6c8fa4-6394-4dd4-98e2-0441dfea042b" containerName="cleanup"
Nov 04 01:15:32.028448 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:32.028423    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz"
Nov 04 01:15:32.037223 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:32.037200    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-6dac99-openshift-default-dockercfg-cxxq4\""
Nov 04 01:15:32.042344 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:32.042313    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz"]
Nov 04 01:15:32.221867 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:32.221834    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52491e2f-36e5-4e6b-ae82-5795752be764-workload-certs\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz\" (UID: \"52491e2f-36e5-4e6b-ae82-5795752be764\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz"
Nov 04 01:15:32.222021 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:32.221880    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52491e2f-36e5-4e6b-ae82-5795752be764-istio-data\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz\" (UID: \"52491e2f-36e5-4e6b-ae82-5795752be764\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz"
Nov 04 01:15:32.222021 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:32.221977    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52491e2f-36e5-4e6b-ae82-5795752be764-istiod-ca-cert\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz\" (UID: \"52491e2f-36e5-4e6b-ae82-5795752be764\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz"
Nov 04 01:15:32.222098 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:32.222016    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52491e2f-36e5-4e6b-ae82-5795752be764-credential-socket\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz\" (UID: \"52491e2f-36e5-4e6b-ae82-5795752be764\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz"
Nov 04 01:15:32.222131 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:32.222109    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lhl4\" (UniqueName: \"kubernetes.io/projected/52491e2f-36e5-4e6b-ae82-5795752be764-kube-api-access-4lhl4\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz\" (UID: \"52491e2f-36e5-4e6b-ae82-5795752be764\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz"
Nov 04 01:15:32.222160 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:32.222150    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52491e2f-36e5-4e6b-ae82-5795752be764-istio-podinfo\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz\" (UID: \"52491e2f-36e5-4e6b-ae82-5795752be764\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz"
Nov 04 01:15:32.222292 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:32.222272    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52491e2f-36e5-4e6b-ae82-5795752be764-workload-socket\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz\" (UID: \"52491e2f-36e5-4e6b-ae82-5795752be764\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz"
Nov 04 01:15:32.222454 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:32.222435    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52491e2f-36e5-4e6b-ae82-5795752be764-istio-token\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz\" (UID: \"52491e2f-36e5-4e6b-ae82-5795752be764\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz"
Nov 04 01:15:32.222534 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:32.222518    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52491e2f-36e5-4e6b-ae82-5795752be764-istio-envoy\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz\" (UID: \"52491e2f-36e5-4e6b-ae82-5795752be764\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz"
Nov 04 01:15:32.285555 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:32.285479    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895915-bgpl8"
Nov 04 01:15:32.308784 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:32.308752    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895915-bgpl8"]
Nov 04 01:15:32.317501 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:32.317463    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895915-bgpl8"]
Nov 04 01:15:32.323184 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:32.323160    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52491e2f-36e5-4e6b-ae82-5795752be764-workload-socket\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz\" (UID: \"52491e2f-36e5-4e6b-ae82-5795752be764\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz"
Nov 04 01:15:32.323315 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:32.323227    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52491e2f-36e5-4e6b-ae82-5795752be764-istio-token\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz\" (UID: \"52491e2f-36e5-4e6b-ae82-5795752be764\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz"
Nov 04 01:15:32.323315 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:32.323280    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52491e2f-36e5-4e6b-ae82-5795752be764-istio-envoy\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz\" (UID: \"52491e2f-36e5-4e6b-ae82-5795752be764\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz"
Nov 04 01:15:32.323426 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:32.323326    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52491e2f-36e5-4e6b-ae82-5795752be764-workload-certs\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz\" (UID: \"52491e2f-36e5-4e6b-ae82-5795752be764\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz"
Nov 04 01:15:32.323426 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:32.323353    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52491e2f-36e5-4e6b-ae82-5795752be764-istio-data\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz\" (UID: \"52491e2f-36e5-4e6b-ae82-5795752be764\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz"
Nov 04 01:15:32.323426 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:32.323392    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52491e2f-36e5-4e6b-ae82-5795752be764-istiod-ca-cert\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz\" (UID: \"52491e2f-36e5-4e6b-ae82-5795752be764\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz"
Nov 04 01:15:32.323579 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:32.323431    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52491e2f-36e5-4e6b-ae82-5795752be764-credential-socket\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz\" (UID: \"52491e2f-36e5-4e6b-ae82-5795752be764\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz"
Nov 04 01:15:32.323579 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:32.323473    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lhl4\" (UniqueName: \"kubernetes.io/projected/52491e2f-36e5-4e6b-ae82-5795752be764-kube-api-access-4lhl4\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz\" (UID: \"52491e2f-36e5-4e6b-ae82-5795752be764\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz"
Nov 04 01:15:32.323579 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:32.323515    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52491e2f-36e5-4e6b-ae82-5795752be764-istio-podinfo\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz\" (UID: \"52491e2f-36e5-4e6b-ae82-5795752be764\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz"
Nov 04 01:15:32.323838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:32.323570    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52491e2f-36e5-4e6b-ae82-5795752be764-workload-socket\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz\" (UID: \"52491e2f-36e5-4e6b-ae82-5795752be764\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz"
Nov 04 01:15:32.323838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:32.323703    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52491e2f-36e5-4e6b-ae82-5795752be764-workload-certs\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz\" (UID: \"52491e2f-36e5-4e6b-ae82-5795752be764\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz"
Nov 04 01:15:32.323945 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:32.323921    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52491e2f-36e5-4e6b-ae82-5795752be764-istio-data\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz\" (UID: \"52491e2f-36e5-4e6b-ae82-5795752be764\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz"
Nov 04 01:15:32.324001 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:32.323941    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52491e2f-36e5-4e6b-ae82-5795752be764-credential-socket\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz\" (UID: \"52491e2f-36e5-4e6b-ae82-5795752be764\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz"
Nov 04 01:15:32.324172 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:32.324151    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52491e2f-36e5-4e6b-ae82-5795752be764-istiod-ca-cert\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz\" (UID: \"52491e2f-36e5-4e6b-ae82-5795752be764\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz"
Nov 04 01:15:32.325819 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:32.325799    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52491e2f-36e5-4e6b-ae82-5795752be764-istio-envoy\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz\" (UID: \"52491e2f-36e5-4e6b-ae82-5795752be764\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz"
Nov 04 01:15:32.325966 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:32.325948    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52491e2f-36e5-4e6b-ae82-5795752be764-istio-podinfo\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz\" (UID: \"52491e2f-36e5-4e6b-ae82-5795752be764\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz"
Nov 04 01:15:32.337230 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:32.337210    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52491e2f-36e5-4e6b-ae82-5795752be764-istio-token\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz\" (UID: \"52491e2f-36e5-4e6b-ae82-5795752be764\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz"
Nov 04 01:15:32.337423 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:32.337404    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lhl4\" (UniqueName: \"kubernetes.io/projected/52491e2f-36e5-4e6b-ae82-5795752be764-kube-api-access-4lhl4\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz\" (UID: \"52491e2f-36e5-4e6b-ae82-5795752be764\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz"
Nov 04 01:15:32.345401 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:32.345380    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz"
Nov 04 01:15:32.661243 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:32.661211    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz"]
Nov 04 01:15:32.663544 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:32.663514    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:15:32.663649 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:32.663593    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:15:32.663705 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:32.663684    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:15:33.294560 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:33.294523    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz" event={"ID":"52491e2f-36e5-4e6b-ae82-5795752be764","Type":"ContainerStarted","Data":"4fc8b329222454fe0d27fff9ece227fe9b9cf9bf10f190d65b6e90478bc9a6c3"}
Nov 04 01:15:33.294754 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:33.294576    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz" event={"ID":"52491e2f-36e5-4e6b-ae82-5795752be764","Type":"ContainerStarted","Data":"d32201e4ccea8433294df57aaeba1df6c5a8a1a8630a2f120a1b43572c220674"}
Nov 04 01:15:33.319415 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:33.319372    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz" podStartSLOduration=1.3193589239999999 podStartE2EDuration="1.319358924s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:15:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:15:33.317154823 +0000 UTC m=+773.363715622" watchObservedRunningTime="2026-11-04 01:15:33.319358924 +0000 UTC m=+773.365919770"
Nov 04 01:15:33.346418 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:33.346392    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz"
Nov 04 01:15:33.348206 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:33.348174    5068 patch_prober.go:29] interesting pod/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 04 01:15:33.348329 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:33.348232    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz" podUID="52491e2f-36e5-4e6b-ae82-5795752be764" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 04 01:15:34.113162 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:34.113125    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6b6c8fa4-6394-4dd4-98e2-0441dfea042b" path="/var/lib/kubelet/pods/6b6c8fa4-6394-4dd4-98e2-0441dfea042b/volumes"
Nov 04 01:15:34.346634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:34.346004    5068 patch_prober.go:29] interesting pod/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 04 01:15:34.346634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:34.346063    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz" podUID="52491e2f-36e5-4e6b-ae82-5795752be764" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 04 01:15:35.353640 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:35.348937    5068 patch_prober.go:29] interesting pod/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 04 01:15:35.353640 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:35.349008    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz" podUID="52491e2f-36e5-4e6b-ae82-5795752be764" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 04 01:15:36.350024 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:36.349991    5068 patch_prober.go:29] interesting pod/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 04 01:15:36.350284 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:36.350255    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz" podUID="52491e2f-36e5-4e6b-ae82-5795752be764" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 04 01:15:37.179742 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:37.179700    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf"]
Nov 04 01:15:37.180523 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:37.180502    5068 cpu_manager.go:414] "containerMap: removing container" podUID="6b6c8fa4-6394-4dd4-98e2-0441dfea042b" containerName="cleanup"
Nov 04 01:15:37.180579 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:37.180527    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6b6c8fa4-6394-4dd4-98e2-0441dfea042b" containerName="cleanup"
Nov 04 01:15:37.185900 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:37.185875    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf"
Nov 04 01:15:37.202406 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:37.195924    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-1036e8\"/\"openshift-service-ca.crt\""
Nov 04 01:15:37.202406 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:37.196003    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-1036e8\"/\"default-dockercfg-769nm\""
Nov 04 01:15:37.202406 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:37.196230    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-1036e8\"/\"distinct-2-w1-1036e8-kserve-self-signed-certs\""
Nov 04 01:15:37.202406 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:37.196254    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf"]
Nov 04 01:15:37.202406 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:37.196346    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-1036e8\"/\"kube-root-ca.crt\""
Nov 04 01:15:37.214996 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:37.214961    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/32f60f59-a98f-46ba-9fb4-8420f77998da-dshm\") pod \"distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf\" (UID: \"32f60f59-a98f-46ba-9fb4-8420f77998da\") " pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf"
Nov 04 01:15:37.215148 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:37.215036    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32f60f59-a98f-46ba-9fb4-8420f77998da-tmp-dir\") pod \"distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf\" (UID: \"32f60f59-a98f-46ba-9fb4-8420f77998da\") " pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf"
Nov 04 01:15:37.215148 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:37.215115    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32f60f59-a98f-46ba-9fb4-8420f77998da-tls-certs\") pod \"distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf\" (UID: \"32f60f59-a98f-46ba-9fb4-8420f77998da\") " pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf"
Nov 04 01:15:37.215266 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:37.215208    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/32f60f59-a98f-46ba-9fb4-8420f77998da-model-cache\") pod \"distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf\" (UID: \"32f60f59-a98f-46ba-9fb4-8420f77998da\") " pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf"
Nov 04 01:15:37.215266 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:37.215247    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsxlk\" (UniqueName: \"kubernetes.io/projected/32f60f59-a98f-46ba-9fb4-8420f77998da-kube-api-access-rsxlk\") pod \"distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf\" (UID: \"32f60f59-a98f-46ba-9fb4-8420f77998da\") " pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf"
Nov 04 01:15:37.215367 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:37.215313    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/32f60f59-a98f-46ba-9fb4-8420f77998da-home\") pod \"distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf\" (UID: \"32f60f59-a98f-46ba-9fb4-8420f77998da\") " pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf"
Nov 04 01:15:37.316657 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:37.316602    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/32f60f59-a98f-46ba-9fb4-8420f77998da-dshm\") pod \"distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf\" (UID: \"32f60f59-a98f-46ba-9fb4-8420f77998da\") " pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf"
Nov 04 01:15:37.317043 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:37.316903    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32f60f59-a98f-46ba-9fb4-8420f77998da-tmp-dir\") pod \"distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf\" (UID: \"32f60f59-a98f-46ba-9fb4-8420f77998da\") " pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf"
Nov 04 01:15:37.317235 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:37.317216    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32f60f59-a98f-46ba-9fb4-8420f77998da-tls-certs\") pod \"distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf\" (UID: \"32f60f59-a98f-46ba-9fb4-8420f77998da\") " pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf"
Nov 04 01:15:37.318166 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:37.317391    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/32f60f59-a98f-46ba-9fb4-8420f77998da-model-cache\") pod \"distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf\" (UID: \"32f60f59-a98f-46ba-9fb4-8420f77998da\") " pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf"
Nov 04 01:15:37.318352 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:37.318330    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsxlk\" (UniqueName: \"kubernetes.io/projected/32f60f59-a98f-46ba-9fb4-8420f77998da-kube-api-access-rsxlk\") pod \"distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf\" (UID: \"32f60f59-a98f-46ba-9fb4-8420f77998da\") " pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf"
Nov 04 01:15:37.318506 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:37.318492    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/32f60f59-a98f-46ba-9fb4-8420f77998da-home\") pod \"distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf\" (UID: \"32f60f59-a98f-46ba-9fb4-8420f77998da\") " pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf"
Nov 04 01:15:37.318965 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:37.317933    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/32f60f59-a98f-46ba-9fb4-8420f77998da-model-cache\") pod \"distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf\" (UID: \"32f60f59-a98f-46ba-9fb4-8420f77998da\") " pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf"
Nov 04 01:15:37.319051 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:37.317712    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32f60f59-a98f-46ba-9fb4-8420f77998da-tmp-dir\") pod \"distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf\" (UID: \"32f60f59-a98f-46ba-9fb4-8420f77998da\") " pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf"
Nov 04 01:15:37.319149 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:37.319130    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/32f60f59-a98f-46ba-9fb4-8420f77998da-home\") pod \"distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf\" (UID: \"32f60f59-a98f-46ba-9fb4-8420f77998da\") " pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf"
Nov 04 01:15:37.325338 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:37.325314    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/32f60f59-a98f-46ba-9fb4-8420f77998da-dshm\") pod \"distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf\" (UID: \"32f60f59-a98f-46ba-9fb4-8420f77998da\") " pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf"
Nov 04 01:15:37.332084 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:37.332057    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsxlk\" (UniqueName: \"kubernetes.io/projected/32f60f59-a98f-46ba-9fb4-8420f77998da-kube-api-access-rsxlk\") pod \"distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf\" (UID: \"32f60f59-a98f-46ba-9fb4-8420f77998da\") " pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf"
Nov 04 01:15:37.332765 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:37.332687    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32f60f59-a98f-46ba-9fb4-8420f77998da-tls-certs\") pod \"distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf\" (UID: \"32f60f59-a98f-46ba-9fb4-8420f77998da\") " pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf"
Nov 04 01:15:37.350652 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:37.348066    5068 patch_prober.go:29] interesting pod/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 04 01:15:37.350652 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:37.348126    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz" podUID="52491e2f-36e5-4e6b-ae82-5795752be764" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 04 01:15:37.508458 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:37.508372    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf"
Nov 04 01:15:37.829291 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:37.828950    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf"]
Nov 04 01:15:38.336345 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:38.336309    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf" event={"ID":"32f60f59-a98f-46ba-9fb4-8420f77998da","Type":"ContainerStarted","Data":"06300cf6bf05237f990c2cf9a7f6a09de989309df2b86f1ecfd8f967ddd31612"}
Nov 04 01:15:38.336767 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:38.336359    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf" event={"ID":"32f60f59-a98f-46ba-9fb4-8420f77998da","Type":"ContainerStarted","Data":"fa23340c1e2eb7708174a662d9abafe9bf9910e61e55c8024f4e80f525dc67df"}
Nov 04 01:15:38.349989 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:38.349960    5068 patch_prober.go:29] interesting pod/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 04 01:15:38.350192 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:38.350168    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz" podUID="52491e2f-36e5-4e6b-ae82-5795752be764" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 04 01:15:38.508950 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:38.508914    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf"
Nov 04 01:15:39.345790 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:39.345745    5068 patch_prober.go:29] interesting pod/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 04 01:15:39.346144 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:39.345810    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz" podUID="52491e2f-36e5-4e6b-ae82-5795752be764" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 04 01:15:40.352154 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:40.352120    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz"
Nov 04 01:15:40.352574 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:40.352549    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz"
Nov 04 01:15:40.353703 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:40.353677    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz"
Nov 04 01:15:40.381648 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:40.381579    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf" podStartSLOduration=3.381567055 podStartE2EDuration="3.381567055s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:15:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:15:38.363230079 +0000 UTC m=+778.409790869" watchObservedRunningTime="2026-11-04 01:15:40.381567055 +0000 UTC m=+780.428127844"
Nov 04 01:15:43.452334 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:15:43.452299    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:15:45.792995 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:45.792948    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:15:46.926478 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:46.926443    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm"]
Nov 04 01:15:46.932777 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:46.932752    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm"
Nov 04 01:15:46.946229 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:46.946202    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-ba844c\"/\"distinct-w4-ba844c-kserve-self-signed-certs\""
Nov 04 01:15:47.025796 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.025713    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm"]
Nov 04 01:15:47.113634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.052282    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9e7abd2-8737-4357-a778-d6492fd12f3f-model-cache\") pod \"distinct-w4-ba844c-kserve-97db78785-cg9xm\" (UID: \"f9e7abd2-8737-4357-a778-d6492fd12f3f\") " pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm"
Nov 04 01:15:47.113634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.052384    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9e7abd2-8737-4357-a778-d6492fd12f3f-home\") pod \"distinct-w4-ba844c-kserve-97db78785-cg9xm\" (UID: \"f9e7abd2-8737-4357-a778-d6492fd12f3f\") " pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm"
Nov 04 01:15:47.113634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.052414    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9e7abd2-8737-4357-a778-d6492fd12f3f-dshm\") pod \"distinct-w4-ba844c-kserve-97db78785-cg9xm\" (UID: \"f9e7abd2-8737-4357-a778-d6492fd12f3f\") " pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm"
Nov 04 01:15:47.113634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.052523    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9e7abd2-8737-4357-a778-d6492fd12f3f-tmp-dir\") pod \"distinct-w4-ba844c-kserve-97db78785-cg9xm\" (UID: \"f9e7abd2-8737-4357-a778-d6492fd12f3f\") " pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm"
Nov 04 01:15:47.113634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.052556    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9e7abd2-8737-4357-a778-d6492fd12f3f-tls-certs\") pod \"distinct-w4-ba844c-kserve-97db78785-cg9xm\" (UID: \"f9e7abd2-8737-4357-a778-d6492fd12f3f\") " pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm"
Nov 04 01:15:47.113634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.052667    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qd6p9\" (UniqueName: \"kubernetes.io/projected/f9e7abd2-8737-4357-a778-d6492fd12f3f-kube-api-access-qd6p9\") pod \"distinct-w4-ba844c-kserve-97db78785-cg9xm\" (UID: \"f9e7abd2-8737-4357-a778-d6492fd12f3f\") " pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm"
Nov 04 01:15:47.159633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.153483    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9e7abd2-8737-4357-a778-d6492fd12f3f-model-cache\") pod \"distinct-w4-ba844c-kserve-97db78785-cg9xm\" (UID: \"f9e7abd2-8737-4357-a778-d6492fd12f3f\") " pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm"
Nov 04 01:15:47.159633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.153586    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9e7abd2-8737-4357-a778-d6492fd12f3f-home\") pod \"distinct-w4-ba844c-kserve-97db78785-cg9xm\" (UID: \"f9e7abd2-8737-4357-a778-d6492fd12f3f\") " pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm"
Nov 04 01:15:47.159633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.153630    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9e7abd2-8737-4357-a778-d6492fd12f3f-dshm\") pod \"distinct-w4-ba844c-kserve-97db78785-cg9xm\" (UID: \"f9e7abd2-8737-4357-a778-d6492fd12f3f\") " pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm"
Nov 04 01:15:47.159633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.153732    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9e7abd2-8737-4357-a778-d6492fd12f3f-tmp-dir\") pod \"distinct-w4-ba844c-kserve-97db78785-cg9xm\" (UID: \"f9e7abd2-8737-4357-a778-d6492fd12f3f\") " pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm"
Nov 04 01:15:47.159633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.153762    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9e7abd2-8737-4357-a778-d6492fd12f3f-tls-certs\") pod \"distinct-w4-ba844c-kserve-97db78785-cg9xm\" (UID: \"f9e7abd2-8737-4357-a778-d6492fd12f3f\") " pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm"
Nov 04 01:15:47.159633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.153849    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qd6p9\" (UniqueName: \"kubernetes.io/projected/f9e7abd2-8737-4357-a778-d6492fd12f3f-kube-api-access-qd6p9\") pod \"distinct-w4-ba844c-kserve-97db78785-cg9xm\" (UID: \"f9e7abd2-8737-4357-a778-d6492fd12f3f\") " pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm"
Nov 04 01:15:47.159633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.154210    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj"]
Nov 04 01:15:47.159633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.156098    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9e7abd2-8737-4357-a778-d6492fd12f3f-home\") pod \"distinct-w4-ba844c-kserve-97db78785-cg9xm\" (UID: \"f9e7abd2-8737-4357-a778-d6492fd12f3f\") " pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm"
Nov 04 01:15:47.193151 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.193078    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9e7abd2-8737-4357-a778-d6492fd12f3f-tmp-dir\") pod \"distinct-w4-ba844c-kserve-97db78785-cg9xm\" (UID: \"f9e7abd2-8737-4357-a778-d6492fd12f3f\") " pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm"
Nov 04 01:15:47.194240 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.194217    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9e7abd2-8737-4357-a778-d6492fd12f3f-model-cache\") pod \"distinct-w4-ba844c-kserve-97db78785-cg9xm\" (UID: \"f9e7abd2-8737-4357-a778-d6492fd12f3f\") " pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm"
Nov 04 01:15:47.213168 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.195588    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj"]
Nov 04 01:15:47.213509 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.195743    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj"
Nov 04 01:15:47.228657 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.219526    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9e7abd2-8737-4357-a778-d6492fd12f3f-tls-certs\") pod \"distinct-w4-ba844c-kserve-97db78785-cg9xm\" (UID: \"f9e7abd2-8737-4357-a778-d6492fd12f3f\") " pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm"
Nov 04 01:15:47.274671 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.267399    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qd6p9\" (UniqueName: \"kubernetes.io/projected/f9e7abd2-8737-4357-a778-d6492fd12f3f-kube-api-access-qd6p9\") pod \"distinct-w4-ba844c-kserve-97db78785-cg9xm\" (UID: \"f9e7abd2-8737-4357-a778-d6492fd12f3f\") " pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm"
Nov 04 01:15:47.291635 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.288918    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9e7abd2-8737-4357-a778-d6492fd12f3f-dshm\") pod \"distinct-w4-ba844c-kserve-97db78785-cg9xm\" (UID: \"f9e7abd2-8737-4357-a778-d6492fd12f3f\") " pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm"
Nov 04 01:15:47.341502 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.328221    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-istio-envoy\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj\" (UID: \"239d3dec-a8fd-45e4-82ec-1ed4e1d01c72\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj"
Nov 04 01:15:47.342062 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.341753    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm"
Nov 04 01:15:47.390634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.351015    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-istio-podinfo\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj\" (UID: \"239d3dec-a8fd-45e4-82ec-1ed4e1d01c72\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj"
Nov 04 01:15:47.390634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.351099    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-istio-data\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj\" (UID: \"239d3dec-a8fd-45e4-82ec-1ed4e1d01c72\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj"
Nov 04 01:15:47.390634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.351174    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-istiod-ca-cert\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj\" (UID: \"239d3dec-a8fd-45e4-82ec-1ed4e1d01c72\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj"
Nov 04 01:15:47.390634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.351248    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-credential-socket\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj\" (UID: \"239d3dec-a8fd-45e4-82ec-1ed4e1d01c72\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj"
Nov 04 01:15:47.390634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.351316    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pvp7\" (UniqueName: \"kubernetes.io/projected/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-kube-api-access-7pvp7\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj\" (UID: \"239d3dec-a8fd-45e4-82ec-1ed4e1d01c72\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj"
Nov 04 01:15:47.390634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.351375    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-workload-certs\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj\" (UID: \"239d3dec-a8fd-45e4-82ec-1ed4e1d01c72\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj"
Nov 04 01:15:47.390634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.351451    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-istio-token\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj\" (UID: \"239d3dec-a8fd-45e4-82ec-1ed4e1d01c72\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj"
Nov 04 01:15:47.390634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.351552    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-workload-socket\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj\" (UID: \"239d3dec-a8fd-45e4-82ec-1ed4e1d01c72\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj"
Nov 04 01:15:47.510934 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.487011    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-istio-data\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj\" (UID: \"239d3dec-a8fd-45e4-82ec-1ed4e1d01c72\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj"
Nov 04 01:15:47.510934 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.487086    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-istiod-ca-cert\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj\" (UID: \"239d3dec-a8fd-45e4-82ec-1ed4e1d01c72\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj"
Nov 04 01:15:47.510934 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.487142    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-credential-socket\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj\" (UID: \"239d3dec-a8fd-45e4-82ec-1ed4e1d01c72\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj"
Nov 04 01:15:47.510934 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.487174    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pvp7\" (UniqueName: \"kubernetes.io/projected/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-kube-api-access-7pvp7\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj\" (UID: \"239d3dec-a8fd-45e4-82ec-1ed4e1d01c72\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj"
Nov 04 01:15:47.510934 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.487213    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-workload-certs\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj\" (UID: \"239d3dec-a8fd-45e4-82ec-1ed4e1d01c72\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj"
Nov 04 01:15:47.510934 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.487247    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-istio-token\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj\" (UID: \"239d3dec-a8fd-45e4-82ec-1ed4e1d01c72\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj"
Nov 04 01:15:47.510934 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.487311    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-workload-socket\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj\" (UID: \"239d3dec-a8fd-45e4-82ec-1ed4e1d01c72\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj"
Nov 04 01:15:47.510934 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.487448    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-istio-envoy\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj\" (UID: \"239d3dec-a8fd-45e4-82ec-1ed4e1d01c72\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj"
Nov 04 01:15:47.510934 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.487480    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-istio-podinfo\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj\" (UID: \"239d3dec-a8fd-45e4-82ec-1ed4e1d01c72\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj"
Nov 04 01:15:47.510934 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.488098    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-istio-data\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj\" (UID: \"239d3dec-a8fd-45e4-82ec-1ed4e1d01c72\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj"
Nov 04 01:15:47.510934 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.488783    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-istiod-ca-cert\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj\" (UID: \"239d3dec-a8fd-45e4-82ec-1ed4e1d01c72\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj"
Nov 04 01:15:47.510934 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.489051    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-credential-socket\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj\" (UID: \"239d3dec-a8fd-45e4-82ec-1ed4e1d01c72\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj"
Nov 04 01:15:47.510934 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.489583    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-workload-certs\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj\" (UID: \"239d3dec-a8fd-45e4-82ec-1ed4e1d01c72\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj"
Nov 04 01:15:47.551574 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.521225    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-workload-socket\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj\" (UID: \"239d3dec-a8fd-45e4-82ec-1ed4e1d01c72\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj"
Nov 04 01:15:47.551574 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.522141    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf"
Nov 04 01:15:47.551574 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.523134    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pvp7\" (UniqueName: \"kubernetes.io/projected/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-kube-api-access-7pvp7\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj\" (UID: \"239d3dec-a8fd-45e4-82ec-1ed4e1d01c72\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj"
Nov 04 01:15:47.551574 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.538863    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-istio-envoy\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj\" (UID: \"239d3dec-a8fd-45e4-82ec-1ed4e1d01c72\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj"
Nov 04 01:15:47.558997 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.556560    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf"]
Nov 04 01:15:47.605635 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.569791    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf"]
Nov 04 01:15:47.624130 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.608324    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf"
Nov 04 01:15:47.625809 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.612525    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-istio-podinfo\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj\" (UID: \"239d3dec-a8fd-45e4-82ec-1ed4e1d01c72\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj"
Nov 04 01:15:47.626396 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.626362    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-istio-token\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj\" (UID: \"239d3dec-a8fd-45e4-82ec-1ed4e1d01c72\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj"
Nov 04 01:15:47.690634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.672771    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf"
Nov 04 01:15:47.710674 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.707921    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-istio-envoy\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf\" (UID: \"c3d5c7b1-d7fe-4a70-aba1-1131171f55c1\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf"
Nov 04 01:15:47.710674 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.707987    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwk77\" (UniqueName: \"kubernetes.io/projected/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-kube-api-access-xwk77\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf\" (UID: \"c3d5c7b1-d7fe-4a70-aba1-1131171f55c1\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf"
Nov 04 01:15:47.710674 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.708035    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-istio-podinfo\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf\" (UID: \"c3d5c7b1-d7fe-4a70-aba1-1131171f55c1\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf"
Nov 04 01:15:47.710674 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.708090    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-istio-data\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf\" (UID: \"c3d5c7b1-d7fe-4a70-aba1-1131171f55c1\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf"
Nov 04 01:15:47.710674 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.708121    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-workload-certs\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf\" (UID: \"c3d5c7b1-d7fe-4a70-aba1-1131171f55c1\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf"
Nov 04 01:15:47.710674 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.708154    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-istiod-ca-cert\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf\" (UID: \"c3d5c7b1-d7fe-4a70-aba1-1131171f55c1\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf"
Nov 04 01:15:47.710674 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.708300    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-credential-socket\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf\" (UID: \"c3d5c7b1-d7fe-4a70-aba1-1131171f55c1\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf"
Nov 04 01:15:47.721390 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.721361    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-istio-token\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf\" (UID: \"c3d5c7b1-d7fe-4a70-aba1-1131171f55c1\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf"
Nov 04 01:15:47.721623 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.721585    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-workload-socket\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf\" (UID: \"c3d5c7b1-d7fe-4a70-aba1-1131171f55c1\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf"
Nov 04 01:15:47.824085 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.823843    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj"
Nov 04 01:15:47.856575 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.835047    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-credential-socket\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf\" (UID: \"c3d5c7b1-d7fe-4a70-aba1-1131171f55c1\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf"
Nov 04 01:15:47.856575 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.835131    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-istio-token\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf\" (UID: \"c3d5c7b1-d7fe-4a70-aba1-1131171f55c1\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf"
Nov 04 01:15:47.856575 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.835176    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-workload-socket\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf\" (UID: \"c3d5c7b1-d7fe-4a70-aba1-1131171f55c1\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf"
Nov 04 01:15:47.856575 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.835249    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-istio-envoy\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf\" (UID: \"c3d5c7b1-d7fe-4a70-aba1-1131171f55c1\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf"
Nov 04 01:15:47.856575 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.835294    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwk77\" (UniqueName: \"kubernetes.io/projected/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-kube-api-access-xwk77\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf\" (UID: \"c3d5c7b1-d7fe-4a70-aba1-1131171f55c1\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf"
Nov 04 01:15:47.856575 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.835336    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-istio-podinfo\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf\" (UID: \"c3d5c7b1-d7fe-4a70-aba1-1131171f55c1\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf"
Nov 04 01:15:47.856575 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.835375    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-istio-data\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf\" (UID: \"c3d5c7b1-d7fe-4a70-aba1-1131171f55c1\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf"
Nov 04 01:15:47.856575 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.835403    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-workload-certs\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf\" (UID: \"c3d5c7b1-d7fe-4a70-aba1-1131171f55c1\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf"
Nov 04 01:15:47.856575 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.835435    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-istiod-ca-cert\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf\" (UID: \"c3d5c7b1-d7fe-4a70-aba1-1131171f55c1\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf"
Nov 04 01:15:47.856575 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.836323    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-istiod-ca-cert\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf\" (UID: \"c3d5c7b1-d7fe-4a70-aba1-1131171f55c1\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf"
Nov 04 01:15:47.856575 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.836634    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-credential-socket\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf\" (UID: \"c3d5c7b1-d7fe-4a70-aba1-1131171f55c1\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf"
Nov 04 01:15:47.856575 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.837406    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-workload-socket\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf\" (UID: \"c3d5c7b1-d7fe-4a70-aba1-1131171f55c1\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf"
Nov 04 01:15:47.856575 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.852703    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-istio-envoy\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf\" (UID: \"c3d5c7b1-d7fe-4a70-aba1-1131171f55c1\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf"
Nov 04 01:15:47.856575 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.852968    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-istio-data\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf\" (UID: \"c3d5c7b1-d7fe-4a70-aba1-1131171f55c1\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf"
Nov 04 01:15:47.856575 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.855932    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-workload-certs\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf\" (UID: \"c3d5c7b1-d7fe-4a70-aba1-1131171f55c1\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf"
Nov 04 01:15:47.887634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.887325    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwk77\" (UniqueName: \"kubernetes.io/projected/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-kube-api-access-xwk77\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf\" (UID: \"c3d5c7b1-d7fe-4a70-aba1-1131171f55c1\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf"
Nov 04 01:15:47.900629 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.899971    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-istio-podinfo\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf\" (UID: \"c3d5c7b1-d7fe-4a70-aba1-1131171f55c1\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf"
Nov 04 01:15:47.943122 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:47.942144    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-istio-token\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf\" (UID: \"c3d5c7b1-d7fe-4a70-aba1-1131171f55c1\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf"
Nov 04 01:15:48.122205 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:48.122161    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf"
Nov 04 01:15:48.318761 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:48.318638    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm"]
Nov 04 01:15:48.330105 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:15:48.330070    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf9e7abd2_8737_4357_a778_d6492fd12f3f.slice/crio-d0c3bca9bd0037a535d862cf3791e5d369a2327ed552664ec679423a20063a74 WatchSource:0}: Error finding container d0c3bca9bd0037a535d862cf3791e5d369a2327ed552664ec679423a20063a74: Status 404 returned error can't find the container with id d0c3bca9bd0037a535d862cf3791e5d369a2327ed552664ec679423a20063a74
Nov 04 01:15:48.484035 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:48.483175    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj"]
Nov 04 01:15:48.524806 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:48.524769    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:15:48.524905 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:48.524853    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:15:48.524905 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:48.524897    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:15:48.535552 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:48.535532    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf"
Nov 04 01:15:48.702665 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:48.702253    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf"]
Nov 04 01:15:48.732121 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:48.732091    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj" event={"ID":"239d3dec-a8fd-45e4-82ec-1ed4e1d01c72","Type":"ContainerStarted","Data":"af0d7e2c1f0f7a0db311015326a5474c1b7ec72d11187d0f9c4c9b70921db89c"}
Nov 04 01:15:48.814880 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:15:48.813312    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3d5c7b1_d7fe_4a70_aba1_1131171f55c1.slice/crio-117915303c947da9f6c28c47d5aaa6c92d14ede62c50facf06ad933da5babdc7 WatchSource:0}: Error finding container 117915303c947da9f6c28c47d5aaa6c92d14ede62c50facf06ad933da5babdc7: Status 404 returned error can't find the container with id 117915303c947da9f6c28c47d5aaa6c92d14ede62c50facf06ad933da5babdc7
Nov 04 01:15:48.822659 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:48.817907    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:15:48.822659 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:48.817993    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:15:48.822659 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:48.818035    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:15:48.822659 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:48.817922    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm" event={"ID":"f9e7abd2-8737-4357-a778-d6492fd12f3f","Type":"ContainerStarted","Data":"d0c3bca9bd0037a535d862cf3791e5d369a2327ed552664ec679423a20063a74"}
Nov 04 01:15:49.827702 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:49.827668    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf" event={"ID":"c3d5c7b1-d7fe-4a70-aba1-1131171f55c1","Type":"ContainerStarted","Data":"7cd84fdd31600ea3f367d1c9b49b1a4bdc85f0d9a894fc11d6104fd55766f601"}
Nov 04 01:15:49.828205 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:49.827725    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf" event={"ID":"c3d5c7b1-d7fe-4a70-aba1-1131171f55c1","Type":"ContainerStarted","Data":"117915303c947da9f6c28c47d5aaa6c92d14ede62c50facf06ad933da5babdc7"}
Nov 04 01:15:49.830429 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:49.830402    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm" event={"ID":"f9e7abd2-8737-4357-a778-d6492fd12f3f","Type":"ContainerStarted","Data":"e9df8cb91833ffe22046558b5ecd859e003ce091e8e0d4f84c4970cf2091e993"}
Nov 04 01:15:49.833129 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:49.833105    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj" event={"ID":"239d3dec-a8fd-45e4-82ec-1ed4e1d01c72","Type":"ContainerStarted","Data":"9f3de262a8c7582a7320d9a0685f7ed27c0e5cdc196de4ffea24d76cbfdada64"}
Nov 04 01:15:49.914307 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:49.912059    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm" podStartSLOduration=3.912039982 podStartE2EDuration="3.912039982s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:15:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:15:49.907013361 +0000 UTC m=+789.953574162" watchObservedRunningTime="2026-11-04 01:15:49.912039982 +0000 UTC m=+789.958600777"
Nov 04 01:15:49.915059 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:49.915003    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf" podStartSLOduration=2.914990472 podStartE2EDuration="2.914990472s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:15:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:15:49.867785476 +0000 UTC m=+789.914346299" watchObservedRunningTime="2026-11-04 01:15:49.914990472 +0000 UTC m=+789.961551260"
Nov 04 01:15:49.933750 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:49.933701    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj" podStartSLOduration=2.933679736 podStartE2EDuration="2.933679736s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:15:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:15:49.930365627 +0000 UTC m=+789.976926415" watchObservedRunningTime="2026-11-04 01:15:49.933679736 +0000 UTC m=+789.980240524"
Nov 04 01:15:50.126711 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:50.126629    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf"
Nov 04 01:15:50.128010 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:50.127980    5068 patch_prober.go:29] interesting pod/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Nov 04 01:15:50.128133 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:50.128026    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf" podUID="c3d5c7b1-d7fe-4a70-aba1-1131171f55c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Nov 04 01:15:50.357636 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:50.353723    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm"
Nov 04 01:15:50.829979 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:50.829946    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj"
Nov 04 01:15:50.831638 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:50.831590    5068 patch_prober.go:29] interesting pod/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 04 01:15:50.831862 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:50.831840    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj" podUID="239d3dec-a8fd-45e4-82ec-1ed4e1d01c72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 04 01:15:51.124008 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:51.123934    5068 patch_prober.go:29] interesting pod/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Nov 04 01:15:51.124008 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:51.123985    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf" podUID="c3d5c7b1-d7fe-4a70-aba1-1131171f55c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Nov 04 01:15:51.830267 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:51.830239    5068 patch_prober.go:29] interesting pod/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 04 01:15:51.830744 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:51.830713    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj" podUID="239d3dec-a8fd-45e4-82ec-1ed4e1d01c72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 04 01:15:52.123389 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:52.123304    5068 patch_prober.go:29] interesting pod/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Nov 04 01:15:52.123389 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:52.123352    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf" podUID="c3d5c7b1-d7fe-4a70-aba1-1131171f55c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Nov 04 01:15:52.829556 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:52.829517    5068 patch_prober.go:29] interesting pod/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 04 01:15:52.829756 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:52.829579    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj" podUID="239d3dec-a8fd-45e4-82ec-1ed4e1d01c72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 04 01:15:53.123783 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:53.123240    5068 patch_prober.go:29] interesting pod/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Nov 04 01:15:53.123783 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:53.123318    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf" podUID="c3d5c7b1-d7fe-4a70-aba1-1131171f55c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Nov 04 01:15:53.829745 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:53.829714    5068 patch_prober.go:29] interesting pod/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 04 01:15:53.829935 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:53.829765    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj" podUID="239d3dec-a8fd-45e4-82ec-1ed4e1d01c72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 04 01:15:54.123110 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:54.123035    5068 patch_prober.go:29] interesting pod/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Nov 04 01:15:54.123110 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:54.123095    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf" podUID="c3d5c7b1-d7fe-4a70-aba1-1131171f55c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Nov 04 01:15:54.829961 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:54.829918    5068 patch_prober.go:29] interesting pod/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 04 01:15:54.830429 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:54.829984    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj" podUID="239d3dec-a8fd-45e4-82ec-1ed4e1d01c72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 04 01:15:55.123449 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:55.123354    5068 patch_prober.go:29] interesting pod/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Nov 04 01:15:55.123449 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:55.123423    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf" podUID="c3d5c7b1-d7fe-4a70-aba1-1131171f55c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Nov 04 01:15:55.868809 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:55.868574    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj"
Nov 04 01:15:55.870375 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:55.870347    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj"
Nov 04 01:15:55.873653 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:55.873028    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj"
Nov 04 01:15:56.128691 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:56.128595    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf"
Nov 04 01:15:56.128935 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:56.128903    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf"
Nov 04 01:15:56.129665 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:56.129647    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf"
Nov 04 01:15:57.351379 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:57.351340    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm"
Nov 04 01:15:57.365072 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:57.365013    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm"
Nov 04 01:15:57.492754 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:57.492718    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8"]
Nov 04 01:15:57.499097 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:57.499060    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8"
Nov 04 01:15:57.501374 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:57.501347    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8"]
Nov 04 01:15:57.503804 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:57.503779    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-875528\"/\"kube-root-ca.crt\""
Nov 04 01:15:57.504019 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:57.503798    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-875528\"/\"openshift-service-ca.crt\""
Nov 04 01:15:57.504138 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:57.503935    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-875528\"/\"distinct-2-w5-875528-kserve-self-signed-certs\""
Nov 04 01:15:57.504138 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:57.504005    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-875528\"/\"default-dockercfg-26cpk\""
Nov 04 01:15:57.641460 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:57.641359    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb9132e5-b90b-46fb-b86c-ca713419906d-home\") pod \"distinct-2-w5-875528-kserve-c595b45c4-rg6h8\" (UID: \"eb9132e5-b90b-46fb-b86c-ca713419906d\") " pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8"
Nov 04 01:15:57.641669 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:57.641477    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb9132e5-b90b-46fb-b86c-ca713419906d-tls-certs\") pod \"distinct-2-w5-875528-kserve-c595b45c4-rg6h8\" (UID: \"eb9132e5-b90b-46fb-b86c-ca713419906d\") " pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8"
Nov 04 01:15:57.641669 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:57.641519    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb9132e5-b90b-46fb-b86c-ca713419906d-model-cache\") pod \"distinct-2-w5-875528-kserve-c595b45c4-rg6h8\" (UID: \"eb9132e5-b90b-46fb-b86c-ca713419906d\") " pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8"
Nov 04 01:15:57.641669 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:57.641582    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vddw\" (UniqueName: \"kubernetes.io/projected/eb9132e5-b90b-46fb-b86c-ca713419906d-kube-api-access-5vddw\") pod \"distinct-2-w5-875528-kserve-c595b45c4-rg6h8\" (UID: \"eb9132e5-b90b-46fb-b86c-ca713419906d\") " pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8"
Nov 04 01:15:57.641883 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:57.641673    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb9132e5-b90b-46fb-b86c-ca713419906d-dshm\") pod \"distinct-2-w5-875528-kserve-c595b45c4-rg6h8\" (UID: \"eb9132e5-b90b-46fb-b86c-ca713419906d\") " pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8"
Nov 04 01:15:57.641883 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:57.641700    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb9132e5-b90b-46fb-b86c-ca713419906d-tmp-dir\") pod \"distinct-2-w5-875528-kserve-c595b45c4-rg6h8\" (UID: \"eb9132e5-b90b-46fb-b86c-ca713419906d\") " pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8"
Nov 04 01:15:57.742795 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:57.742763    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb9132e5-b90b-46fb-b86c-ca713419906d-model-cache\") pod \"distinct-2-w5-875528-kserve-c595b45c4-rg6h8\" (UID: \"eb9132e5-b90b-46fb-b86c-ca713419906d\") " pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8"
Nov 04 01:15:57.743007 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:57.742835    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vddw\" (UniqueName: \"kubernetes.io/projected/eb9132e5-b90b-46fb-b86c-ca713419906d-kube-api-access-5vddw\") pod \"distinct-2-w5-875528-kserve-c595b45c4-rg6h8\" (UID: \"eb9132e5-b90b-46fb-b86c-ca713419906d\") " pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8"
Nov 04 01:15:57.743007 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:57.742905    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb9132e5-b90b-46fb-b86c-ca713419906d-dshm\") pod \"distinct-2-w5-875528-kserve-c595b45c4-rg6h8\" (UID: \"eb9132e5-b90b-46fb-b86c-ca713419906d\") " pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8"
Nov 04 01:15:57.743007 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:57.742927    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb9132e5-b90b-46fb-b86c-ca713419906d-tmp-dir\") pod \"distinct-2-w5-875528-kserve-c595b45c4-rg6h8\" (UID: \"eb9132e5-b90b-46fb-b86c-ca713419906d\") " pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8"
Nov 04 01:15:57.743165 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:57.743008    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb9132e5-b90b-46fb-b86c-ca713419906d-home\") pod \"distinct-2-w5-875528-kserve-c595b45c4-rg6h8\" (UID: \"eb9132e5-b90b-46fb-b86c-ca713419906d\") " pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8"
Nov 04 01:15:57.743165 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:57.743102    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb9132e5-b90b-46fb-b86c-ca713419906d-tls-certs\") pod \"distinct-2-w5-875528-kserve-c595b45c4-rg6h8\" (UID: \"eb9132e5-b90b-46fb-b86c-ca713419906d\") " pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8"
Nov 04 01:15:57.743277 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:57.743245    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb9132e5-b90b-46fb-b86c-ca713419906d-model-cache\") pod \"distinct-2-w5-875528-kserve-c595b45c4-rg6h8\" (UID: \"eb9132e5-b90b-46fb-b86c-ca713419906d\") " pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8"
Nov 04 01:15:57.743448 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:57.743421    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb9132e5-b90b-46fb-b86c-ca713419906d-tmp-dir\") pod \"distinct-2-w5-875528-kserve-c595b45c4-rg6h8\" (UID: \"eb9132e5-b90b-46fb-b86c-ca713419906d\") " pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8"
Nov 04 01:15:57.743701 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:57.743675    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb9132e5-b90b-46fb-b86c-ca713419906d-home\") pod \"distinct-2-w5-875528-kserve-c595b45c4-rg6h8\" (UID: \"eb9132e5-b90b-46fb-b86c-ca713419906d\") " pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8"
Nov 04 01:15:57.745733 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:57.745710    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb9132e5-b90b-46fb-b86c-ca713419906d-dshm\") pod \"distinct-2-w5-875528-kserve-c595b45c4-rg6h8\" (UID: \"eb9132e5-b90b-46fb-b86c-ca713419906d\") " pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8"
Nov 04 01:15:57.746258 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:57.746234    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb9132e5-b90b-46fb-b86c-ca713419906d-tls-certs\") pod \"distinct-2-w5-875528-kserve-c595b45c4-rg6h8\" (UID: \"eb9132e5-b90b-46fb-b86c-ca713419906d\") " pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8"
Nov 04 01:15:57.755579 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:57.755552    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vddw\" (UniqueName: \"kubernetes.io/projected/eb9132e5-b90b-46fb-b86c-ca713419906d-kube-api-access-5vddw\") pod \"distinct-2-w5-875528-kserve-c595b45c4-rg6h8\" (UID: \"eb9132e5-b90b-46fb-b86c-ca713419906d\") " pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8"
Nov 04 01:15:57.829032 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:57.829001    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8"
Nov 04 01:15:57.993094 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:57.985519    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm"
Nov 04 01:15:58.126193 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:58.126166    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8"]
Nov 04 01:15:58.127408 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:15:58.127386    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeb9132e5_b90b_46fb_b86c_ca713419906d.slice/crio-000556ba2d7d1aa6d188040a4b6a38ead1078b6e3f6825d30b74f875e447ac43 WatchSource:0}: Error finding container 000556ba2d7d1aa6d188040a4b6a38ead1078b6e3f6825d30b74f875e447ac43: Status 404 returned error can't find the container with id 000556ba2d7d1aa6d188040a4b6a38ead1078b6e3f6825d30b74f875e447ac43
Nov 04 01:15:58.970288 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:58.970252    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8" event={"ID":"eb9132e5-b90b-46fb-b86c-ca713419906d","Type":"ContainerStarted","Data":"13561f0b11491d1e4e982cc0706fd0cbd8391a6375e8139f534e65f151e709a8"}
Nov 04 01:15:58.970746 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:58.970299    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8" event={"ID":"eb9132e5-b90b-46fb-b86c-ca713419906d","Type":"ContainerStarted","Data":"000556ba2d7d1aa6d188040a4b6a38ead1078b6e3f6825d30b74f875e447ac43"}
Nov 04 01:15:58.990049 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:58.989999    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8" podStartSLOduration=1.9899630560000001 podStartE2EDuration="1.989963056s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:15:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:15:58.988881102 +0000 UTC m=+799.035441893" watchObservedRunningTime="2026-11-04 01:15:58.989963056 +0000 UTC m=+799.036523867"
Nov 04 01:15:59.829286 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:15:59.829250    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8"
Nov 04 01:16:00.413718 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:00.413682    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89"]
Nov 04 01:16:00.417953 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:00.417931    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89"
Nov 04 01:16:00.425693 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:00.425541    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-cffa80\"/\"openshift-service-ca.crt\""
Nov 04 01:16:00.428528 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:00.428506    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-cffa80\"/\"default-dockercfg-qvhgx\""
Nov 04 01:16:00.429098 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:00.428591    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-cffa80\"/\"distinct-w2-cffa80-kserve-self-signed-certs\""
Nov 04 01:16:00.429958 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:00.429939    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89"]
Nov 04 01:16:00.524720 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:00.524691    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-cffa80\"/\"kube-root-ca.crt\""
Nov 04 01:16:00.600631 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:00.599350    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d-tls-certs\") pod \"distinct-w2-cffa80-kserve-64996cfcbc-z7w89\" (UID: \"6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d\") " pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89"
Nov 04 01:16:00.600631 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:00.599406    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d-home\") pod \"distinct-w2-cffa80-kserve-64996cfcbc-z7w89\" (UID: \"6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d\") " pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89"
Nov 04 01:16:00.600631 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:00.599472    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d-tmp-dir\") pod \"distinct-w2-cffa80-kserve-64996cfcbc-z7w89\" (UID: \"6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d\") " pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89"
Nov 04 01:16:00.600631 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:00.599508    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xc45k\" (UniqueName: \"kubernetes.io/projected/6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d-kube-api-access-xc45k\") pod \"distinct-w2-cffa80-kserve-64996cfcbc-z7w89\" (UID: \"6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d\") " pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89"
Nov 04 01:16:00.600631 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:00.599557    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d-model-cache\") pod \"distinct-w2-cffa80-kserve-64996cfcbc-z7w89\" (UID: \"6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d\") " pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89"
Nov 04 01:16:00.600631 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:00.599710    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d-dshm\") pod \"distinct-w2-cffa80-kserve-64996cfcbc-z7w89\" (UID: \"6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d\") " pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89"
Nov 04 01:16:00.701332 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:00.701244    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d-dshm\") pod \"distinct-w2-cffa80-kserve-64996cfcbc-z7w89\" (UID: \"6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d\") " pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89"
Nov 04 01:16:00.701496 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:00.701340    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d-tls-certs\") pod \"distinct-w2-cffa80-kserve-64996cfcbc-z7w89\" (UID: \"6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d\") " pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89"
Nov 04 01:16:00.701496 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:00.701377    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d-home\") pod \"distinct-w2-cffa80-kserve-64996cfcbc-z7w89\" (UID: \"6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d\") " pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89"
Nov 04 01:16:00.701496 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:00.701433    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d-tmp-dir\") pod \"distinct-w2-cffa80-kserve-64996cfcbc-z7w89\" (UID: \"6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d\") " pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89"
Nov 04 01:16:00.701496 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:00.701462    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xc45k\" (UniqueName: \"kubernetes.io/projected/6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d-kube-api-access-xc45k\") pod \"distinct-w2-cffa80-kserve-64996cfcbc-z7w89\" (UID: \"6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d\") " pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89"
Nov 04 01:16:00.701742 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:00.701506    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d-model-cache\") pod \"distinct-w2-cffa80-kserve-64996cfcbc-z7w89\" (UID: \"6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d\") " pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89"
Nov 04 01:16:00.701918 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:00.701894    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d-model-cache\") pod \"distinct-w2-cffa80-kserve-64996cfcbc-z7w89\" (UID: \"6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d\") " pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89"
Nov 04 01:16:00.701997 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:00.701966    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d-tmp-dir\") pod \"distinct-w2-cffa80-kserve-64996cfcbc-z7w89\" (UID: \"6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d\") " pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89"
Nov 04 01:16:00.702126 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:00.702107    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d-home\") pod \"distinct-w2-cffa80-kserve-64996cfcbc-z7w89\" (UID: \"6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d\") " pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89"
Nov 04 01:16:00.703804 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:00.703743    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d-dshm\") pod \"distinct-w2-cffa80-kserve-64996cfcbc-z7w89\" (UID: \"6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d\") " pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89"
Nov 04 01:16:00.704308 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:00.704270    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d-tls-certs\") pod \"distinct-w2-cffa80-kserve-64996cfcbc-z7w89\" (UID: \"6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d\") " pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89"
Nov 04 01:16:00.714711 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:00.714681    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xc45k\" (UniqueName: \"kubernetes.io/projected/6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d-kube-api-access-xc45k\") pod \"distinct-w2-cffa80-kserve-64996cfcbc-z7w89\" (UID: \"6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d\") " pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89"
Nov 04 01:16:00.741490 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:00.741468    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89"
Nov 04 01:16:01.034730 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:01.034705    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89"]
Nov 04 01:16:01.036918 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:16:01.036889    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6cff90d2_d1f7_4dc1_b2be_5ca5c71c125d.slice/crio-4847263e8d8805b13ed0e2bcdba5d2bcb0880fbc41f8d5925b2d13bfc4efd4f5 WatchSource:0}: Error finding container 4847263e8d8805b13ed0e2bcdba5d2bcb0880fbc41f8d5925b2d13bfc4efd4f5: Status 404 returned error can't find the container with id 4847263e8d8805b13ed0e2bcdba5d2bcb0880fbc41f8d5925b2d13bfc4efd4f5
Nov 04 01:16:02.013131 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.007395    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89" event={"ID":"6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d","Type":"ContainerStarted","Data":"4d663e0a2b93d554eeac27ea1992e39fd5390fbca572ce885849dd2d09a9d98c"}
Nov 04 01:16:02.013131 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.007446    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89" event={"ID":"6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d","Type":"ContainerStarted","Data":"4847263e8d8805b13ed0e2bcdba5d2bcb0880fbc41f8d5925b2d13bfc4efd4f5"}
Nov 04 01:16:02.030478 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.030418    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89" podStartSLOduration=2.030399134 podStartE2EDuration="2.030399134s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:16:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:16:02.027088828 +0000 UTC m=+802.073649635" watchObservedRunningTime="2026-11-04 01:16:02.030399134 +0000 UTC m=+802.076959925"
Nov 04 01:16:02.407648 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.407529    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64"]
Nov 04 01:16:02.412942 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.412915    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64"
Nov 04 01:16:02.430381 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.430351    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64"]
Nov 04 01:16:02.629744 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.622176    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aae6d603-4415-436c-917f-448a09092e29-istiod-ca-cert\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64\" (UID: \"aae6d603-4415-436c-917f-448a09092e29\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64"
Nov 04 01:16:02.629744 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.622247    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aae6d603-4415-436c-917f-448a09092e29-istio-envoy\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64\" (UID: \"aae6d603-4415-436c-917f-448a09092e29\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64"
Nov 04 01:16:02.629744 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.622277    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aae6d603-4415-436c-917f-448a09092e29-credential-socket\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64\" (UID: \"aae6d603-4415-436c-917f-448a09092e29\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64"
Nov 04 01:16:02.629744 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.622311    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aae6d603-4415-436c-917f-448a09092e29-istio-data\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64\" (UID: \"aae6d603-4415-436c-917f-448a09092e29\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64"
Nov 04 01:16:02.629744 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.622354    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aae6d603-4415-436c-917f-448a09092e29-istio-podinfo\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64\" (UID: \"aae6d603-4415-436c-917f-448a09092e29\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64"
Nov 04 01:16:02.629744 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.622381    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aae6d603-4415-436c-917f-448a09092e29-workload-certs\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64\" (UID: \"aae6d603-4415-436c-917f-448a09092e29\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64"
Nov 04 01:16:02.629744 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.622467    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aae6d603-4415-436c-917f-448a09092e29-istio-token\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64\" (UID: \"aae6d603-4415-436c-917f-448a09092e29\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64"
Nov 04 01:16:02.629744 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.622511    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aae6d603-4415-436c-917f-448a09092e29-workload-socket\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64\" (UID: \"aae6d603-4415-436c-917f-448a09092e29\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64"
Nov 04 01:16:02.629744 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.622582    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfw9r\" (UniqueName: \"kubernetes.io/projected/aae6d603-4415-436c-917f-448a09092e29-kube-api-access-xfw9r\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64\" (UID: \"aae6d603-4415-436c-917f-448a09092e29\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64"
Nov 04 01:16:02.629744 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.626025    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg"]
Nov 04 01:16:02.645284 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.643436    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg"]
Nov 04 01:16:02.645284 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.643550    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg"
Nov 04 01:16:02.724968 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.724918    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11913ebb-58e6-4b68-83ba-f4618a61e457-istio-data\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg\" (UID: \"11913ebb-58e6-4b68-83ba-f4618a61e457\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg"
Nov 04 01:16:02.725159 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.725079    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aae6d603-4415-436c-917f-448a09092e29-istiod-ca-cert\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64\" (UID: \"aae6d603-4415-436c-917f-448a09092e29\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64"
Nov 04 01:16:02.725159 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.725126    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aae6d603-4415-436c-917f-448a09092e29-istio-envoy\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64\" (UID: \"aae6d603-4415-436c-917f-448a09092e29\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64"
Nov 04 01:16:02.725159 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.725151    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11913ebb-58e6-4b68-83ba-f4618a61e457-istio-podinfo\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg\" (UID: \"11913ebb-58e6-4b68-83ba-f4618a61e457\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg"
Nov 04 01:16:02.725343 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.725180    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aae6d603-4415-436c-917f-448a09092e29-credential-socket\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64\" (UID: \"aae6d603-4415-436c-917f-448a09092e29\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64"
Nov 04 01:16:02.725343 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.725214    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aae6d603-4415-436c-917f-448a09092e29-istio-data\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64\" (UID: \"aae6d603-4415-436c-917f-448a09092e29\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64"
Nov 04 01:16:02.725343 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.725240    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11913ebb-58e6-4b68-83ba-f4618a61e457-istio-envoy\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg\" (UID: \"11913ebb-58e6-4b68-83ba-f4618a61e457\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg"
Nov 04 01:16:02.725343 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.725271    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11913ebb-58e6-4b68-83ba-f4618a61e457-credential-socket\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg\" (UID: \"11913ebb-58e6-4b68-83ba-f4618a61e457\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg"
Nov 04 01:16:02.725343 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.725301    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8nnk\" (UniqueName: \"kubernetes.io/projected/11913ebb-58e6-4b68-83ba-f4618a61e457-kube-api-access-j8nnk\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg\" (UID: \"11913ebb-58e6-4b68-83ba-f4618a61e457\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg"
Nov 04 01:16:02.725343 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.725330    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aae6d603-4415-436c-917f-448a09092e29-istio-podinfo\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64\" (UID: \"aae6d603-4415-436c-917f-448a09092e29\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64"
Nov 04 01:16:02.725768 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.725358    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11913ebb-58e6-4b68-83ba-f4618a61e457-istio-token\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg\" (UID: \"11913ebb-58e6-4b68-83ba-f4618a61e457\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg"
Nov 04 01:16:02.725768 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.725388    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aae6d603-4415-436c-917f-448a09092e29-workload-certs\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64\" (UID: \"aae6d603-4415-436c-917f-448a09092e29\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64"
Nov 04 01:16:02.725768 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.725468    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11913ebb-58e6-4b68-83ba-f4618a61e457-istiod-ca-cert\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg\" (UID: \"11913ebb-58e6-4b68-83ba-f4618a61e457\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg"
Nov 04 01:16:02.725768 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.725522    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aae6d603-4415-436c-917f-448a09092e29-istio-token\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64\" (UID: \"aae6d603-4415-436c-917f-448a09092e29\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64"
Nov 04 01:16:02.725768 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.725562    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11913ebb-58e6-4b68-83ba-f4618a61e457-workload-certs\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg\" (UID: \"11913ebb-58e6-4b68-83ba-f4618a61e457\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg"
Nov 04 01:16:02.727504 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.725600    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aae6d603-4415-436c-917f-448a09092e29-workload-socket\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64\" (UID: \"aae6d603-4415-436c-917f-448a09092e29\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64"
Nov 04 01:16:02.727504 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.727024    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11913ebb-58e6-4b68-83ba-f4618a61e457-workload-socket\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg\" (UID: \"11913ebb-58e6-4b68-83ba-f4618a61e457\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg"
Nov 04 01:16:02.727504 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.725914    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aae6d603-4415-436c-917f-448a09092e29-istiod-ca-cert\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64\" (UID: \"aae6d603-4415-436c-917f-448a09092e29\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64"
Nov 04 01:16:02.727504 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.726202    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aae6d603-4415-436c-917f-448a09092e29-istio-data\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64\" (UID: \"aae6d603-4415-436c-917f-448a09092e29\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64"
Nov 04 01:16:02.727504 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.726431    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aae6d603-4415-436c-917f-448a09092e29-credential-socket\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64\" (UID: \"aae6d603-4415-436c-917f-448a09092e29\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64"
Nov 04 01:16:02.727504 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.726969    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aae6d603-4415-436c-917f-448a09092e29-workload-socket\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64\" (UID: \"aae6d603-4415-436c-917f-448a09092e29\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64"
Nov 04 01:16:02.727504 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.726148    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aae6d603-4415-436c-917f-448a09092e29-workload-certs\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64\" (UID: \"aae6d603-4415-436c-917f-448a09092e29\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64"
Nov 04 01:16:02.727504 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.727151    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfw9r\" (UniqueName: \"kubernetes.io/projected/aae6d603-4415-436c-917f-448a09092e29-kube-api-access-xfw9r\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64\" (UID: \"aae6d603-4415-436c-917f-448a09092e29\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64"
Nov 04 01:16:02.739657 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.734875    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aae6d603-4415-436c-917f-448a09092e29-istio-podinfo\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64\" (UID: \"aae6d603-4415-436c-917f-448a09092e29\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64"
Nov 04 01:16:02.765640 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.747874    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89"
Nov 04 01:16:02.782644 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.781884    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aae6d603-4415-436c-917f-448a09092e29-istio-envoy\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64\" (UID: \"aae6d603-4415-436c-917f-448a09092e29\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64"
Nov 04 01:16:02.784172 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.784095    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aae6d603-4415-436c-917f-448a09092e29-istio-token\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64\" (UID: \"aae6d603-4415-436c-917f-448a09092e29\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64"
Nov 04 01:16:02.819078 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.818992    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfw9r\" (UniqueName: \"kubernetes.io/projected/aae6d603-4415-436c-917f-448a09092e29-kube-api-access-xfw9r\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64\" (UID: \"aae6d603-4415-436c-917f-448a09092e29\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64"
Nov 04 01:16:02.851288 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.849427    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11913ebb-58e6-4b68-83ba-f4618a61e457-workload-socket\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg\" (UID: \"11913ebb-58e6-4b68-83ba-f4618a61e457\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg"
Nov 04 01:16:02.851288 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.849522    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11913ebb-58e6-4b68-83ba-f4618a61e457-istio-data\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg\" (UID: \"11913ebb-58e6-4b68-83ba-f4618a61e457\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg"
Nov 04 01:16:02.851288 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.849622    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11913ebb-58e6-4b68-83ba-f4618a61e457-istio-podinfo\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg\" (UID: \"11913ebb-58e6-4b68-83ba-f4618a61e457\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg"
Nov 04 01:16:02.851288 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.849656    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11913ebb-58e6-4b68-83ba-f4618a61e457-istio-envoy\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg\" (UID: \"11913ebb-58e6-4b68-83ba-f4618a61e457\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg"
Nov 04 01:16:02.851288 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.849690    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11913ebb-58e6-4b68-83ba-f4618a61e457-credential-socket\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg\" (UID: \"11913ebb-58e6-4b68-83ba-f4618a61e457\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg"
Nov 04 01:16:02.851288 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.849718    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8nnk\" (UniqueName: \"kubernetes.io/projected/11913ebb-58e6-4b68-83ba-f4618a61e457-kube-api-access-j8nnk\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg\" (UID: \"11913ebb-58e6-4b68-83ba-f4618a61e457\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg"
Nov 04 01:16:02.851288 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.849749    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11913ebb-58e6-4b68-83ba-f4618a61e457-istio-token\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg\" (UID: \"11913ebb-58e6-4b68-83ba-f4618a61e457\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg"
Nov 04 01:16:02.851288 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.849817    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11913ebb-58e6-4b68-83ba-f4618a61e457-istiod-ca-cert\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg\" (UID: \"11913ebb-58e6-4b68-83ba-f4618a61e457\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg"
Nov 04 01:16:02.851288 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.849874    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11913ebb-58e6-4b68-83ba-f4618a61e457-workload-certs\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg\" (UID: \"11913ebb-58e6-4b68-83ba-f4618a61e457\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg"
Nov 04 01:16:02.851288 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.850356    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11913ebb-58e6-4b68-83ba-f4618a61e457-workload-certs\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg\" (UID: \"11913ebb-58e6-4b68-83ba-f4618a61e457\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg"
Nov 04 01:16:02.851288 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.850581    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11913ebb-58e6-4b68-83ba-f4618a61e457-workload-socket\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg\" (UID: \"11913ebb-58e6-4b68-83ba-f4618a61e457\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg"
Nov 04 01:16:02.851288 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.850900    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11913ebb-58e6-4b68-83ba-f4618a61e457-istio-data\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg\" (UID: \"11913ebb-58e6-4b68-83ba-f4618a61e457\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg"
Nov 04 01:16:02.855894 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.853882    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11913ebb-58e6-4b68-83ba-f4618a61e457-credential-socket\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg\" (UID: \"11913ebb-58e6-4b68-83ba-f4618a61e457\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg"
Nov 04 01:16:02.855894 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.854966    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11913ebb-58e6-4b68-83ba-f4618a61e457-istiod-ca-cert\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg\" (UID: \"11913ebb-58e6-4b68-83ba-f4618a61e457\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg"
Nov 04 01:16:02.917705 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.903185    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11913ebb-58e6-4b68-83ba-f4618a61e457-istio-podinfo\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg\" (UID: \"11913ebb-58e6-4b68-83ba-f4618a61e457\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg"
Nov 04 01:16:02.917705 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.903682    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11913ebb-58e6-4b68-83ba-f4618a61e457-istio-envoy\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg\" (UID: \"11913ebb-58e6-4b68-83ba-f4618a61e457\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg"
Nov 04 01:16:02.917705 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.909256    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11913ebb-58e6-4b68-83ba-f4618a61e457-istio-token\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg\" (UID: \"11913ebb-58e6-4b68-83ba-f4618a61e457\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg"
Nov 04 01:16:02.931638 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:02.929449    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8nnk\" (UniqueName: \"kubernetes.io/projected/11913ebb-58e6-4b68-83ba-f4618a61e457-kube-api-access-j8nnk\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg\" (UID: \"11913ebb-58e6-4b68-83ba-f4618a61e457\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg"
Nov 04 01:16:03.005173 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:03.004860    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg"
Nov 04 01:16:03.092829 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:03.092795    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64"
Nov 04 01:16:04.030317 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:16:04.028707    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11913ebb_58e6_4b68_83ba_f4618a61e457.slice/crio-189548a7706956037081b13c805ee5bc137331e6edd36b58fd5d1160e1e1371e WatchSource:0}: Error finding container 189548a7706956037081b13c805ee5bc137331e6edd36b58fd5d1160e1e1371e: Status 404 returned error can't find the container with id 189548a7706956037081b13c805ee5bc137331e6edd36b58fd5d1160e1e1371e
Nov 04 01:16:04.080750 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:04.067748    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg"]
Nov 04 01:16:04.384633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:04.382170    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:16:04.384633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:04.382334    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:16:04.384633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:04.382429    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:16:04.513973 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:04.513942    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64"]
Nov 04 01:16:04.558054 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:04.558023    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg" event={"ID":"11913ebb-58e6-4b68-83ba-f4618a61e457","Type":"ContainerStarted","Data":"189548a7706956037081b13c805ee5bc137331e6edd36b58fd5d1160e1e1371e"}
Nov 04 01:16:04.567333 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:04.567299    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:16:04.567799 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:04.567773    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:16:04.567947 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:04.567932    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:16:04.570127 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:04.570099    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf" podUID="32f60f59-a98f-46ba-9fb4-8420f77998da" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:16:05.656472 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:05.656385    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg" event={"ID":"11913ebb-58e6-4b68-83ba-f4618a61e457","Type":"ContainerStarted","Data":"d1da1b14b02280e26da09a101c39b1e54e0fab795030b56bca3d381cbb0e2ab5"}
Nov 04 01:16:05.659818 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:05.659792    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64" event={"ID":"aae6d603-4415-436c-917f-448a09092e29","Type":"ContainerStarted","Data":"1a3e56562a94d0f7be583e9a7c03fc053544819101fa7a4d16b5180779e38379"}
Nov 04 01:16:05.659983 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:05.659969    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64" event={"ID":"aae6d603-4415-436c-917f-448a09092e29","Type":"ContainerStarted","Data":"f0a93a037d23befb62a29b6089dd2c194fa23f282a092e22cbdf7e2015d0cb23"}
Nov 04 01:16:05.689704 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:05.689624    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg" podStartSLOduration=3.689591225 podStartE2EDuration="3.689591225s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:16:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:16:05.685532947 +0000 UTC m=+805.732093760" watchObservedRunningTime="2026-11-04 01:16:05.689591225 +0000 UTC m=+805.736152017"
Nov 04 01:16:06.005735 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:06.005656    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg"
Nov 04 01:16:06.006886 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:06.006854    5068 patch_prober.go:29] interesting pod/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Nov 04 01:16:06.007008 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:06.006913    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg" podUID="11913ebb-58e6-4b68-83ba-f4618a61e457" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Nov 04 01:16:06.093754 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:06.093725    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64"
Nov 04 01:16:06.094832 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:06.094807    5068 patch_prober.go:29] interesting pod/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 04 01:16:06.094955 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:06.094850    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64" podUID="aae6d603-4415-436c-917f-448a09092e29" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 04 01:16:07.006238 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:07.006200    5068 patch_prober.go:29] interesting pod/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Nov 04 01:16:07.006681 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:07.006270    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg" podUID="11913ebb-58e6-4b68-83ba-f4618a61e457" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Nov 04 01:16:07.101641 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:07.093748    5068 patch_prober.go:29] interesting pod/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 04 01:16:07.101641 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:07.093806    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64" podUID="aae6d603-4415-436c-917f-448a09092e29" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 04 01:16:07.829782 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:07.829751    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8"
Nov 04 01:16:07.875998 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:07.875964    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8"
Nov 04 01:16:07.908256 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:07.908197    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64" podStartSLOduration=5.908184616 podStartE2EDuration="5.908184616s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:16:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:16:05.726295234 +0000 UTC m=+805.772856035" watchObservedRunningTime="2026-11-04 01:16:07.908184616 +0000 UTC m=+807.954745406"
Nov 04 01:16:08.006821 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:08.006784    5068 patch_prober.go:29] interesting pod/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Nov 04 01:16:08.007207 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:08.006846    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg" podUID="11913ebb-58e6-4b68-83ba-f4618a61e457" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Nov 04 01:16:08.094821 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:08.094696    5068 patch_prober.go:29] interesting pod/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 04 01:16:08.094821 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:08.094766    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64" podUID="aae6d603-4415-436c-917f-448a09092e29" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 04 01:16:08.815447 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:08.815417    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8"
Nov 04 01:16:09.007180 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:09.007138    5068 patch_prober.go:29] interesting pod/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Nov 04 01:16:09.007826 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:09.007796    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg" podUID="11913ebb-58e6-4b68-83ba-f4618a61e457" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Nov 04 01:16:09.114683 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:09.113897    5068 patch_prober.go:29] interesting pod/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 04 01:16:09.114683 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:09.113983    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64" podUID="aae6d603-4415-436c-917f-448a09092e29" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 04 01:16:10.010649 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:10.009987    5068 patch_prober.go:29] interesting pod/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Nov 04 01:16:10.010649 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:10.010049    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg" podUID="11913ebb-58e6-4b68-83ba-f4618a61e457" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Nov 04 01:16:10.095644 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:10.094679    5068 patch_prober.go:29] interesting pod/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 04 01:16:10.095644 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:10.094739    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64" podUID="aae6d603-4415-436c-917f-448a09092e29" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 04 01:16:10.746200 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:10.746169    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89"
Nov 04 01:16:10.806315 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:10.806286    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89"
Nov 04 01:16:11.007021 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:11.006938    5068 patch_prober.go:29] interesting pod/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Nov 04 01:16:11.007021 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:11.006994    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg" podUID="11913ebb-58e6-4b68-83ba-f4618a61e457" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Nov 04 01:16:11.094946 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:11.094911    5068 patch_prober.go:29] interesting pod/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 04 01:16:11.095481 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:11.095449    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64" podUID="aae6d603-4415-436c-917f-448a09092e29" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 04 01:16:11.773740 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:11.773708    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89"
Nov 04 01:16:12.006458 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:12.006417    5068 patch_prober.go:29] interesting pod/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Nov 04 01:16:12.006687 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:12.006481    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg" podUID="11913ebb-58e6-4b68-83ba-f4618a61e457" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Nov 04 01:16:12.093808 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:12.093717    5068 patch_prober.go:29] interesting pod/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 04 01:16:12.093808 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:12.093774    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64" podUID="aae6d603-4415-436c-917f-448a09092e29" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 04 01:16:13.088943 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:13.088904    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg"
Nov 04 01:16:13.089751 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:13.089726    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg"
Nov 04 01:16:13.097104 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:13.097083    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg"
Nov 04 01:16:13.169177 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:13.169141    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64"
Nov 04 01:16:13.170178 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:13.170155    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64"
Nov 04 01:16:13.175128 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:13.175107    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64"
Nov 04 01:16:13.452306 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:16:13.452217    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:16:16.981086 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:16.981051    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv"]
Nov 04 01:16:16.990388 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:16.990361    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv"
Nov 04 01:16:17.001205 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.000832    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv"]
Nov 04 01:16:17.169409 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.169372    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3bde9301-d2f8-4820-9044-463e387b3c3d-istio-token\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv\" (UID: \"3bde9301-d2f8-4820-9044-463e387b3c3d\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv"
Nov 04 01:16:17.169409 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.169413    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bde9301-d2f8-4820-9044-463e387b3c3d-workload-socket\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv\" (UID: \"3bde9301-d2f8-4820-9044-463e387b3c3d\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv"
Nov 04 01:16:17.169678 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.169546    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3bde9301-d2f8-4820-9044-463e387b3c3d-workload-certs\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv\" (UID: \"3bde9301-d2f8-4820-9044-463e387b3c3d\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv"
Nov 04 01:16:17.169678 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.169617    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3bde9301-d2f8-4820-9044-463e387b3c3d-istio-podinfo\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv\" (UID: \"3bde9301-d2f8-4820-9044-463e387b3c3d\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv"
Nov 04 01:16:17.169789 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.169673    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bg7bd\" (UniqueName: \"kubernetes.io/projected/3bde9301-d2f8-4820-9044-463e387b3c3d-kube-api-access-bg7bd\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv\" (UID: \"3bde9301-d2f8-4820-9044-463e387b3c3d\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv"
Nov 04 01:16:17.169789 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.169738    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3bde9301-d2f8-4820-9044-463e387b3c3d-istio-data\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv\" (UID: \"3bde9301-d2f8-4820-9044-463e387b3c3d\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv"
Nov 04 01:16:17.169789 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.169757    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3bde9301-d2f8-4820-9044-463e387b3c3d-istiod-ca-cert\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv\" (UID: \"3bde9301-d2f8-4820-9044-463e387b3c3d\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv"
Nov 04 01:16:17.169942 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.169858    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bde9301-d2f8-4820-9044-463e387b3c3d-credential-socket\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv\" (UID: \"3bde9301-d2f8-4820-9044-463e387b3c3d\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv"
Nov 04 01:16:17.169942 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.169892    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3bde9301-d2f8-4820-9044-463e387b3c3d-istio-envoy\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv\" (UID: \"3bde9301-d2f8-4820-9044-463e387b3c3d\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv"
Nov 04 01:16:17.271498 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.271470    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3bde9301-d2f8-4820-9044-463e387b3c3d-istio-podinfo\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv\" (UID: \"3bde9301-d2f8-4820-9044-463e387b3c3d\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv"
Nov 04 01:16:17.271727 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.271505    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bg7bd\" (UniqueName: \"kubernetes.io/projected/3bde9301-d2f8-4820-9044-463e387b3c3d-kube-api-access-bg7bd\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv\" (UID: \"3bde9301-d2f8-4820-9044-463e387b3c3d\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv"
Nov 04 01:16:17.271727 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.271540    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3bde9301-d2f8-4820-9044-463e387b3c3d-istio-data\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv\" (UID: \"3bde9301-d2f8-4820-9044-463e387b3c3d\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv"
Nov 04 01:16:17.271727 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.271566    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3bde9301-d2f8-4820-9044-463e387b3c3d-istiod-ca-cert\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv\" (UID: \"3bde9301-d2f8-4820-9044-463e387b3c3d\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv"
Nov 04 01:16:17.271727 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.271704    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bde9301-d2f8-4820-9044-463e387b3c3d-credential-socket\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv\" (UID: \"3bde9301-d2f8-4820-9044-463e387b3c3d\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv"
Nov 04 01:16:17.271988 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.271784    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3bde9301-d2f8-4820-9044-463e387b3c3d-istio-envoy\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv\" (UID: \"3bde9301-d2f8-4820-9044-463e387b3c3d\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv"
Nov 04 01:16:17.271988 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.271820    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3bde9301-d2f8-4820-9044-463e387b3c3d-istio-token\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv\" (UID: \"3bde9301-d2f8-4820-9044-463e387b3c3d\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv"
Nov 04 01:16:17.271988 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.271850    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bde9301-d2f8-4820-9044-463e387b3c3d-workload-socket\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv\" (UID: \"3bde9301-d2f8-4820-9044-463e387b3c3d\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv"
Nov 04 01:16:17.271988 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.271961    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3bde9301-d2f8-4820-9044-463e387b3c3d-workload-certs\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv\" (UID: \"3bde9301-d2f8-4820-9044-463e387b3c3d\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv"
Nov 04 01:16:17.272173 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.272032    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3bde9301-d2f8-4820-9044-463e387b3c3d-istio-data\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv\" (UID: \"3bde9301-d2f8-4820-9044-463e387b3c3d\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv"
Nov 04 01:16:17.272293 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.272268    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bde9301-d2f8-4820-9044-463e387b3c3d-workload-socket\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv\" (UID: \"3bde9301-d2f8-4820-9044-463e387b3c3d\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv"
Nov 04 01:16:17.272349 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.272320    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3bde9301-d2f8-4820-9044-463e387b3c3d-workload-certs\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv\" (UID: \"3bde9301-d2f8-4820-9044-463e387b3c3d\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv"
Nov 04 01:16:17.272482 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.272456    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3bde9301-d2f8-4820-9044-463e387b3c3d-istiod-ca-cert\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv\" (UID: \"3bde9301-d2f8-4820-9044-463e387b3c3d\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv"
Nov 04 01:16:17.272686 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.272500    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bde9301-d2f8-4820-9044-463e387b3c3d-credential-socket\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv\" (UID: \"3bde9301-d2f8-4820-9044-463e387b3c3d\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv"
Nov 04 01:16:17.274278 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.274247    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3bde9301-d2f8-4820-9044-463e387b3c3d-istio-envoy\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv\" (UID: \"3bde9301-d2f8-4820-9044-463e387b3c3d\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv"
Nov 04 01:16:17.274358 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.274302    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3bde9301-d2f8-4820-9044-463e387b3c3d-istio-podinfo\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv\" (UID: \"3bde9301-d2f8-4820-9044-463e387b3c3d\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv"
Nov 04 01:16:17.283298 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.283275    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3bde9301-d2f8-4820-9044-463e387b3c3d-istio-token\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv\" (UID: \"3bde9301-d2f8-4820-9044-463e387b3c3d\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv"
Nov 04 01:16:17.283395 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.283373    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bg7bd\" (UniqueName: \"kubernetes.io/projected/3bde9301-d2f8-4820-9044-463e387b3c3d-kube-api-access-bg7bd\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv\" (UID: \"3bde9301-d2f8-4820-9044-463e387b3c3d\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv"
Nov 04 01:16:17.313295 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.313272    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv"
Nov 04 01:16:17.322350 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.322331    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c"]
Nov 04 01:16:17.326872 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.326854    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c"
Nov 04 01:16:17.340405 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.340378    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c"]
Nov 04 01:16:17.481630 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.478865    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f777506-99a6-4278-a678-de005b0722e6-workload-certs\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c\" (UID: \"5f777506-99a6-4278-a678-de005b0722e6\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c"
Nov 04 01:16:17.481630 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.478922    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f777506-99a6-4278-a678-de005b0722e6-istiod-ca-cert\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c\" (UID: \"5f777506-99a6-4278-a678-de005b0722e6\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c"
Nov 04 01:16:17.481630 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.479016    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzjgg\" (UniqueName: \"kubernetes.io/projected/5f777506-99a6-4278-a678-de005b0722e6-kube-api-access-qzjgg\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c\" (UID: \"5f777506-99a6-4278-a678-de005b0722e6\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c"
Nov 04 01:16:17.481630 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.479064    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f777506-99a6-4278-a678-de005b0722e6-credential-socket\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c\" (UID: \"5f777506-99a6-4278-a678-de005b0722e6\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c"
Nov 04 01:16:17.481630 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.479113    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f777506-99a6-4278-a678-de005b0722e6-istio-envoy\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c\" (UID: \"5f777506-99a6-4278-a678-de005b0722e6\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c"
Nov 04 01:16:17.481630 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.479197    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f777506-99a6-4278-a678-de005b0722e6-workload-socket\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c\" (UID: \"5f777506-99a6-4278-a678-de005b0722e6\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c"
Nov 04 01:16:17.481630 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.479225    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f777506-99a6-4278-a678-de005b0722e6-istio-podinfo\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c\" (UID: \"5f777506-99a6-4278-a678-de005b0722e6\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c"
Nov 04 01:16:17.481630 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.479272    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f777506-99a6-4278-a678-de005b0722e6-istio-data\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c\" (UID: \"5f777506-99a6-4278-a678-de005b0722e6\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c"
Nov 04 01:16:17.481630 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.479333    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f777506-99a6-4278-a678-de005b0722e6-istio-token\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c\" (UID: \"5f777506-99a6-4278-a678-de005b0722e6\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c"
Nov 04 01:16:17.580038 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.579966    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f777506-99a6-4278-a678-de005b0722e6-istio-envoy\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c\" (UID: \"5f777506-99a6-4278-a678-de005b0722e6\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c"
Nov 04 01:16:17.580038 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.580037    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f777506-99a6-4278-a678-de005b0722e6-workload-socket\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c\" (UID: \"5f777506-99a6-4278-a678-de005b0722e6\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c"
Nov 04 01:16:17.580249 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.580057    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f777506-99a6-4278-a678-de005b0722e6-istio-podinfo\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c\" (UID: \"5f777506-99a6-4278-a678-de005b0722e6\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c"
Nov 04 01:16:17.580249 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.580087    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f777506-99a6-4278-a678-de005b0722e6-istio-data\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c\" (UID: \"5f777506-99a6-4278-a678-de005b0722e6\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c"
Nov 04 01:16:17.580360 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.580341    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f777506-99a6-4278-a678-de005b0722e6-istio-token\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c\" (UID: \"5f777506-99a6-4278-a678-de005b0722e6\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c"
Nov 04 01:16:17.580424 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.580393    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f777506-99a6-4278-a678-de005b0722e6-workload-certs\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c\" (UID: \"5f777506-99a6-4278-a678-de005b0722e6\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c"
Nov 04 01:16:17.580482 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.580427    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f777506-99a6-4278-a678-de005b0722e6-istio-data\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c\" (UID: \"5f777506-99a6-4278-a678-de005b0722e6\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c"
Nov 04 01:16:17.580482 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.580432    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f777506-99a6-4278-a678-de005b0722e6-istiod-ca-cert\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c\" (UID: \"5f777506-99a6-4278-a678-de005b0722e6\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c"
Nov 04 01:16:17.580584 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.580489    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f777506-99a6-4278-a678-de005b0722e6-workload-socket\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c\" (UID: \"5f777506-99a6-4278-a678-de005b0722e6\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c"
Nov 04 01:16:17.580584 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.580560    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzjgg\" (UniqueName: \"kubernetes.io/projected/5f777506-99a6-4278-a678-de005b0722e6-kube-api-access-qzjgg\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c\" (UID: \"5f777506-99a6-4278-a678-de005b0722e6\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c"
Nov 04 01:16:17.580710 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.580634    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f777506-99a6-4278-a678-de005b0722e6-credential-socket\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c\" (UID: \"5f777506-99a6-4278-a678-de005b0722e6\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c"
Nov 04 01:16:17.581000 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.580978    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f777506-99a6-4278-a678-de005b0722e6-credential-socket\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c\" (UID: \"5f777506-99a6-4278-a678-de005b0722e6\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c"
Nov 04 01:16:17.581095 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.581079    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f777506-99a6-4278-a678-de005b0722e6-istiod-ca-cert\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c\" (UID: \"5f777506-99a6-4278-a678-de005b0722e6\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c"
Nov 04 01:16:17.581153 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.581134    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f777506-99a6-4278-a678-de005b0722e6-workload-certs\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c\" (UID: \"5f777506-99a6-4278-a678-de005b0722e6\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c"
Nov 04 01:16:17.582743 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.582721    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f777506-99a6-4278-a678-de005b0722e6-istio-envoy\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c\" (UID: \"5f777506-99a6-4278-a678-de005b0722e6\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c"
Nov 04 01:16:17.582920 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.582906    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f777506-99a6-4278-a678-de005b0722e6-istio-podinfo\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c\" (UID: \"5f777506-99a6-4278-a678-de005b0722e6\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c"
Nov 04 01:16:17.593002 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.592975    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f777506-99a6-4278-a678-de005b0722e6-istio-token\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c\" (UID: \"5f777506-99a6-4278-a678-de005b0722e6\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c"
Nov 04 01:16:17.594219 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.594198    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzjgg\" (UniqueName: \"kubernetes.io/projected/5f777506-99a6-4278-a678-de005b0722e6-kube-api-access-qzjgg\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c\" (UID: \"5f777506-99a6-4278-a678-de005b0722e6\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c"
Nov 04 01:16:17.618338 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:16:17.618314    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bde9301_d2f8_4820_9044_463e387b3c3d.slice/crio-52875d83d3168be23e945f667fced37b28e2ad23c02dd6d4f8984aa03a4959a9 WatchSource:0}: Error finding container 52875d83d3168be23e945f667fced37b28e2ad23c02dd6d4f8984aa03a4959a9: Status 404 returned error can't find the container with id 52875d83d3168be23e945f667fced37b28e2ad23c02dd6d4f8984aa03a4959a9
Nov 04 01:16:17.618432 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.618341    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv"]
Nov 04 01:16:17.621736 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.621700    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:16:17.621829 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.621812    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:16:17.621877 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.621862    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:16:17.685599 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.685575    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c"
Nov 04 01:16:17.881720 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.875311    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv" event={"ID":"3bde9301-d2f8-4820-9044-463e387b3c3d","Type":"ContainerStarted","Data":"02d30ff6c58243a35c9e1fc8ab24caa86f435f0a65fd42b4b9d0f9624cc95537"}
Nov 04 01:16:17.881720 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.875377    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv" event={"ID":"3bde9301-d2f8-4820-9044-463e387b3c3d","Type":"ContainerStarted","Data":"52875d83d3168be23e945f667fced37b28e2ad23c02dd6d4f8984aa03a4959a9"}
Nov 04 01:16:17.901848 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:17.901789    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv" podStartSLOduration=1.901770441 podStartE2EDuration="1.901770441s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:16:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:16:17.898050205 +0000 UTC m=+817.944610994" watchObservedRunningTime="2026-11-04 01:16:17.901770441 +0000 UTC m=+817.948331230"
Nov 04 01:16:18.000353 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:18.000327    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c"]
Nov 04 01:16:18.001299 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:16:18.001276    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f777506_99a6_4278_a678_de005b0722e6.slice/crio-9f4f70f1c6854ca2df53a4a5fd7dbb4b50e069f2acbcde6ae42c7af0ce025102 WatchSource:0}: Error finding container 9f4f70f1c6854ca2df53a4a5fd7dbb4b50e069f2acbcde6ae42c7af0ce025102: Status 404 returned error can't find the container with id 9f4f70f1c6854ca2df53a4a5fd7dbb4b50e069f2acbcde6ae42c7af0ce025102
Nov 04 01:16:18.004682 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:18.004645    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:16:18.004784 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:18.004732    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:16:18.004848 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:18.004786    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:16:18.314092 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:18.314059    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv"
Nov 04 01:16:18.315367 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:18.315345    5068 patch_prober.go:29] interesting pod/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Nov 04 01:16:18.315550 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:18.315528    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv" podUID="3bde9301-d2f8-4820-9044-463e387b3c3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Nov 04 01:16:18.920631 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:18.919798    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c" event={"ID":"5f777506-99a6-4278-a678-de005b0722e6","Type":"ContainerStarted","Data":"1a0fcf14d916e01ec286797ac9a9c930dcdb360aff9738c5bb38be96356d8502"}
Nov 04 01:16:18.920631 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:18.919899    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c" event={"ID":"5f777506-99a6-4278-a678-de005b0722e6","Type":"ContainerStarted","Data":"9f4f70f1c6854ca2df53a4a5fd7dbb4b50e069f2acbcde6ae42c7af0ce025102"}
Nov 04 01:16:18.954633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:18.953648    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c" podStartSLOduration=1.953629132 podStartE2EDuration="1.953629132s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:16:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:16:18.951502989 +0000 UTC m=+818.998063783" watchObservedRunningTime="2026-11-04 01:16:18.953629132 +0000 UTC m=+819.000189922"
Nov 04 01:16:19.315247 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:19.315189    5068 patch_prober.go:29] interesting pod/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Nov 04 01:16:19.315943 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:19.315910    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv" podUID="3bde9301-d2f8-4820-9044-463e387b3c3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Nov 04 01:16:19.689038 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:19.688940    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c"
Nov 04 01:16:19.692751 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:19.692718    5068 patch_prober.go:29] interesting pod/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 04 01:16:19.693003 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:19.692964    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c" podUID="5f777506-99a6-4278-a678-de005b0722e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 04 01:16:20.313990 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:20.313943    5068 patch_prober.go:29] interesting pod/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Nov 04 01:16:20.314169 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:20.313996    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv" podUID="3bde9301-d2f8-4820-9044-463e387b3c3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Nov 04 01:16:20.690690 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:20.688022    5068 patch_prober.go:29] interesting pod/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 04 01:16:20.690690 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:20.688087    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c" podUID="5f777506-99a6-4278-a678-de005b0722e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 04 01:16:21.314121 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:21.314083    5068 patch_prober.go:29] interesting pod/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Nov 04 01:16:21.314398 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:21.314374    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv" podUID="3bde9301-d2f8-4820-9044-463e387b3c3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Nov 04 01:16:21.689687 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:21.689477    5068 patch_prober.go:29] interesting pod/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 04 01:16:21.689687 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:21.689556    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c" podUID="5f777506-99a6-4278-a678-de005b0722e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 04 01:16:22.314203 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:22.314168    5068 patch_prober.go:29] interesting pod/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Nov 04 01:16:22.314652 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:22.314225    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv" podUID="3bde9301-d2f8-4820-9044-463e387b3c3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Nov 04 01:16:22.698687 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:22.694982    5068 patch_prober.go:29] interesting pod/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 04 01:16:22.698687 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:22.695061    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c" podUID="5f777506-99a6-4278-a678-de005b0722e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 04 01:16:23.313981 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:23.313950    5068 patch_prober.go:29] interesting pod/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Nov 04 01:16:23.314184 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:23.314024    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv" podUID="3bde9301-d2f8-4820-9044-463e387b3c3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Nov 04 01:16:23.702684 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:23.697004    5068 patch_prober.go:29] interesting pod/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 04 01:16:23.702684 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:23.697080    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c" podUID="5f777506-99a6-4278-a678-de005b0722e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 04 01:16:24.314094 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:24.314056    5068 patch_prober.go:29] interesting pod/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Nov 04 01:16:24.314277 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:24.314144    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv" podUID="3bde9301-d2f8-4820-9044-463e387b3c3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Nov 04 01:16:24.686111 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:24.686019    5068 patch_prober.go:29] interesting pod/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 04 01:16:24.686111 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:24.686080    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c" podUID="5f777506-99a6-4278-a678-de005b0722e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 04 01:16:25.471467 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:25.471435    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv"
Nov 04 01:16:25.473432 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:25.473411    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv"
Nov 04 01:16:25.475732 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:25.475708    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv"
Nov 04 01:16:25.892344 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:25.892310    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c"
Nov 04 01:16:25.892740 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:25.892715    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c"
Nov 04 01:16:25.893587 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:25.893570    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c"
Nov 04 01:16:27.383359 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:27.383313    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82"]
Nov 04 01:16:27.388424 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:27.388396    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82"]
Nov 04 01:16:27.388583 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:27.388506    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82"
Nov 04 01:16:27.393542 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:27.393516    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-ba844c\"/\"distinct-2-w4-ba844c-kserve-self-signed-certs\""
Nov 04 01:16:27.482388 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:27.482346    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6cdd01de-ff61-494a-bdb3-c67025a49440-home\") pod \"distinct-2-w4-ba844c-kserve-748d585646-m9p82\" (UID: \"6cdd01de-ff61-494a-bdb3-c67025a49440\") " pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82"
Nov 04 01:16:27.482721 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:27.482696    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6cdd01de-ff61-494a-bdb3-c67025a49440-tls-certs\") pod \"distinct-2-w4-ba844c-kserve-748d585646-m9p82\" (UID: \"6cdd01de-ff61-494a-bdb3-c67025a49440\") " pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82"
Nov 04 01:16:27.483010 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:27.482990    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxrl7\" (UniqueName: \"kubernetes.io/projected/6cdd01de-ff61-494a-bdb3-c67025a49440-kube-api-access-lxrl7\") pod \"distinct-2-w4-ba844c-kserve-748d585646-m9p82\" (UID: \"6cdd01de-ff61-494a-bdb3-c67025a49440\") " pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82"
Nov 04 01:16:27.483143 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:27.483127    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6cdd01de-ff61-494a-bdb3-c67025a49440-model-cache\") pod \"distinct-2-w4-ba844c-kserve-748d585646-m9p82\" (UID: \"6cdd01de-ff61-494a-bdb3-c67025a49440\") " pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82"
Nov 04 01:16:27.483260 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:27.483246    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6cdd01de-ff61-494a-bdb3-c67025a49440-tmp-dir\") pod \"distinct-2-w4-ba844c-kserve-748d585646-m9p82\" (UID: \"6cdd01de-ff61-494a-bdb3-c67025a49440\") " pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82"
Nov 04 01:16:27.483387 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:27.483372    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6cdd01de-ff61-494a-bdb3-c67025a49440-dshm\") pod \"distinct-2-w4-ba844c-kserve-748d585646-m9p82\" (UID: \"6cdd01de-ff61-494a-bdb3-c67025a49440\") " pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82"
Nov 04 01:16:27.585436 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:27.585389    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6cdd01de-ff61-494a-bdb3-c67025a49440-tmp-dir\") pod \"distinct-2-w4-ba844c-kserve-748d585646-m9p82\" (UID: \"6cdd01de-ff61-494a-bdb3-c67025a49440\") " pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82"
Nov 04 01:16:27.586163 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:27.586138    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6cdd01de-ff61-494a-bdb3-c67025a49440-dshm\") pod \"distinct-2-w4-ba844c-kserve-748d585646-m9p82\" (UID: \"6cdd01de-ff61-494a-bdb3-c67025a49440\") " pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82"
Nov 04 01:16:27.586839 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:27.586818    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6cdd01de-ff61-494a-bdb3-c67025a49440-home\") pod \"distinct-2-w4-ba844c-kserve-748d585646-m9p82\" (UID: \"6cdd01de-ff61-494a-bdb3-c67025a49440\") " pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82"
Nov 04 01:16:27.587016 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:27.586999    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6cdd01de-ff61-494a-bdb3-c67025a49440-tls-certs\") pod \"distinct-2-w4-ba844c-kserve-748d585646-m9p82\" (UID: \"6cdd01de-ff61-494a-bdb3-c67025a49440\") " pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82"
Nov 04 01:16:27.587272 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:27.587257    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxrl7\" (UniqueName: \"kubernetes.io/projected/6cdd01de-ff61-494a-bdb3-c67025a49440-kube-api-access-lxrl7\") pod \"distinct-2-w4-ba844c-kserve-748d585646-m9p82\" (UID: \"6cdd01de-ff61-494a-bdb3-c67025a49440\") " pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82"
Nov 04 01:16:27.587404 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:27.587378    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6cdd01de-ff61-494a-bdb3-c67025a49440-model-cache\") pod \"distinct-2-w4-ba844c-kserve-748d585646-m9p82\" (UID: \"6cdd01de-ff61-494a-bdb3-c67025a49440\") " pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82"
Nov 04 01:16:27.591460 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:27.586042    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6cdd01de-ff61-494a-bdb3-c67025a49440-tmp-dir\") pod \"distinct-2-w4-ba844c-kserve-748d585646-m9p82\" (UID: \"6cdd01de-ff61-494a-bdb3-c67025a49440\") " pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82"
Nov 04 01:16:27.591914 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:27.591869    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6cdd01de-ff61-494a-bdb3-c67025a49440-home\") pod \"distinct-2-w4-ba844c-kserve-748d585646-m9p82\" (UID: \"6cdd01de-ff61-494a-bdb3-c67025a49440\") " pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82"
Nov 04 01:16:27.595628 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:27.592787    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6cdd01de-ff61-494a-bdb3-c67025a49440-model-cache\") pod \"distinct-2-w4-ba844c-kserve-748d585646-m9p82\" (UID: \"6cdd01de-ff61-494a-bdb3-c67025a49440\") " pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82"
Nov 04 01:16:27.602155 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:27.598036    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6cdd01de-ff61-494a-bdb3-c67025a49440-dshm\") pod \"distinct-2-w4-ba844c-kserve-748d585646-m9p82\" (UID: \"6cdd01de-ff61-494a-bdb3-c67025a49440\") " pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82"
Nov 04 01:16:27.602155 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:27.600248    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6cdd01de-ff61-494a-bdb3-c67025a49440-tls-certs\") pod \"distinct-2-w4-ba844c-kserve-748d585646-m9p82\" (UID: \"6cdd01de-ff61-494a-bdb3-c67025a49440\") " pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82"
Nov 04 01:16:27.604863 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:27.604839    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxrl7\" (UniqueName: \"kubernetes.io/projected/6cdd01de-ff61-494a-bdb3-c67025a49440-kube-api-access-lxrl7\") pod \"distinct-2-w4-ba844c-kserve-748d585646-m9p82\" (UID: \"6cdd01de-ff61-494a-bdb3-c67025a49440\") " pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82"
Nov 04 01:16:27.706207 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:27.706112    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82"
Nov 04 01:16:28.006086 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:28.006055    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82"]
Nov 04 01:16:28.006902 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:16:28.006875    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6cdd01de_ff61_494a_bdb3_c67025a49440.slice/crio-f2e472238d1374a293eceb5065faea219edb94c5f6e30e8777da87018ecdd7ec WatchSource:0}: Error finding container f2e472238d1374a293eceb5065faea219edb94c5f6e30e8777da87018ecdd7ec: Status 404 returned error can't find the container with id f2e472238d1374a293eceb5065faea219edb94c5f6e30e8777da87018ecdd7ec
Nov 04 01:16:28.035055 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:28.035029    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82" event={"ID":"6cdd01de-ff61-494a-bdb3-c67025a49440","Type":"ContainerStarted","Data":"f2e472238d1374a293eceb5065faea219edb94c5f6e30e8777da87018ecdd7ec"}
Nov 04 01:16:28.438514 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:28.438486    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:16:29.046182 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:29.045855    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82" event={"ID":"6cdd01de-ff61-494a-bdb3-c67025a49440","Type":"ContainerStarted","Data":"58804afceec88781291367b70788b6abd8d2a0ab7cfe27e8f62decaf5086d7df"}
Nov 04 01:16:29.068813 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:29.068763    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82" podStartSLOduration=2.068748013 podStartE2EDuration="2.068748013s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:16:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:16:29.064955638 +0000 UTC m=+829.111516429" watchObservedRunningTime="2026-11-04 01:16:29.068748013 +0000 UTC m=+829.115308803"
Nov 04 01:16:29.706389 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:29.706357    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82"
Nov 04 01:16:31.515633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:31.514698    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf" podUID="32f60f59-a98f-46ba-9fb4-8420f77998da" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:16:31.665755 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:31.664698    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9" podUID="530fc578-6622-4c42-a384-02e73bf17b1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:16:31.701716 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:31.700693    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5" podUID="60f5c87a-48f8-4ad9-847b-cff571cc9340" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:16:31.736695 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:31.735711    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8" podUID="eb9132e5-b90b-46fb-b86c-ca713419906d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:16:31.744985 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:31.744934    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89" podUID="6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": context deadline exceeded"
Nov 04 01:16:31.807579 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:31.807453    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx" podUID="46d4ec7b-558d-4fb5-8011-177760a3291c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Nov 04 01:16:31.854739 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:31.853702    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz" podUID="4ccafa61-1c9b-4595-a511-3404d7479526" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:16:31.864803 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:31.864490    5068 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5" podUID="60f5c87a-48f8-4ad9-847b-cff571cc9340" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:16:31.894788 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:31.894749    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n" podUID="b3223811-a857-4279-a671-189e65383481" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:16:32.469031 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.468995    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp"]
Nov 04 01:16:32.515827 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.515776    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf" podUID="32f60f59-a98f-46ba-9fb4-8420f77998da" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:16:32.669635 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.665931    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9" podUID="530fc578-6622-4c42-a384-02e73bf17b1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:16:32.683632 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.677878    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf"
Nov 04 01:16:32.698188 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.683925    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9"
Nov 04 01:16:32.698484 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.698460    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp"]
Nov 04 01:16:32.702632 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.698648    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v"]
Nov 04 01:16:32.707911 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.707887    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v"]
Nov 04 01:16:32.717699 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.701525    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5" podUID="60f5c87a-48f8-4ad9-847b-cff571cc9340" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:16:32.731894 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.726506    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5"
Nov 04 01:16:32.731894 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.713267    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v"
Nov 04 01:16:32.731894 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.684024    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp"
Nov 04 01:16:32.812720 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.812664    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx" podUID="46d4ec7b-558d-4fb5-8011-177760a3291c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Nov 04 01:16:32.901633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.829817    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx"
Nov 04 01:16:32.901633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.832405    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73f6a679-d6e5-45d1-a885-ebea2ed4f253-istio-token\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp\" (UID: \"73f6a679-d6e5-45d1-a885-ebea2ed4f253\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp"
Nov 04 01:16:32.901633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.832449    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73f6a679-d6e5-45d1-a885-ebea2ed4f253-workload-certs\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp\" (UID: \"73f6a679-d6e5-45d1-a885-ebea2ed4f253\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp"
Nov 04 01:16:32.901633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.832481    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73f6a679-d6e5-45d1-a885-ebea2ed4f253-istio-data\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp\" (UID: \"73f6a679-d6e5-45d1-a885-ebea2ed4f253\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp"
Nov 04 01:16:32.901633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.832521    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73f6a679-d6e5-45d1-a885-ebea2ed4f253-istiod-ca-cert\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp\" (UID: \"73f6a679-d6e5-45d1-a885-ebea2ed4f253\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp"
Nov 04 01:16:32.901633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.832582    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b28aa81-39ec-4761-a076-0e5a2db936cd-istio-podinfo\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v\" (UID: \"9b28aa81-39ec-4761-a076-0e5a2db936cd\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v"
Nov 04 01:16:32.901633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.832659    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73f6a679-d6e5-45d1-a885-ebea2ed4f253-workload-socket\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp\" (UID: \"73f6a679-d6e5-45d1-a885-ebea2ed4f253\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp"
Nov 04 01:16:32.901633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.832695    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b28aa81-39ec-4761-a076-0e5a2db936cd-istio-data\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v\" (UID: \"9b28aa81-39ec-4761-a076-0e5a2db936cd\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v"
Nov 04 01:16:32.901633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.832749    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73f6a679-d6e5-45d1-a885-ebea2ed4f253-credential-socket\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp\" (UID: \"73f6a679-d6e5-45d1-a885-ebea2ed4f253\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp"
Nov 04 01:16:32.901633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.832793    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b28aa81-39ec-4761-a076-0e5a2db936cd-workload-certs\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v\" (UID: \"9b28aa81-39ec-4761-a076-0e5a2db936cd\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v"
Nov 04 01:16:32.901633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.832822    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73f6a679-d6e5-45d1-a885-ebea2ed4f253-istio-envoy\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp\" (UID: \"73f6a679-d6e5-45d1-a885-ebea2ed4f253\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp"
Nov 04 01:16:32.901633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.832850    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z828v\" (UniqueName: \"kubernetes.io/projected/73f6a679-d6e5-45d1-a885-ebea2ed4f253-kube-api-access-z828v\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp\" (UID: \"73f6a679-d6e5-45d1-a885-ebea2ed4f253\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp"
Nov 04 01:16:32.901633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.832893    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b28aa81-39ec-4761-a076-0e5a2db936cd-credential-socket\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v\" (UID: \"9b28aa81-39ec-4761-a076-0e5a2db936cd\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v"
Nov 04 01:16:32.901633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.832969    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4f2q\" (UniqueName: \"kubernetes.io/projected/9b28aa81-39ec-4761-a076-0e5a2db936cd-kube-api-access-p4f2q\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v\" (UID: \"9b28aa81-39ec-4761-a076-0e5a2db936cd\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v"
Nov 04 01:16:32.901633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.833064    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b28aa81-39ec-4761-a076-0e5a2db936cd-istio-token\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v\" (UID: \"9b28aa81-39ec-4761-a076-0e5a2db936cd\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v"
Nov 04 01:16:32.901633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.833154    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b28aa81-39ec-4761-a076-0e5a2db936cd-workload-socket\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v\" (UID: \"9b28aa81-39ec-4761-a076-0e5a2db936cd\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v"
Nov 04 01:16:32.901633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.833179    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b28aa81-39ec-4761-a076-0e5a2db936cd-istiod-ca-cert\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v\" (UID: \"9b28aa81-39ec-4761-a076-0e5a2db936cd\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v"
Nov 04 01:16:32.901633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.833210    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73f6a679-d6e5-45d1-a885-ebea2ed4f253-istio-podinfo\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp\" (UID: \"73f6a679-d6e5-45d1-a885-ebea2ed4f253\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp"
Nov 04 01:16:32.901633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.833275    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b28aa81-39ec-4761-a076-0e5a2db936cd-istio-envoy\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v\" (UID: \"9b28aa81-39ec-4761-a076-0e5a2db936cd\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v"
Nov 04 01:16:32.901633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.859891    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz" podUID="4ccafa61-1c9b-4595-a511-3404d7479526" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:16:32.901633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.861362    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz"
Nov 04 01:16:32.901633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.879074    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947"]
Nov 04 01:16:32.932822 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.932516    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947"]
Nov 04 01:16:32.932822 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.932662    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947"
Nov 04 01:16:32.966647 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.964929    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b28aa81-39ec-4761-a076-0e5a2db936cd-istio-podinfo\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v\" (UID: \"9b28aa81-39ec-4761-a076-0e5a2db936cd\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v"
Nov 04 01:16:32.966647 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.964988    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73f6a679-d6e5-45d1-a885-ebea2ed4f253-workload-socket\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp\" (UID: \"73f6a679-d6e5-45d1-a885-ebea2ed4f253\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp"
Nov 04 01:16:32.966647 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.965029    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b28aa81-39ec-4761-a076-0e5a2db936cd-istio-data\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v\" (UID: \"9b28aa81-39ec-4761-a076-0e5a2db936cd\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v"
Nov 04 01:16:32.966647 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.965066    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73f6a679-d6e5-45d1-a885-ebea2ed4f253-credential-socket\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp\" (UID: \"73f6a679-d6e5-45d1-a885-ebea2ed4f253\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp"
Nov 04 01:16:32.966647 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.965113    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b28aa81-39ec-4761-a076-0e5a2db936cd-workload-certs\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v\" (UID: \"9b28aa81-39ec-4761-a076-0e5a2db936cd\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v"
Nov 04 01:16:32.966647 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.965139    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73f6a679-d6e5-45d1-a885-ebea2ed4f253-istio-envoy\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp\" (UID: \"73f6a679-d6e5-45d1-a885-ebea2ed4f253\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp"
Nov 04 01:16:32.966647 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.965166    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z828v\" (UniqueName: \"kubernetes.io/projected/73f6a679-d6e5-45d1-a885-ebea2ed4f253-kube-api-access-z828v\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp\" (UID: \"73f6a679-d6e5-45d1-a885-ebea2ed4f253\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp"
Nov 04 01:16:32.966647 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.965207    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b28aa81-39ec-4761-a076-0e5a2db936cd-credential-socket\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v\" (UID: \"9b28aa81-39ec-4761-a076-0e5a2db936cd\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v"
Nov 04 01:16:32.966647 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.965266    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4f2q\" (UniqueName: \"kubernetes.io/projected/9b28aa81-39ec-4761-a076-0e5a2db936cd-kube-api-access-p4f2q\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v\" (UID: \"9b28aa81-39ec-4761-a076-0e5a2db936cd\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v"
Nov 04 01:16:32.966647 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.965390    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b28aa81-39ec-4761-a076-0e5a2db936cd-istio-token\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v\" (UID: \"9b28aa81-39ec-4761-a076-0e5a2db936cd\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v"
Nov 04 01:16:32.966647 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.965493    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b28aa81-39ec-4761-a076-0e5a2db936cd-workload-socket\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v\" (UID: \"9b28aa81-39ec-4761-a076-0e5a2db936cd\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v"
Nov 04 01:16:32.966647 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.965520    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b28aa81-39ec-4761-a076-0e5a2db936cd-istiod-ca-cert\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v\" (UID: \"9b28aa81-39ec-4761-a076-0e5a2db936cd\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v"
Nov 04 01:16:32.966647 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.965550    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73f6a679-d6e5-45d1-a885-ebea2ed4f253-istio-podinfo\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp\" (UID: \"73f6a679-d6e5-45d1-a885-ebea2ed4f253\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp"
Nov 04 01:16:32.966647 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.965599    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b28aa81-39ec-4761-a076-0e5a2db936cd-istio-envoy\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v\" (UID: \"9b28aa81-39ec-4761-a076-0e5a2db936cd\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v"
Nov 04 01:16:32.966647 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.965672    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73f6a679-d6e5-45d1-a885-ebea2ed4f253-istio-token\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp\" (UID: \"73f6a679-d6e5-45d1-a885-ebea2ed4f253\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp"
Nov 04 01:16:32.966647 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.965701    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73f6a679-d6e5-45d1-a885-ebea2ed4f253-workload-certs\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp\" (UID: \"73f6a679-d6e5-45d1-a885-ebea2ed4f253\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp"
Nov 04 01:16:32.966647 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.965723    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73f6a679-d6e5-45d1-a885-ebea2ed4f253-istio-data\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp\" (UID: \"73f6a679-d6e5-45d1-a885-ebea2ed4f253\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp"
Nov 04 01:16:32.966647 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.965787    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73f6a679-d6e5-45d1-a885-ebea2ed4f253-istiod-ca-cert\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp\" (UID: \"73f6a679-d6e5-45d1-a885-ebea2ed4f253\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp"
Nov 04 01:16:32.986917 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.966743    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73f6a679-d6e5-45d1-a885-ebea2ed4f253-istiod-ca-cert\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp\" (UID: \"73f6a679-d6e5-45d1-a885-ebea2ed4f253\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp"
Nov 04 01:16:32.986917 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.977702    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73f6a679-d6e5-45d1-a885-ebea2ed4f253-workload-socket\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp\" (UID: \"73f6a679-d6e5-45d1-a885-ebea2ed4f253\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp"
Nov 04 01:16:32.986917 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.978094    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b28aa81-39ec-4761-a076-0e5a2db936cd-istio-data\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v\" (UID: \"9b28aa81-39ec-4761-a076-0e5a2db936cd\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v"
Nov 04 01:16:32.986917 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.978342    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73f6a679-d6e5-45d1-a885-ebea2ed4f253-credential-socket\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp\" (UID: \"73f6a679-d6e5-45d1-a885-ebea2ed4f253\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp"
Nov 04 01:16:32.986917 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.978584    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b28aa81-39ec-4761-a076-0e5a2db936cd-workload-certs\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v\" (UID: \"9b28aa81-39ec-4761-a076-0e5a2db936cd\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v"
Nov 04 01:16:32.986917 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.978979    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b28aa81-39ec-4761-a076-0e5a2db936cd-credential-socket\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v\" (UID: \"9b28aa81-39ec-4761-a076-0e5a2db936cd\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v"
Nov 04 01:16:33.007669 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.995060    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b28aa81-39ec-4761-a076-0e5a2db936cd-workload-socket\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v\" (UID: \"9b28aa81-39ec-4761-a076-0e5a2db936cd\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v"
Nov 04 01:16:33.007669 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.995370    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73f6a679-d6e5-45d1-a885-ebea2ed4f253-workload-certs\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp\" (UID: \"73f6a679-d6e5-45d1-a885-ebea2ed4f253\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp"
Nov 04 01:16:33.007669 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.996942    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73f6a679-d6e5-45d1-a885-ebea2ed4f253-istio-data\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp\" (UID: \"73f6a679-d6e5-45d1-a885-ebea2ed4f253\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp"
Nov 04 01:16:33.007669 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:32.997961    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b28aa81-39ec-4761-a076-0e5a2db936cd-istiod-ca-cert\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v\" (UID: \"9b28aa81-39ec-4761-a076-0e5a2db936cd\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v"
Nov 04 01:16:33.070051 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:33.066947    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5f098fc-01c1-4e24-8611-e2b6aed954dc-workload-socket\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947\" (UID: \"a5f098fc-01c1-4e24-8611-e2b6aed954dc\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947"
Nov 04 01:16:33.070051 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:33.067059    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5f098fc-01c1-4e24-8611-e2b6aed954dc-istio-token\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947\" (UID: \"a5f098fc-01c1-4e24-8611-e2b6aed954dc\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947"
Nov 04 01:16:33.070051 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:33.067161    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5f098fc-01c1-4e24-8611-e2b6aed954dc-credential-socket\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947\" (UID: \"a5f098fc-01c1-4e24-8611-e2b6aed954dc\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947"
Nov 04 01:16:33.070051 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:33.067242    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5f098fc-01c1-4e24-8611-e2b6aed954dc-workload-certs\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947\" (UID: \"a5f098fc-01c1-4e24-8611-e2b6aed954dc\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947"
Nov 04 01:16:33.070051 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:33.067268    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5f098fc-01c1-4e24-8611-e2b6aed954dc-istio-podinfo\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947\" (UID: \"a5f098fc-01c1-4e24-8611-e2b6aed954dc\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947"
Nov 04 01:16:33.070051 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:33.067322    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5f098fc-01c1-4e24-8611-e2b6aed954dc-istio-data\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947\" (UID: \"a5f098fc-01c1-4e24-8611-e2b6aed954dc\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947"
Nov 04 01:16:33.070051 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:33.067356    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5f098fc-01c1-4e24-8611-e2b6aed954dc-istio-envoy\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947\" (UID: \"a5f098fc-01c1-4e24-8611-e2b6aed954dc\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947"
Nov 04 01:16:33.070051 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:33.067388    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpx7g\" (UniqueName: \"kubernetes.io/projected/a5f098fc-01c1-4e24-8611-e2b6aed954dc-kube-api-access-gpx7g\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947\" (UID: \"a5f098fc-01c1-4e24-8611-e2b6aed954dc\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947"
Nov 04 01:16:33.070051 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:33.067431    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5f098fc-01c1-4e24-8611-e2b6aed954dc-istiod-ca-cert\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947\" (UID: \"a5f098fc-01c1-4e24-8611-e2b6aed954dc\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947"
Nov 04 01:16:33.071561 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:33.071510    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73f6a679-d6e5-45d1-a885-ebea2ed4f253-istio-token\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp\" (UID: \"73f6a679-d6e5-45d1-a885-ebea2ed4f253\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp"
Nov 04 01:16:33.076566 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:33.076542    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b28aa81-39ec-4761-a076-0e5a2db936cd-istio-token\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v\" (UID: \"9b28aa81-39ec-4761-a076-0e5a2db936cd\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v"
Nov 04 01:16:33.107993 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:33.107954    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq" podUID="b87aafac-9ab8-40e5-8d08-72eafc4187dd" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:16:33.111096 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:33.111026    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b28aa81-39ec-4761-a076-0e5a2db936cd-istio-podinfo\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v\" (UID: \"9b28aa81-39ec-4761-a076-0e5a2db936cd\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v"
Nov 04 01:16:33.127644 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:33.113133    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73f6a679-d6e5-45d1-a885-ebea2ed4f253-istio-podinfo\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp\" (UID: \"73f6a679-d6e5-45d1-a885-ebea2ed4f253\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp"
Nov 04 01:16:33.220852 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:33.172673    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5f098fc-01c1-4e24-8611-e2b6aed954dc-istio-token\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947\" (UID: \"a5f098fc-01c1-4e24-8611-e2b6aed954dc\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947"
Nov 04 01:16:33.220852 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:33.172788    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5f098fc-01c1-4e24-8611-e2b6aed954dc-credential-socket\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947\" (UID: \"a5f098fc-01c1-4e24-8611-e2b6aed954dc\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947"
Nov 04 01:16:33.220852 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:33.172868    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5f098fc-01c1-4e24-8611-e2b6aed954dc-workload-certs\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947\" (UID: \"a5f098fc-01c1-4e24-8611-e2b6aed954dc\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947"
Nov 04 01:16:33.220852 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:33.172896    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5f098fc-01c1-4e24-8611-e2b6aed954dc-istio-podinfo\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947\" (UID: \"a5f098fc-01c1-4e24-8611-e2b6aed954dc\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947"
Nov 04 01:16:33.220852 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:33.172947    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5f098fc-01c1-4e24-8611-e2b6aed954dc-istio-data\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947\" (UID: \"a5f098fc-01c1-4e24-8611-e2b6aed954dc\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947"
Nov 04 01:16:33.220852 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:33.172980    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5f098fc-01c1-4e24-8611-e2b6aed954dc-istio-envoy\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947\" (UID: \"a5f098fc-01c1-4e24-8611-e2b6aed954dc\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947"
Nov 04 01:16:33.220852 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:33.173008    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpx7g\" (UniqueName: \"kubernetes.io/projected/a5f098fc-01c1-4e24-8611-e2b6aed954dc-kube-api-access-gpx7g\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947\" (UID: \"a5f098fc-01c1-4e24-8611-e2b6aed954dc\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947"
Nov 04 01:16:33.220852 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:33.173062    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5f098fc-01c1-4e24-8611-e2b6aed954dc-istiod-ca-cert\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947\" (UID: \"a5f098fc-01c1-4e24-8611-e2b6aed954dc\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947"
Nov 04 01:16:33.220852 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:33.173231    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5f098fc-01c1-4e24-8611-e2b6aed954dc-workload-socket\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947\" (UID: \"a5f098fc-01c1-4e24-8611-e2b6aed954dc\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947"
Nov 04 01:16:33.220852 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:33.173841    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5f098fc-01c1-4e24-8611-e2b6aed954dc-workload-socket\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947\" (UID: \"a5f098fc-01c1-4e24-8611-e2b6aed954dc\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947"
Nov 04 01:16:33.220852 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:33.174458    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5f098fc-01c1-4e24-8611-e2b6aed954dc-credential-socket\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947\" (UID: \"a5f098fc-01c1-4e24-8611-e2b6aed954dc\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947"
Nov 04 01:16:33.220852 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:33.174735    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5f098fc-01c1-4e24-8611-e2b6aed954dc-workload-certs\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947\" (UID: \"a5f098fc-01c1-4e24-8611-e2b6aed954dc\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947"
Nov 04 01:16:33.220852 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:33.178958    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5f098fc-01c1-4e24-8611-e2b6aed954dc-istio-data\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947\" (UID: \"a5f098fc-01c1-4e24-8611-e2b6aed954dc\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947"
Nov 04 01:16:33.220852 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:33.179976    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5f098fc-01c1-4e24-8611-e2b6aed954dc-istiod-ca-cert\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947\" (UID: \"a5f098fc-01c1-4e24-8611-e2b6aed954dc\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947"
Nov 04 01:16:33.220852 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:33.182936    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5f098fc-01c1-4e24-8611-e2b6aed954dc-istio-envoy\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947\" (UID: \"a5f098fc-01c1-4e24-8611-e2b6aed954dc\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947"
Nov 04 01:16:33.294704 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:33.257158    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9"
Nov 04 01:16:33.309729 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:33.304352    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpx7g\" (UniqueName: \"kubernetes.io/projected/a5f098fc-01c1-4e24-8611-e2b6aed954dc-kube-api-access-gpx7g\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947\" (UID: \"a5f098fc-01c1-4e24-8611-e2b6aed954dc\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947"
Nov 04 01:16:33.309729 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:33.304886    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5f098fc-01c1-4e24-8611-e2b6aed954dc-istio-token\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947\" (UID: \"a5f098fc-01c1-4e24-8611-e2b6aed954dc\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947"
Nov 04 01:16:33.444028 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:33.443994    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b28aa81-39ec-4761-a076-0e5a2db936cd-istio-envoy\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v\" (UID: \"9b28aa81-39ec-4761-a076-0e5a2db936cd\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v"
Nov 04 01:16:33.446663 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:33.446589    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73f6a679-d6e5-45d1-a885-ebea2ed4f253-istio-envoy\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp\" (UID: \"73f6a679-d6e5-45d1-a885-ebea2ed4f253\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp"
Nov 04 01:16:33.464380 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:33.462393    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4f2q\" (UniqueName: \"kubernetes.io/projected/9b28aa81-39ec-4761-a076-0e5a2db936cd-kube-api-access-p4f2q\") pod \"e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v\" (UID: \"9b28aa81-39ec-4761-a076-0e5a2db936cd\") " pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v"
Nov 04 01:16:33.464380 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:33.464296    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z828v\" (UniqueName: \"kubernetes.io/projected/73f6a679-d6e5-45d1-a885-ebea2ed4f253-kube-api-access-z828v\") pod \"e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp\" (UID: \"73f6a679-d6e5-45d1-a885-ebea2ed4f253\") " pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp"
Nov 04 01:16:33.569946 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:33.548027    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf" podUID="32f60f59-a98f-46ba-9fb4-8420f77998da" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:16:33.576633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:33.576561    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf"
Nov 04 01:16:33.726495 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:33.726367    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp"
Nov 04 01:16:33.896694 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:33.873078    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz" podUID="4ccafa61-1c9b-4595-a511-3404d7479526" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": context deadline exceeded"
Nov 04 01:16:33.896694 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:33.729060    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5f098fc-01c1-4e24-8611-e2b6aed954dc-istio-podinfo\") pod \"e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947\" (UID: \"a5f098fc-01c1-4e24-8611-e2b6aed954dc\") " pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947"
Nov 04 01:16:33.896694 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:33.788049    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5" podUID="60f5c87a-48f8-4ad9-847b-cff571cc9340" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:16:33.986689 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:33.971541    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v"
Nov 04 01:16:33.994645 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:33.993464    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947"
Nov 04 01:16:34.061728 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:34.061693    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz"
Nov 04 01:16:34.091762 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:34.091726    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5"
Nov 04 01:16:34.168933 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:34.128476    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq" podUID="b87aafac-9ab8-40e5-8d08-72eafc4187dd" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Nov 04 01:16:34.457872 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:34.400748    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx" podUID="46d4ec7b-558d-4fb5-8011-177760a3291c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:16:34.599633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:34.588580    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9" podUID="530fc578-6622-4c42-a384-02e73bf17b1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:16:34.599633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:34.588998    5068 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz" podUID="4ccafa61-1c9b-4595-a511-3404d7479526" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:16:34.749166 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:34.739312    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq"
Nov 04 01:16:34.798733 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:34.798689    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8" podUID="eb9132e5-b90b-46fb-b86c-ca713419906d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:16:35.008696 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:35.005719    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm" podUID="f9e7abd2-8737-4357-a778-d6492fd12f3f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": context deadline exceeded"
Nov 04 01:16:35.125966 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:35.125041    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq"
Nov 04 01:16:35.129019 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:35.128994    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx"
Nov 04 01:16:35.393738 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:35.393683    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp"]
Nov 04 01:16:35.398834 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:35.398798    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:16:35.398956 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:35.398866    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:16:35.398956 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:35.398902    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:16:35.429542 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:16:35.429514    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b28aa81_39ec_4761_a076_0e5a2db936cd.slice/crio-b2e29f63f14a2edaa795cfe7ef53f6c3e1f02ff7e46500bb6d5a6ceb3dcb05f1 WatchSource:0}: Error finding container b2e29f63f14a2edaa795cfe7ef53f6c3e1f02ff7e46500bb6d5a6ceb3dcb05f1: Status 404 returned error can't find the container with id b2e29f63f14a2edaa795cfe7ef53f6c3e1f02ff7e46500bb6d5a6ceb3dcb05f1
Nov 04 01:16:35.430367 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:35.430340    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v"]
Nov 04 01:16:35.433255 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:35.433218    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:16:35.433362 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:35.433302    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:16:35.433362 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:35.433336    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:16:35.490843 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:16:35.490815    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5f098fc_01c1_4e24_8611_e2b6aed954dc.slice/crio-039d1dbbfdfb13e1ab8bfe8ce380964f9bfa6796f84c655bedb0bc11b72f3458 WatchSource:0}: Error finding container 039d1dbbfdfb13e1ab8bfe8ce380964f9bfa6796f84c655bedb0bc11b72f3458: Status 404 returned error can't find the container with id 039d1dbbfdfb13e1ab8bfe8ce380964f9bfa6796f84c655bedb0bc11b72f3458
Nov 04 01:16:35.491999 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:35.491967    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947"]
Nov 04 01:16:35.494665 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:35.494362    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:16:35.494665 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:35.494430    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:16:35.494665 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:35.494460    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:16:36.237812 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:36.237771    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947" event={"ID":"a5f098fc-01c1-4e24-8611-e2b6aed954dc","Type":"ContainerStarted","Data":"52538cce5f64b8e64773a33fb18c455f844d3e14e49cbd8b598453fc39e9bb46"}
Nov 04 01:16:36.238361 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:36.238331    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947" event={"ID":"a5f098fc-01c1-4e24-8611-e2b6aed954dc","Type":"ContainerStarted","Data":"039d1dbbfdfb13e1ab8bfe8ce380964f9bfa6796f84c655bedb0bc11b72f3458"}
Nov 04 01:16:36.240727 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:36.240696    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp" event={"ID":"73f6a679-d6e5-45d1-a885-ebea2ed4f253","Type":"ContainerStarted","Data":"e7d6118c5c88d2a64ed465cc3ec0a56bb3879bcf99cc063f749d792e5d7ed748"}
Nov 04 01:16:36.241604 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:36.241583    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp" event={"ID":"73f6a679-d6e5-45d1-a885-ebea2ed4f253","Type":"ContainerStarted","Data":"4021509b22dd20d8bb5caffbbfd20d43a59130465050dfeb0f3eae5c4245131a"}
Nov 04 01:16:36.243791 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:36.243767    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v" event={"ID":"9b28aa81-39ec-4761-a076-0e5a2db936cd","Type":"ContainerStarted","Data":"192b0defb9cd1abc1b2ab2408f5e12dc2ee1612bc6af3470f72a482a2d34d0d2"}
Nov 04 01:16:36.243915 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:36.243811    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v" event={"ID":"9b28aa81-39ec-4761-a076-0e5a2db936cd","Type":"ContainerStarted","Data":"b2e29f63f14a2edaa795cfe7ef53f6c3e1f02ff7e46500bb6d5a6ceb3dcb05f1"}
Nov 04 01:16:36.296357 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:36.296285    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947" podStartSLOduration=4.2962640069999996 podStartE2EDuration="4.296264007s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:16:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:16:36.267735185 +0000 UTC m=+836.314295981" watchObservedRunningTime="2026-11-04 01:16:36.296264007 +0000 UTC m=+836.342824800"
Nov 04 01:16:36.310896 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:36.310828    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v" podStartSLOduration=4.31081181 podStartE2EDuration="4.31081181s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:16:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:16:36.308692433 +0000 UTC m=+836.355253252" watchObservedRunningTime="2026-11-04 01:16:36.31081181 +0000 UTC m=+836.357372601"
Nov 04 01:16:36.332655 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:36.332591    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp" podStartSLOduration=5.332577595 podStartE2EDuration="5.332577595s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:16:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:16:36.330322611 +0000 UTC m=+836.376883418" watchObservedRunningTime="2026-11-04 01:16:36.332577595 +0000 UTC m=+836.379138384"
Nov 04 01:16:36.731185 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:36.731144    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp"
Nov 04 01:16:36.731546 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:36.731522    5068 patch_prober.go:29] interesting pod/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 04 01:16:36.731734 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:36.731701    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp" podUID="73f6a679-d6e5-45d1-a885-ebea2ed4f253" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 04 01:16:37.028680 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:37.019274    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947"
Nov 04 01:16:37.028680 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:37.019347    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v"
Nov 04 01:16:37.028680 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:37.020733    5068 patch_prober.go:29] interesting pod/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 04 01:16:37.028680 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:37.020784    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v" podUID="9b28aa81-39ec-4761-a076-0e5a2db936cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 04 01:16:37.028680 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:37.021084    5068 patch_prober.go:29] interesting pod/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 04 01:16:37.028680 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:37.021118    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947" podUID="a5f098fc-01c1-4e24-8611-e2b6aed954dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 04 01:16:37.710635 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:37.709913    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82"
Nov 04 01:16:37.733634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:37.732995    5068 patch_prober.go:29] interesting pod/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 04 01:16:37.733634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:37.733068    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp" podUID="73f6a679-d6e5-45d1-a885-ebea2ed4f253" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 04 01:16:38.006116 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:38.006023    5068 patch_prober.go:29] interesting pod/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 04 01:16:38.006395 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:38.006365    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v" podUID="9b28aa81-39ec-4761-a076-0e5a2db936cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 04 01:16:38.009976 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:38.009946    5068 patch_prober.go:29] interesting pod/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 04 01:16:38.010169 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:38.010147    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947" podUID="a5f098fc-01c1-4e24-8611-e2b6aed954dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 04 01:16:38.544634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:38.538412    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82"
Nov 04 01:16:38.636968 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:38.636935    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82"
Nov 04 01:16:38.734102 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:38.734068    5068 patch_prober.go:29] interesting pod/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 04 01:16:38.734670 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:38.734636    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp" podUID="73f6a679-d6e5-45d1-a885-ebea2ed4f253" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 04 01:16:38.947693 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:38.909000    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz" podUID="4ccafa61-1c9b-4595-a511-3404d7479526" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:16:38.967743 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:38.967693    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm" podUID="f9e7abd2-8737-4357-a778-d6492fd12f3f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": context deadline exceeded"
Nov 04 01:16:39.008178 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:39.008141    5068 patch_prober.go:29] interesting pod/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 04 01:16:39.008988 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:39.008955    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v" podUID="9b28aa81-39ec-4761-a076-0e5a2db936cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 04 01:16:39.009169 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:39.008400    5068 patch_prober.go:29] interesting pod/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 04 01:16:39.009275 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:39.009255    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947" podUID="a5f098fc-01c1-4e24-8611-e2b6aed954dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 04 01:16:39.612791 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:39.611755    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf" podUID="32f60f59-a98f-46ba-9fb4-8420f77998da" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:16:39.806670 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:39.804992    5068 patch_prober.go:29] interesting pod/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 04 01:16:39.806670 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:39.805076    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp" podUID="73f6a679-d6e5-45d1-a885-ebea2ed4f253" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 04 01:16:40.015724 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:40.015157    5068 patch_prober.go:29] interesting pod/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 04 01:16:40.015724 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:40.015227    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v" podUID="9b28aa81-39ec-4761-a076-0e5a2db936cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 04 01:16:40.016131 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:40.016028    5068 patch_prober.go:29] interesting pod/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 04 01:16:40.016131 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:40.016071    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947" podUID="a5f098fc-01c1-4e24-8611-e2b6aed954dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 04 01:16:40.778632 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:40.771112    5068 patch_prober.go:29] interesting pod/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 04 01:16:40.778632 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:40.771267    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp" podUID="73f6a679-d6e5-45d1-a885-ebea2ed4f253" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 04 01:16:40.790637 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:40.783545    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89" podUID="6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:16:41.022585 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:41.022173    5068 patch_prober.go:29] interesting pod/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 04 01:16:41.022585 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:41.022232    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v" podUID="9b28aa81-39ec-4761-a076-0e5a2db936cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 04 01:16:41.022585 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:41.022291    5068 patch_prober.go:29] interesting pod/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 04 01:16:41.022585 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:41.022319    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947" podUID="a5f098fc-01c1-4e24-8611-e2b6aed954dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 04 01:16:41.258490 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:41.258364    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq" podUID="b87aafac-9ab8-40e5-8d08-72eafc4187dd" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:16:41.744753 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:41.742676    5068 patch_prober.go:29] interesting pod/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 04 01:16:41.744753 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:41.742744    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp" podUID="73f6a679-d6e5-45d1-a885-ebea2ed4f253" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 04 01:16:42.015696 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:42.009220    5068 patch_prober.go:29] interesting pod/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 04 01:16:42.015696 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:42.009281    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v" podUID="9b28aa81-39ec-4761-a076-0e5a2db936cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 04 01:16:42.015696 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:42.009708    5068 patch_prober.go:29] interesting pod/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 04 01:16:42.015696 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:42.009741    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947" podUID="a5f098fc-01c1-4e24-8611-e2b6aed954dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 04 01:16:42.728715 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:42.728675    5068 patch_prober.go:29] interesting pod/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 04 01:16:42.729168 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:42.729146    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp" podUID="73f6a679-d6e5-45d1-a885-ebea2ed4f253" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 04 01:16:43.005711 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:43.005634    5068 patch_prober.go:29] interesting pod/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 04 01:16:43.005917 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:43.005896    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v" podUID="9b28aa81-39ec-4761-a076-0e5a2db936cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 04 01:16:43.007909 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:43.007884    5068 patch_prober.go:29] interesting pod/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 04 01:16:43.008021 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:43.007921    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947" podUID="a5f098fc-01c1-4e24-8611-e2b6aed954dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 04 01:16:43.463324 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:16:43.463281    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:16:43.731179 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:43.728909    5068 patch_prober.go:29] interesting pod/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 04 01:16:43.731179 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:43.728968    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp" podUID="73f6a679-d6e5-45d1-a885-ebea2ed4f253" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 04 01:16:44.263685 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:44.261554    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947"
Nov 04 01:16:44.263685 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:44.263158    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947"
Nov 04 01:16:44.274642 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:44.272835    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947"
Nov 04 01:16:44.795379 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:44.795350    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v"
Nov 04 01:16:44.795913 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:44.795817    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v"
Nov 04 01:16:44.796827 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:44.796804    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v"
Nov 04 01:16:44.838157 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:44.838125    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp"
Nov 04 01:16:44.838916 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:44.838889    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp"
Nov 04 01:16:44.841965 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:16:44.841943    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp"
Nov 04 01:17:02.053634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:02.031003    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp"]
Nov 04 01:17:02.208651 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:02.180742    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp"
Nov 04 01:17:02.392803 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:02.392691    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9" podUID="530fc578-6622-4c42-a384-02e73bf17b1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:02.431882 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:02.430400    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx" podUID="46d4ec7b-558d-4fb5-8011-177760a3291c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:02.488206 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:02.488167    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20fba30d-0142-49f8-8489-957bad3db95e-credential-socket\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp\" (UID: \"20fba30d-0142-49f8-8489-957bad3db95e\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp"
Nov 04 01:17:02.506872 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:02.506831    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20fba30d-0142-49f8-8489-957bad3db95e-istio-token\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp\" (UID: \"20fba30d-0142-49f8-8489-957bad3db95e\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp"
Nov 04 01:17:02.511788 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:02.511755    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f94wk\" (UniqueName: \"kubernetes.io/projected/20fba30d-0142-49f8-8489-957bad3db95e-kube-api-access-f94wk\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp\" (UID: \"20fba30d-0142-49f8-8489-957bad3db95e\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp"
Nov 04 01:17:02.512043 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:02.512023    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20fba30d-0142-49f8-8489-957bad3db95e-workload-certs\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp\" (UID: \"20fba30d-0142-49f8-8489-957bad3db95e\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp"
Nov 04 01:17:02.515851 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:02.515823    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20fba30d-0142-49f8-8489-957bad3db95e-istio-envoy\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp\" (UID: \"20fba30d-0142-49f8-8489-957bad3db95e\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp"
Nov 04 01:17:02.516023 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:02.516005    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20fba30d-0142-49f8-8489-957bad3db95e-istiod-ca-cert\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp\" (UID: \"20fba30d-0142-49f8-8489-957bad3db95e\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp"
Nov 04 01:17:02.516181 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:02.516162    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20fba30d-0142-49f8-8489-957bad3db95e-istio-podinfo\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp\" (UID: \"20fba30d-0142-49f8-8489-957bad3db95e\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp"
Nov 04 01:17:02.516320 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:02.516301    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20fba30d-0142-49f8-8489-957bad3db95e-workload-socket\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp\" (UID: \"20fba30d-0142-49f8-8489-957bad3db95e\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp"
Nov 04 01:17:02.516484 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:02.516468    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20fba30d-0142-49f8-8489-957bad3db95e-istio-data\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp\" (UID: \"20fba30d-0142-49f8-8489-957bad3db95e\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp"
Nov 04 01:17:02.545283 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:02.542868    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82" podUID="6cdd01de-ff61-494a-bdb3-c67025a49440" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": context deadline exceeded"
Nov 04 01:17:02.545283 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:02.544718    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp"]
Nov 04 01:17:02.570840 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:02.570684    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f"]
Nov 04 01:17:02.968278 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:02.964315    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz" podUID="4ccafa61-1c9b-4595-a511-3404d7479526" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": context deadline exceeded"
Nov 04 01:17:02.968278 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:02.964760    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5" podUID="60f5c87a-48f8-4ad9-847b-cff571cc9340" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Nov 04 01:17:02.968278 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:02.965483    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8" podUID="eb9132e5-b90b-46fb-b86c-ca713419906d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded"
Nov 04 01:17:03.001633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:02.986353    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89" podUID="6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:03.001633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:02.989938    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf" podUID="32f60f59-a98f-46ba-9fb4-8420f77998da" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:03.001633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:02.996884    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f"
Nov 04 01:17:03.099347 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.099306    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f"]
Nov 04 01:17:03.142647 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.141392    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm" podUID="f9e7abd2-8737-4357-a778-d6492fd12f3f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:03.212079 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.212028    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f94wk\" (UniqueName: \"kubernetes.io/projected/20fba30d-0142-49f8-8489-957bad3db95e-kube-api-access-f94wk\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp\" (UID: \"20fba30d-0142-49f8-8489-957bad3db95e\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp"
Nov 04 01:17:03.221246 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.221163    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20fba30d-0142-49f8-8489-957bad3db95e-workload-certs\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp\" (UID: \"20fba30d-0142-49f8-8489-957bad3db95e\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp"
Nov 04 01:17:03.221465 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.212348    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20fba30d-0142-49f8-8489-957bad3db95e-workload-certs\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp\" (UID: \"20fba30d-0142-49f8-8489-957bad3db95e\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp"
Nov 04 01:17:03.221724 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.221703    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20fba30d-0142-49f8-8489-957bad3db95e-istio-envoy\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp\" (UID: \"20fba30d-0142-49f8-8489-957bad3db95e\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp"
Nov 04 01:17:03.221876 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.221861    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20fba30d-0142-49f8-8489-957bad3db95e-istiod-ca-cert\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp\" (UID: \"20fba30d-0142-49f8-8489-957bad3db95e\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp"
Nov 04 01:17:03.222012 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.221998    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20fba30d-0142-49f8-8489-957bad3db95e-istio-podinfo\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp\" (UID: \"20fba30d-0142-49f8-8489-957bad3db95e\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp"
Nov 04 01:17:03.222134 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.222119    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20fba30d-0142-49f8-8489-957bad3db95e-workload-socket\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp\" (UID: \"20fba30d-0142-49f8-8489-957bad3db95e\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp"
Nov 04 01:17:03.222296 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.222268    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20fba30d-0142-49f8-8489-957bad3db95e-istio-data\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp\" (UID: \"20fba30d-0142-49f8-8489-957bad3db95e\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp"
Nov 04 01:17:03.222468 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.222455    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20fba30d-0142-49f8-8489-957bad3db95e-credential-socket\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp\" (UID: \"20fba30d-0142-49f8-8489-957bad3db95e\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp"
Nov 04 01:17:03.222773 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.222753    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20fba30d-0142-49f8-8489-957bad3db95e-istio-token\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp\" (UID: \"20fba30d-0142-49f8-8489-957bad3db95e\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp"
Nov 04 01:17:03.230645 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.224972    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20fba30d-0142-49f8-8489-957bad3db95e-workload-socket\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp\" (UID: \"20fba30d-0142-49f8-8489-957bad3db95e\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp"
Nov 04 01:17:03.231123 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.231099    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20fba30d-0142-49f8-8489-957bad3db95e-istio-data\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp\" (UID: \"20fba30d-0142-49f8-8489-957bad3db95e\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp"
Nov 04 01:17:03.231542 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.231521    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20fba30d-0142-49f8-8489-957bad3db95e-credential-socket\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp\" (UID: \"20fba30d-0142-49f8-8489-957bad3db95e\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp"
Nov 04 01:17:03.233817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.233791    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20fba30d-0142-49f8-8489-957bad3db95e-istiod-ca-cert\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp\" (UID: \"20fba30d-0142-49f8-8489-957bad3db95e\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp"
Nov 04 01:17:03.270852 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.270808    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7"]
Nov 04 01:17:03.366102 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.366074    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7"
Nov 04 01:17:03.398636 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.377300    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ca2a53a-ada2-4565-bc5a-66c7eefef997-istio-envoy\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f\" (UID: \"7ca2a53a-ada2-4565-bc5a-66c7eefef997\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f"
Nov 04 01:17:03.398636 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.377411    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ca2a53a-ada2-4565-bc5a-66c7eefef997-istio-data\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f\" (UID: \"7ca2a53a-ada2-4565-bc5a-66c7eefef997\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f"
Nov 04 01:17:03.398636 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.377589    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ca2a53a-ada2-4565-bc5a-66c7eefef997-workload-socket\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f\" (UID: \"7ca2a53a-ada2-4565-bc5a-66c7eefef997\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f"
Nov 04 01:17:03.455565 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.455524    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq" podUID="b87aafac-9ab8-40e5-8d08-72eafc4187dd" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:03.459490 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.459461    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbsxw\" (UniqueName: \"kubernetes.io/projected/7ca2a53a-ada2-4565-bc5a-66c7eefef997-kube-api-access-dbsxw\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f\" (UID: \"7ca2a53a-ada2-4565-bc5a-66c7eefef997\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f"
Nov 04 01:17:03.459730 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.459702    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ca2a53a-ada2-4565-bc5a-66c7eefef997-istio-token\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f\" (UID: \"7ca2a53a-ada2-4565-bc5a-66c7eefef997\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f"
Nov 04 01:17:03.459880 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.459863    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ca2a53a-ada2-4565-bc5a-66c7eefef997-istio-podinfo\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f\" (UID: \"7ca2a53a-ada2-4565-bc5a-66c7eefef997\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f"
Nov 04 01:17:03.460020 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.460002    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ca2a53a-ada2-4565-bc5a-66c7eefef997-credential-socket\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f\" (UID: \"7ca2a53a-ada2-4565-bc5a-66c7eefef997\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f"
Nov 04 01:17:03.460164 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.460149    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ca2a53a-ada2-4565-bc5a-66c7eefef997-workload-certs\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f\" (UID: \"7ca2a53a-ada2-4565-bc5a-66c7eefef997\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f"
Nov 04 01:17:03.460291 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.460278    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ca2a53a-ada2-4565-bc5a-66c7eefef997-istiod-ca-cert\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f\" (UID: \"7ca2a53a-ada2-4565-bc5a-66c7eefef997\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f"
Nov 04 01:17:03.569044 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.563561    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-istiod-ca-cert\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7\" (UID: \"7d1dc8f1-3238-4b12-b579-ec8179bc4e2a\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7"
Nov 04 01:17:03.569044 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.563640    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ca2a53a-ada2-4565-bc5a-66c7eefef997-istio-envoy\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f\" (UID: \"7ca2a53a-ada2-4565-bc5a-66c7eefef997\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f"
Nov 04 01:17:03.569044 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.563764    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ca2a53a-ada2-4565-bc5a-66c7eefef997-istio-data\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f\" (UID: \"7ca2a53a-ada2-4565-bc5a-66c7eefef997\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f"
Nov 04 01:17:03.569044 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.563805    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-credential-socket\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7\" (UID: \"7d1dc8f1-3238-4b12-b579-ec8179bc4e2a\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7"
Nov 04 01:17:03.569044 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.563834    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-istio-podinfo\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7\" (UID: \"7d1dc8f1-3238-4b12-b579-ec8179bc4e2a\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7"
Nov 04 01:17:03.569044 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.563899    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-workload-socket\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7\" (UID: \"7d1dc8f1-3238-4b12-b579-ec8179bc4e2a\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7"
Nov 04 01:17:03.569044 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.563975    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-istio-envoy\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7\" (UID: \"7d1dc8f1-3238-4b12-b579-ec8179bc4e2a\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7"
Nov 04 01:17:03.569044 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.564029    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ca2a53a-ada2-4565-bc5a-66c7eefef997-workload-socket\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f\" (UID: \"7ca2a53a-ada2-4565-bc5a-66c7eefef997\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f"
Nov 04 01:17:03.569044 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.564056    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbsxw\" (UniqueName: \"kubernetes.io/projected/7ca2a53a-ada2-4565-bc5a-66c7eefef997-kube-api-access-dbsxw\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f\" (UID: \"7ca2a53a-ada2-4565-bc5a-66c7eefef997\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f"
Nov 04 01:17:03.569044 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.564084    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ca2a53a-ada2-4565-bc5a-66c7eefef997-istio-token\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f\" (UID: \"7ca2a53a-ada2-4565-bc5a-66c7eefef997\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f"
Nov 04 01:17:03.569044 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.564126    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ca2a53a-ada2-4565-bc5a-66c7eefef997-istio-podinfo\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f\" (UID: \"7ca2a53a-ada2-4565-bc5a-66c7eefef997\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f"
Nov 04 01:17:03.569044 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.564173    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ca2a53a-ada2-4565-bc5a-66c7eefef997-credential-socket\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f\" (UID: \"7ca2a53a-ada2-4565-bc5a-66c7eefef997\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f"
Nov 04 01:17:03.569044 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.564207    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgwvl\" (UniqueName: \"kubernetes.io/projected/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-kube-api-access-xgwvl\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7\" (UID: \"7d1dc8f1-3238-4b12-b579-ec8179bc4e2a\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7"
Nov 04 01:17:03.569044 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.564248    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ca2a53a-ada2-4565-bc5a-66c7eefef997-workload-certs\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f\" (UID: \"7ca2a53a-ada2-4565-bc5a-66c7eefef997\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f"
Nov 04 01:17:03.569044 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.564276    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-istio-token\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7\" (UID: \"7d1dc8f1-3238-4b12-b579-ec8179bc4e2a\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7"
Nov 04 01:17:03.569044 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.564313    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-istio-data\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7\" (UID: \"7d1dc8f1-3238-4b12-b579-ec8179bc4e2a\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7"
Nov 04 01:17:03.569044 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.564347    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-workload-certs\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7\" (UID: \"7d1dc8f1-3238-4b12-b579-ec8179bc4e2a\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7"
Nov 04 01:17:03.569044 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.564375    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ca2a53a-ada2-4565-bc5a-66c7eefef997-istiod-ca-cert\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f\" (UID: \"7ca2a53a-ada2-4565-bc5a-66c7eefef997\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f"
Nov 04 01:17:03.609643 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.598437    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ca2a53a-ada2-4565-bc5a-66c7eefef997-istiod-ca-cert\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f\" (UID: \"7ca2a53a-ada2-4565-bc5a-66c7eefef997\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f"
Nov 04 01:17:03.609643 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.598570    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ca2a53a-ada2-4565-bc5a-66c7eefef997-workload-socket\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f\" (UID: \"7ca2a53a-ada2-4565-bc5a-66c7eefef997\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f"
Nov 04 01:17:03.609643 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.609626    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ca2a53a-ada2-4565-bc5a-66c7eefef997-credential-socket\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f\" (UID: \"7ca2a53a-ada2-4565-bc5a-66c7eefef997\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f"
Nov 04 01:17:03.619657 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.610034    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ca2a53a-ada2-4565-bc5a-66c7eefef997-workload-certs\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f\" (UID: \"7ca2a53a-ada2-4565-bc5a-66c7eefef997\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f"
Nov 04 01:17:03.645690 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.635561    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ca2a53a-ada2-4565-bc5a-66c7eefef997-istio-data\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f\" (UID: \"7ca2a53a-ada2-4565-bc5a-66c7eefef997\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f"
Nov 04 01:17:03.645690 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.636837    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n" podUID="b3223811-a857-4279-a671-189e65383481" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:03.660765 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.660235    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ca2a53a-ada2-4565-bc5a-66c7eefef997-istio-podinfo\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f\" (UID: \"7ca2a53a-ada2-4565-bc5a-66c7eefef997\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f"
Nov 04 01:17:03.664709 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.664672    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ca2a53a-ada2-4565-bc5a-66c7eefef997-istio-token\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f\" (UID: \"7ca2a53a-ada2-4565-bc5a-66c7eefef997\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f"
Nov 04 01:17:03.665990 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.665956    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-credential-socket\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7\" (UID: \"7d1dc8f1-3238-4b12-b579-ec8179bc4e2a\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7"
Nov 04 01:17:03.666186 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.666166    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-istio-podinfo\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7\" (UID: \"7d1dc8f1-3238-4b12-b579-ec8179bc4e2a\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7"
Nov 04 01:17:03.666366 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.666349    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-workload-socket\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7\" (UID: \"7d1dc8f1-3238-4b12-b579-ec8179bc4e2a\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7"
Nov 04 01:17:03.666548 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.666527    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-istio-envoy\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7\" (UID: \"7d1dc8f1-3238-4b12-b579-ec8179bc4e2a\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7"
Nov 04 01:17:03.672641 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.666752    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgwvl\" (UniqueName: \"kubernetes.io/projected/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-kube-api-access-xgwvl\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7\" (UID: \"7d1dc8f1-3238-4b12-b579-ec8179bc4e2a\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7"
Nov 04 01:17:03.672858 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.672835    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-istio-token\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7\" (UID: \"7d1dc8f1-3238-4b12-b579-ec8179bc4e2a\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7"
Nov 04 01:17:03.673016 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.672997    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-istio-data\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7\" (UID: \"7d1dc8f1-3238-4b12-b579-ec8179bc4e2a\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7"
Nov 04 01:17:03.673155 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.673135    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-workload-certs\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7\" (UID: \"7d1dc8f1-3238-4b12-b579-ec8179bc4e2a\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7"
Nov 04 01:17:03.676636 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.673395    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-istiod-ca-cert\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7\" (UID: \"7d1dc8f1-3238-4b12-b579-ec8179bc4e2a\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7"
Nov 04 01:17:03.775693 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.701680    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-credential-socket\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7\" (UID: \"7d1dc8f1-3238-4b12-b579-ec8179bc4e2a\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7"
Nov 04 01:17:03.775693 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.701964    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-workload-certs\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7\" (UID: \"7d1dc8f1-3238-4b12-b579-ec8179bc4e2a\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7"
Nov 04 01:17:03.775693 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.702202    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-workload-socket\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7\" (UID: \"7d1dc8f1-3238-4b12-b579-ec8179bc4e2a\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7"
Nov 04 01:17:03.775693 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.704219    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-istiod-ca-cert\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7\" (UID: \"7d1dc8f1-3238-4b12-b579-ec8179bc4e2a\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7"
Nov 04 01:17:03.780895 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.779668    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7"]
Nov 04 01:17:03.784645 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.781591    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgwvl\" (UniqueName: \"kubernetes.io/projected/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-kube-api-access-xgwvl\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7\" (UID: \"7d1dc8f1-3238-4b12-b579-ec8179bc4e2a\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7"
Nov 04 01:17:03.799160 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.799128    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-istio-data\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7\" (UID: \"7d1dc8f1-3238-4b12-b579-ec8179bc4e2a\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7"
Nov 04 01:17:03.853636 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.850645    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-istio-envoy\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7\" (UID: \"7d1dc8f1-3238-4b12-b579-ec8179bc4e2a\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7"
Nov 04 01:17:03.870316 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.870278    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-istio-podinfo\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7\" (UID: \"7d1dc8f1-3238-4b12-b579-ec8179bc4e2a\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7"
Nov 04 01:17:03.909727 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.888794    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20fba30d-0142-49f8-8489-957bad3db95e-istio-podinfo\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp\" (UID: \"20fba30d-0142-49f8-8489-957bad3db95e\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp"
Nov 04 01:17:03.972632 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.971789    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5" podUID="60f5c87a-48f8-4ad9-847b-cff571cc9340" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Nov 04 01:17:03.972824 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.972628    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8" podUID="eb9132e5-b90b-46fb-b86c-ca713419906d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded"
Nov 04 01:17:03.974639 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.974056    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5"
Nov 04 01:17:03.974639 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:03.974117    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8"
Nov 04 01:17:04.141717 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:04.001682    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz" podUID="4ccafa61-1c9b-4595-a511-3404d7479526" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:04.141717 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:04.002324    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz"
Nov 04 01:17:04.141717 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:04.002887    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf" podUID="32f60f59-a98f-46ba-9fb4-8420f77998da" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded"
Nov 04 01:17:04.141717 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:04.003112    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf"
Nov 04 01:17:04.141717 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:04.012811    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89" podUID="6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:04.141717 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:04.057890    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89"
Nov 04 01:17:04.141717 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:04.139021    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5"
Nov 04 01:17:04.144044 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:04.143681    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm" podUID="f9e7abd2-8737-4357-a778-d6492fd12f3f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:04.240839 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:04.240781    5068 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz" podUID="4ccafa61-1c9b-4595-a511-3404d7479526" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:04.264563 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:04.258802    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9" podUID="530fc578-6622-4c42-a384-02e73bf17b1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:04.384642 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:04.381404    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbsxw\" (UniqueName: \"kubernetes.io/projected/7ca2a53a-ada2-4565-bc5a-66c7eefef997-kube-api-access-dbsxw\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f\" (UID: \"7ca2a53a-ada2-4565-bc5a-66c7eefef997\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f"
Nov 04 01:17:04.384642 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:04.382038    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f94wk\" (UniqueName: \"kubernetes.io/projected/20fba30d-0142-49f8-8489-957bad3db95e-kube-api-access-f94wk\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp\" (UID: \"20fba30d-0142-49f8-8489-957bad3db95e\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp"
Nov 04 01:17:04.384642 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:04.382256    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20fba30d-0142-49f8-8489-957bad3db95e-istio-envoy\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp\" (UID: \"20fba30d-0142-49f8-8489-957bad3db95e\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp"
Nov 04 01:17:04.384642 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:04.382707    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20fba30d-0142-49f8-8489-957bad3db95e-istio-token\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp\" (UID: \"20fba30d-0142-49f8-8489-957bad3db95e\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp"
Nov 04 01:17:04.400587 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:04.389288    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm"
Nov 04 01:17:04.469966 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:04.468787    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8"
Nov 04 01:17:04.486642 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:04.471129    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq" podUID="b87aafac-9ab8-40e5-8d08-72eafc4187dd" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:04.486642 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:04.471555    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx" podUID="46d4ec7b-558d-4fb5-8011-177760a3291c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Nov 04 01:17:04.486642 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:04.472154    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq"
Nov 04 01:17:04.486642 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:04.363977    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ca2a53a-ada2-4565-bc5a-66c7eefef997-istio-envoy\") pod \"e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f\" (UID: \"7ca2a53a-ada2-4565-bc5a-66c7eefef997\") " pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f"
Nov 04 01:17:04.487123 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:04.487098    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm"
Nov 04 01:17:04.506632 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:04.505962    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f"
Nov 04 01:17:04.556635 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:04.545973    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82" podUID="6cdd01de-ff61-494a-bdb3-c67025a49440" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:04.593319 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:04.592740    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-istio-token\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7\" (UID: \"7d1dc8f1-3238-4b12-b579-ec8179bc4e2a\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7"
Nov 04 01:17:04.642147 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:04.641904    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp"
Nov 04 01:17:04.747689 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:04.743511    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7"
Nov 04 01:17:05.057667 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:05.030559    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89" podUID="6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": context deadline exceeded"
Nov 04 01:17:05.057667 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:05.031954    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz" podUID="4ccafa61-1c9b-4595-a511-3404d7479526" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": context deadline exceeded"
Nov 04 01:17:05.091639 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:05.061694    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf" podUID="32f60f59-a98f-46ba-9fb4-8420f77998da" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:05.127815 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:05.127774    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5" podUID="60f5c87a-48f8-4ad9-847b-cff571cc9340" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Nov 04 01:17:05.471002 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:05.460234    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8" podUID="eb9132e5-b90b-46fb-b86c-ca713419906d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:05.474440 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:05.471596    5068 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9" podUID="530fc578-6622-4c42-a384-02e73bf17b1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/health\": context deadline exceeded"
Nov 04 01:17:05.474440 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:05.471736    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx" podUID="46d4ec7b-558d-4fb5-8011-177760a3291c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:05.474440 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:05.471963    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq" podUID="b87aafac-9ab8-40e5-8d08-72eafc4187dd" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Nov 04 01:17:05.474440 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:05.472181    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx"
Nov 04 01:17:05.582638 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:05.578948    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82" podUID="6cdd01de-ff61-494a-bdb3-c67025a49440" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": context deadline exceeded"
Nov 04 01:17:05.624645 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:05.591028    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82"
Nov 04 01:17:05.644057 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:05.642578    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n" podUID="b3223811-a857-4279-a671-189e65383481" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:06.077957 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:06.077900    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:17:06.078254 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:06.078231    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:17:06.078477 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:06.078455    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:17:06.102314 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:06.102264    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf" podUID="32f60f59-a98f-46ba-9fb4-8420f77998da" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:06.339701 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:06.329698    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9" podUID="530fc578-6622-4c42-a384-02e73bf17b1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:06.360861 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:06.346984    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5" podUID="60f5c87a-48f8-4ad9-847b-cff571cc9340" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:06.605684 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:06.601639    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8" podUID="eb9132e5-b90b-46fb-b86c-ca713419906d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded"
Nov 04 01:17:06.631404 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:06.631358    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx" podUID="46d4ec7b-558d-4fb5-8011-177760a3291c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Nov 04 01:17:06.632167 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:06.632135    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq" podUID="b87aafac-9ab8-40e5-8d08-72eafc4187dd" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Nov 04 01:17:06.632878 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:06.632853    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm" podUID="f9e7abd2-8737-4357-a778-d6492fd12f3f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": context deadline exceeded"
Nov 04 01:17:06.641822 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:06.641779    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82" podUID="6cdd01de-ff61-494a-bdb3-c67025a49440" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": context deadline exceeded"
Nov 04 01:17:06.667888 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:06.667838    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89" podUID="6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:06.694769 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:06.671778    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n" podUID="b3223811-a857-4279-a671-189e65383481" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:06.695109 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:06.695070    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz" podUID="4ccafa61-1c9b-4595-a511-3404d7479526" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": context deadline exceeded"
Nov 04 01:17:07.347857 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:07.347807    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5" podUID="60f5c87a-48f8-4ad9-847b-cff571cc9340" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:07.353665 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:07.349953    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf" podUID="32f60f59-a98f-46ba-9fb4-8420f77998da" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded"
Nov 04 01:17:07.670544 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:07.670486    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89" podUID="6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": context deadline exceeded"
Nov 04 01:17:07.711736 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:07.672423    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82" podUID="6cdd01de-ff61-494a-bdb3-c67025a49440" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": context deadline exceeded"
Nov 04 01:17:07.724667 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:07.713745    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9" podUID="530fc578-6622-4c42-a384-02e73bf17b1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Nov 04 01:17:07.724667 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:07.714790    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8" podUID="eb9132e5-b90b-46fb-b86c-ca713419906d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:07.724667 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:07.714937    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx" podUID="46d4ec7b-558d-4fb5-8011-177760a3291c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Nov 04 01:17:07.724667 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:07.715026    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq" podUID="b87aafac-9ab8-40e5-8d08-72eafc4187dd" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:07.724667 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:07.715145    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n" podUID="b3223811-a857-4279-a671-189e65383481" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:08.060734 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:08.060000    5068 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx" podUID="46d4ec7b-558d-4fb5-8011-177760a3291c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/health\": context deadline exceeded"
Nov 04 01:17:08.060734 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:08.060093    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm" podUID="f9e7abd2-8737-4357-a778-d6492fd12f3f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": context deadline exceeded"
Nov 04 01:17:08.065916 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:08.061349    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz" podUID="4ccafa61-1c9b-4595-a511-3404d7479526" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": context deadline exceeded"
Nov 04 01:17:08.207891 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:08.207835    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:17:08.208165 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:08.208143    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:17:08.208339 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:08.208313    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:17:08.221126 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:08.221068    5068 prober.go:122] "Probe failed" probeType="Liveness" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8ff7fd4ff-65jvg" podUID="781d0e29-3d65-4a97-ac09-aa8f3973e32a" containerName="manager" probeResult="failure" output="Get \"http://10.132.0.37:8081/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:08.555298 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:08.544171    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5" podUID="60f5c87a-48f8-4ad9-847b-cff571cc9340" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Nov 04 01:17:08.555298 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:08.544469    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf" podUID="32f60f59-a98f-46ba-9fb4-8420f77998da" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded"
Nov 04 01:17:08.814700 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:08.788325    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8" podUID="eb9132e5-b90b-46fb-b86c-ca713419906d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:08.852077 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:08.811706    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89" podUID="6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:09.047635 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:08.811768    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx" podUID="46d4ec7b-558d-4fb5-8011-177760a3291c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Nov 04 01:17:09.073113 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:08.811810    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq" podUID="b87aafac-9ab8-40e5-8d08-72eafc4187dd" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Nov 04 01:17:09.098756 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:08.811833    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n" podUID="b3223811-a857-4279-a671-189e65383481" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Nov 04 01:17:09.099156 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:08.812359    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9" podUID="530fc578-6622-4c42-a384-02e73bf17b1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Nov 04 01:17:09.099314 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:08.812411    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82" podUID="6cdd01de-ff61-494a-bdb3-c67025a49440" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:09.099555 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:08.878726    5068 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8" podUID="eb9132e5-b90b-46fb-b86c-ca713419906d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/health\": context deadline exceeded"
Nov 04 01:17:09.099666 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:17:08.880817    5068 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="2.632s"
Nov 04 01:17:09.162314 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:09.061238    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm" podUID="f9e7abd2-8737-4357-a778-d6492fd12f3f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:09.162314 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:09.061546    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz" podUID="4ccafa61-1c9b-4595-a511-3404d7479526" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:09.346749 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:09.307696    5068 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq" podUID="b87aafac-9ab8-40e5-8d08-72eafc4187dd" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/health\": context deadline exceeded"
Nov 04 01:17:09.346749 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:09.324366    5068 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm" podUID="f9e7abd2-8737-4357-a778-d6492fd12f3f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:09.346749 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:09.324451    5068 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf" podUID="32f60f59-a98f-46ba-9fb4-8420f77998da" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/health\": context deadline exceeded"
Nov 04 01:17:09.369114 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:09.369057    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:17:09.369304 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:09.369167    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:17:09.369304 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:09.369213    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:17:09.440633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:09.435747    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f" event={"ID":"7ca2a53a-ada2-4565-bc5a-66c7eefef997","Type":"ContainerStarted","Data":"b4f393fd7c4336b02f80c0fe04cf92066cd9310a4b8c2700b959eef122a69a64"}
Nov 04 01:17:09.440633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:09.435841    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5"
Nov 04 01:17:09.440633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:09.435882    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8"
Nov 04 01:17:09.440633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:09.435913    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f"]
Nov 04 01:17:09.440633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:09.435959    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n"
Nov 04 01:17:09.440633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:09.435992    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9"
Nov 04 01:17:09.440633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:09.436012    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp"]
Nov 04 01:17:09.440633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:09.436037    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7"]
Nov 04 01:17:09.440633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:09.436081    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm"
Nov 04 01:17:09.448653 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:09.445739    5068 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82" podUID="6cdd01de-ff61-494a-bdb3-c67025a49440" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:09.546793 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:09.546744    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf" podUID="32f60f59-a98f-46ba-9fb4-8420f77998da" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:09.547478 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:09.547435    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5" podUID="60f5c87a-48f8-4ad9-847b-cff571cc9340" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Nov 04 01:17:09.864894 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:09.864776    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8" podUID="eb9132e5-b90b-46fb-b86c-ca713419906d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:10.044020 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:10.043918    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89" podUID="6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:10.062644 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:10.055708    5068 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n" podUID="b3223811-a857-4279-a671-189e65383481" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:10.077632 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:10.077572    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx" podUID="46d4ec7b-558d-4fb5-8011-177760a3291c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:10.108193 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:10.108065    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82" podUID="6cdd01de-ff61-494a-bdb3-c67025a49440" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:10.109699 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:10.108367    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n" podUID="b3223811-a857-4279-a671-189e65383481" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:10.109699 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:10.108560    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9" podUID="530fc578-6622-4c42-a384-02e73bf17b1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Nov 04 01:17:10.152871 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:10.120954    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq" podUID="b87aafac-9ab8-40e5-8d08-72eafc4187dd" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:10.159842 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:10.159802    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm" podUID="f9e7abd2-8737-4357-a778-d6492fd12f3f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": context deadline exceeded"
Nov 04 01:17:10.160546 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:10.160077    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz" podUID="4ccafa61-1c9b-4595-a511-3404d7479526" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:10.490219 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:10.462957    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7" event={"ID":"7d1dc8f1-3238-4b12-b579-ec8179bc4e2a","Type":"ContainerStarted","Data":"f499661d4f10b5e4546f357483e519f23d6eda941d34bd01aca47dfa15dff956"}
Nov 04 01:17:10.490219 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:10.487775    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89"
Nov 04 01:17:10.682464 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:10.580738    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf" podUID="32f60f59-a98f-46ba-9fb4-8420f77998da" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded"
Nov 04 01:17:10.682464 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:10.595104    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp" event={"ID":"20fba30d-0142-49f8-8489-957bad3db95e","Type":"ContainerStarted","Data":"e657c85e63f817eabb701068b7d9bced58da25d0dc1874fcfeee0d84494083bf"}
Nov 04 01:17:10.682464 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:10.641729    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5" podUID="60f5c87a-48f8-4ad9-847b-cff571cc9340" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:10.682464 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:10.643041    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8"
Nov 04 01:17:10.732395 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:10.732185    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f" event={"ID":"7ca2a53a-ada2-4565-bc5a-66c7eefef997","Type":"ContainerStarted","Data":"c442438a5f770f4b9326055ad3c246e0146284879f99db5e5d1d1865b1e22f3a"}
Nov 04 01:17:10.748949 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:10.744537    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp" podStartSLOduration=9.744517623 podStartE2EDuration="9.744517623s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:17:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:17:10.656917391 +0000 UTC m=+870.703478181" watchObservedRunningTime="2026-11-04 01:17:10.744517623 +0000 UTC m=+870.791078414"
Nov 04 01:17:10.831739 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:10.831699    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp"
Nov 04 01:17:10.842473 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:10.842442    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82"
Nov 04 01:17:10.844635 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:10.844579    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9"
Nov 04 01:17:10.857815 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:10.855840    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm"
Nov 04 01:17:10.991420 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:10.991291    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f" podStartSLOduration=8.991271853 podStartE2EDuration="8.991271853s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:17:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:17:10.881156257 +0000 UTC m=+870.927717048" watchObservedRunningTime="2026-11-04 01:17:10.991271853 +0000 UTC m=+871.037832644"
Nov 04 01:17:11.081405 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:11.006813    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n"
Nov 04 01:17:11.081866 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:11.081796    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq"
Nov 04 01:17:11.108379 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:11.108118    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz"
Nov 04 01:17:11.110679 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:11.109716    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf"
Nov 04 01:17:11.126602 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:11.123728    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx" podUID="46d4ec7b-558d-4fb5-8011-177760a3291c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:11.201720 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:11.139658    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5"
Nov 04 01:17:11.216630 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:11.216011    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx"
Nov 04 01:17:11.508269 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:11.508237    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f"
Nov 04 01:17:11.509500 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:11.509472    5068 patch_prober.go:29] interesting pod/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Nov 04 01:17:11.509670 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:11.509524    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f" podUID="7ca2a53a-ada2-4565-bc5a-66c7eefef997" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Nov 04 01:17:11.663635 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:11.655774    5068 patch_prober.go:29] interesting pod/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Nov 04 01:17:11.663635 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:11.655848    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp" podUID="20fba30d-0142-49f8-8489-957bad3db95e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Nov 04 01:17:11.704049 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:11.703998    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7" event={"ID":"7d1dc8f1-3238-4b12-b579-ec8179bc4e2a","Type":"ContainerStarted","Data":"22a27310c8b2a4b2312973876da0ab42d52308596edfa2bb1a3864033939da24"}
Nov 04 01:17:11.709962 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:11.709907    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp" event={"ID":"20fba30d-0142-49f8-8489-957bad3db95e","Type":"ContainerStarted","Data":"40b5f1c0fb186e12538dfe099ccece193a8e0d5aa1ec63ce5734c65849bba32d"}
Nov 04 01:17:11.750382 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:11.750298    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7"
Nov 04 01:17:11.755298 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:11.755260    5068 patch_prober.go:29] interesting pod/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Nov 04 01:17:11.755560 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:11.755533    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7" podUID="7d1dc8f1-3238-4b12-b579-ec8179bc4e2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Nov 04 01:17:11.757407 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:11.757350    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7" podStartSLOduration=9.757333498 podStartE2EDuration="9.757333498s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:17:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:17:11.73070414 +0000 UTC m=+871.777264931" watchObservedRunningTime="2026-11-04 01:17:11.757333498 +0000 UTC m=+871.803894290"
Nov 04 01:17:12.508157 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:12.508120    5068 patch_prober.go:29] interesting pod/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Nov 04 01:17:12.508852 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:12.508803    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f" podUID="7ca2a53a-ada2-4565-bc5a-66c7eefef997" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Nov 04 01:17:12.651574 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:12.651525    5068 patch_prober.go:29] interesting pod/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Nov 04 01:17:12.651875 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:12.651842    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp" podUID="20fba30d-0142-49f8-8489-957bad3db95e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Nov 04 01:17:12.762705 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:12.752801    5068 patch_prober.go:29] interesting pod/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Nov 04 01:17:12.762705 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:12.752858    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7" podUID="7d1dc8f1-3238-4b12-b579-ec8179bc4e2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Nov 04 01:17:13.511634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:13.511036    5068 patch_prober.go:29] interesting pod/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Nov 04 01:17:13.511634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:13.511107    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f" podUID="7ca2a53a-ada2-4565-bc5a-66c7eefef997" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Nov 04 01:17:13.648142 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:13.648108    5068 patch_prober.go:29] interesting pod/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Nov 04 01:17:13.648450 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:13.648409    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp" podUID="20fba30d-0142-49f8-8489-957bad3db95e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Nov 04 01:17:13.710637 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:17:13.702846    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:17:13.751218 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:13.751166    5068 patch_prober.go:29] interesting pod/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Nov 04 01:17:13.751514 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:13.751463    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7" podUID="7d1dc8f1-3238-4b12-b579-ec8179bc4e2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Nov 04 01:17:14.520281 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:14.520236    5068 patch_prober.go:29] interesting pod/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Nov 04 01:17:14.684025 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:14.683981    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f" podUID="7ca2a53a-ada2-4565-bc5a-66c7eefef997" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Nov 04 01:17:14.714634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:14.687264    5068 patch_prober.go:29] interesting pod/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Nov 04 01:17:14.714634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:14.687323    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp" podUID="20fba30d-0142-49f8-8489-957bad3db95e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Nov 04 01:17:14.767663 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:14.759645    5068 patch_prober.go:29] interesting pod/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Nov 04 01:17:14.767663 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:14.759735    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7" podUID="7d1dc8f1-3238-4b12-b579-ec8179bc4e2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Nov 04 01:17:15.526088 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:15.526052    5068 patch_prober.go:29] interesting pod/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Nov 04 01:17:15.537760 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:15.537713    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f" podUID="7ca2a53a-ada2-4565-bc5a-66c7eefef997" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Nov 04 01:17:15.659102 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:15.657901    5068 patch_prober.go:29] interesting pod/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Nov 04 01:17:15.659102 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:15.658008    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp" podUID="20fba30d-0142-49f8-8489-957bad3db95e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Nov 04 01:17:15.756659 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:15.754995    5068 patch_prober.go:29] interesting pod/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Nov 04 01:17:15.756659 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:15.755076    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7" podUID="7d1dc8f1-3238-4b12-b579-ec8179bc4e2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Nov 04 01:17:16.134638 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:16.123781    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82" podUID="6cdd01de-ff61-494a-bdb3-c67025a49440" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:16.522698 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:16.517349    5068 patch_prober.go:29] interesting pod/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Nov 04 01:17:16.522698 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:16.517410    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f" podUID="7ca2a53a-ada2-4565-bc5a-66c7eefef997" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Nov 04 01:17:16.647678 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:16.644936    5068 patch_prober.go:29] interesting pod/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Nov 04 01:17:16.647678 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:16.645009    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp" podUID="20fba30d-0142-49f8-8489-957bad3db95e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Nov 04 01:17:16.772075 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:16.772029    5068 patch_prober.go:29] interesting pod/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Nov 04 01:17:16.772904 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:16.772830    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7" podUID="7d1dc8f1-3238-4b12-b579-ec8179bc4e2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Nov 04 01:17:17.005639 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:16.996666    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94"]
Nov 04 01:17:17.091712 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:17.091413    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94"]
Nov 04 01:17:17.091712 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:17.091553    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94"
Nov 04 01:17:17.280634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:17.268817    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45b260dc-8b00-4f55-a649-58220ed083bb-workload-certs\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94\" (UID: \"45b260dc-8b00-4f55-a649-58220ed083bb\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94"
Nov 04 01:17:17.280634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:17.268916    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45b260dc-8b00-4f55-a649-58220ed083bb-istio-podinfo\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94\" (UID: \"45b260dc-8b00-4f55-a649-58220ed083bb\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94"
Nov 04 01:17:17.280634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:17.268988    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9tdk\" (UniqueName: \"kubernetes.io/projected/45b260dc-8b00-4f55-a649-58220ed083bb-kube-api-access-l9tdk\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94\" (UID: \"45b260dc-8b00-4f55-a649-58220ed083bb\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94"
Nov 04 01:17:17.280634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:17.269018    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45b260dc-8b00-4f55-a649-58220ed083bb-istiod-ca-cert\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94\" (UID: \"45b260dc-8b00-4f55-a649-58220ed083bb\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94"
Nov 04 01:17:17.280634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:17.269117    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45b260dc-8b00-4f55-a649-58220ed083bb-istio-token\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94\" (UID: \"45b260dc-8b00-4f55-a649-58220ed083bb\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94"
Nov 04 01:17:17.280634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:17.269326    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45b260dc-8b00-4f55-a649-58220ed083bb-istio-data\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94\" (UID: \"45b260dc-8b00-4f55-a649-58220ed083bb\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94"
Nov 04 01:17:17.280634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:17.269356    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45b260dc-8b00-4f55-a649-58220ed083bb-credential-socket\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94\" (UID: \"45b260dc-8b00-4f55-a649-58220ed083bb\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94"
Nov 04 01:17:17.280634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:17.269386    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45b260dc-8b00-4f55-a649-58220ed083bb-istio-envoy\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94\" (UID: \"45b260dc-8b00-4f55-a649-58220ed083bb\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94"
Nov 04 01:17:17.280634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:17.269417    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45b260dc-8b00-4f55-a649-58220ed083bb-workload-socket\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94\" (UID: \"45b260dc-8b00-4f55-a649-58220ed083bb\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94"
Nov 04 01:17:17.378513 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:17.378415    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45b260dc-8b00-4f55-a649-58220ed083bb-istio-token\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94\" (UID: \"45b260dc-8b00-4f55-a649-58220ed083bb\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94"
Nov 04 01:17:17.378935 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:17.378904    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45b260dc-8b00-4f55-a649-58220ed083bb-istio-data\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94\" (UID: \"45b260dc-8b00-4f55-a649-58220ed083bb\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94"
Nov 04 01:17:17.379079 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:17.379061    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45b260dc-8b00-4f55-a649-58220ed083bb-credential-socket\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94\" (UID: \"45b260dc-8b00-4f55-a649-58220ed083bb\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94"
Nov 04 01:17:17.379210 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:17.379194    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45b260dc-8b00-4f55-a649-58220ed083bb-istio-envoy\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94\" (UID: \"45b260dc-8b00-4f55-a649-58220ed083bb\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94"
Nov 04 01:17:17.379334 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:17.379320    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45b260dc-8b00-4f55-a649-58220ed083bb-workload-socket\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94\" (UID: \"45b260dc-8b00-4f55-a649-58220ed083bb\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94"
Nov 04 01:17:17.379524 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:17.379509    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45b260dc-8b00-4f55-a649-58220ed083bb-workload-certs\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94\" (UID: \"45b260dc-8b00-4f55-a649-58220ed083bb\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94"
Nov 04 01:17:17.379669 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:17.379652    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45b260dc-8b00-4f55-a649-58220ed083bb-istio-podinfo\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94\" (UID: \"45b260dc-8b00-4f55-a649-58220ed083bb\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94"
Nov 04 01:17:17.379840 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:17.379821    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9tdk\" (UniqueName: \"kubernetes.io/projected/45b260dc-8b00-4f55-a649-58220ed083bb-kube-api-access-l9tdk\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94\" (UID: \"45b260dc-8b00-4f55-a649-58220ed083bb\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94"
Nov 04 01:17:17.379962 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:17.379944    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45b260dc-8b00-4f55-a649-58220ed083bb-istiod-ca-cert\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94\" (UID: \"45b260dc-8b00-4f55-a649-58220ed083bb\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94"
Nov 04 01:17:17.384632 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:17.381073    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45b260dc-8b00-4f55-a649-58220ed083bb-istiod-ca-cert\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94\" (UID: \"45b260dc-8b00-4f55-a649-58220ed083bb\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94"
Nov 04 01:17:17.408246 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:17.385859    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45b260dc-8b00-4f55-a649-58220ed083bb-istio-data\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94\" (UID: \"45b260dc-8b00-4f55-a649-58220ed083bb\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94"
Nov 04 01:17:17.408246 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:17.395088    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45b260dc-8b00-4f55-a649-58220ed083bb-credential-socket\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94\" (UID: \"45b260dc-8b00-4f55-a649-58220ed083bb\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94"
Nov 04 01:17:17.408246 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:17.404348    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45b260dc-8b00-4f55-a649-58220ed083bb-workload-certs\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94\" (UID: \"45b260dc-8b00-4f55-a649-58220ed083bb\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94"
Nov 04 01:17:17.408246 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:17.404582    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45b260dc-8b00-4f55-a649-58220ed083bb-workload-socket\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94\" (UID: \"45b260dc-8b00-4f55-a649-58220ed083bb\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94"
Nov 04 01:17:17.410275 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:17.410235    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45b260dc-8b00-4f55-a649-58220ed083bb-istio-podinfo\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94\" (UID: \"45b260dc-8b00-4f55-a649-58220ed083bb\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94"
Nov 04 01:17:17.411101 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:17.411068    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45b260dc-8b00-4f55-a649-58220ed083bb-istio-envoy\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94\" (UID: \"45b260dc-8b00-4f55-a649-58220ed083bb\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94"
Nov 04 01:17:17.414579 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:17.414532    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45b260dc-8b00-4f55-a649-58220ed083bb-istio-token\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94\" (UID: \"45b260dc-8b00-4f55-a649-58220ed083bb\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94"
Nov 04 01:17:17.418094 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:17.418070    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9tdk\" (UniqueName: \"kubernetes.io/projected/45b260dc-8b00-4f55-a649-58220ed083bb-kube-api-access-l9tdk\") pod \"e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94\" (UID: \"45b260dc-8b00-4f55-a649-58220ed083bb\") " pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94"
Nov 04 01:17:17.504714 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:17.500858    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94"
Nov 04 01:17:17.509581 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:17.509134    5068 patch_prober.go:29] interesting pod/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Nov 04 01:17:17.509581 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:17.509190    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f" podUID="7ca2a53a-ada2-4565-bc5a-66c7eefef997" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Nov 04 01:17:17.645701 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:17.644985    5068 patch_prober.go:29] interesting pod/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Nov 04 01:17:17.645701 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:17.645046    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp" podUID="20fba30d-0142-49f8-8489-957bad3db95e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Nov 04 01:17:17.754641 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:17.753973    5068 patch_prober.go:29] interesting pod/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Nov 04 01:17:17.754641 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:17.754033    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7" podUID="7d1dc8f1-3238-4b12-b579-ec8179bc4e2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Nov 04 01:17:18.222356 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:18.222317    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94"]
Nov 04 01:17:18.224753 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:17:18.224718    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45b260dc_8b00_4f55_a649_58220ed083bb.slice/crio-039dbbe110ece46360f374831722f2898778f87bc0e1fedf01b2c60f02f80306 WatchSource:0}: Error finding container 039dbbe110ece46360f374831722f2898778f87bc0e1fedf01b2c60f02f80306: Status 404 returned error can't find the container with id 039dbbe110ece46360f374831722f2898778f87bc0e1fedf01b2c60f02f80306
Nov 04 01:17:18.228649 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:18.228561    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:17:18.228831 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:18.228678    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:17:18.228831 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:18.228722    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:17:18.508336 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:18.508236    5068 patch_prober.go:29] interesting pod/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Nov 04 01:17:18.508624 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:18.508568    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f" podUID="7ca2a53a-ada2-4565-bc5a-66c7eefef997" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Nov 04 01:17:18.644969 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:18.644902    5068 patch_prober.go:29] interesting pod/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Nov 04 01:17:18.645255 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:18.645222    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp" podUID="20fba30d-0142-49f8-8489-957bad3db95e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Nov 04 01:17:18.753620 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:18.749860    5068 patch_prober.go:29] interesting pod/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Nov 04 01:17:18.753620 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:18.749919    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7" podUID="7d1dc8f1-3238-4b12-b579-ec8179bc4e2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Nov 04 01:17:19.288635 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:19.276265    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94" event={"ID":"45b260dc-8b00-4f55-a649-58220ed083bb","Type":"ContainerStarted","Data":"ab4ccb934f6a263df1ac0d8019a4d7b4957b6b7e4a02be7117c68056ba5131e2"}
Nov 04 01:17:19.288635 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:19.276341    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94" event={"ID":"45b260dc-8b00-4f55-a649-58220ed083bb","Type":"ContainerStarted","Data":"039dbbe110ece46360f374831722f2898778f87bc0e1fedf01b2c60f02f80306"}
Nov 04 01:17:19.320883 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:19.320817    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94" podStartSLOduration=3.320800371 podStartE2EDuration="3.320800371s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:17:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:17:19.317969948 +0000 UTC m=+879.364530739" watchObservedRunningTime="2026-11-04 01:17:19.320800371 +0000 UTC m=+879.367361162"
Nov 04 01:17:19.511228 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:19.507072    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94"
Nov 04 01:17:19.511228 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:19.507799    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f"
Nov 04 01:17:19.511228 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:19.508886    5068 patch_prober.go:29] interesting pod/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Nov 04 01:17:19.511228 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:19.508935    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94" podUID="45b260dc-8b00-4f55-a649-58220ed083bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Nov 04 01:17:19.517760 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:19.517738    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f"
Nov 04 01:17:19.645846 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:19.645750    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp"
Nov 04 01:17:19.758636 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:19.757431    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7"
Nov 04 01:17:20.400515 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:20.399936    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f"
Nov 04 01:17:20.479901 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:20.479868    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp"
Nov 04 01:17:20.506827 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:20.505234    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp"
Nov 04 01:17:20.506827 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:20.505349    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7"
Nov 04 01:17:20.509792 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:20.509268    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7"
Nov 04 01:17:20.509792 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:20.509279    5068 patch_prober.go:29] interesting pod/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Nov 04 01:17:20.509792 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:20.509412    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94" podUID="45b260dc-8b00-4f55-a649-58220ed083bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Nov 04 01:17:21.506305 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:21.506253    5068 patch_prober.go:29] interesting pod/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Nov 04 01:17:21.506917 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:21.506331    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94" podUID="45b260dc-8b00-4f55-a649-58220ed083bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Nov 04 01:17:22.505292 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:22.505251    5068 patch_prober.go:29] interesting pod/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Nov 04 01:17:22.505497 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:22.505327    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94" podUID="45b260dc-8b00-4f55-a649-58220ed083bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Nov 04 01:17:23.505520 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:23.505476    5068 patch_prober.go:29] interesting pod/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Nov 04 01:17:23.505924 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:23.505545    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94" podUID="45b260dc-8b00-4f55-a649-58220ed083bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Nov 04 01:17:24.505570 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:24.505533    5068 patch_prober.go:29] interesting pod/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Nov 04 01:17:24.506015 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:24.505589    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94" podUID="45b260dc-8b00-4f55-a649-58220ed083bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Nov 04 01:17:25.506028 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:25.505989    5068 patch_prober.go:29] interesting pod/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Nov 04 01:17:25.506575 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:25.506059    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94" podUID="45b260dc-8b00-4f55-a649-58220ed083bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Nov 04 01:17:26.509970 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:26.509930    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94"
Nov 04 01:17:26.510399 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:26.510275    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94"
Nov 04 01:17:26.511009 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:26.510988    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94"
Nov 04 01:17:31.939907 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:31.939868    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:17:32.575992 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:32.575951    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq"]
Nov 04 01:17:32.581895 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:32.581857    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq"
Nov 04 01:17:32.596282 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:32.596254    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq"]
Nov 04 01:17:32.711634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:32.693328    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cbbfd346-7a24-49e7-8853-b60e0dbf754c-istiod-ca-cert\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq\" (UID: \"cbbfd346-7a24-49e7-8853-b60e0dbf754c\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq"
Nov 04 01:17:32.798226 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:32.798188    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cbbfd346-7a24-49e7-8853-b60e0dbf754c-workload-certs\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq\" (UID: \"cbbfd346-7a24-49e7-8853-b60e0dbf754c\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq"
Nov 04 01:17:32.798556 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:32.798536    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cbbfd346-7a24-49e7-8853-b60e0dbf754c-istio-podinfo\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq\" (UID: \"cbbfd346-7a24-49e7-8853-b60e0dbf754c\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq"
Nov 04 01:17:32.798781 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:32.798763    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cbbfd346-7a24-49e7-8853-b60e0dbf754c-istio-envoy\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq\" (UID: \"cbbfd346-7a24-49e7-8853-b60e0dbf754c\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq"
Nov 04 01:17:32.798961 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:32.798946    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clfmk\" (UniqueName: \"kubernetes.io/projected/cbbfd346-7a24-49e7-8853-b60e0dbf754c-kube-api-access-clfmk\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq\" (UID: \"cbbfd346-7a24-49e7-8853-b60e0dbf754c\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq"
Nov 04 01:17:32.799122 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:32.799108    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cbbfd346-7a24-49e7-8853-b60e0dbf754c-istio-token\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq\" (UID: \"cbbfd346-7a24-49e7-8853-b60e0dbf754c\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq"
Nov 04 01:17:32.799296 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:32.799284    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbbfd346-7a24-49e7-8853-b60e0dbf754c-workload-socket\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq\" (UID: \"cbbfd346-7a24-49e7-8853-b60e0dbf754c\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq"
Nov 04 01:17:32.799511 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:32.799496    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbbfd346-7a24-49e7-8853-b60e0dbf754c-credential-socket\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq\" (UID: \"cbbfd346-7a24-49e7-8853-b60e0dbf754c\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq"
Nov 04 01:17:32.799754 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:32.799738    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cbbfd346-7a24-49e7-8853-b60e0dbf754c-istio-data\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq\" (UID: \"cbbfd346-7a24-49e7-8853-b60e0dbf754c\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq"
Nov 04 01:17:32.902276 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:32.902155    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbbfd346-7a24-49e7-8853-b60e0dbf754c-credential-socket\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq\" (UID: \"cbbfd346-7a24-49e7-8853-b60e0dbf754c\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq"
Nov 04 01:17:32.902463 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:32.902323    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cbbfd346-7a24-49e7-8853-b60e0dbf754c-istio-data\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq\" (UID: \"cbbfd346-7a24-49e7-8853-b60e0dbf754c\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq"
Nov 04 01:17:32.902463 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:32.902377    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cbbfd346-7a24-49e7-8853-b60e0dbf754c-istiod-ca-cert\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq\" (UID: \"cbbfd346-7a24-49e7-8853-b60e0dbf754c\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq"
Nov 04 01:17:32.902463 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:32.902427    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cbbfd346-7a24-49e7-8853-b60e0dbf754c-workload-certs\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq\" (UID: \"cbbfd346-7a24-49e7-8853-b60e0dbf754c\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq"
Nov 04 01:17:32.902702 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:32.902464    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cbbfd346-7a24-49e7-8853-b60e0dbf754c-istio-podinfo\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq\" (UID: \"cbbfd346-7a24-49e7-8853-b60e0dbf754c\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq"
Nov 04 01:17:32.902702 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:32.902536    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cbbfd346-7a24-49e7-8853-b60e0dbf754c-istio-envoy\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq\" (UID: \"cbbfd346-7a24-49e7-8853-b60e0dbf754c\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq"
Nov 04 01:17:32.902702 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:32.902626    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clfmk\" (UniqueName: \"kubernetes.io/projected/cbbfd346-7a24-49e7-8853-b60e0dbf754c-kube-api-access-clfmk\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq\" (UID: \"cbbfd346-7a24-49e7-8853-b60e0dbf754c\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq"
Nov 04 01:17:32.902702 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:32.902681    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cbbfd346-7a24-49e7-8853-b60e0dbf754c-istio-token\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq\" (UID: \"cbbfd346-7a24-49e7-8853-b60e0dbf754c\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq"
Nov 04 01:17:32.902925 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:32.902792    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbbfd346-7a24-49e7-8853-b60e0dbf754c-workload-socket\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq\" (UID: \"cbbfd346-7a24-49e7-8853-b60e0dbf754c\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq"
Nov 04 01:17:32.903400 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:32.903376    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbbfd346-7a24-49e7-8853-b60e0dbf754c-workload-socket\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq\" (UID: \"cbbfd346-7a24-49e7-8853-b60e0dbf754c\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq"
Nov 04 01:17:32.904345 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:32.904320    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cbbfd346-7a24-49e7-8853-b60e0dbf754c-workload-certs\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq\" (UID: \"cbbfd346-7a24-49e7-8853-b60e0dbf754c\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq"
Nov 04 01:17:32.905388 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:32.905332    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cbbfd346-7a24-49e7-8853-b60e0dbf754c-istio-data\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq\" (UID: \"cbbfd346-7a24-49e7-8853-b60e0dbf754c\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq"
Nov 04 01:17:32.905594 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:32.905542    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cbbfd346-7a24-49e7-8853-b60e0dbf754c-istiod-ca-cert\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq\" (UID: \"cbbfd346-7a24-49e7-8853-b60e0dbf754c\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq"
Nov 04 01:17:32.905872 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:32.905852    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbbfd346-7a24-49e7-8853-b60e0dbf754c-credential-socket\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq\" (UID: \"cbbfd346-7a24-49e7-8853-b60e0dbf754c\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq"
Nov 04 01:17:32.907287 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:32.907264    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cbbfd346-7a24-49e7-8853-b60e0dbf754c-istio-podinfo\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq\" (UID: \"cbbfd346-7a24-49e7-8853-b60e0dbf754c\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq"
Nov 04 01:17:32.907642 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:32.907603    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cbbfd346-7a24-49e7-8853-b60e0dbf754c-istio-envoy\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq\" (UID: \"cbbfd346-7a24-49e7-8853-b60e0dbf754c\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq"
Nov 04 01:17:32.916600 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:32.916570    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cbbfd346-7a24-49e7-8853-b60e0dbf754c-istio-token\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq\" (UID: \"cbbfd346-7a24-49e7-8853-b60e0dbf754c\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq"
Nov 04 01:17:32.916867 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:32.916851    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-clfmk\" (UniqueName: \"kubernetes.io/projected/cbbfd346-7a24-49e7-8853-b60e0dbf754c-kube-api-access-clfmk\") pod \"e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq\" (UID: \"cbbfd346-7a24-49e7-8853-b60e0dbf754c\") " pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq"
Nov 04 01:17:33.206208 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:33.206094    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq"
Nov 04 01:17:34.164631 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:34.160053    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz" podUID="4ccafa61-1c9b-4595-a511-3404d7479526" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:34.210205 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:34.209727    5068 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz" podUID="4ccafa61-1c9b-4595-a511-3404d7479526" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:34.614281 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:34.550779    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5" podUID="60f5c87a-48f8-4ad9-847b-cff571cc9340" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:34.846061 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:34.818268    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx" podUID="46d4ec7b-558d-4fb5-8011-177760a3291c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Nov 04 01:17:34.846061 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:34.819023    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89" podUID="6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": context deadline exceeded"
Nov 04 01:17:34.846061 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:34.819244    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq" podUID="b87aafac-9ab8-40e5-8d08-72eafc4187dd" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:34.902714 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:34.875107    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8" podUID="eb9132e5-b90b-46fb-b86c-ca713419906d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:34.942531 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:34.942477    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq"]
Nov 04 01:17:35.002314 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:17:35.002271    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbbfd346_7a24_49e7_8853_b60e0dbf754c.slice/crio-a110569051177d1472234db77e9eb6d414e16bf54b2b23bf9b53baba14a80f01 WatchSource:0}: Error finding container a110569051177d1472234db77e9eb6d414e16bf54b2b23bf9b53baba14a80f01: Status 404 returned error can't find the container with id a110569051177d1472234db77e9eb6d414e16bf54b2b23bf9b53baba14a80f01
Nov 04 01:17:35.046637 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:35.043716    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:17:35.046637 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:35.043818    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:17:35.046637 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:35.043863    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:17:35.083639 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:35.072468    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n" podUID="b3223811-a857-4279-a671-189e65383481" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Nov 04 01:17:35.128638 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:35.123916    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82" podUID="6cdd01de-ff61-494a-bdb3-c67025a49440" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:35.128638 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:35.126017    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9" podUID="530fc578-6622-4c42-a384-02e73bf17b1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:35.180076 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:35.171667    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm" podUID="f9e7abd2-8737-4357-a778-d6492fd12f3f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:35.183637 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:35.170899    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz" podUID="4ccafa61-1c9b-4595-a511-3404d7479526" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:35.184666 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:35.184641    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz"
Nov 04 01:17:35.358472 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:35.356881    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf" podUID="32f60f59-a98f-46ba-9fb4-8420f77998da" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded"
Nov 04 01:17:35.478105 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:35.477741    5068 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9" podUID="530fc578-6622-4c42-a384-02e73bf17b1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:35.553793 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:35.553742    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5" podUID="60f5c87a-48f8-4ad9-847b-cff571cc9340" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:35.554760 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:35.554737    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5"
Nov 04 01:17:35.875643 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:35.875566    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx" podUID="46d4ec7b-558d-4fb5-8011-177760a3291c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Nov 04 01:17:35.889316 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:35.879101    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89" podUID="6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:35.889949 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:35.879142    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq" podUID="b87aafac-9ab8-40e5-8d08-72eafc4187dd" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:35.891171 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:35.890907    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx"
Nov 04 01:17:35.891357 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:35.879176    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8" podUID="eb9132e5-b90b-46fb-b86c-ca713419906d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:36.089725 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:36.089512    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n" podUID="b3223811-a857-4279-a671-189e65383481" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Nov 04 01:17:36.341698 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:36.315367    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82" podUID="6cdd01de-ff61-494a-bdb3-c67025a49440" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": context deadline exceeded"
Nov 04 01:17:36.855590 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:36.343675    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89"
Nov 04 01:17:36.959239 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:36.386764    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9" podUID="530fc578-6622-4c42-a384-02e73bf17b1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Nov 04 01:17:37.022072 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:36.430301    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf" podUID="32f60f59-a98f-46ba-9fb4-8420f77998da" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:37.022072 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:36.430925    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm" podUID="f9e7abd2-8737-4357-a778-d6492fd12f3f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:37.022072 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:36.456447    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz" podUID="4ccafa61-1c9b-4595-a511-3404d7479526" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": context deadline exceeded"
Nov 04 01:17:37.022072 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:36.858191    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5" podUID="60f5c87a-48f8-4ad9-847b-cff571cc9340" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:37.022072 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:36.902777    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx" podUID="46d4ec7b-558d-4fb5-8011-177760a3291c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:37.022072 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:36.918870    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89" podUID="6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": context deadline exceeded"
Nov 04 01:17:37.022072 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:36.919128    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq" podUID="b87aafac-9ab8-40e5-8d08-72eafc4187dd" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Nov 04 01:17:37.022072 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:36.919345    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8" podUID="eb9132e5-b90b-46fb-b86c-ca713419906d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:37.098467 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:37.098364    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n" podUID="b3223811-a857-4279-a671-189e65383481" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:37.448114 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:37.448014    5068 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx" podUID="46d4ec7b-558d-4fb5-8011-177760a3291c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:37.862629 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:37.859255    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq" event={"ID":"cbbfd346-7a24-49e7-8853-b60e0dbf754c","Type":"ContainerStarted","Data":"a110569051177d1472234db77e9eb6d414e16bf54b2b23bf9b53baba14a80f01"}
Nov 04 01:17:37.862629 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:37.859349    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq"
Nov 04 01:17:37.862629 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:37.859390    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8"
Nov 04 01:17:37.862629 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:37.859431    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n"
Nov 04 01:17:37.862629 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:37.859468    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82"
Nov 04 01:17:37.862629 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:37.859500    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9"
Nov 04 01:17:37.862629 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:37.859532    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf"
Nov 04 01:17:37.862629 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:37.859563    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm"
Nov 04 01:17:37.939671 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:37.909648    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82" podUID="6cdd01de-ff61-494a-bdb3-c67025a49440" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": context deadline exceeded"
Nov 04 01:17:38.020285 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:38.020243    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9" podUID="530fc578-6622-4c42-a384-02e73bf17b1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Nov 04 01:17:38.128727 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:38.096053    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq" event={"ID":"cbbfd346-7a24-49e7-8853-b60e0dbf754c","Type":"ContainerStarted","Data":"b72360f764e028842daaf82dee7cb79611ee053f643c1dbccec8dd5f2344cebb"}
Nov 04 01:17:38.151908 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:38.151864    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n" podUID="b3223811-a857-4279-a671-189e65383481" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Nov 04 01:17:38.319852 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:38.319812    5068 patch_prober.go:29] interesting pod/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Nov 04 01:17:38.320144 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:38.320097    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq" podUID="cbbfd346-7a24-49e7-8853-b60e0dbf754c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Nov 04 01:17:38.351130 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:38.351073    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf" podUID="32f60f59-a98f-46ba-9fb4-8420f77998da" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded"
Nov 04 01:17:38.354639 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:38.352095    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq" podUID="b87aafac-9ab8-40e5-8d08-72eafc4187dd" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Nov 04 01:17:38.354639 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:38.352457    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm" podUID="f9e7abd2-8737-4357-a778-d6492fd12f3f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": context deadline exceeded"
Nov 04 01:17:38.354639 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:38.352754    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz" podUID="4ccafa61-1c9b-4595-a511-3404d7479526" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": context deadline exceeded"
Nov 04 01:17:38.354639 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:38.352989    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5" podUID="60f5c87a-48f8-4ad9-847b-cff571cc9340" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Nov 04 01:17:38.354639 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:38.353232    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx" podUID="46d4ec7b-558d-4fb5-8011-177760a3291c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Nov 04 01:17:38.354639 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:38.353477    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89" podUID="6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": context deadline exceeded"
Nov 04 01:17:38.404722 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:38.362742    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8" podUID="eb9132e5-b90b-46fb-b86c-ca713419906d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded"
Nov 04 01:17:38.404722 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:38.363901    5068 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq" podUID="b87aafac-9ab8-40e5-8d08-72eafc4187dd" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:38.514036 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:38.504269    5068 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm" podUID="f9e7abd2-8737-4357-a778-d6492fd12f3f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/health\": context deadline exceeded"
Nov 04 01:17:38.536874 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:38.536820    5068 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf" podUID="32f60f59-a98f-46ba-9fb4-8420f77998da" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:38.902576 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:38.902520    5068 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82" podUID="6cdd01de-ff61-494a-bdb3-c67025a49440" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:38.962585 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:38.962529    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82" podUID="6cdd01de-ff61-494a-bdb3-c67025a49440" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": context deadline exceeded"
Nov 04 01:17:39.067635 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:39.042508    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9" podUID="530fc578-6622-4c42-a384-02e73bf17b1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:39.250769 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:39.070911    5068 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8" podUID="eb9132e5-b90b-46fb-b86c-ca713419906d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:39.251016 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:39.138914    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq"
Nov 04 01:17:39.278783 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:39.268831    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n" podUID="b3223811-a857-4279-a671-189e65383481" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:39.314664 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:39.291200    5068 patch_prober.go:29] interesting pod/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Nov 04 01:17:39.314664 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:39.291265    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq" podUID="cbbfd346-7a24-49e7-8853-b60e0dbf754c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Nov 04 01:17:39.375645 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:39.372497    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8" podUID="eb9132e5-b90b-46fb-b86c-ca713419906d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded"
Nov 04 01:17:39.391754 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:39.379694    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq" podUID="b87aafac-9ab8-40e5-8d08-72eafc4187dd" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Nov 04 01:17:39.392029 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:39.379744    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm" podUID="f9e7abd2-8737-4357-a778-d6492fd12f3f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": context deadline exceeded"
Nov 04 01:17:39.396717 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:39.379764    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz" podUID="4ccafa61-1c9b-4595-a511-3404d7479526" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": context deadline exceeded"
Nov 04 01:17:39.421719 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:39.379783    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5" podUID="60f5c87a-48f8-4ad9-847b-cff571cc9340" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Nov 04 01:17:39.421969 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:39.379806    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx" podUID="46d4ec7b-558d-4fb5-8011-177760a3291c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Nov 04 01:17:39.422087 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:39.379825    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89" podUID="6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": context deadline exceeded"
Nov 04 01:17:39.422199 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:39.380338    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf" podUID="32f60f59-a98f-46ba-9fb4-8420f77998da" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:39.986774 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:39.964085    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82" podUID="6cdd01de-ff61-494a-bdb3-c67025a49440" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:40.009867 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:39.991062    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq" podStartSLOduration=7.99103828 podStartE2EDuration="7.99103828s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:17:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:17:39.515349414 +0000 UTC m=+899.561910205" watchObservedRunningTime="2026-11-04 01:17:39.99103828 +0000 UTC m=+900.037599071"
Nov 04 01:17:40.104636 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:40.087055    5068 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n" podUID="b3223811-a857-4279-a671-189e65383481" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:40.361681 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:40.354023    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9" podUID="530fc578-6622-4c42-a384-02e73bf17b1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:40.442636 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:40.396996    5068 patch_prober.go:29] interesting pod/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Nov 04 01:17:40.442636 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:40.397077    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq" podUID="cbbfd346-7a24-49e7-8853-b60e0dbf754c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Nov 04 01:17:40.442636 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:40.409919    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n" podUID="b3223811-a857-4279-a671-189e65383481" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Nov 04 01:17:40.487853 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:40.476130    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8" podUID="eb9132e5-b90b-46fb-b86c-ca713419906d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded"
Nov 04 01:17:40.487853 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:40.476929    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq" podUID="b87aafac-9ab8-40e5-8d08-72eafc4187dd" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Nov 04 01:17:40.522637 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:40.491229    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm" podUID="f9e7abd2-8737-4357-a778-d6492fd12f3f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": context deadline exceeded"
Nov 04 01:17:40.625888 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:40.625778    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89" podUID="6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": context deadline exceeded"
Nov 04 01:17:40.660660 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:40.638055    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf" podUID="32f60f59-a98f-46ba-9fb4-8420f77998da" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:40.698635 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:40.678602    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz" podUID="4ccafa61-1c9b-4595-a511-3404d7479526" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:40.698635 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:40.693578    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5" podUID="60f5c87a-48f8-4ad9-847b-cff571cc9340" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:40.698635 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:40.695165    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx" podUID="46d4ec7b-558d-4fb5-8011-177760a3291c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:41.008694 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:41.002845    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82" podUID="6cdd01de-ff61-494a-bdb3-c67025a49440" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:41.242781 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:41.227931    5068 patch_prober.go:29] interesting pod/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Nov 04 01:17:41.242781 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:41.227995    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq" podUID="cbbfd346-7a24-49e7-8853-b60e0dbf754c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Nov 04 01:17:41.244503 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:41.244467    5068 patch_prober.go:29] interesting pod/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:17:41.265849 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:41.265808    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds" podUID="84f5c469-8fea-4b88-969b-d1eed184c268" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:41.359637 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:17:41.353979    5068 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.197s"
Nov 04 01:17:41.395639 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:41.390981    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9" podUID="530fc578-6622-4c42-a384-02e73bf17b1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:41.443453 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:41.415728    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n" podUID="b3223811-a857-4279-a671-189e65383481" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:41.489307 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:41.488874    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq" podUID="b87aafac-9ab8-40e5-8d08-72eafc4187dd" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:41.495629 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:41.491979    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm" podUID="f9e7abd2-8737-4357-a778-d6492fd12f3f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:41.545700 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:41.529490    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8" podUID="eb9132e5-b90b-46fb-b86c-ca713419906d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded"
Nov 04 01:17:41.545700 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:41.539706    5068 patch_prober.go:29] interesting pod/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:17:41.545700 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:41.539771    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz" podUID="52491e2f-36e5-4e6b-ae82-5795752be764" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:41.546635 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:41.546589    5068 patch_prober.go:29] interesting pod/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:17:41.546840 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:41.546813    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv" podUID="3bde9301-d2f8-4820-9044-463e387b3c3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:41.596657 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:41.580733    5068 patch_prober.go:29] interesting pod/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:17:41.596657 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:41.580840    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr" podUID="b21786c9-e7fa-43eb-bb36-f2633486b8b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:41.725793 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:41.713652    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89" podUID="6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": context deadline exceeded"
Nov 04 01:17:41.725793 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:41.714062    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf" podUID="32f60f59-a98f-46ba-9fb4-8420f77998da" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded"
Nov 04 01:17:41.761488 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:41.761446    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5" podUID="60f5c87a-48f8-4ad9-847b-cff571cc9340" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Nov 04 01:17:41.790266 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:41.788511    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx" podUID="46d4ec7b-558d-4fb5-8011-177760a3291c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Nov 04 01:17:41.790266 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:41.788928    5068 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89" podUID="6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:41.872449 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:41.872344    5068 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5" podUID="60f5c87a-48f8-4ad9-847b-cff571cc9340" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:41.955136 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:41.955093    5068 patch_prober.go:29] interesting pod/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:17:41.955423 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:41.955391    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj" podUID="239d3dec-a8fd-45e4-82ec-1ed4e1d01c72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:42.068633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:42.003570    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz" podUID="4ccafa61-1c9b-4595-a511-3404d7479526" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:42.225694 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:42.014050    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82" podUID="6cdd01de-ff61-494a-bdb3-c67025a49440" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:42.225694 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:42.215211    5068 patch_prober.go:29] interesting pod/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Nov 04 01:17:42.225694 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:42.219946    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq" podUID="cbbfd346-7a24-49e7-8853-b60e0dbf754c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Nov 04 01:17:42.447642 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:42.427919    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n" podUID="b3223811-a857-4279-a671-189e65383481" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:42.496694 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:42.468995    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9" podUID="530fc578-6622-4c42-a384-02e73bf17b1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Nov 04 01:17:42.514420 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:42.496966    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq" podUID="b87aafac-9ab8-40e5-8d08-72eafc4187dd" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:42.514420 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:42.498379    5068 patch_prober.go:29] interesting pod/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:17:42.514420 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:42.499436    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p" podUID="fa8debd2-b803-412c-9e4c-53917a748a59" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:42.514420 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:42.498428    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm" podUID="f9e7abd2-8737-4357-a778-d6492fd12f3f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:42.535645 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:42.534757    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8" podUID="eb9132e5-b90b-46fb-b86c-ca713419906d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded"
Nov 04 01:17:42.769861 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:42.766919    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89" podUID="6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:42.769861 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:42.768159    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf" podUID="32f60f59-a98f-46ba-9fb4-8420f77998da" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:42.769861 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:42.769149    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5" podUID="60f5c87a-48f8-4ad9-847b-cff571cc9340" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Nov 04 01:17:42.806779 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:42.794656    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx" podUID="46d4ec7b-558d-4fb5-8011-177760a3291c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Nov 04 01:17:42.823851 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:42.822750    5068 patch_prober.go:29] interesting pod/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:17:42.823851 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:42.822818    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n" podUID="d8725463-9c1f-4cee-b5ec-28fb3ca9f939" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:43.219855 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:43.219705    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82" podUID="6cdd01de-ff61-494a-bdb3-c67025a49440" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:43.227645 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:43.221032    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz" podUID="4ccafa61-1c9b-4595-a511-3404d7479526" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": context deadline exceeded"
Nov 04 01:17:43.317631 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:43.288734    5068 patch_prober.go:29] interesting pod/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Nov 04 01:17:43.317631 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:43.313810    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq" podUID="cbbfd346-7a24-49e7-8853-b60e0dbf754c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Nov 04 01:17:43.442665 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:43.435870    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n" podUID="b3223811-a857-4279-a671-189e65383481" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:43.517767 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:43.506767    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm" podUID="f9e7abd2-8737-4357-a778-d6492fd12f3f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:43.517767 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:43.506919    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9" podUID="530fc578-6622-4c42-a384-02e73bf17b1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:43.517767 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:43.506981    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq" podUID="b87aafac-9ab8-40e5-8d08-72eafc4187dd" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Nov 04 01:17:43.533044 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:43.532929    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tr8b2_21d1d23b-7b6e-4ab5-9593-cf7784fffea2/console-operator/2.log"
Nov 04 01:17:43.538002 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:43.534944    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8" podUID="eb9132e5-b90b-46fb-b86c-ca713419906d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:43.799162 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:43.799069    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89" podUID="6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:43.812627 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:43.800252    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5" podUID="60f5c87a-48f8-4ad9-847b-cff571cc9340" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:43.917117 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:43.830255    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9"
Nov 04 01:17:43.919685 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:43.911644    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf" podUID="32f60f59-a98f-46ba-9fb4-8420f77998da" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded"
Nov 04 01:17:43.970665 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:43.964203    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq"
Nov 04 01:17:44.042216 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:17:44.042173    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:17:44.057195 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:44.057098    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n"
Nov 04 01:17:44.059872 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:44.059844    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz"
Nov 04 01:17:44.061094 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:43.797783    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx" podUID="46d4ec7b-558d-4fb5-8011-177760a3291c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:44.121736 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:44.121700    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82"
Nov 04 01:17:44.183558 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:44.183527    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tr8b2_21d1d23b-7b6e-4ab5-9593-cf7784fffea2/console-operator/2.log"
Nov 04 01:17:44.204191 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:44.204135    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm"
Nov 04 01:17:44.218628 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:44.214674    5068 patch_prober.go:29] interesting pod/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Nov 04 01:17:44.218628 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:44.214732    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq" podUID="cbbfd346-7a24-49e7-8853-b60e0dbf754c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Nov 04 01:17:44.218628 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:44.216039    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8"
Nov 04 01:17:44.327027 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:44.326943    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89"
Nov 04 01:17:44.328036 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:44.328013    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf"
Nov 04 01:17:44.332790 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:44.332768    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx"
Nov 04 01:17:44.333015 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:44.332991    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5"
Nov 04 01:17:45.209150 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:45.209110    5068 patch_prober.go:29] interesting pod/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Nov 04 01:17:45.209369 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:45.209162    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq" podUID="cbbfd346-7a24-49e7-8853-b60e0dbf754c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Nov 04 01:17:46.209135 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:46.209095    5068 patch_prober.go:29] interesting pod/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Nov 04 01:17:46.209662 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:46.209163    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq" podUID="cbbfd346-7a24-49e7-8853-b60e0dbf754c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Nov 04 01:17:47.208887 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:47.208849    5068 patch_prober.go:29] interesting pod/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Nov 04 01:17:47.209249 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:47.209216    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq" podUID="cbbfd346-7a24-49e7-8853-b60e0dbf754c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Nov 04 01:17:48.208439 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:48.208404    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq"
Nov 04 01:17:48.208712 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:48.208685    5068 patch_prober.go:29] interesting pod/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Nov 04 01:17:48.208812 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:48.208738    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq" podUID="cbbfd346-7a24-49e7-8853-b60e0dbf754c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Nov 04 01:17:49.687011 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:49.686970    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq"
Nov 04 01:17:49.689264 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:49.689239    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq"
Nov 04 01:17:56.152640 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:56.149472    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7005-0af584"]
Nov 04 01:17:56.233630 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:56.209860    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7005-0af584"]
Nov 04 01:17:56.233630 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:56.210341    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7005-0af584"
Nov 04 01:17:56.233630 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:56.222019    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-lnpw7\""
Nov 04 01:17:56.443637 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:56.429877    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knktf\" (UniqueName: \"kubernetes.io/projected/d0487680-6528-4f82-a6a3-2ed48688c266-kube-api-access-knktf\") pod \"test-curl-7005-0af584\" (UID: \"d0487680-6528-4f82-a6a3-2ed48688c266\") " pod="opendatahub/test-curl-7005-0af584"
Nov 04 01:17:56.661659 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:56.540905    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knktf\" (UniqueName: \"kubernetes.io/projected/d0487680-6528-4f82-a6a3-2ed48688c266-kube-api-access-knktf\") pod \"test-curl-7005-0af584\" (UID: \"d0487680-6528-4f82-a6a3-2ed48688c266\") " pod="opendatahub/test-curl-7005-0af584"
Nov 04 01:17:56.685342 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:56.655779    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-knktf\" (UniqueName: \"kubernetes.io/projected/d0487680-6528-4f82-a6a3-2ed48688c266-kube-api-access-knktf\") pod \"test-curl-7005-0af584\" (UID: \"d0487680-6528-4f82-a6a3-2ed48688c266\") " pod="opendatahub/test-curl-7005-0af584"
Nov 04 01:17:56.715790 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:56.686836    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7005-0af584"
Nov 04 01:17:57.392195 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:57.391030    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz" podUID="4ccafa61-1c9b-4595-a511-3404d7479526" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:57.405200 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:57.404531    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm" podUID="f9e7abd2-8737-4357-a778-d6492fd12f3f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:57.432115 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:57.430756    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8" podUID="eb9132e5-b90b-46fb-b86c-ca713419906d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:57.457011 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:57.456969    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9" podUID="530fc578-6622-4c42-a384-02e73bf17b1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:57.459147 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:57.457254    5068 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx" podUID="46d4ec7b-558d-4fb5-8011-177760a3291c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:57.459147 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:57.457280    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89" podUID="6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:58.380687 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:58.372840    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf" podUID="32f60f59-a98f-46ba-9fb4-8420f77998da" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded"
Nov 04 01:17:58.396163 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:58.382460    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq" podUID="b87aafac-9ab8-40e5-8d08-72eafc4187dd" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:58.423936 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:58.410115    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm" podUID="f9e7abd2-8737-4357-a778-d6492fd12f3f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:58.423936 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:58.395746    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz" podUID="4ccafa61-1c9b-4595-a511-3404d7479526" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": context deadline exceeded"
Nov 04 01:17:58.423936 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:58.382493    5068 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm" podUID="f9e7abd2-8737-4357-a778-d6492fd12f3f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/health\": context deadline exceeded"
Nov 04 01:17:58.432317 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:58.432259    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8" podUID="eb9132e5-b90b-46fb-b86c-ca713419906d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded"
Nov 04 01:17:58.474658 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:58.467219    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89" podUID="6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:58.474658 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:58.467598    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9" podUID="530fc578-6622-4c42-a384-02e73bf17b1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Nov 04 01:17:58.588635 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:58.544785    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm"
Nov 04 01:17:58.589357 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:58.589331    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz"
Nov 04 01:17:58.663759 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:58.663721    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8"
Nov 04 01:17:58.693756 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:58.693726    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89"
Nov 04 01:17:58.694019 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:58.694006    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9"
Nov 04 01:17:58.694126 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:58.694115    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7005-0af584"]
Nov 04 01:17:58.727636 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:58.640397    5068 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 01:17:58.877799 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:58.563298    5068 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf" podUID="32f60f59-a98f-46ba-9fb4-8420f77998da" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/health\": context deadline exceeded"
Nov 04 01:17:59.122637 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:59.114718    5068 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8" podUID="eb9132e5-b90b-46fb-b86c-ca713419906d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/health\": context deadline exceeded"
Nov 04 01:17:59.278887 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:59.278830    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx" podUID="46d4ec7b-558d-4fb5-8011-177760a3291c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Nov 04 01:17:59.307514 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:59.284985    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82" podUID="6cdd01de-ff61-494a-bdb3-c67025a49440" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": context deadline exceeded"
Nov 04 01:17:59.307514 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:59.285191    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n" podUID="b3223811-a857-4279-a671-189e65383481" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Nov 04 01:17:59.307514 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:59.285233    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5" podUID="60f5c87a-48f8-4ad9-847b-cff571cc9340" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Nov 04 01:17:59.307514 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:59.286372    5068 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82" podUID="6cdd01de-ff61-494a-bdb3-c67025a49440" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/health\": context deadline exceeded"
Nov 04 01:17:59.404491 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:59.404378    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf" podUID="32f60f59-a98f-46ba-9fb4-8420f77998da" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded"
Nov 04 01:17:59.405094 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:59.404795    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf"
Nov 04 01:17:59.453122 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:59.450858    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm" podUID="f9e7abd2-8737-4357-a778-d6492fd12f3f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": context deadline exceeded"
Nov 04 01:17:59.453122 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:59.451235    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz" podUID="4ccafa61-1c9b-4595-a511-3404d7479526" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": context deadline exceeded"
Nov 04 01:17:59.453122 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:59.451471    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8" podUID="eb9132e5-b90b-46fb-b86c-ca713419906d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded"
Nov 04 01:17:59.495632 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:59.463021    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq" podUID="b87aafac-9ab8-40e5-8d08-72eafc4187dd" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:59.495632 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:59.477516    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq"
Nov 04 01:17:59.495632 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:59.479223    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9" podUID="530fc578-6622-4c42-a384-02e73bf17b1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:17:59.495632 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:59.479575    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89" podUID="6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": context deadline exceeded"
Nov 04 01:17:59.810691 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:17:59.810652    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7005-0af584" event={"ID":"d0487680-6528-4f82-a6a3-2ed48688c266","Type":"ContainerStarted","Data":"3327db473a50341a00a32fd25ee0929a51aa2cdf817ccfebf255c45a25dec489"}
Nov 04 01:18:00.058784 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:00.054933    5068 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n" podUID="b3223811-a857-4279-a671-189e65383481" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:00.307781 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:00.307722    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5" podUID="60f5c87a-48f8-4ad9-847b-cff571cc9340" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:00.312754 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:00.312714    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx" podUID="46d4ec7b-558d-4fb5-8011-177760a3291c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:00.343414 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:00.341790    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n" podUID="b3223811-a857-4279-a671-189e65383481" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Nov 04 01:18:00.343414 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:00.341830    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82" podUID="6cdd01de-ff61-494a-bdb3-c67025a49440" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": context deadline exceeded"
Nov 04 01:18:00.406913 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:00.404550    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf" podUID="32f60f59-a98f-46ba-9fb4-8420f77998da" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:00.451176 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:00.451132    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm" podUID="f9e7abd2-8737-4357-a778-d6492fd12f3f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:00.459307 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:00.459259    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8" podUID="eb9132e5-b90b-46fb-b86c-ca713419906d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:00.460868 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:00.460196    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5"
Nov 04 01:18:00.461534 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:00.461507    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx"
Nov 04 01:18:00.465993 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:00.465972    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n"
Nov 04 01:18:00.466229 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:00.466212    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82"
Nov 04 01:18:00.466822 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:00.466793    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq" podUID="b87aafac-9ab8-40e5-8d08-72eafc4187dd" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Nov 04 01:18:00.471306 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:00.460383    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz" podUID="4ccafa61-1c9b-4595-a511-3404d7479526" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": context deadline exceeded"
Nov 04 01:18:00.484359 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:00.484319    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89" podUID="6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:00.485872 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:00.485843    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9" podUID="530fc578-6622-4c42-a384-02e73bf17b1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Nov 04 01:18:00.940634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:00.933877    5068 patch_prober.go:29] interesting pod/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:18:00.940634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:00.933946    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v" podUID="9b28aa81-39ec-4761-a076-0e5a2db936cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:00.941155 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:00.941029    5068 patch_prober.go:29] interesting pod/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:18:00.941155 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:00.941087    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp" podUID="73f6a679-d6e5-45d1-a885-ebea2ed4f253" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:01.180197 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:01.180161    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5"
Nov 04 01:18:01.203703 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:01.199947    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf"
Nov 04 01:18:01.506688 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:01.439897    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx" podUID="46d4ec7b-558d-4fb5-8011-177760a3291c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:01.597109 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:01.450798    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n" podUID="b3223811-a857-4279-a671-189e65383481" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:01.599278 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:01.450865    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82" podUID="6cdd01de-ff61-494a-bdb3-c67025a49440" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:01.655706 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:01.457787    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm" podUID="f9e7abd2-8737-4357-a778-d6492fd12f3f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:01.655706 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:01.459762    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8" podUID="eb9132e5-b90b-46fb-b86c-ca713419906d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded"
Nov 04 01:18:01.655706 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:01.470766    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq" podUID="b87aafac-9ab8-40e5-8d08-72eafc4187dd" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:01.655706 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:01.471781    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz" podUID="4ccafa61-1c9b-4595-a511-3404d7479526" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": context deadline exceeded"
Nov 04 01:18:01.655706 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:01.488089    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9" podUID="530fc578-6622-4c42-a384-02e73bf17b1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:01.655706 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:01.488125    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89" podUID="6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": context deadline exceeded"
Nov 04 01:18:01.759689 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:01.755182    5068 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89" podUID="6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/health\": context deadline exceeded"
Nov 04 01:18:02.431635 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:02.431391    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf" podUID="32f60f59-a98f-46ba-9fb4-8420f77998da" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:02.598435 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:02.597362    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx" podUID="46d4ec7b-558d-4fb5-8011-177760a3291c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:02.599887 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:02.599106    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n" podUID="b3223811-a857-4279-a671-189e65383481" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:02.660886 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:02.658756    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89" podUID="6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:02.660886 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:02.659119    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm" podUID="f9e7abd2-8737-4357-a778-d6492fd12f3f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:02.660886 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:02.659377    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8" podUID="eb9132e5-b90b-46fb-b86c-ca713419906d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded"
Nov 04 01:18:02.660886 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:02.659649    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq" podUID="b87aafac-9ab8-40e5-8d08-72eafc4187dd" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Nov 04 01:18:02.660886 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:02.659902    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz" podUID="4ccafa61-1c9b-4595-a511-3404d7479526" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": context deadline exceeded"
Nov 04 01:18:02.660886 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:02.660155    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9" podUID="530fc578-6622-4c42-a384-02e73bf17b1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Nov 04 01:18:02.694429 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:02.693737    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82" podUID="6cdd01de-ff61-494a-bdb3-c67025a49440" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:03.193764 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:03.193710    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5" podUID="60f5c87a-48f8-4ad9-847b-cff571cc9340" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:03.434740 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:03.434694    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf" podUID="32f60f59-a98f-46ba-9fb4-8420f77998da" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:03.451687 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:03.435452    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf"
Nov 04 01:18:03.645682 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:03.643134    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n" podUID="b3223811-a857-4279-a671-189e65383481" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:03.683640 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:03.676185    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx" podUID="46d4ec7b-558d-4fb5-8011-177760a3291c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Nov 04 01:18:03.711094 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:03.683921    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9" podUID="530fc578-6622-4c42-a384-02e73bf17b1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:03.747633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:03.688086    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8" podUID="eb9132e5-b90b-46fb-b86c-ca713419906d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded"
Nov 04 01:18:03.747633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:03.688122    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89" podUID="6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": context deadline exceeded"
Nov 04 01:18:03.747633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:03.688144    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm" podUID="f9e7abd2-8737-4357-a778-d6492fd12f3f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": context deadline exceeded"
Nov 04 01:18:03.747633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:03.688169    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz" podUID="4ccafa61-1c9b-4595-a511-3404d7479526" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": context deadline exceeded"
Nov 04 01:18:03.747633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:03.688187    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq" podUID="b87aafac-9ab8-40e5-8d08-72eafc4187dd" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Nov 04 01:18:03.747633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:03.710679    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82" podUID="6cdd01de-ff61-494a-bdb3-c67025a49440" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:04.271630 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:04.270744    5068 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz" podUID="4ccafa61-1c9b-4595-a511-3404d7479526" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/health\": context deadline exceeded"
Nov 04 01:18:04.271630 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:04.270927    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5" podUID="60f5c87a-48f8-4ad9-847b-cff571cc9340" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Nov 04 01:18:04.271630 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:04.271541    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5"
Nov 04 01:18:04.436763 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:04.436708    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf" podUID="32f60f59-a98f-46ba-9fb4-8420f77998da" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:04.658294 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:04.658186    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n" podUID="b3223811-a857-4279-a671-189e65383481" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Nov 04 01:18:04.716122 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:04.716068    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx" podUID="46d4ec7b-558d-4fb5-8011-177760a3291c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Nov 04 01:18:04.758685 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:04.746754    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82" podUID="6cdd01de-ff61-494a-bdb3-c67025a49440" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": context deadline exceeded"
Nov 04 01:18:04.758685 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:04.747213    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9" podUID="530fc578-6622-4c42-a384-02e73bf17b1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Nov 04 01:18:04.758685 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:04.747556    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8" podUID="eb9132e5-b90b-46fb-b86c-ca713419906d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded"
Nov 04 01:18:04.758685 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:04.747882    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89" podUID="6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": context deadline exceeded"
Nov 04 01:18:04.758685 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:04.748152    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm" podUID="f9e7abd2-8737-4357-a778-d6492fd12f3f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": context deadline exceeded"
Nov 04 01:18:04.758685 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:04.748433    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz" podUID="4ccafa61-1c9b-4595-a511-3404d7479526" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": context deadline exceeded"
Nov 04 01:18:04.758685 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:04.748745    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq" podUID="b87aafac-9ab8-40e5-8d08-72eafc4187dd" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Nov 04 01:18:05.373770 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:05.373725    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5" podUID="60f5c87a-48f8-4ad9-847b-cff571cc9340" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:05.509855 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:05.509810    5068 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9" podUID="530fc578-6622-4c42-a384-02e73bf17b1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:05.529968 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:05.529925    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf" podUID="32f60f59-a98f-46ba-9fb4-8420f77998da" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:05.659461 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:05.659382    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n"
Nov 04 01:18:05.695004 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:05.694975    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz"
Nov 04 01:18:05.709690 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:05.701991    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5"
Nov 04 01:18:05.744108 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:05.744080    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm"
Nov 04 01:18:05.752756 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:05.751795    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82" podUID="6cdd01de-ff61-494a-bdb3-c67025a49440" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": context deadline exceeded"
Nov 04 01:18:05.752756 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:05.751825    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89" podUID="6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": context deadline exceeded"
Nov 04 01:18:05.752756 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:05.751910    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq" podUID="b87aafac-9ab8-40e5-8d08-72eafc4187dd" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:05.752756 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:05.752122    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9" podUID="530fc578-6622-4c42-a384-02e73bf17b1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Nov 04 01:18:05.752756 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:05.752369    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8" podUID="eb9132e5-b90b-46fb-b86c-ca713419906d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded"
Nov 04 01:18:05.753322 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:05.753297    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx" podUID="46d4ec7b-558d-4fb5-8011-177760a3291c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:05.820052 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:05.820021    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf"
Nov 04 01:18:05.820634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:05.820536    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8"
Nov 04 01:18:05.888458 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:05.884412    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89"
Nov 04 01:18:05.888458 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:05.887419    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq"
Nov 04 01:18:05.894036 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:05.894004    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9"
Nov 04 01:18:05.895057 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:05.895022    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx"
Nov 04 01:18:05.895270 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:05.895255    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82"
Nov 04 01:18:09.784075 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:09.782646    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7005-0af584" event={"ID":"d0487680-6528-4f82-a6a3-2ed48688c266","Type":"ContainerStarted","Data":"84bb5eb9fb432b5fbf2ce758b915401b9e6f14795fb6b58cf55bee402515f2c5"}
Nov 04 01:18:09.809584 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:09.809536    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7005-0af584" podStartSLOduration=3.811410547 podStartE2EDuration="13.809521638s" totalImagesPullingTime="9.998111091s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:17:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:18:09.807589542 +0000 UTC m=+929.854150335" watchObservedRunningTime="2026-11-04 01:18:09.809521638 +0000 UTC m=+929.856082428"
Nov 04 01:18:10.595417 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:10.595375    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7005-0af584"]
Nov 04 01:18:10.608631 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:10.606342    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7005-0af584"]
Nov 04 01:18:10.765199 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:10.765163    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7005-ffcb5a"]
Nov 04 01:18:10.766167 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:10.766140    5068 cpu_manager.go:414] "containerMap: removing container" podUID="d0487680-6528-4f82-a6a3-2ed48688c266" containerName="test-curl-7005-0af584"
Nov 04 01:18:10.766167 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:10.766164    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d0487680-6528-4f82-a6a3-2ed48688c266" containerName="test-curl-7005-0af584"
Nov 04 01:18:10.766394 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:10.766307    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="d0487680-6528-4f82-a6a3-2ed48688c266" containerName="test-curl-7005-0af584"
Nov 04 01:18:10.770684 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:10.770584    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7005-ffcb5a"
Nov 04 01:18:10.774820 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:10.774791    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7005-ffcb5a"]
Nov 04 01:18:10.849851 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:10.849748    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2gqd\" (UniqueName: \"kubernetes.io/projected/1268daed-a99c-4de8-b0e6-fd96b67b18b6-kube-api-access-v2gqd\") pod \"test-curl-7005-ffcb5a\" (UID: \"1268daed-a99c-4de8-b0e6-fd96b67b18b6\") " pod="opendatahub/test-curl-7005-ffcb5a"
Nov 04 01:18:10.951498 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:10.951443    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2gqd\" (UniqueName: \"kubernetes.io/projected/1268daed-a99c-4de8-b0e6-fd96b67b18b6-kube-api-access-v2gqd\") pod \"test-curl-7005-ffcb5a\" (UID: \"1268daed-a99c-4de8-b0e6-fd96b67b18b6\") " pod="opendatahub/test-curl-7005-ffcb5a"
Nov 04 01:18:10.969384 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:10.969354    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2gqd\" (UniqueName: \"kubernetes.io/projected/1268daed-a99c-4de8-b0e6-fd96b67b18b6-kube-api-access-v2gqd\") pod \"test-curl-7005-ffcb5a\" (UID: \"1268daed-a99c-4de8-b0e6-fd96b67b18b6\") " pod="opendatahub/test-curl-7005-ffcb5a"
Nov 04 01:18:11.085130 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:11.085099    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7005-ffcb5a"
Nov 04 01:18:11.371884 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:11.371851    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7005-ffcb5a"]
Nov 04 01:18:11.372276 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:18:11.372252    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1268daed_a99c_4de8_b0e6_fd96b67b18b6.slice/crio-9c00dbae1cdb8ec313ac0df7cbed7133312155bc3cb1d2b906589974f396fd23 WatchSource:0}: Error finding container 9c00dbae1cdb8ec313ac0df7cbed7133312155bc3cb1d2b906589974f396fd23: Status 404 returned error can't find the container with id 9c00dbae1cdb8ec313ac0df7cbed7133312155bc3cb1d2b906589974f396fd23
Nov 04 01:18:11.811769 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:11.811731    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7005-ffcb5a" event={"ID":"1268daed-a99c-4de8-b0e6-fd96b67b18b6","Type":"ContainerStarted","Data":"3203d6d96c069809309a0ee121df3fb6b67a3db8be86bca21a7eb5570b22405b"}
Nov 04 01:18:11.811991 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:11.811783    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7005-ffcb5a" event={"ID":"1268daed-a99c-4de8-b0e6-fd96b67b18b6","Type":"ContainerStarted","Data":"9c00dbae1cdb8ec313ac0df7cbed7133312155bc3cb1d2b906589974f396fd23"}
Nov 04 01:18:11.813342 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:11.812316    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7005-0af584" podUID="d0487680-6528-4f82-a6a3-2ed48688c266" containerName="test-curl-7005-0af584" containerID="cri-o://84bb5eb9fb432b5fbf2ce758b915401b9e6f14795fb6b58cf55bee402515f2c5" gracePeriod=2
Nov 04 01:18:11.835095 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:11.835025    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7005-ffcb5a" podStartSLOduration=1.835009047 podStartE2EDuration="1.835009047s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:18:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:18:11.830510708 +0000 UTC m=+931.877071499" watchObservedRunningTime="2026-11-04 01:18:11.835009047 +0000 UTC m=+931.881569838"
Nov 04 01:18:11.836495 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:18:11.836462    5068 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7005-0af584\" is forbidden: User \"system:node:ip-10-0-132-67.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-67.us-east-2.compute.internal' and this object" podUID="d0487680-6528-4f82-a6a3-2ed48688c266" pod="opendatahub/test-curl-7005-0af584"
Nov 04 01:18:12.251865 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:12.248331    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-383a864b-8f97d964c-svcwd"]
Nov 04 01:18:12.260022 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:12.259771    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-383a864b-8f97d964c-svcwd"
Nov 04 01:18:12.262863 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:12.262272    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-383a864b-8f97d964c-svcwd"]
Nov 04 01:18:12.274365 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:12.270567    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-383a864b\""
Nov 04 01:18:12.274365 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:12.270668    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-383a864b-dockercfg-q49x9\""
Nov 04 01:18:12.281878 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:12.281849    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/606d1ef1-1350-4025-9017-e784ccddace7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-383a864b-8f97d964c-svcwd\" (UID: \"606d1ef1-1350-4025-9017-e784ccddace7\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-383a864b-8f97d964c-svcwd"
Nov 04 01:18:12.282050 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:12.282036    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qnwt\" (UniqueName: \"kubernetes.io/projected/606d1ef1-1350-4025-9017-e784ccddace7-kube-api-access-8qnwt\") pod \"payload-pre-processing-e2e-ait-383a864b-8f97d964c-svcwd\" (UID: \"606d1ef1-1350-4025-9017-e784ccddace7\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-383a864b-8f97d964c-svcwd"
Nov 04 01:18:12.402649 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:12.389721    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qnwt\" (UniqueName: \"kubernetes.io/projected/606d1ef1-1350-4025-9017-e784ccddace7-kube-api-access-8qnwt\") pod \"payload-pre-processing-e2e-ait-383a864b-8f97d964c-svcwd\" (UID: \"606d1ef1-1350-4025-9017-e784ccddace7\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-383a864b-8f97d964c-svcwd"
Nov 04 01:18:12.402649 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:12.397877    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/606d1ef1-1350-4025-9017-e784ccddace7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-383a864b-8f97d964c-svcwd\" (UID: \"606d1ef1-1350-4025-9017-e784ccddace7\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-383a864b-8f97d964c-svcwd"
Nov 04 01:18:12.402649 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:12.398821    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/606d1ef1-1350-4025-9017-e784ccddace7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-383a864b-8f97d964c-svcwd\" (UID: \"606d1ef1-1350-4025-9017-e784ccddace7\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-383a864b-8f97d964c-svcwd"
Nov 04 01:18:12.551551 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:12.551498    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qnwt\" (UniqueName: \"kubernetes.io/projected/606d1ef1-1350-4025-9017-e784ccddace7-kube-api-access-8qnwt\") pod \"payload-pre-processing-e2e-ait-383a864b-8f97d964c-svcwd\" (UID: \"606d1ef1-1350-4025-9017-e784ccddace7\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-383a864b-8f97d964c-svcwd"
Nov 04 01:18:12.629740 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:12.608134    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-383a864b-8f97d964c-svcwd"
Nov 04 01:18:12.918370 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:12.918274    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7005-ffcb5a"]
Nov 04 01:18:12.924798 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:12.924767    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7005-ffcb5a"]
Nov 04 01:18:12.968471 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:12.968440    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-383a864b-8f97d964c-svcwd"]
Nov 04 01:18:13.224696 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:13.224602    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8f95141e-5dc7877c9b-79xw5"]
Nov 04 01:18:13.225353 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:13.225334    5068 cpu_manager.go:414] "containerMap: removing container" podUID="1268daed-a99c-4de8-b0e6-fd96b67b18b6" containerName="test-curl-7005-ffcb5a"
Nov 04 01:18:13.225400 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:13.225358    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1268daed-a99c-4de8-b0e6-fd96b67b18b6" containerName="test-curl-7005-ffcb5a"
Nov 04 01:18:13.225568 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:13.225557    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="1268daed-a99c-4de8-b0e6-fd96b67b18b6" containerName="test-curl-7005-ffcb5a"
Nov 04 01:18:13.229245 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:13.229226    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-8f95141e-5dc7877c9b-79xw5"
Nov 04 01:18:13.233862 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:13.233823    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-8f95141e\""
Nov 04 01:18:13.235598 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:13.234602    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-8f95141e-dockercfg-wcsmg\""
Nov 04 01:18:13.240238 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:13.240220    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8f95141e-5dc7877c9b-79xw5"]
Nov 04 01:18:13.247026 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:13.247004    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-8f95141e-5fdd4bfb8c-rf5pd"]
Nov 04 01:18:13.252142 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:13.252124    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-8f95141e-5fdd4bfb8c-rf5pd"
Nov 04 01:18:13.263189 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:13.263160    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-8f95141e-5fdd4bfb8c-rf5pd"]
Nov 04 01:18:13.371633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:13.363676    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svhg8\" (UniqueName: \"kubernetes.io/projected/90c7d91f-388a-4cb1-9669-e2686768af46-kube-api-access-svhg8\") pod \"payload-pre-processing-e2e-ait-8f95141e-5fdd4bfb8c-rf5pd\" (UID: \"90c7d91f-388a-4cb1-9669-e2686768af46\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-8f95141e-5fdd4bfb8c-rf5pd"
Nov 04 01:18:13.371633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:13.363788    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tpt2\" (UniqueName: \"kubernetes.io/projected/38a6ee6d-4cac-4204-a973-30bb8cc22102-kube-api-access-8tpt2\") pod \"payload-processing-e2e-ait-8f95141e-5dc7877c9b-79xw5\" (UID: \"38a6ee6d-4cac-4204-a973-30bb8cc22102\") " pod="openshift-ingress/payload-processing-e2e-ait-8f95141e-5dc7877c9b-79xw5"
Nov 04 01:18:13.371633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:13.363824    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/90c7d91f-388a-4cb1-9669-e2686768af46-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-8f95141e-5fdd4bfb8c-rf5pd\" (UID: \"90c7d91f-388a-4cb1-9669-e2686768af46\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-8f95141e-5fdd4bfb8c-rf5pd"
Nov 04 01:18:13.371633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:13.364119    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38a6ee6d-4cac-4204-a973-30bb8cc22102-plugins-config-volume\") pod \"payload-processing-e2e-ait-8f95141e-5dc7877c9b-79xw5\" (UID: \"38a6ee6d-4cac-4204-a973-30bb8cc22102\") " pod="openshift-ingress/payload-processing-e2e-ait-8f95141e-5dc7877c9b-79xw5"
Nov 04 01:18:13.498749 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:13.485846    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38a6ee6d-4cac-4204-a973-30bb8cc22102-plugins-config-volume\") pod \"payload-processing-e2e-ait-8f95141e-5dc7877c9b-79xw5\" (UID: \"38a6ee6d-4cac-4204-a973-30bb8cc22102\") " pod="openshift-ingress/payload-processing-e2e-ait-8f95141e-5dc7877c9b-79xw5"
Nov 04 01:18:13.498749 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:13.487167    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38a6ee6d-4cac-4204-a973-30bb8cc22102-plugins-config-volume\") pod \"payload-processing-e2e-ait-8f95141e-5dc7877c9b-79xw5\" (UID: \"38a6ee6d-4cac-4204-a973-30bb8cc22102\") " pod="openshift-ingress/payload-processing-e2e-ait-8f95141e-5dc7877c9b-79xw5"
Nov 04 01:18:13.498749 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:13.487275    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svhg8\" (UniqueName: \"kubernetes.io/projected/90c7d91f-388a-4cb1-9669-e2686768af46-kube-api-access-svhg8\") pod \"payload-pre-processing-e2e-ait-8f95141e-5fdd4bfb8c-rf5pd\" (UID: \"90c7d91f-388a-4cb1-9669-e2686768af46\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-8f95141e-5fdd4bfb8c-rf5pd"
Nov 04 01:18:13.498749 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:13.487714    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tpt2\" (UniqueName: \"kubernetes.io/projected/38a6ee6d-4cac-4204-a973-30bb8cc22102-kube-api-access-8tpt2\") pod \"payload-processing-e2e-ait-8f95141e-5dc7877c9b-79xw5\" (UID: \"38a6ee6d-4cac-4204-a973-30bb8cc22102\") " pod="openshift-ingress/payload-processing-e2e-ait-8f95141e-5dc7877c9b-79xw5"
Nov 04 01:18:13.509634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:13.504129    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/90c7d91f-388a-4cb1-9669-e2686768af46-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-8f95141e-5fdd4bfb8c-rf5pd\" (UID: \"90c7d91f-388a-4cb1-9669-e2686768af46\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-8f95141e-5fdd4bfb8c-rf5pd"
Nov 04 01:18:13.509634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:13.505725    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/90c7d91f-388a-4cb1-9669-e2686768af46-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-8f95141e-5fdd4bfb8c-rf5pd\" (UID: \"90c7d91f-388a-4cb1-9669-e2686768af46\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-8f95141e-5fdd4bfb8c-rf5pd"
Nov 04 01:18:13.567636 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:13.556355    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tpt2\" (UniqueName: \"kubernetes.io/projected/38a6ee6d-4cac-4204-a973-30bb8cc22102-kube-api-access-8tpt2\") pod \"payload-processing-e2e-ait-8f95141e-5dc7877c9b-79xw5\" (UID: \"38a6ee6d-4cac-4204-a973-30bb8cc22102\") " pod="openshift-ingress/payload-processing-e2e-ait-8f95141e-5dc7877c9b-79xw5"
Nov 04 01:18:13.579884 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:13.579851    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-svhg8\" (UniqueName: \"kubernetes.io/projected/90c7d91f-388a-4cb1-9669-e2686768af46-kube-api-access-svhg8\") pod \"payload-pre-processing-e2e-ait-8f95141e-5fdd4bfb8c-rf5pd\" (UID: \"90c7d91f-388a-4cb1-9669-e2686768af46\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-8f95141e-5fdd4bfb8c-rf5pd"
Nov 04 01:18:13.610634 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:18:13.610582    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:18:13.835567 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:13.835535    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-383a864b-8f97d964c-svcwd" event={"ID":"606d1ef1-1350-4025-9017-e784ccddace7","Type":"ContainerStarted","Data":"2220c36b0bb863d114c482f6c57b73c09d8c08647a01134ce1e0863550cc2df5"}
Nov 04 01:18:13.835716 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:13.835594    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-383a864b-8f97d964c-svcwd" event={"ID":"606d1ef1-1350-4025-9017-e784ccddace7","Type":"ContainerStarted","Data":"ae3eecd1e92b3f83e4ecb24dfa92190fb021f02dbbd4cc8bb1d24b18d9245900"}
Nov 04 01:18:13.835716 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:13.835638    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-383a864b-8f97d964c-svcwd"
Nov 04 01:18:13.835873 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:13.835824    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7005-ffcb5a" podUID="1268daed-a99c-4de8-b0e6-fd96b67b18b6" containerName="test-curl-7005-ffcb5a" containerID="cri-o://3203d6d96c069809309a0ee121df3fb6b67a3db8be86bca21a7eb5570b22405b" gracePeriod=2
Nov 04 01:18:13.846464 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:13.846439    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-8f95141e-5dc7877c9b-79xw5"
Nov 04 01:18:13.864157 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:18:13.862287    5068 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7005-ffcb5a\" is forbidden: User \"system:node:ip-10-0-132-67.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-67.us-east-2.compute.internal' and this object" podUID="1268daed-a99c-4de8-b0e6-fd96b67b18b6" pod="opendatahub/test-curl-7005-ffcb5a"
Nov 04 01:18:13.864157 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:13.863628    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-383a864b-8f97d964c-svcwd" podStartSLOduration=1.863596524 podStartE2EDuration="1.863596524s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:18:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:18:13.859840989 +0000 UTC m=+933.906401794" watchObservedRunningTime="2026-11-04 01:18:13.863596524 +0000 UTC m=+933.910157314"
Nov 04 01:18:13.870651 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:13.870584    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-8f95141e-5fdd4bfb8c-rf5pd"
Nov 04 01:18:14.162107 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:18:14.162075    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38a6ee6d_4cac_4204_a973_30bb8cc22102.slice/crio-d254b183d1e72e8d1eb1175ac5f54d4825ee9277b50c00ff19be0bcc24379079 WatchSource:0}: Error finding container d254b183d1e72e8d1eb1175ac5f54d4825ee9277b50c00ff19be0bcc24379079: Status 404 returned error can't find the container with id d254b183d1e72e8d1eb1175ac5f54d4825ee9277b50c00ff19be0bcc24379079
Nov 04 01:18:14.165385 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:14.165356    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8f95141e-5dc7877c9b-79xw5"]
Nov 04 01:18:14.171738 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:14.171698    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7005-0af584"
Nov 04 01:18:14.218697 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:14.218671    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d0487680-6528-4f82-a6a3-2ed48688c266-kube-api-access-knktf\" (UniqueName: \"kubernetes.io/projected/d0487680-6528-4f82-a6a3-2ed48688c266-kube-api-access-knktf\") pod \"d0487680-6528-4f82-a6a3-2ed48688c266\" (UID: \"d0487680-6528-4f82-a6a3-2ed48688c266\") "
Nov 04 01:18:14.221822 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:14.221788    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0487680-6528-4f82-a6a3-2ed48688c266-kube-api-access-knktf" pod "d0487680-6528-4f82-a6a3-2ed48688c266" (UID: "d0487680-6528-4f82-a6a3-2ed48688c266"). InnerVolumeSpecName "kube-api-access-knktf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:18:14.231764 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:18:14.231735    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90c7d91f_388a_4cb1_9669_e2686768af46.slice/crio-f27a5dc0b78ca89d1fe3edba02d5a87fadf3aa7457bb5c068cff9db150a0e239 WatchSource:0}: Error finding container f27a5dc0b78ca89d1fe3edba02d5a87fadf3aa7457bb5c068cff9db150a0e239: Status 404 returned error can't find the container with id f27a5dc0b78ca89d1fe3edba02d5a87fadf3aa7457bb5c068cff9db150a0e239
Nov 04 01:18:14.233324 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:14.233299    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-8f95141e-5fdd4bfb8c-rf5pd"]
Nov 04 01:18:14.321056 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:14.321026    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-knktf\" (UniqueName: \"kubernetes.io/projected/d0487680-6528-4f82-a6a3-2ed48688c266-kube-api-access-knktf\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:18:14.873687 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:14.867397    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-8f95141e-5fdd4bfb8c-rf5pd" event={"ID":"90c7d91f-388a-4cb1-9669-e2686768af46","Type":"ContainerStarted","Data":"34babe53f254b3d09736a2dd9db1cfc5bb8086dc3917f056168d5ee7a1694cad"}
Nov 04 01:18:14.873687 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:14.867459    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-8f95141e-5fdd4bfb8c-rf5pd" event={"ID":"90c7d91f-388a-4cb1-9669-e2686768af46","Type":"ContainerStarted","Data":"f27a5dc0b78ca89d1fe3edba02d5a87fadf3aa7457bb5c068cff9db150a0e239"}
Nov 04 01:18:14.873687 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:14.868580    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-8f95141e-5fdd4bfb8c-rf5pd"
Nov 04 01:18:14.874794 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:14.874669    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-8f95141e-5dc7877c9b-79xw5" event={"ID":"38a6ee6d-4cac-4204-a973-30bb8cc22102","Type":"ContainerStarted","Data":"63e3c2745678c50955285a60e34ec507ca683d3ae701a3e7dc8e4e8f989ae94b"}
Nov 04 01:18:14.874794 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:14.874736    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-8f95141e-5dc7877c9b-79xw5" event={"ID":"38a6ee6d-4cac-4204-a973-30bb8cc22102","Type":"ContainerStarted","Data":"d254b183d1e72e8d1eb1175ac5f54d4825ee9277b50c00ff19be0bcc24379079"}
Nov 04 01:18:14.875110 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:14.875060    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-8f95141e-5dc7877c9b-79xw5"
Nov 04 01:18:14.875379 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:14.875343    5068 generic.go:356] "Generic (PLEG): container finished" podID="d0487680-6528-4f82-a6a3-2ed48688c266" containerID="84bb5eb9fb432b5fbf2ce758b915401b9e6f14795fb6b58cf55bee402515f2c5" exitCode=137
Nov 04 01:18:14.880006 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:14.875845    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7005-0af584"
Nov 04 01:18:14.880006 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:14.875960    5068 scope.go:122] "RemoveContainer" containerID="84bb5eb9fb432b5fbf2ce758b915401b9e6f14795fb6b58cf55bee402515f2c5"
Nov 04 01:18:14.945653 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:18:14.939010    5068 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7005-0af584\" is forbidden: User \"system:node:ip-10-0-132-67.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-67.us-east-2.compute.internal' and this object" podUID="d0487680-6528-4f82-a6a3-2ed48688c266" pod="opendatahub/test-curl-7005-0af584"
Nov 04 01:18:14.959644 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:14.959565    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-8f95141e-5fdd4bfb8c-rf5pd" podStartSLOduration=1.959545668 podStartE2EDuration="1.959545668s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:18:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:18:14.894097947 +0000 UTC m=+934.940658737" watchObservedRunningTime="2026-11-04 01:18:14.959545668 +0000 UTC m=+935.006106457"
Nov 04 01:18:15.007852 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:15.003793    5068 scope.go:122] "RemoveContainer" containerID="84bb5eb9fb432b5fbf2ce758b915401b9e6f14795fb6b58cf55bee402515f2c5"
Nov 04 01:18:15.007852 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:18:15.004238    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84bb5eb9fb432b5fbf2ce758b915401b9e6f14795fb6b58cf55bee402515f2c5\": container with ID starting with 84bb5eb9fb432b5fbf2ce758b915401b9e6f14795fb6b58cf55bee402515f2c5 not found: ID does not exist" containerID="84bb5eb9fb432b5fbf2ce758b915401b9e6f14795fb6b58cf55bee402515f2c5"
Nov 04 01:18:15.007852 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:15.004291    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84bb5eb9fb432b5fbf2ce758b915401b9e6f14795fb6b58cf55bee402515f2c5"} err="failed to get container status \"84bb5eb9fb432b5fbf2ce758b915401b9e6f14795fb6b58cf55bee402515f2c5\": rpc error: code = NotFound desc = could not find container \"84bb5eb9fb432b5fbf2ce758b915401b9e6f14795fb6b58cf55bee402515f2c5\": container with ID starting with 84bb5eb9fb432b5fbf2ce758b915401b9e6f14795fb6b58cf55bee402515f2c5 not found: ID does not exist"
Nov 04 01:18:15.009493 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:15.009438    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-8f95141e-5dc7877c9b-79xw5" podStartSLOduration=2.009423475 podStartE2EDuration="2.009423475s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:18:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:18:14.93315539 +0000 UTC m=+934.979716184" watchObservedRunningTime="2026-11-04 01:18:15.009423475 +0000 UTC m=+935.055984267"
Nov 04 01:18:15.736387 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:15.736350    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9cc22d46-6f7fdfbcf8-4chb2"]
Nov 04 01:18:15.744109 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:15.744071    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9cc22d46-6f7fdfbcf8-4chb2"
Nov 04 01:18:15.755542 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:15.754875    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-9cc22d46-serving-cert\""
Nov 04 01:18:15.755542 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:15.755469    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9cc22d46-6f7fdfbcf8-4chb2"]
Nov 04 01:18:15.773630 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:18:15.773582    5068 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7005-0af584\" is forbidden: User \"system:node:ip-10-0-132-67.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-67.us-east-2.compute.internal' and this object" podUID="d0487680-6528-4f82-a6a3-2ed48688c266" pod="opendatahub/test-curl-7005-0af584"
Nov 04 01:18:15.891791 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:15.891755    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e18d5c36-c7e9-4785-b665-a425ea01ef97-metrics-certs\") pod \"maas-api-e2e-ait-9cc22d46-6f7fdfbcf8-4chb2\" (UID: \"e18d5c36-c7e9-4785-b665-a425ea01ef97\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9cc22d46-6f7fdfbcf8-4chb2"
Nov 04 01:18:15.892131 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:15.892102    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7mf9\" (UniqueName: \"kubernetes.io/projected/e18d5c36-c7e9-4785-b665-a425ea01ef97-kube-api-access-b7mf9\") pod \"maas-api-e2e-ait-9cc22d46-6f7fdfbcf8-4chb2\" (UID: \"e18d5c36-c7e9-4785-b665-a425ea01ef97\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9cc22d46-6f7fdfbcf8-4chb2"
Nov 04 01:18:15.892440 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:15.892424    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e18d5c36-c7e9-4785-b665-a425ea01ef97-maas-api-tls\") pod \"maas-api-e2e-ait-9cc22d46-6f7fdfbcf8-4chb2\" (UID: \"e18d5c36-c7e9-4785-b665-a425ea01ef97\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9cc22d46-6f7fdfbcf8-4chb2"
Nov 04 01:18:15.892572 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:15.892559    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e18d5c36-c7e9-4785-b665-a425ea01ef97-openshift-service-ca\") pod \"maas-api-e2e-ait-9cc22d46-6f7fdfbcf8-4chb2\" (UID: \"e18d5c36-c7e9-4785-b665-a425ea01ef97\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9cc22d46-6f7fdfbcf8-4chb2"
Nov 04 01:18:15.994830 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:15.994512    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e18d5c36-c7e9-4785-b665-a425ea01ef97-maas-api-tls\") pod \"maas-api-e2e-ait-9cc22d46-6f7fdfbcf8-4chb2\" (UID: \"e18d5c36-c7e9-4785-b665-a425ea01ef97\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9cc22d46-6f7fdfbcf8-4chb2"
Nov 04 01:18:15.995108 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:15.995078    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e18d5c36-c7e9-4785-b665-a425ea01ef97-openshift-service-ca\") pod \"maas-api-e2e-ait-9cc22d46-6f7fdfbcf8-4chb2\" (UID: \"e18d5c36-c7e9-4785-b665-a425ea01ef97\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9cc22d46-6f7fdfbcf8-4chb2"
Nov 04 01:18:15.996239 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:15.995306    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e18d5c36-c7e9-4785-b665-a425ea01ef97-metrics-certs\") pod \"maas-api-e2e-ait-9cc22d46-6f7fdfbcf8-4chb2\" (UID: \"e18d5c36-c7e9-4785-b665-a425ea01ef97\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9cc22d46-6f7fdfbcf8-4chb2"
Nov 04 01:18:15.996239 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:15.995349    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7mf9\" (UniqueName: \"kubernetes.io/projected/e18d5c36-c7e9-4785-b665-a425ea01ef97-kube-api-access-b7mf9\") pod \"maas-api-e2e-ait-9cc22d46-6f7fdfbcf8-4chb2\" (UID: \"e18d5c36-c7e9-4785-b665-a425ea01ef97\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9cc22d46-6f7fdfbcf8-4chb2"
Nov 04 01:18:15.996239 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:15.995988    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e18d5c36-c7e9-4785-b665-a425ea01ef97-openshift-service-ca\") pod \"maas-api-e2e-ait-9cc22d46-6f7fdfbcf8-4chb2\" (UID: \"e18d5c36-c7e9-4785-b665-a425ea01ef97\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9cc22d46-6f7fdfbcf8-4chb2"
Nov 04 01:18:15.997824 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:15.997778    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e18d5c36-c7e9-4785-b665-a425ea01ef97-maas-api-tls\") pod \"maas-api-e2e-ait-9cc22d46-6f7fdfbcf8-4chb2\" (UID: \"e18d5c36-c7e9-4785-b665-a425ea01ef97\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9cc22d46-6f7fdfbcf8-4chb2"
Nov 04 01:18:16.000038 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:16.000011    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e18d5c36-c7e9-4785-b665-a425ea01ef97-metrics-certs\") pod \"maas-api-e2e-ait-9cc22d46-6f7fdfbcf8-4chb2\" (UID: \"e18d5c36-c7e9-4785-b665-a425ea01ef97\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9cc22d46-6f7fdfbcf8-4chb2"
Nov 04 01:18:16.011053 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:16.010999    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7mf9\" (UniqueName: \"kubernetes.io/projected/e18d5c36-c7e9-4785-b665-a425ea01ef97-kube-api-access-b7mf9\") pod \"maas-api-e2e-ait-9cc22d46-6f7fdfbcf8-4chb2\" (UID: \"e18d5c36-c7e9-4785-b665-a425ea01ef97\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9cc22d46-6f7fdfbcf8-4chb2"
Nov 04 01:18:16.076245 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:16.076211    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9cc22d46-6f7fdfbcf8-4chb2"
Nov 04 01:18:16.120845 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:16.120809    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d0487680-6528-4f82-a6a3-2ed48688c266" path="/var/lib/kubelet/pods/d0487680-6528-4f82-a6a3-2ed48688c266/volumes"
Nov 04 01:18:16.129810 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:16.129782    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7005-ffcb5a"
Nov 04 01:18:16.205436 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:16.203316    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1268daed-a99c-4de8-b0e6-fd96b67b18b6-kube-api-access-v2gqd\" (UniqueName: \"kubernetes.io/projected/1268daed-a99c-4de8-b0e6-fd96b67b18b6-kube-api-access-v2gqd\") pod \"1268daed-a99c-4de8-b0e6-fd96b67b18b6\" (UID: \"1268daed-a99c-4de8-b0e6-fd96b67b18b6\") "
Nov 04 01:18:16.209825 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:16.209782    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1268daed-a99c-4de8-b0e6-fd96b67b18b6-kube-api-access-v2gqd" pod "1268daed-a99c-4de8-b0e6-fd96b67b18b6" (UID: "1268daed-a99c-4de8-b0e6-fd96b67b18b6"). InnerVolumeSpecName "kube-api-access-v2gqd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:18:16.305921 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:16.305887    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v2gqd\" (UniqueName: \"kubernetes.io/projected/1268daed-a99c-4de8-b0e6-fd96b67b18b6-kube-api-access-v2gqd\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:18:16.369035 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:18:16.369003    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode18d5c36_c7e9_4785_b665_a425ea01ef97.slice/crio-a2dd018747bb89d736c930a4f9b3f9ce1b273a00939a913d99889f9c0b9373fd WatchSource:0}: Error finding container a2dd018747bb89d736c930a4f9b3f9ce1b273a00939a913d99889f9c0b9373fd: Status 404 returned error can't find the container with id a2dd018747bb89d736c930a4f9b3f9ce1b273a00939a913d99889f9c0b9373fd
Nov 04 01:18:16.369681 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:16.369646    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9cc22d46-6f7fdfbcf8-4chb2"]
Nov 04 01:18:16.958325 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:16.958291    5068 generic.go:356] "Generic (PLEG): container finished" podID="1268daed-a99c-4de8-b0e6-fd96b67b18b6" containerID="3203d6d96c069809309a0ee121df3fb6b67a3db8be86bca21a7eb5570b22405b" exitCode=137
Nov 04 01:18:16.959110 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:16.959087    5068 scope.go:122] "RemoveContainer" containerID="3203d6d96c069809309a0ee121df3fb6b67a3db8be86bca21a7eb5570b22405b"
Nov 04 01:18:16.959352 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:16.959335    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7005-ffcb5a"
Nov 04 01:18:16.985984 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:16.985952    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9cc22d46-6f7fdfbcf8-4chb2" event={"ID":"e18d5c36-c7e9-4785-b665-a425ea01ef97","Type":"ContainerStarted","Data":"a2dd018747bb89d736c930a4f9b3f9ce1b273a00939a913d99889f9c0b9373fd"}
Nov 04 01:18:17.066639 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:17.066594    5068 scope.go:122] "RemoveContainer" containerID="3203d6d96c069809309a0ee121df3fb6b67a3db8be86bca21a7eb5570b22405b"
Nov 04 01:18:17.067070 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:18:17.067037    5068 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7005-ffcb5a\" is forbidden: User \"system:node:ip-10-0-132-67.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-67.us-east-2.compute.internal' and this object" podUID="1268daed-a99c-4de8-b0e6-fd96b67b18b6" pod="opendatahub/test-curl-7005-ffcb5a"
Nov 04 01:18:17.067663 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:18:17.067636    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3203d6d96c069809309a0ee121df3fb6b67a3db8be86bca21a7eb5570b22405b\": container with ID starting with 3203d6d96c069809309a0ee121df3fb6b67a3db8be86bca21a7eb5570b22405b not found: ID does not exist" containerID="3203d6d96c069809309a0ee121df3fb6b67a3db8be86bca21a7eb5570b22405b"
Nov 04 01:18:17.067832 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:17.067808    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3203d6d96c069809309a0ee121df3fb6b67a3db8be86bca21a7eb5570b22405b"} err="failed to get container status \"3203d6d96c069809309a0ee121df3fb6b67a3db8be86bca21a7eb5570b22405b\": rpc error: code = NotFound desc = could not find container \"3203d6d96c069809309a0ee121df3fb6b67a3db8be86bca21a7eb5570b22405b\": container with ID starting with 3203d6d96c069809309a0ee121df3fb6b67a3db8be86bca21a7eb5570b22405b not found: ID does not exist"
Nov 04 01:18:17.995976 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:17.995927    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9cc22d46-6f7fdfbcf8-4chb2" event={"ID":"e18d5c36-c7e9-4785-b665-a425ea01ef97","Type":"ContainerStarted","Data":"0ddebabfb5fc74d6fd2a8caddee675a2164c7558818539c18ead44830ea52483"}
Nov 04 01:18:17.996605 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:17.996245    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9cc22d46-6f7fdfbcf8-4chb2"
Nov 04 01:18:18.021211 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:18:18.021172    5068 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7005-ffcb5a\" is forbidden: User \"system:node:ip-10-0-132-67.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-67.us-east-2.compute.internal' and this object" podUID="1268daed-a99c-4de8-b0e6-fd96b67b18b6" pod="opendatahub/test-curl-7005-ffcb5a"
Nov 04 01:18:18.022068 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:18.022016    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9cc22d46-6f7fdfbcf8-4chb2" podStartSLOduration=2.490953192 podStartE2EDuration="3.022003797s" totalImagesPullingTime="531.050605ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:18:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:18:18.018992871 +0000 UTC m=+938.065553662" watchObservedRunningTime="2026-11-04 01:18:18.022003797 +0000 UTC m=+938.068564587"
Nov 04 01:18:18.118364 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:18.118330    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1268daed-a99c-4de8-b0e6-fd96b67b18b6" path="/var/lib/kubelet/pods/1268daed-a99c-4de8-b0e6-fd96b67b18b6/volumes"
Nov 04 01:18:20.797509 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:20.797417    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8f95141e-667b749fc6-8b8zv"]
Nov 04 01:18:20.804493 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:20.804463    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f95141e-667b749fc6-8b8zv"
Nov 04 01:18:20.818921 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:20.818876    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-8f95141e-serving-cert\""
Nov 04 01:18:20.821093 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:20.821063    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8f95141e-667b749fc6-8b8zv"]
Nov 04 01:18:20.899624 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:20.899582    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bncbm\" (UniqueName: \"kubernetes.io/projected/ef7dacdd-803b-4207-b4c3-d75cc50541d8-kube-api-access-bncbm\") pod \"maas-api-e2e-ait-8f95141e-667b749fc6-8b8zv\" (UID: \"ef7dacdd-803b-4207-b4c3-d75cc50541d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f95141e-667b749fc6-8b8zv"
Nov 04 01:18:20.899814 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:20.899788    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef7dacdd-803b-4207-b4c3-d75cc50541d8-metrics-certs\") pod \"maas-api-e2e-ait-8f95141e-667b749fc6-8b8zv\" (UID: \"ef7dacdd-803b-4207-b4c3-d75cc50541d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f95141e-667b749fc6-8b8zv"
Nov 04 01:18:20.899867 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:20.899852    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef7dacdd-803b-4207-b4c3-d75cc50541d8-openshift-service-ca\") pod \"maas-api-e2e-ait-8f95141e-667b749fc6-8b8zv\" (UID: \"ef7dacdd-803b-4207-b4c3-d75cc50541d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f95141e-667b749fc6-8b8zv"
Nov 04 01:18:20.899998 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:20.899976    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef7dacdd-803b-4207-b4c3-d75cc50541d8-maas-api-tls\") pod \"maas-api-e2e-ait-8f95141e-667b749fc6-8b8zv\" (UID: \"ef7dacdd-803b-4207-b4c3-d75cc50541d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f95141e-667b749fc6-8b8zv"
Nov 04 01:18:21.001198 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:21.001165    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bncbm\" (UniqueName: \"kubernetes.io/projected/ef7dacdd-803b-4207-b4c3-d75cc50541d8-kube-api-access-bncbm\") pod \"maas-api-e2e-ait-8f95141e-667b749fc6-8b8zv\" (UID: \"ef7dacdd-803b-4207-b4c3-d75cc50541d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f95141e-667b749fc6-8b8zv"
Nov 04 01:18:21.001398 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:21.001314    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef7dacdd-803b-4207-b4c3-d75cc50541d8-metrics-certs\") pod \"maas-api-e2e-ait-8f95141e-667b749fc6-8b8zv\" (UID: \"ef7dacdd-803b-4207-b4c3-d75cc50541d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f95141e-667b749fc6-8b8zv"
Nov 04 01:18:21.001398 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:21.001349    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef7dacdd-803b-4207-b4c3-d75cc50541d8-openshift-service-ca\") pod \"maas-api-e2e-ait-8f95141e-667b749fc6-8b8zv\" (UID: \"ef7dacdd-803b-4207-b4c3-d75cc50541d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f95141e-667b749fc6-8b8zv"
Nov 04 01:18:21.002139 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:21.002111    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef7dacdd-803b-4207-b4c3-d75cc50541d8-openshift-service-ca\") pod \"maas-api-e2e-ait-8f95141e-667b749fc6-8b8zv\" (UID: \"ef7dacdd-803b-4207-b4c3-d75cc50541d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f95141e-667b749fc6-8b8zv"
Nov 04 01:18:21.002278 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:21.002204    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef7dacdd-803b-4207-b4c3-d75cc50541d8-maas-api-tls\") pod \"maas-api-e2e-ait-8f95141e-667b749fc6-8b8zv\" (UID: \"ef7dacdd-803b-4207-b4c3-d75cc50541d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f95141e-667b749fc6-8b8zv"
Nov 04 01:18:21.002491 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:18:21.002472    5068 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-8f95141e-serving-cert: secret "maas-api-e2e-ait-8f95141e-serving-cert" not found
Nov 04 01:18:21.002557 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:18:21.002535    5068 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef7dacdd-803b-4207-b4c3-d75cc50541d8-maas-api-tls podName:ef7dacdd-803b-4207-b4c3-d75cc50541d8 nodeName:}" failed. No retries permitted until 2026-11-04 01:18:21.502516972 +0000 UTC m=+941.549077755 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ef7dacdd-803b-4207-b4c3-d75cc50541d8-maas-api-tls") pod "maas-api-e2e-ait-8f95141e-667b749fc6-8b8zv" (UID: "ef7dacdd-803b-4207-b4c3-d75cc50541d8") : secret "maas-api-e2e-ait-8f95141e-serving-cert" not found
Nov 04 01:18:21.006526 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:21.005228    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef7dacdd-803b-4207-b4c3-d75cc50541d8-metrics-certs\") pod \"maas-api-e2e-ait-8f95141e-667b749fc6-8b8zv\" (UID: \"ef7dacdd-803b-4207-b4c3-d75cc50541d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f95141e-667b749fc6-8b8zv"
Nov 04 01:18:21.016966 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:21.016937    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bncbm\" (UniqueName: \"kubernetes.io/projected/ef7dacdd-803b-4207-b4c3-d75cc50541d8-kube-api-access-bncbm\") pod \"maas-api-e2e-ait-8f95141e-667b749fc6-8b8zv\" (UID: \"ef7dacdd-803b-4207-b4c3-d75cc50541d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f95141e-667b749fc6-8b8zv"
Nov 04 01:18:21.509190 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:21.509156    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef7dacdd-803b-4207-b4c3-d75cc50541d8-maas-api-tls\") pod \"maas-api-e2e-ait-8f95141e-667b749fc6-8b8zv\" (UID: \"ef7dacdd-803b-4207-b4c3-d75cc50541d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f95141e-667b749fc6-8b8zv"
Nov 04 01:18:21.511872 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:21.511845    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef7dacdd-803b-4207-b4c3-d75cc50541d8-maas-api-tls\") pod \"maas-api-e2e-ait-8f95141e-667b749fc6-8b8zv\" (UID: \"ef7dacdd-803b-4207-b4c3-d75cc50541d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f95141e-667b749fc6-8b8zv"
Nov 04 01:18:21.737504 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:21.737180    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f95141e-667b749fc6-8b8zv"
Nov 04 01:18:22.110925 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:18:22.110891    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef7dacdd_803b_4207_b4c3_d75cc50541d8.slice/crio-f9db8c930d4b5042ebce92fba28096082bfa6efa6df92167693bb5c01e3e0673 WatchSource:0}: Error finding container f9db8c930d4b5042ebce92fba28096082bfa6efa6df92167693bb5c01e3e0673: Status 404 returned error can't find the container with id f9db8c930d4b5042ebce92fba28096082bfa6efa6df92167693bb5c01e3e0673
Nov 04 01:18:22.116668 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:22.116641    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8f95141e-667b749fc6-8b8zv"]
Nov 04 01:18:23.065566 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:23.065527    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f95141e-667b749fc6-8b8zv" event={"ID":"ef7dacdd-803b-4207-b4c3-d75cc50541d8","Type":"ContainerStarted","Data":"499c29397e07020caba4dcbaa8198e79789b9cecdd89377ce19be1a657d8c055"}
Nov 04 01:18:23.065763 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:23.065646    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f95141e-667b749fc6-8b8zv" event={"ID":"ef7dacdd-803b-4207-b4c3-d75cc50541d8","Type":"ContainerStarted","Data":"f9db8c930d4b5042ebce92fba28096082bfa6efa6df92167693bb5c01e3e0673"}
Nov 04 01:18:23.065866 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:23.065788    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f95141e-667b749fc6-8b8zv"
Nov 04 01:18:23.087183 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:23.087131    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f95141e-667b749fc6-8b8zv" podStartSLOduration=2.500828453 podStartE2EDuration="3.087116791s" totalImagesPullingTime="586.288338ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:18:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:18:23.085031025 +0000 UTC m=+943.131591816" watchObservedRunningTime="2026-11-04 01:18:23.087116791 +0000 UTC m=+943.133677581"
Nov 04 01:18:24.016299 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:24.016272    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9cc22d46-6f7fdfbcf8-4chb2"
Nov 04 01:18:24.879569 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:24.879541    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-383a864b-8f97d964c-svcwd"
Nov 04 01:18:25.912256 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:25.912228    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-8f95141e-5dc7877c9b-79xw5"
Nov 04 01:18:26.232912 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:26.232827    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-gs6sr"]
Nov 04 01:18:26.237376 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:26.237356    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-gs6sr"
Nov 04 01:18:26.246302 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:26.246276    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Nov 04 01:18:26.247220 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:26.247198    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Nov 04 01:18:26.247442 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:26.247413    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-mdkcc\""
Nov 04 01:18:26.247549 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:26.247493    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Nov 04 01:18:26.253565 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:26.253540    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-gs6sr"]
Nov 04 01:18:26.325394 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:26.325346    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9fe2e235-4232-4929-b61c-e31045e8e3f0-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-gs6sr\" (UID: \"9fe2e235-4232-4929-b61c-e31045e8e3f0\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-gs6sr"
Nov 04 01:18:26.325572 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:26.325528    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64vr2\" (UniqueName: \"kubernetes.io/projected/9fe2e235-4232-4929-b61c-e31045e8e3f0-kube-api-access-64vr2\") pod \"istiod-openshift-gateway-7ddd9cfdc8-gs6sr\" (UID: \"9fe2e235-4232-4929-b61c-e31045e8e3f0\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-gs6sr"
Nov 04 01:18:26.325676 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:26.325584    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fe2e235-4232-4929-b61c-e31045e8e3f0-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-gs6sr\" (UID: \"9fe2e235-4232-4929-b61c-e31045e8e3f0\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-gs6sr"
Nov 04 01:18:26.325676 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:26.325643    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9fe2e235-4232-4929-b61c-e31045e8e3f0-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-gs6sr\" (UID: \"9fe2e235-4232-4929-b61c-e31045e8e3f0\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-gs6sr"
Nov 04 01:18:26.325892 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:26.325859    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9fe2e235-4232-4929-b61c-e31045e8e3f0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-gs6sr\" (UID: \"9fe2e235-4232-4929-b61c-e31045e8e3f0\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-gs6sr"
Nov 04 01:18:26.325942 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:26.325929    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fe2e235-4232-4929-b61c-e31045e8e3f0-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-gs6sr\" (UID: \"9fe2e235-4232-4929-b61c-e31045e8e3f0\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-gs6sr"
Nov 04 01:18:26.326010 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:26.325994    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9fe2e235-4232-4929-b61c-e31045e8e3f0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-gs6sr\" (UID: \"9fe2e235-4232-4929-b61c-e31045e8e3f0\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-gs6sr"
Nov 04 01:18:26.427401 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:26.427362    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9fe2e235-4232-4929-b61c-e31045e8e3f0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-gs6sr\" (UID: \"9fe2e235-4232-4929-b61c-e31045e8e3f0\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-gs6sr"
Nov 04 01:18:26.427575 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:26.427547    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9fe2e235-4232-4929-b61c-e31045e8e3f0-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-gs6sr\" (UID: \"9fe2e235-4232-4929-b61c-e31045e8e3f0\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-gs6sr"
Nov 04 01:18:26.427715 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:26.427698    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64vr2\" (UniqueName: \"kubernetes.io/projected/9fe2e235-4232-4929-b61c-e31045e8e3f0-kube-api-access-64vr2\") pod \"istiod-openshift-gateway-7ddd9cfdc8-gs6sr\" (UID: \"9fe2e235-4232-4929-b61c-e31045e8e3f0\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-gs6sr"
Nov 04 01:18:26.427767 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:26.427754    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fe2e235-4232-4929-b61c-e31045e8e3f0-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-gs6sr\" (UID: \"9fe2e235-4232-4929-b61c-e31045e8e3f0\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-gs6sr"
Nov 04 01:18:26.427808 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:26.427789    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9fe2e235-4232-4929-b61c-e31045e8e3f0-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-gs6sr\" (UID: \"9fe2e235-4232-4929-b61c-e31045e8e3f0\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-gs6sr"
Nov 04 01:18:26.427881 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:26.427867    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9fe2e235-4232-4929-b61c-e31045e8e3f0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-gs6sr\" (UID: \"9fe2e235-4232-4929-b61c-e31045e8e3f0\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-gs6sr"
Nov 04 01:18:26.427959 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:26.427902    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fe2e235-4232-4929-b61c-e31045e8e3f0-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-gs6sr\" (UID: \"9fe2e235-4232-4929-b61c-e31045e8e3f0\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-gs6sr"
Nov 04 01:18:26.429729 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:26.429689    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9fe2e235-4232-4929-b61c-e31045e8e3f0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-gs6sr\" (UID: \"9fe2e235-4232-4929-b61c-e31045e8e3f0\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-gs6sr"
Nov 04 01:18:26.430169 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:26.430140    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fe2e235-4232-4929-b61c-e31045e8e3f0-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-gs6sr\" (UID: \"9fe2e235-4232-4929-b61c-e31045e8e3f0\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-gs6sr"
Nov 04 01:18:26.430273 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:26.430140    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9fe2e235-4232-4929-b61c-e31045e8e3f0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-gs6sr\" (UID: \"9fe2e235-4232-4929-b61c-e31045e8e3f0\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-gs6sr"
Nov 04 01:18:26.430322 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:26.430267    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9fe2e235-4232-4929-b61c-e31045e8e3f0-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-gs6sr\" (UID: \"9fe2e235-4232-4929-b61c-e31045e8e3f0\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-gs6sr"
Nov 04 01:18:26.430450 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:26.430429    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9fe2e235-4232-4929-b61c-e31045e8e3f0-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-gs6sr\" (UID: \"9fe2e235-4232-4929-b61c-e31045e8e3f0\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-gs6sr"
Nov 04 01:18:26.439649 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:26.439584    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fe2e235-4232-4929-b61c-e31045e8e3f0-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-gs6sr\" (UID: \"9fe2e235-4232-4929-b61c-e31045e8e3f0\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-gs6sr"
Nov 04 01:18:26.439902 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:26.439885    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-64vr2\" (UniqueName: \"kubernetes.io/projected/9fe2e235-4232-4929-b61c-e31045e8e3f0-kube-api-access-64vr2\") pod \"istiod-openshift-gateway-7ddd9cfdc8-gs6sr\" (UID: \"9fe2e235-4232-4929-b61c-e31045e8e3f0\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-gs6sr"
Nov 04 01:18:26.556026 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:26.555994    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-gs6sr"
Nov 04 01:18:26.927050 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:26.916143    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-gs6sr"]
Nov 04 01:18:26.990442 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:26.990413    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-8f95141e-5fdd4bfb8c-rf5pd"
Nov 04 01:18:27.107281 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:27.107247    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-gs6sr" event={"ID":"9fe2e235-4232-4929-b61c-e31045e8e3f0","Type":"ContainerStarted","Data":"5458afecb29883af59fb21684358f38bc8a9345aca0a18436973b9723c831748"}
Nov 04 01:18:29.088705 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:29.088666    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f95141e-667b749fc6-8b8zv"
Nov 04 01:18:29.951062 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:29.951024    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:18:30.751130 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:30.751099    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:18:30.751420 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:30.751165    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:18:31.210756 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:31.210602    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-gs6sr" event={"ID":"9fe2e235-4232-4929-b61c-e31045e8e3f0","Type":"ContainerStarted","Data":"36580e421920bc1cd7b144c3f9b79517821d203bfd8695acc64b95e475f0b495"}
Nov 04 01:18:31.214790 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:31.214762    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-gs6sr"
Nov 04 01:18:31.227175 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:31.224869    5068 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-gs6sr container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 01:18:31.227175 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:31.224953    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-gs6sr" podUID="9fe2e235-4232-4929-b61c-e31045e8e3f0" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 01:18:31.251506 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:31.251428    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-gs6sr" podStartSLOduration=1.445776451 podStartE2EDuration="5.251408744s" totalImagesPullingTime="3.805632293s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:18:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:18:31.249147547 +0000 UTC m=+951.295708351" watchObservedRunningTime="2026-11-04 01:18:31.251408744 +0000 UTC m=+951.297969535"
Nov 04 01:18:32.289524 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:32.288978    5068 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-gs6sr container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 01:18:32.289524 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:32.289062    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-gs6sr" podUID="9fe2e235-4232-4929-b61c-e31045e8e3f0" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 01:18:32.461637 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:32.460799    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm" podUID="f9e7abd2-8737-4357-a778-d6492fd12f3f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:32.461637 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:32.461242    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz" podUID="4ccafa61-1c9b-4595-a511-3404d7479526" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": context deadline exceeded"
Nov 04 01:18:32.480787 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:32.478709    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq" podUID="b87aafac-9ab8-40e5-8d08-72eafc4187dd" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:32.496635 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:32.483688    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9" podUID="530fc578-6622-4c42-a384-02e73bf17b1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:32.566693 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:32.553198    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf" podUID="32f60f59-a98f-46ba-9fb4-8420f77998da" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded"
Nov 04 01:18:32.612635 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:32.611755    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n" podUID="b3223811-a857-4279-a671-189e65383481" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:32.612635 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:32.612216    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx" podUID="46d4ec7b-558d-4fb5-8011-177760a3291c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Nov 04 01:18:32.673721 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:32.672743    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89" podUID="6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:32.707409 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:32.707358    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82" podUID="6cdd01de-ff61-494a-bdb3-c67025a49440" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:32.751499 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:32.751395    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8" podUID="eb9132e5-b90b-46fb-b86c-ca713419906d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:33.481644 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:33.473283    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5" podUID="60f5c87a-48f8-4ad9-847b-cff571cc9340" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Nov 04 01:18:33.544528 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:33.482244    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq" podUID="b87aafac-9ab8-40e5-8d08-72eafc4187dd" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:33.546638 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:33.484255    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9" podUID="530fc578-6622-4c42-a384-02e73bf17b1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:33.548787 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:33.547446    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq"
Nov 04 01:18:33.548787 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:33.547525    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9"
Nov 04 01:18:33.793930 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:33.484763    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz" podUID="4ccafa61-1c9b-4595-a511-3404d7479526" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": context deadline exceeded"
Nov 04 01:18:33.800754 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:33.484735    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm" podUID="f9e7abd2-8737-4357-a778-d6492fd12f3f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:33.937645 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:33.623179    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx" podUID="46d4ec7b-558d-4fb5-8011-177760a3291c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:33.992497 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:33.623217    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n" podUID="b3223811-a857-4279-a671-189e65383481" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Nov 04 01:18:34.000768 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:33.653758    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf" podUID="32f60f59-a98f-46ba-9fb4-8420f77998da" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:34.000768 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:33.687287    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89" podUID="6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:34.000768 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:33.708676    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82" podUID="6cdd01de-ff61-494a-bdb3-c67025a49440" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:34.000768 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:33.793867    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8" podUID="eb9132e5-b90b-46fb-b86c-ca713419906d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded"
Nov 04 01:18:34.000768 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:33.802013    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz"
Nov 04 01:18:34.128242 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:34.027787    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm"
Nov 04 01:18:34.349914 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:34.349865    5068 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz" podUID="4ccafa61-1c9b-4595-a511-3404d7479526" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:34.549641 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:34.548180    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9" podUID="530fc578-6622-4c42-a384-02e73bf17b1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:34.579068 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:34.574681    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq" podUID="b87aafac-9ab8-40e5-8d08-72eafc4187dd" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:34.579068 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:34.575636    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5" podUID="60f5c87a-48f8-4ad9-847b-cff571cc9340" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Nov 04 01:18:34.579068 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:34.578462    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx"
Nov 04 01:18:34.579068 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:34.578511    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n"
Nov 04 01:18:34.579068 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:34.578548    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf"
Nov 04 01:18:34.579068 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:34.578585    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89"
Nov 04 01:18:34.579068 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:34.578639    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82"
Nov 04 01:18:34.579068 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:34.578676    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8"
Nov 04 01:18:34.579068 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:34.578712    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-gs6sr"
Nov 04 01:18:34.579068 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:34.578757    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5"
Nov 04 01:18:34.799994 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:34.799892    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz" podUID="4ccafa61-1c9b-4595-a511-3404d7479526" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": context deadline exceeded"
Nov 04 01:18:34.819491 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:34.819450    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm" podUID="f9e7abd2-8737-4357-a778-d6492fd12f3f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:35.020642 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:35.008038    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8" podUID="eb9132e5-b90b-46fb-b86c-ca713419906d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:35.037035 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:35.036984    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx" podUID="46d4ec7b-558d-4fb5-8011-177760a3291c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:35.039196 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:35.039159    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n" podUID="b3223811-a857-4279-a671-189e65383481" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:35.228196 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:35.220148    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf" podUID="32f60f59-a98f-46ba-9fb4-8420f77998da" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded"
Nov 04 01:18:35.245679 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:35.243541    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89" podUID="6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": context deadline exceeded"
Nov 04 01:18:35.245679 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:35.245237    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82" podUID="6cdd01de-ff61-494a-bdb3-c67025a49440" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": context deadline exceeded"
Nov 04 01:18:35.604861 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:35.604812    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq" podUID="b87aafac-9ab8-40e5-8d08-72eafc4187dd" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:35.686668 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:35.609040    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9" podUID="530fc578-6622-4c42-a384-02e73bf17b1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Nov 04 01:18:35.818636 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:35.797547    5068 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9" podUID="530fc578-6622-4c42-a384-02e73bf17b1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/health\": context deadline exceeded"
Nov 04 01:18:35.823495 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:35.823442    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz" podUID="4ccafa61-1c9b-4595-a511-3404d7479526" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:35.864495 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:35.826190    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5" podUID="60f5c87a-48f8-4ad9-847b-cff571cc9340" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:35.868072 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:35.834909    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm" podUID="f9e7abd2-8737-4357-a778-d6492fd12f3f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:35.876105 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:35.858363    5068 patch_prober.go:29] interesting pod/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:18:35.876381 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:35.876350    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq" podUID="cbbfd346-7a24-49e7-8853-b60e0dbf754c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:36.013865 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:36.013817    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8" podUID="eb9132e5-b90b-46fb-b86c-ca713419906d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded"
Nov 04 01:18:36.071810 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:36.055492    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx" podUID="46d4ec7b-558d-4fb5-8011-177760a3291c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Nov 04 01:18:36.111219 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:36.111171    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n" podUID="b3223811-a857-4279-a671-189e65383481" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:36.226687 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:36.222882    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf" podUID="32f60f59-a98f-46ba-9fb4-8420f77998da" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:36.264157 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:36.250047    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82" podUID="6cdd01de-ff61-494a-bdb3-c67025a49440" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:36.264157 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:36.250482    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89" podUID="6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:36.662082 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:36.662038    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq" podUID="b87aafac-9ab8-40e5-8d08-72eafc4187dd" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:36.706637 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:36.698723    5068 patch_prober.go:29] interesting pod/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:18:36.706637 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:36.698790    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f" podUID="7ca2a53a-ada2-4565-bc5a-66c7eefef997" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:36.770830 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:36.770788    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9" podUID="530fc578-6622-4c42-a384-02e73bf17b1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:36.865590 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:36.865544    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz" podUID="4ccafa61-1c9b-4595-a511-3404d7479526" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:36.865978 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:36.865949    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5" podUID="60f5c87a-48f8-4ad9-847b-cff571cc9340" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:36.878988 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:36.878943    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm" podUID="f9e7abd2-8737-4357-a778-d6492fd12f3f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:37.015574 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:37.015044    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8" podUID="eb9132e5-b90b-46fb-b86c-ca713419906d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded"
Nov 04 01:18:37.081765 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:37.081710    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx" podUID="46d4ec7b-558d-4fb5-8011-177760a3291c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:37.126830 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:37.126758    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n" podUID="b3223811-a857-4279-a671-189e65383481" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Nov 04 01:18:37.233783 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:37.233722    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf" podUID="32f60f59-a98f-46ba-9fb4-8420f77998da" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded"
Nov 04 01:18:37.255236 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:37.255189    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82" podUID="6cdd01de-ff61-494a-bdb3-c67025a49440" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:37.255957 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:37.255488    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89" podUID="6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": context deadline exceeded"
Nov 04 01:18:37.517261 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:37.516268    5068 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx" podUID="46d4ec7b-558d-4fb5-8011-177760a3291c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:37.691646 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:37.686289    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq" podUID="b87aafac-9ab8-40e5-8d08-72eafc4187dd" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:37.782695 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:37.781094    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9" podUID="530fc578-6622-4c42-a384-02e73bf17b1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:37.867023 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:37.866977    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5" podUID="60f5c87a-48f8-4ad9-847b-cff571cc9340" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Nov 04 01:18:37.867638 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:37.867257    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz" podUID="4ccafa61-1c9b-4595-a511-3404d7479526" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": context deadline exceeded"
Nov 04 01:18:37.930680 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:37.930648    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89"
Nov 04 01:18:37.935287 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:37.931443    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm" podUID="f9e7abd2-8737-4357-a778-d6492fd12f3f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:38.023333 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:38.022769    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8" podUID="eb9132e5-b90b-46fb-b86c-ca713419906d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded"
Nov 04 01:18:38.095242 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:38.094628    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx" podUID="46d4ec7b-558d-4fb5-8011-177760a3291c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:38.138369 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:38.135741    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n" podUID="b3223811-a857-4279-a671-189e65383481" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Nov 04 01:18:38.189978 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:38.189925    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8"
Nov 04 01:18:38.235188 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:38.234250    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf" podUID="32f60f59-a98f-46ba-9fb4-8420f77998da" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:38.268004 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:38.267636    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82" podUID="6cdd01de-ff61-494a-bdb3-c67025a49440" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:38.529263 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:38.523696    5068 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf" podUID="32f60f59-a98f-46ba-9fb4-8420f77998da" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:38.714206 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:38.714167    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq" podUID="b87aafac-9ab8-40e5-8d08-72eafc4187dd" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:38.725050 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:38.724640    5068 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82" podUID="6cdd01de-ff61-494a-bdb3-c67025a49440" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:38.806075 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:38.805973    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9" podUID="530fc578-6622-4c42-a384-02e73bf17b1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Nov 04 01:18:38.867785 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:38.867732    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5" podUID="60f5c87a-48f8-4ad9-847b-cff571cc9340" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:38.868803 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:38.868768    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz" podUID="4ccafa61-1c9b-4595-a511-3404d7479526" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:39.027490 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:39.027441    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm" podUID="f9e7abd2-8737-4357-a778-d6492fd12f3f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": context deadline exceeded"
Nov 04 01:18:39.097756 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:39.097662    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx" podUID="46d4ec7b-558d-4fb5-8011-177760a3291c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:39.143685 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:39.142706    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n" podUID="b3223811-a857-4279-a671-189e65383481" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Nov 04 01:18:39.201773 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:39.200504    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm"
Nov 04 01:18:39.239100 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:39.239062    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf" podUID="32f60f59-a98f-46ba-9fb4-8420f77998da" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:39.268737 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:39.268691    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82" podUID="6cdd01de-ff61-494a-bdb3-c67025a49440" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": context deadline exceeded"
Nov 04 01:18:39.738786 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:39.738732    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq" podUID="b87aafac-9ab8-40e5-8d08-72eafc4187dd" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:39.883251 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:39.865936    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9" podUID="530fc578-6622-4c42-a384-02e73bf17b1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Nov 04 01:18:39.894877 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:39.883592    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz" podUID="4ccafa61-1c9b-4595-a511-3404d7479526" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:39.894877 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:39.883991    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5" podUID="60f5c87a-48f8-4ad9-847b-cff571cc9340" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Nov 04 01:18:40.255634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:40.254376    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n" podUID="b3223811-a857-4279-a671-189e65383481" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Nov 04 01:18:40.263640 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:40.255945    5068 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n" podUID="b3223811-a857-4279-a671-189e65383481" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:40.264645 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:40.264233    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx" podUID="46d4ec7b-558d-4fb5-8011-177760a3291c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:40.312636 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:40.299784    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf" podUID="32f60f59-a98f-46ba-9fb4-8420f77998da" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:40.393634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:40.298726    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82" podUID="6cdd01de-ff61-494a-bdb3-c67025a49440" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:40.753631 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:40.753294    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq" podUID="b87aafac-9ab8-40e5-8d08-72eafc4187dd" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Nov 04 01:18:40.939634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:40.871851    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9" podUID="530fc578-6622-4c42-a384-02e73bf17b1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:40.939634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:40.874384    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm" podUID="f9e7abd2-8737-4357-a778-d6492fd12f3f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": context deadline exceeded"
Nov 04 01:18:41.028888 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:41.013548    5068 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-848f6bf8bc-mdxsf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:18:41.028888 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:41.013633    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-mdxsf" podUID="8fea1dc2-7dc8-44b2-ae40-e84791006ba0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:41.087741 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:41.071176    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz" podUID="4ccafa61-1c9b-4595-a511-3404d7479526" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": context deadline exceeded"
Nov 04 01:18:41.087741 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:41.071541    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5" podUID="60f5c87a-48f8-4ad9-847b-cff571cc9340" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Nov 04 01:18:41.090125 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:41.090088    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8" podUID="eb9132e5-b90b-46fb-b86c-ca713419906d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded"
Nov 04 01:18:41.213796 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:41.213757    5068 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-848f6bf8bc-5g7t8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:18:41.236636 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:41.214055    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-5g7t8" podUID="5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:41.271635 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:18:41.271145    5068 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.033s"
Nov 04 01:18:41.271635 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:41.271245    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n"
Nov 04 01:18:41.275633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:41.273156    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89" podUID="6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:41.275633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:41.274105    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx" podUID="46d4ec7b-558d-4fb5-8011-177760a3291c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Nov 04 01:18:41.333589 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:41.333448    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf" podUID="32f60f59-a98f-46ba-9fb4-8420f77998da" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:41.415635 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:41.387992    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82" podUID="6cdd01de-ff61-494a-bdb3-c67025a49440" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:41.777856 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:41.761773    5068 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89" podUID="6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:41.897844 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:41.886870    5068 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5" podUID="60f5c87a-48f8-4ad9-847b-cff571cc9340" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/health\": context deadline exceeded"
Nov 04 01:18:41.937020 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:41.936982    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm" podUID="f9e7abd2-8737-4357-a778-d6492fd12f3f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:41.969151 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:41.969104    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq" podUID="b87aafac-9ab8-40e5-8d08-72eafc4187dd" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Nov 04 01:18:41.970882 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:41.970757    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9" podUID="530fc578-6622-4c42-a384-02e73bf17b1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:41.997634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:41.997543    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm"
Nov 04 01:18:42.120871 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:42.108292    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8" podUID="eb9132e5-b90b-46fb-b86c-ca713419906d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded"
Nov 04 01:18:42.120871 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:42.108975    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz" podUID="4ccafa61-1c9b-4595-a511-3404d7479526" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:42.120871 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:42.113122    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5" podUID="60f5c87a-48f8-4ad9-847b-cff571cc9340" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:42.274642 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:42.273802    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89" podUID="6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:42.278635 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:42.275559    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx" podUID="46d4ec7b-558d-4fb5-8011-177760a3291c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:42.356758 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:42.356694    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf" podUID="32f60f59-a98f-46ba-9fb4-8420f77998da" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:42.424838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:42.424779    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82" podUID="6cdd01de-ff61-494a-bdb3-c67025a49440" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:42.602247 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:42.583991    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8"
Nov 04 01:18:42.755631 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:42.752996    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89"
Nov 04 01:18:43.022134 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:42.938106    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm" podUID="f9e7abd2-8737-4357-a778-d6492fd12f3f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:43.022134 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:42.989183    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm"
Nov 04 01:18:43.022134 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:42.990764    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq" podUID="b87aafac-9ab8-40e5-8d08-72eafc4187dd" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Nov 04 01:18:43.022134 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:42.991770    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9" podUID="530fc578-6622-4c42-a384-02e73bf17b1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:43.109661 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:43.109555    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz" podUID="4ccafa61-1c9b-4595-a511-3404d7479526" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:43.110163 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:43.110129    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8" podUID="eb9132e5-b90b-46fb-b86c-ca713419906d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded"
Nov 04 01:18:43.113298 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:43.113267    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5" podUID="60f5c87a-48f8-4ad9-847b-cff571cc9340" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:43.130625 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:43.124151    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n" podUID="b3223811-a857-4279-a671-189e65383481" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:43.164932 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:43.164902    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8"
Nov 04 01:18:43.331159 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:43.331055    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx" podUID="46d4ec7b-558d-4fb5-8011-177760a3291c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:43.335926 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:43.333604    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89" podUID="6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:43.468275 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:43.432768    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf" podUID="32f60f59-a98f-46ba-9fb4-8420f77998da" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:43.468275 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:43.435233    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82" podUID="6cdd01de-ff61-494a-bdb3-c67025a49440" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:18:43.661149 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:43.660401    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89"
Nov 04 01:18:43.672126 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:18:43.671867    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:18:43.680514 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:43.679738    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9"
Nov 04 01:18:43.802768 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:43.800677    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82"
Nov 04 01:18:43.809762 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:43.806982    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf"
Nov 04 01:18:43.809762 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:43.808726    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx"
Nov 04 01:18:43.914678 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:43.913259    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5"
Nov 04 01:18:43.917159 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:43.915969    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz"
Nov 04 01:18:43.919709 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:43.919063    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq"
Nov 04 01:18:49.720509 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:49.719874    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9cc22d46-6f7fdfbcf8-4chb2"]
Nov 04 01:18:49.721348 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:49.721289    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9cc22d46-6f7fdfbcf8-4chb2" podUID="e18d5c36-c7e9-4785-b665-a425ea01ef97" containerName="maas-api" containerID="cri-o://0ddebabfb5fc74d6fd2a8caddee675a2164c7558818539c18ead44830ea52483" gracePeriod=30
Nov 04 01:18:50.383111 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:50.383072    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9cc22d46-6f7fdfbcf8-4chb2"
Nov 04 01:18:50.477706 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:50.471704    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e18d5c36-c7e9-4785-b665-a425ea01ef97-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e18d5c36-c7e9-4785-b665-a425ea01ef97-openshift-service-ca\") pod \"e18d5c36-c7e9-4785-b665-a425ea01ef97\" (UID: \"e18d5c36-c7e9-4785-b665-a425ea01ef97\") "
Nov 04 01:18:50.477706 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:50.471838    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e18d5c36-c7e9-4785-b665-a425ea01ef97-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e18d5c36-c7e9-4785-b665-a425ea01ef97-maas-api-tls\") pod \"e18d5c36-c7e9-4785-b665-a425ea01ef97\" (UID: \"e18d5c36-c7e9-4785-b665-a425ea01ef97\") "
Nov 04 01:18:50.477706 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:50.472012    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e18d5c36-c7e9-4785-b665-a425ea01ef97-metrics-certs\" (UniqueName: \"kubernetes.io/secret/e18d5c36-c7e9-4785-b665-a425ea01ef97-metrics-certs\") pod \"e18d5c36-c7e9-4785-b665-a425ea01ef97\" (UID: \"e18d5c36-c7e9-4785-b665-a425ea01ef97\") "
Nov 04 01:18:50.477706 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:50.472160    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e18d5c36-c7e9-4785-b665-a425ea01ef97-openshift-service-ca" pod "e18d5c36-c7e9-4785-b665-a425ea01ef97" (UID: "e18d5c36-c7e9-4785-b665-a425ea01ef97"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:18:50.477706 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:50.472188    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e18d5c36-c7e9-4785-b665-a425ea01ef97-kube-api-access-b7mf9\" (UniqueName: \"kubernetes.io/projected/e18d5c36-c7e9-4785-b665-a425ea01ef97-kube-api-access-b7mf9\") pod \"e18d5c36-c7e9-4785-b665-a425ea01ef97\" (UID: \"e18d5c36-c7e9-4785-b665-a425ea01ef97\") "
Nov 04 01:18:50.477706 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:50.472960    5068 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e18d5c36-c7e9-4785-b665-a425ea01ef97-openshift-service-ca\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:18:50.479524 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:50.479493    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e18d5c36-c7e9-4785-b665-a425ea01ef97-maas-api-tls" pod "e18d5c36-c7e9-4785-b665-a425ea01ef97" (UID: "e18d5c36-c7e9-4785-b665-a425ea01ef97"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:18:50.479885 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:50.479857    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e18d5c36-c7e9-4785-b665-a425ea01ef97-kube-api-access-b7mf9" pod "e18d5c36-c7e9-4785-b665-a425ea01ef97" (UID: "e18d5c36-c7e9-4785-b665-a425ea01ef97"). InnerVolumeSpecName "kube-api-access-b7mf9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:18:50.483018 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:50.482079    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e18d5c36-c7e9-4785-b665-a425ea01ef97-metrics-certs" pod "e18d5c36-c7e9-4785-b665-a425ea01ef97" (UID: "e18d5c36-c7e9-4785-b665-a425ea01ef97"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:18:50.579689 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:50.576052    5068 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e18d5c36-c7e9-4785-b665-a425ea01ef97-maas-api-tls\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:18:50.579689 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:50.576106    5068 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e18d5c36-c7e9-4785-b665-a425ea01ef97-metrics-certs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:18:50.579689 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:50.576135    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b7mf9\" (UniqueName: \"kubernetes.io/projected/e18d5c36-c7e9-4785-b665-a425ea01ef97-kube-api-access-b7mf9\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:18:50.828476 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:50.825669    5068 generic.go:356] "Generic (PLEG): container finished" podID="e18d5c36-c7e9-4785-b665-a425ea01ef97" containerID="0ddebabfb5fc74d6fd2a8caddee675a2164c7558818539c18ead44830ea52483" exitCode=0
Nov 04 01:18:50.828476 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:50.825805    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9cc22d46-6f7fdfbcf8-4chb2" event={"ID":"e18d5c36-c7e9-4785-b665-a425ea01ef97","Type":"ContainerDied","Data":"0ddebabfb5fc74d6fd2a8caddee675a2164c7558818539c18ead44830ea52483"}
Nov 04 01:18:50.828476 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:50.825865    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9cc22d46-6f7fdfbcf8-4chb2" event={"ID":"e18d5c36-c7e9-4785-b665-a425ea01ef97","Type":"ContainerDied","Data":"a2dd018747bb89d736c930a4f9b3f9ce1b273a00939a913d99889f9c0b9373fd"}
Nov 04 01:18:50.828476 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:50.825898    5068 scope.go:122] "RemoveContainer" containerID="0ddebabfb5fc74d6fd2a8caddee675a2164c7558818539c18ead44830ea52483"
Nov 04 01:18:50.828476 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:50.825853    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9cc22d46-6f7fdfbcf8-4chb2"
Nov 04 01:18:50.842241 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:50.842221    5068 scope.go:122] "RemoveContainer" containerID="0ddebabfb5fc74d6fd2a8caddee675a2164c7558818539c18ead44830ea52483"
Nov 04 01:18:50.842547 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:18:50.842524    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ddebabfb5fc74d6fd2a8caddee675a2164c7558818539c18ead44830ea52483\": container with ID starting with 0ddebabfb5fc74d6fd2a8caddee675a2164c7558818539c18ead44830ea52483 not found: ID does not exist" containerID="0ddebabfb5fc74d6fd2a8caddee675a2164c7558818539c18ead44830ea52483"
Nov 04 01:18:50.842623 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:50.842556    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ddebabfb5fc74d6fd2a8caddee675a2164c7558818539c18ead44830ea52483"} err="failed to get container status \"0ddebabfb5fc74d6fd2a8caddee675a2164c7558818539c18ead44830ea52483\": rpc error: code = NotFound desc = could not find container \"0ddebabfb5fc74d6fd2a8caddee675a2164c7558818539c18ead44830ea52483\": container with ID starting with 0ddebabfb5fc74d6fd2a8caddee675a2164c7558818539c18ead44830ea52483 not found: ID does not exist"
Nov 04 01:18:50.855468 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:50.855437    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9cc22d46-6f7fdfbcf8-4chb2"]
Nov 04 01:18:50.864524 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:50.862501    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9cc22d46-6f7fdfbcf8-4chb2"]
Nov 04 01:18:52.125265 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:52.125233    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e18d5c36-c7e9-4785-b665-a425ea01ef97" path="/var/lib/kubelet/pods/e18d5c36-c7e9-4785-b665-a425ea01ef97/volumes"
Nov 04 01:18:57.909161 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:57.909122    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-8f95141e-5fdd4bfb8c-rf5pd"]
Nov 04 01:18:57.914186 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:57.914159    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8f95141e-5dc7877c9b-79xw5"]
Nov 04 01:18:58.845634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:18:58.845583    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-383a864b-8f97d964c-svcwd"]
Nov 04 01:19:02.910458 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:02.910398    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-8f95141e-5fdd4bfb8c-rf5pd" podUID="90c7d91f-388a-4cb1-9669-e2686768af46" containerName="payload-pre-processing" containerID="cri-o://34babe53f254b3d09736a2dd9db1cfc5bb8086dc3917f056168d5ee7a1694cad" gracePeriod=25
Nov 04 01:19:02.915260 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:02.914977    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-8f95141e-5dc7877c9b-79xw5" podUID="38a6ee6d-4cac-4204-a973-30bb8cc22102" containerName="payload-processing" containerID="cri-o://63e3c2745678c50955285a60e34ec507ca683d3ae701a3e7dc8e4e8f989ae94b" gracePeriod=25
Nov 04 01:19:03.329634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:03.326987    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-8f95141e-5dc7877c9b-79xw5"
Nov 04 01:19:03.418198 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:03.418176    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-8f95141e-5fdd4bfb8c-rf5pd"
Nov 04 01:19:03.471128 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:03.471098    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/38a6ee6d-4cac-4204-a973-30bb8cc22102-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38a6ee6d-4cac-4204-a973-30bb8cc22102-plugins-config-volume\") pod \"38a6ee6d-4cac-4204-a973-30bb8cc22102\" (UID: \"38a6ee6d-4cac-4204-a973-30bb8cc22102\") "
Nov 04 01:19:03.471420 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:03.471397    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/38a6ee6d-4cac-4204-a973-30bb8cc22102-kube-api-access-8tpt2\" (UniqueName: \"kubernetes.io/projected/38a6ee6d-4cac-4204-a973-30bb8cc22102-kube-api-access-8tpt2\") pod \"38a6ee6d-4cac-4204-a973-30bb8cc22102\" (UID: \"38a6ee6d-4cac-4204-a973-30bb8cc22102\") "
Nov 04 01:19:03.471944 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:03.471770    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38a6ee6d-4cac-4204-a973-30bb8cc22102-plugins-config-volume" pod "38a6ee6d-4cac-4204-a973-30bb8cc22102" (UID: "38a6ee6d-4cac-4204-a973-30bb8cc22102"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:19:03.472179 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:03.472153    5068 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38a6ee6d-4cac-4204-a973-30bb8cc22102-plugins-config-volume\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:19:03.474280 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:03.474247    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38a6ee6d-4cac-4204-a973-30bb8cc22102-kube-api-access-8tpt2" pod "38a6ee6d-4cac-4204-a973-30bb8cc22102" (UID: "38a6ee6d-4cac-4204-a973-30bb8cc22102"). InnerVolumeSpecName "kube-api-access-8tpt2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:19:03.573361 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:03.573327    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/90c7d91f-388a-4cb1-9669-e2686768af46-kube-api-access-svhg8\" (UniqueName: \"kubernetes.io/projected/90c7d91f-388a-4cb1-9669-e2686768af46-kube-api-access-svhg8\") pod \"90c7d91f-388a-4cb1-9669-e2686768af46\" (UID: \"90c7d91f-388a-4cb1-9669-e2686768af46\") "
Nov 04 01:19:03.573758 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:03.573729    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/90c7d91f-388a-4cb1-9669-e2686768af46-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/90c7d91f-388a-4cb1-9669-e2686768af46-plugins-config-volume\") pod \"90c7d91f-388a-4cb1-9669-e2686768af46\" (UID: \"90c7d91f-388a-4cb1-9669-e2686768af46\") "
Nov 04 01:19:03.574294 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:03.574260    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90c7d91f-388a-4cb1-9669-e2686768af46-plugins-config-volume" pod "90c7d91f-388a-4cb1-9669-e2686768af46" (UID: "90c7d91f-388a-4cb1-9669-e2686768af46"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:19:03.574416 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:03.574395    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8tpt2\" (UniqueName: \"kubernetes.io/projected/38a6ee6d-4cac-4204-a973-30bb8cc22102-kube-api-access-8tpt2\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:19:03.574471 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:03.574423    5068 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/90c7d91f-388a-4cb1-9669-e2686768af46-plugins-config-volume\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:19:03.575698 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:03.575670    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90c7d91f-388a-4cb1-9669-e2686768af46-kube-api-access-svhg8" pod "90c7d91f-388a-4cb1-9669-e2686768af46" (UID: "90c7d91f-388a-4cb1-9669-e2686768af46"). InnerVolumeSpecName "kube-api-access-svhg8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:19:03.676168 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:03.676137    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-svhg8\" (UniqueName: \"kubernetes.io/projected/90c7d91f-388a-4cb1-9669-e2686768af46-kube-api-access-svhg8\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:19:03.850241 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:03.848679    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-383a864b-8f97d964c-svcwd" podUID="606d1ef1-1350-4025-9017-e784ccddace7" containerName="payload-pre-processing" containerID="cri-o://2220c36b0bb863d114c482f6c57b73c09d8c08647a01134ce1e0863550cc2df5" gracePeriod=25
Nov 04 01:19:03.968701 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:03.968663    5068 generic.go:356] "Generic (PLEG): container finished" podID="90c7d91f-388a-4cb1-9669-e2686768af46" containerID="34babe53f254b3d09736a2dd9db1cfc5bb8086dc3917f056168d5ee7a1694cad" exitCode=0
Nov 04 01:19:03.969277 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:03.969250    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-8f95141e-5fdd4bfb8c-rf5pd" event={"ID":"90c7d91f-388a-4cb1-9669-e2686768af46","Type":"ContainerDied","Data":"34babe53f254b3d09736a2dd9db1cfc5bb8086dc3917f056168d5ee7a1694cad"}
Nov 04 01:19:03.969764 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:03.969741    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-8f95141e-5fdd4bfb8c-rf5pd" event={"ID":"90c7d91f-388a-4cb1-9669-e2686768af46","Type":"ContainerDied","Data":"f27a5dc0b78ca89d1fe3edba02d5a87fadf3aa7457bb5c068cff9db150a0e239"}
Nov 04 01:19:03.969916 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:03.969902    5068 scope.go:122] "RemoveContainer" containerID="34babe53f254b3d09736a2dd9db1cfc5bb8086dc3917f056168d5ee7a1694cad"
Nov 04 01:19:03.970101 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:03.969401    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-8f95141e-5fdd4bfb8c-rf5pd"
Nov 04 01:19:03.978765 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:03.977320    5068 generic.go:356] "Generic (PLEG): container finished" podID="38a6ee6d-4cac-4204-a973-30bb8cc22102" containerID="63e3c2745678c50955285a60e34ec507ca683d3ae701a3e7dc8e4e8f989ae94b" exitCode=0
Nov 04 01:19:03.978765 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:03.977595    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-8f95141e-5dc7877c9b-79xw5" event={"ID":"38a6ee6d-4cac-4204-a973-30bb8cc22102","Type":"ContainerDied","Data":"63e3c2745678c50955285a60e34ec507ca683d3ae701a3e7dc8e4e8f989ae94b"}
Nov 04 01:19:03.978765 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:03.977652    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-8f95141e-5dc7877c9b-79xw5" event={"ID":"38a6ee6d-4cac-4204-a973-30bb8cc22102","Type":"ContainerDied","Data":"d254b183d1e72e8d1eb1175ac5f54d4825ee9277b50c00ff19be0bcc24379079"}
Nov 04 01:19:03.978765 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:03.977729    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-8f95141e-5dc7877c9b-79xw5"
Nov 04 01:19:04.000582 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:04.000512    5068 scope.go:122] "RemoveContainer" containerID="34babe53f254b3d09736a2dd9db1cfc5bb8086dc3917f056168d5ee7a1694cad"
Nov 04 01:19:04.001200 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:19:04.001170    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34babe53f254b3d09736a2dd9db1cfc5bb8086dc3917f056168d5ee7a1694cad\": container with ID starting with 34babe53f254b3d09736a2dd9db1cfc5bb8086dc3917f056168d5ee7a1694cad not found: ID does not exist" containerID="34babe53f254b3d09736a2dd9db1cfc5bb8086dc3917f056168d5ee7a1694cad"
Nov 04 01:19:04.001312 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:04.001212    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34babe53f254b3d09736a2dd9db1cfc5bb8086dc3917f056168d5ee7a1694cad"} err="failed to get container status \"34babe53f254b3d09736a2dd9db1cfc5bb8086dc3917f056168d5ee7a1694cad\": rpc error: code = NotFound desc = could not find container \"34babe53f254b3d09736a2dd9db1cfc5bb8086dc3917f056168d5ee7a1694cad\": container with ID starting with 34babe53f254b3d09736a2dd9db1cfc5bb8086dc3917f056168d5ee7a1694cad not found: ID does not exist"
Nov 04 01:19:04.001312 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:04.001237    5068 scope.go:122] "RemoveContainer" containerID="63e3c2745678c50955285a60e34ec507ca683d3ae701a3e7dc8e4e8f989ae94b"
Nov 04 01:19:04.038551 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:04.037798    5068 scope.go:122] "RemoveContainer" containerID="63e3c2745678c50955285a60e34ec507ca683d3ae701a3e7dc8e4e8f989ae94b"
Nov 04 01:19:04.038735 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:19:04.038586    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63e3c2745678c50955285a60e34ec507ca683d3ae701a3e7dc8e4e8f989ae94b\": container with ID starting with 63e3c2745678c50955285a60e34ec507ca683d3ae701a3e7dc8e4e8f989ae94b not found: ID does not exist" containerID="63e3c2745678c50955285a60e34ec507ca683d3ae701a3e7dc8e4e8f989ae94b"
Nov 04 01:19:04.038735 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:04.038649    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63e3c2745678c50955285a60e34ec507ca683d3ae701a3e7dc8e4e8f989ae94b"} err="failed to get container status \"63e3c2745678c50955285a60e34ec507ca683d3ae701a3e7dc8e4e8f989ae94b\": rpc error: code = NotFound desc = could not find container \"63e3c2745678c50955285a60e34ec507ca683d3ae701a3e7dc8e4e8f989ae94b\": container with ID starting with 63e3c2745678c50955285a60e34ec507ca683d3ae701a3e7dc8e4e8f989ae94b not found: ID does not exist"
Nov 04 01:19:04.039672 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:04.039601    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8f95141e-5dc7877c9b-79xw5"]
Nov 04 01:19:04.050326 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:04.050265    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8f95141e-5dc7877c9b-79xw5"]
Nov 04 01:19:04.060674 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:04.060647    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-8f95141e-5fdd4bfb8c-rf5pd"]
Nov 04 01:19:04.067582 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:04.067515    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-8f95141e-5fdd4bfb8c-rf5pd"]
Nov 04 01:19:04.123833 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:04.122997    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="38a6ee6d-4cac-4204-a973-30bb8cc22102" path="/var/lib/kubelet/pods/38a6ee6d-4cac-4204-a973-30bb8cc22102/volumes"
Nov 04 01:19:04.123833 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:04.123791    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="90c7d91f-388a-4cb1-9669-e2686768af46" path="/var/lib/kubelet/pods/90c7d91f-388a-4cb1-9669-e2686768af46/volumes"
Nov 04 01:19:04.292183 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:04.292159    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-383a864b-8f97d964c-svcwd"
Nov 04 01:19:04.427683 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:04.418152    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/606d1ef1-1350-4025-9017-e784ccddace7-kube-api-access-8qnwt\" (UniqueName: \"kubernetes.io/projected/606d1ef1-1350-4025-9017-e784ccddace7-kube-api-access-8qnwt\") pod \"606d1ef1-1350-4025-9017-e784ccddace7\" (UID: \"606d1ef1-1350-4025-9017-e784ccddace7\") "
Nov 04 01:19:04.427683 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:04.418254    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/606d1ef1-1350-4025-9017-e784ccddace7-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/606d1ef1-1350-4025-9017-e784ccddace7-plugins-config-volume\") pod \"606d1ef1-1350-4025-9017-e784ccddace7\" (UID: \"606d1ef1-1350-4025-9017-e784ccddace7\") "
Nov 04 01:19:04.431630 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:04.428763    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/606d1ef1-1350-4025-9017-e784ccddace7-plugins-config-volume" pod "606d1ef1-1350-4025-9017-e784ccddace7" (UID: "606d1ef1-1350-4025-9017-e784ccddace7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:19:04.499657 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:04.497270    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/606d1ef1-1350-4025-9017-e784ccddace7-kube-api-access-8qnwt" pod "606d1ef1-1350-4025-9017-e784ccddace7" (UID: "606d1ef1-1350-4025-9017-e784ccddace7"). InnerVolumeSpecName "kube-api-access-8qnwt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:19:04.521144 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:04.521109    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8qnwt\" (UniqueName: \"kubernetes.io/projected/606d1ef1-1350-4025-9017-e784ccddace7-kube-api-access-8qnwt\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:19:04.521144 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:04.521136    5068 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/606d1ef1-1350-4025-9017-e784ccddace7-plugins-config-volume\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:19:04.992956 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:04.992918    5068 generic.go:356] "Generic (PLEG): container finished" podID="606d1ef1-1350-4025-9017-e784ccddace7" containerID="2220c36b0bb863d114c482f6c57b73c09d8c08647a01134ce1e0863550cc2df5" exitCode=0
Nov 04 01:19:04.992956 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:04.992964    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-383a864b-8f97d964c-svcwd"
Nov 04 01:19:04.993529 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:04.993004    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-383a864b-8f97d964c-svcwd" event={"ID":"606d1ef1-1350-4025-9017-e784ccddace7","Type":"ContainerDied","Data":"2220c36b0bb863d114c482f6c57b73c09d8c08647a01134ce1e0863550cc2df5"}
Nov 04 01:19:04.993529 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:04.993057    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-383a864b-8f97d964c-svcwd" event={"ID":"606d1ef1-1350-4025-9017-e784ccddace7","Type":"ContainerDied","Data":"ae3eecd1e92b3f83e4ecb24dfa92190fb021f02dbbd4cc8bb1d24b18d9245900"}
Nov 04 01:19:04.993529 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:04.993082    5068 scope.go:122] "RemoveContainer" containerID="2220c36b0bb863d114c482f6c57b73c09d8c08647a01134ce1e0863550cc2df5"
Nov 04 01:19:05.006442 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:05.006423    5068 scope.go:122] "RemoveContainer" containerID="2220c36b0bb863d114c482f6c57b73c09d8c08647a01134ce1e0863550cc2df5"
Nov 04 01:19:05.006742 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:19:05.006715    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2220c36b0bb863d114c482f6c57b73c09d8c08647a01134ce1e0863550cc2df5\": container with ID starting with 2220c36b0bb863d114c482f6c57b73c09d8c08647a01134ce1e0863550cc2df5 not found: ID does not exist" containerID="2220c36b0bb863d114c482f6c57b73c09d8c08647a01134ce1e0863550cc2df5"
Nov 04 01:19:05.006813 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:05.006741    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2220c36b0bb863d114c482f6c57b73c09d8c08647a01134ce1e0863550cc2df5"} err="failed to get container status \"2220c36b0bb863d114c482f6c57b73c09d8c08647a01134ce1e0863550cc2df5\": rpc error: code = NotFound desc = could not find container \"2220c36b0bb863d114c482f6c57b73c09d8c08647a01134ce1e0863550cc2df5\": container with ID starting with 2220c36b0bb863d114c482f6c57b73c09d8c08647a01134ce1e0863550cc2df5 not found: ID does not exist"
Nov 04 01:19:05.020149 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:05.020124    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-383a864b-8f97d964c-svcwd"]
Nov 04 01:19:05.024622 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:05.024591    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-383a864b-8f97d964c-svcwd"]
Nov 04 01:19:05.056649 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:05.052735    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8f95141e-667b749fc6-8b8zv"]
Nov 04 01:19:05.056649 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:05.053050    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f95141e-667b749fc6-8b8zv" podUID="ef7dacdd-803b-4207-b4c3-d75cc50541d8" containerName="maas-api" containerID="cri-o://499c29397e07020caba4dcbaa8198e79789b9cecdd89377ce19be1a657d8c055" gracePeriod=30
Nov 04 01:19:05.604250 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:05.603815    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f95141e-667b749fc6-8b8zv"
Nov 04 01:19:05.687669 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:05.649180    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ef7dacdd-803b-4207-b4c3-d75cc50541d8-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef7dacdd-803b-4207-b4c3-d75cc50541d8-maas-api-tls\") pod \"ef7dacdd-803b-4207-b4c3-d75cc50541d8\" (UID: \"ef7dacdd-803b-4207-b4c3-d75cc50541d8\") "
Nov 04 01:19:05.690339 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:05.690311    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ef7dacdd-803b-4207-b4c3-d75cc50541d8-kube-api-access-bncbm\" (UniqueName: \"kubernetes.io/projected/ef7dacdd-803b-4207-b4c3-d75cc50541d8-kube-api-access-bncbm\") pod \"ef7dacdd-803b-4207-b4c3-d75cc50541d8\" (UID: \"ef7dacdd-803b-4207-b4c3-d75cc50541d8\") "
Nov 04 01:19:05.690627 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:05.690591    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ef7dacdd-803b-4207-b4c3-d75cc50541d8-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef7dacdd-803b-4207-b4c3-d75cc50541d8-openshift-service-ca\") pod \"ef7dacdd-803b-4207-b4c3-d75cc50541d8\" (UID: \"ef7dacdd-803b-4207-b4c3-d75cc50541d8\") "
Nov 04 01:19:05.690792 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:05.690775    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ef7dacdd-803b-4207-b4c3-d75cc50541d8-metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef7dacdd-803b-4207-b4c3-d75cc50541d8-metrics-certs\") pod \"ef7dacdd-803b-4207-b4c3-d75cc50541d8\" (UID: \"ef7dacdd-803b-4207-b4c3-d75cc50541d8\") "
Nov 04 01:19:05.698521 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:05.698396    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef7dacdd-803b-4207-b4c3-d75cc50541d8-openshift-service-ca" pod "ef7dacdd-803b-4207-b4c3-d75cc50541d8" (UID: "ef7dacdd-803b-4207-b4c3-d75cc50541d8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:19:05.703169 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:05.703126    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef7dacdd-803b-4207-b4c3-d75cc50541d8-metrics-certs" pod "ef7dacdd-803b-4207-b4c3-d75cc50541d8" (UID: "ef7dacdd-803b-4207-b4c3-d75cc50541d8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:19:05.703787 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:05.703749    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef7dacdd-803b-4207-b4c3-d75cc50541d8-kube-api-access-bncbm" pod "ef7dacdd-803b-4207-b4c3-d75cc50541d8" (UID: "ef7dacdd-803b-4207-b4c3-d75cc50541d8"). InnerVolumeSpecName "kube-api-access-bncbm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:19:05.704744 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:05.704714    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef7dacdd-803b-4207-b4c3-d75cc50541d8-maas-api-tls" pod "ef7dacdd-803b-4207-b4c3-d75cc50541d8" (UID: "ef7dacdd-803b-4207-b4c3-d75cc50541d8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:19:05.796120 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:05.795967    5068 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef7dacdd-803b-4207-b4c3-d75cc50541d8-maas-api-tls\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:19:05.796120 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:05.796009    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bncbm\" (UniqueName: \"kubernetes.io/projected/ef7dacdd-803b-4207-b4c3-d75cc50541d8-kube-api-access-bncbm\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:19:05.796120 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:05.796025    5068 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef7dacdd-803b-4207-b4c3-d75cc50541d8-openshift-service-ca\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:19:05.796120 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:05.796037    5068 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef7dacdd-803b-4207-b4c3-d75cc50541d8-metrics-certs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:19:06.003313 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:06.003217    5068 generic.go:356] "Generic (PLEG): container finished" podID="ef7dacdd-803b-4207-b4c3-d75cc50541d8" containerID="499c29397e07020caba4dcbaa8198e79789b9cecdd89377ce19be1a657d8c055" exitCode=0
Nov 04 01:19:06.003313 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:06.003278    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f95141e-667b749fc6-8b8zv"
Nov 04 01:19:06.004227 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:06.003304    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f95141e-667b749fc6-8b8zv" event={"ID":"ef7dacdd-803b-4207-b4c3-d75cc50541d8","Type":"ContainerDied","Data":"499c29397e07020caba4dcbaa8198e79789b9cecdd89377ce19be1a657d8c055"}
Nov 04 01:19:06.004227 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:06.003406    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f95141e-667b749fc6-8b8zv" event={"ID":"ef7dacdd-803b-4207-b4c3-d75cc50541d8","Type":"ContainerDied","Data":"f9db8c930d4b5042ebce92fba28096082bfa6efa6df92167693bb5c01e3e0673"}
Nov 04 01:19:06.004227 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:06.003442    5068 scope.go:122] "RemoveContainer" containerID="499c29397e07020caba4dcbaa8198e79789b9cecdd89377ce19be1a657d8c055"
Nov 04 01:19:06.018747 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:06.018727    5068 scope.go:122] "RemoveContainer" containerID="499c29397e07020caba4dcbaa8198e79789b9cecdd89377ce19be1a657d8c055"
Nov 04 01:19:06.019094 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:19:06.019066    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"499c29397e07020caba4dcbaa8198e79789b9cecdd89377ce19be1a657d8c055\": container with ID starting with 499c29397e07020caba4dcbaa8198e79789b9cecdd89377ce19be1a657d8c055 not found: ID does not exist" containerID="499c29397e07020caba4dcbaa8198e79789b9cecdd89377ce19be1a657d8c055"
Nov 04 01:19:06.019160 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:06.019104    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"499c29397e07020caba4dcbaa8198e79789b9cecdd89377ce19be1a657d8c055"} err="failed to get container status \"499c29397e07020caba4dcbaa8198e79789b9cecdd89377ce19be1a657d8c055\": rpc error: code = NotFound desc = could not find container \"499c29397e07020caba4dcbaa8198e79789b9cecdd89377ce19be1a657d8c055\": container with ID starting with 499c29397e07020caba4dcbaa8198e79789b9cecdd89377ce19be1a657d8c055 not found: ID does not exist"
Nov 04 01:19:06.031931 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:06.031910    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8f95141e-667b749fc6-8b8zv"]
Nov 04 01:19:06.040950 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:06.040882    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8f95141e-667b749fc6-8b8zv"]
Nov 04 01:19:06.120107 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:06.120072    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="606d1ef1-1350-4025-9017-e784ccddace7" path="/var/lib/kubelet/pods/606d1ef1-1350-4025-9017-e784ccddace7/volumes"
Nov 04 01:19:06.120729 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:06.120709    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ef7dacdd-803b-4207-b4c3-d75cc50541d8" path="/var/lib/kubelet/pods/ef7dacdd-803b-4207-b4c3-d75cc50541d8/volumes"
Nov 04 01:19:07.763683 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:07.763647    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w4-029e8c-574bd8d98-79ssg"]
Nov 04 01:19:07.764675 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:07.764653    5068 cpu_manager.go:414] "containerMap: removing container" podUID="e18d5c36-c7e9-4785-b665-a425ea01ef97" containerName="maas-api"
Nov 04 01:19:07.764773 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:07.764676    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e18d5c36-c7e9-4785-b665-a425ea01ef97" containerName="maas-api"
Nov 04 01:19:07.764773 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:07.764690    5068 cpu_manager.go:414] "containerMap: removing container" podUID="38a6ee6d-4cac-4204-a973-30bb8cc22102" containerName="payload-processing"
Nov 04 01:19:07.764773 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:07.764698    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="38a6ee6d-4cac-4204-a973-30bb8cc22102" containerName="payload-processing"
Nov 04 01:19:07.764773 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:07.764724    5068 cpu_manager.go:414] "containerMap: removing container" podUID="606d1ef1-1350-4025-9017-e784ccddace7" containerName="payload-pre-processing"
Nov 04 01:19:07.764773 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:07.764733    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="606d1ef1-1350-4025-9017-e784ccddace7" containerName="payload-pre-processing"
Nov 04 01:19:07.764773 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:07.764755    5068 cpu_manager.go:414] "containerMap: removing container" podUID="ef7dacdd-803b-4207-b4c3-d75cc50541d8" containerName="maas-api"
Nov 04 01:19:07.764773 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:07.764762    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ef7dacdd-803b-4207-b4c3-d75cc50541d8" containerName="maas-api"
Nov 04 01:19:07.764773 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:07.764774    5068 cpu_manager.go:414] "containerMap: removing container" podUID="90c7d91f-388a-4cb1-9669-e2686768af46" containerName="payload-pre-processing"
Nov 04 01:19:07.765157 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:07.764782    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="90c7d91f-388a-4cb1-9669-e2686768af46" containerName="payload-pre-processing"
Nov 04 01:19:07.765157 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:07.764980    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="e18d5c36-c7e9-4785-b665-a425ea01ef97" containerName="maas-api"
Nov 04 01:19:07.765157 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:07.764994    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="38a6ee6d-4cac-4204-a973-30bb8cc22102" containerName="payload-processing"
Nov 04 01:19:07.765157 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:07.765007    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="606d1ef1-1350-4025-9017-e784ccddace7" containerName="payload-pre-processing"
Nov 04 01:19:07.765157 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:07.765019    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="ef7dacdd-803b-4207-b4c3-d75cc50541d8" containerName="maas-api"
Nov 04 01:19:07.765157 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:07.765028    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="90c7d91f-388a-4cb1-9669-e2686768af46" containerName="payload-pre-processing"
Nov 04 01:19:07.770874 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:07.770851    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-029e8c-574bd8d98-79ssg"
Nov 04 01:19:07.782288 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:07.781921    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w4-029e8c\""
Nov 04 01:19:07.783109 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:07.783041    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w4-029e8c-dockercfg-ss5sw\""
Nov 04 01:19:07.785013 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:07.784988    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w4-029e8c-574bd8d98-79ssg"]
Nov 04 01:19:07.858631 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:07.857378    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f176eb7-f53c-48c8-a7dc-0e9212448556-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w4-029e8c-574bd8d98-79ssg\" (UID: \"1f176eb7-f53c-48c8-a7dc-0e9212448556\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w4-029e8c-574bd8d98-79ssg"
Nov 04 01:19:07.858819 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:07.858655    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsqhh\" (UniqueName: \"kubernetes.io/projected/1f176eb7-f53c-48c8-a7dc-0e9212448556-kube-api-access-fsqhh\") pod \"payload-processing-e2e-shared-a-w4-029e8c-574bd8d98-79ssg\" (UID: \"1f176eb7-f53c-48c8-a7dc-0e9212448556\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w4-029e8c-574bd8d98-79ssg"
Nov 04 01:19:07.962633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:07.962363    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f176eb7-f53c-48c8-a7dc-0e9212448556-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w4-029e8c-574bd8d98-79ssg\" (UID: \"1f176eb7-f53c-48c8-a7dc-0e9212448556\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w4-029e8c-574bd8d98-79ssg"
Nov 04 01:19:07.962633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:07.962447    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsqhh\" (UniqueName: \"kubernetes.io/projected/1f176eb7-f53c-48c8-a7dc-0e9212448556-kube-api-access-fsqhh\") pod \"payload-processing-e2e-shared-a-w4-029e8c-574bd8d98-79ssg\" (UID: \"1f176eb7-f53c-48c8-a7dc-0e9212448556\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w4-029e8c-574bd8d98-79ssg"
Nov 04 01:19:07.990637 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:07.977700    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f176eb7-f53c-48c8-a7dc-0e9212448556-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w4-029e8c-574bd8d98-79ssg\" (UID: \"1f176eb7-f53c-48c8-a7dc-0e9212448556\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w4-029e8c-574bd8d98-79ssg"
Nov 04 01:19:08.003644 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:08.001371    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsqhh\" (UniqueName: \"kubernetes.io/projected/1f176eb7-f53c-48c8-a7dc-0e9212448556-kube-api-access-fsqhh\") pod \"payload-processing-e2e-shared-a-w4-029e8c-574bd8d98-79ssg\" (UID: \"1f176eb7-f53c-48c8-a7dc-0e9212448556\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w4-029e8c-574bd8d98-79ssg"
Nov 04 01:19:08.099687 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:08.099238    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-029e8c-574bd8d98-79ssg"
Nov 04 01:19:08.573819 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:08.573694    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w4-029e8c-574bd8d98-79ssg"]
Nov 04 01:19:09.085809 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:09.085734    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-029e8c-574bd8d98-79ssg" event={"ID":"1f176eb7-f53c-48c8-a7dc-0e9212448556","Type":"ContainerStarted","Data":"3d73c9ce8d931148f2f166487090aa05731180187966087300f707241c65887a"}
Nov 04 01:19:09.085809 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:09.085810    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-029e8c-574bd8d98-79ssg" event={"ID":"1f176eb7-f53c-48c8-a7dc-0e9212448556","Type":"ContainerStarted","Data":"fbc146bdffc9e198c0c8eb4f95979e607a5060a0fae4340279dbf2fd5522094a"}
Nov 04 01:19:09.086679 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:09.086654    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-029e8c-574bd8d98-79ssg"
Nov 04 01:19:09.117255 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:09.117197    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-029e8c-574bd8d98-79ssg" podStartSLOduration=2.117179882 podStartE2EDuration="2.117179882s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:19:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:19:09.114372032 +0000 UTC m=+989.160932824" watchObservedRunningTime="2026-11-04 01:19:09.117179882 +0000 UTC m=+989.163740674"
Nov 04 01:19:10.645943 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:10.645911    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:19:13.786636 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:13.774175    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9" podUID="530fc578-6622-4c42-a384-02e73bf17b1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:19:13.875637 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:13.870801    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz" podUID="4ccafa61-1c9b-4595-a511-3404d7479526" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:19:13.875637 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:13.871477    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5" podUID="60f5c87a-48f8-4ad9-847b-cff571cc9340" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Nov 04 01:19:13.946278 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:13.945770    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm" podUID="f9e7abd2-8737-4357-a778-d6492fd12f3f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:19:13.987351 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:19:13.987269    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:19:14.164626 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:14.152863    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n" podUID="b3223811-a857-4279-a671-189e65383481" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:19:14.176517 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:14.176264    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx" podUID="46d4ec7b-558d-4fb5-8011-177760a3291c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Nov 04 01:19:14.288430 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:14.288385    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf" podUID="32f60f59-a98f-46ba-9fb4-8420f77998da" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded"
Nov 04 01:19:14.325004 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:14.324962    5068 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz" podUID="4ccafa61-1c9b-4595-a511-3404d7479526" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/health\": context deadline exceeded"
Nov 04 01:19:14.335638 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:14.334876    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89" podUID="6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:19:14.345508 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:14.345470    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82" podUID="6cdd01de-ff61-494a-bdb3-c67025a49440" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:19:14.775631 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:14.774599    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9" podUID="530fc578-6622-4c42-a384-02e73bf17b1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:19:14.775631 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:14.775219    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9"
Nov 04 01:19:14.888635 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:14.871133    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz" podUID="4ccafa61-1c9b-4595-a511-3404d7479526" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:19:14.888635 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:14.871792    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz"
Nov 04 01:19:14.888635 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:14.872395    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5" podUID="60f5c87a-48f8-4ad9-847b-cff571cc9340" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:19:14.888635 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:14.872903    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5"
Nov 04 01:19:15.345647 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:15.323177    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf" podUID="32f60f59-a98f-46ba-9fb4-8420f77998da" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:19:15.345647 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:15.323793    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf"
Nov 04 01:19:15.368100 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:15.368058    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89" podUID="6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:19:15.368823 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:15.368799    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89"
Nov 04 01:19:15.446660 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:15.426745    5068 patch_prober.go:29] interesting pod/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:19:15.471630 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:15.446925    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947" podUID="a5f098fc-01c1-4e24-8611-e2b6aed954dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:19:15.487633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:15.484579    5068 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9" podUID="530fc578-6622-4c42-a384-02e73bf17b1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/health\": context deadline exceeded"
Nov 04 01:19:15.654297 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:15.624232    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq" podUID="b87aafac-9ab8-40e5-8d08-72eafc4187dd" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:19:15.789635 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:15.780681    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9" podUID="530fc578-6622-4c42-a384-02e73bf17b1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Nov 04 01:19:15.881144 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:15.881100    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5" podUID="60f5c87a-48f8-4ad9-847b-cff571cc9340" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Nov 04 01:19:15.881794 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:15.881762    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz" podUID="4ccafa61-1c9b-4595-a511-3404d7479526" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:19:16.083633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:16.062825    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm" podUID="f9e7abd2-8737-4357-a778-d6492fd12f3f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": context deadline exceeded"
Nov 04 01:19:16.211778 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:16.211728    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n" podUID="b3223811-a857-4279-a671-189e65383481" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:19:16.301443 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:16.226762    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8" podUID="eb9132e5-b90b-46fb-b86c-ca713419906d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:19:16.301443 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:16.234852    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx" podUID="46d4ec7b-558d-4fb5-8011-177760a3291c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:19:16.301443 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:16.265320    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82" podUID="6cdd01de-ff61-494a-bdb3-c67025a49440" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": context deadline exceeded"
Nov 04 01:19:16.333427 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:16.330549    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf" podUID="32f60f59-a98f-46ba-9fb4-8420f77998da" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:19:16.425685 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:16.400474    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89" podUID="6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:19:16.653836 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:16.653779    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq" podUID="b87aafac-9ab8-40e5-8d08-72eafc4187dd" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:19:16.665641 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:16.654875    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq"
Nov 04 01:19:16.802633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:16.787835    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9" podUID="530fc578-6622-4c42-a384-02e73bf17b1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Nov 04 01:19:16.882475 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:16.882429    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz" podUID="4ccafa61-1c9b-4595-a511-3404d7479526" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:19:16.893669 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:16.882698    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5" podUID="60f5c87a-48f8-4ad9-847b-cff571cc9340" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:19:17.163688 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:17.163544    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm" podUID="f9e7abd2-8737-4357-a778-d6492fd12f3f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": context deadline exceeded"
Nov 04 01:19:17.192632 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:17.164894    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm"
Nov 04 01:19:17.284633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:17.274993    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82" podUID="6cdd01de-ff61-494a-bdb3-c67025a49440" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": context deadline exceeded"
Nov 04 01:19:17.285901 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:17.284899    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8" podUID="eb9132e5-b90b-46fb-b86c-ca713419906d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded"
Nov 04 01:19:17.286131 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:17.286099    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx" podUID="46d4ec7b-558d-4fb5-8011-177760a3291c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Nov 04 01:19:17.286979 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:17.286956    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82"
Nov 04 01:19:17.287166 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:17.287150    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8"
Nov 04 01:19:17.287311 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:17.287297    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx"
Nov 04 01:19:17.318656 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:17.313410    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n" podUID="b3223811-a857-4279-a671-189e65383481" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Nov 04 01:19:17.320112 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:17.320085    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n"
Nov 04 01:19:17.360738 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:17.347291    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf" podUID="32f60f59-a98f-46ba-9fb4-8420f77998da" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:19:17.419872 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:17.419770    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89" podUID="6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:19:17.459651 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:17.438885    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5"
Nov 04 01:19:17.459651 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:17.454190    5068 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx" podUID="46d4ec7b-558d-4fb5-8011-177760a3291c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:19:17.711408 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:17.711324    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89"
Nov 04 01:19:17.714980 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:17.714944    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq" podUID="b87aafac-9ab8-40e5-8d08-72eafc4187dd" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Nov 04 01:19:17.835467 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:17.835060    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9" podUID="530fc578-6622-4c42-a384-02e73bf17b1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Nov 04 01:19:17.885066 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:17.884895    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz" podUID="4ccafa61-1c9b-4595-a511-3404d7479526" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:19:17.970692 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:17.968927    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9"
Nov 04 01:19:18.227691 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:18.188359    5068 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq" podUID="b87aafac-9ab8-40e5-8d08-72eafc4187dd" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:19:18.426898 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:18.227800    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm" podUID="f9e7abd2-8737-4357-a778-d6492fd12f3f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:19:18.427442 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:18.353831    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n" podUID="b3223811-a857-4279-a671-189e65383481" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:19:18.427796 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:18.378369    5068 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm" podUID="f9e7abd2-8737-4357-a778-d6492fd12f3f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:19:18.427953 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:18.378406    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82" podUID="6cdd01de-ff61-494a-bdb3-c67025a49440" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": context deadline exceeded"
Nov 04 01:19:18.428333 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:18.378428    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8" podUID="eb9132e5-b90b-46fb-b86c-ca713419906d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": context deadline exceeded"
Nov 04 01:19:18.428747 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:18.378447    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx" podUID="46d4ec7b-558d-4fb5-8011-177760a3291c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Nov 04 01:19:18.429127 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:18.379875    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf" podUID="32f60f59-a98f-46ba-9fb4-8420f77998da" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:19:18.445759 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:18.445709    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5" podUID="60f5c87a-48f8-4ad9-847b-cff571cc9340" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:19:18.783633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:18.780544    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq" podUID="b87aafac-9ab8-40e5-8d08-72eafc4187dd" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:19:18.783633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:18.780728    5068 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82" podUID="6cdd01de-ff61-494a-bdb3-c67025a49440" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/health\": context deadline exceeded"
Nov 04 01:19:18.902669 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:18.902626    5068 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8" podUID="eb9132e5-b90b-46fb-b86c-ca713419906d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/health\": context deadline exceeded"
Nov 04 01:19:18.918193 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:18.918156    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz" podUID="4ccafa61-1c9b-4595-a511-3404d7479526" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:19:19.097534 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:19.093231    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n"
Nov 04 01:19:19.170671 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:19.166044    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq"
Nov 04 01:19:19.191085 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:19.191041    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz"
Nov 04 01:19:19.241460 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:19.241433    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82"
Nov 04 01:19:19.242117 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:19.242098    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx"
Nov 04 01:19:19.323690 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:19.323659    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8"
Nov 04 01:19:19.393391 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:19.393111    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm"
Nov 04 01:19:19.399228 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:19.399193    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf"
Nov 04 01:19:19.969311 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:19.969278    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-029e8c-8c7f458c5-gvzpd"]
Nov 04 01:19:19.975164 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:19.975141    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-029e8c-8c7f458c5-gvzpd"
Nov 04 01:19:19.984321 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:19.984299    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w4-029e8c-serving-cert\""
Nov 04 01:19:19.987311 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:19.986968    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-029e8c-8c7f458c5-gvzpd"]
Nov 04 01:19:20.013098 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:20.013070    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39c35ec4-5c1b-472e-9e73-991258bb835d-metrics-certs\") pod \"maas-api-e2e-shared-a-w4-029e8c-8c7f458c5-gvzpd\" (UID: \"39c35ec4-5c1b-472e-9e73-991258bb835d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-029e8c-8c7f458c5-gvzpd"
Nov 04 01:19:20.013246 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:20.013205    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39c35ec4-5c1b-472e-9e73-991258bb835d-maas-api-tls\") pod \"maas-api-e2e-shared-a-w4-029e8c-8c7f458c5-gvzpd\" (UID: \"39c35ec4-5c1b-472e-9e73-991258bb835d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-029e8c-8c7f458c5-gvzpd"
Nov 04 01:19:20.013311 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:20.013262    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39c35ec4-5c1b-472e-9e73-991258bb835d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w4-029e8c-8c7f458c5-gvzpd\" (UID: \"39c35ec4-5c1b-472e-9e73-991258bb835d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-029e8c-8c7f458c5-gvzpd"
Nov 04 01:19:20.013544 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:20.013520    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gj2l8\" (UniqueName: \"kubernetes.io/projected/39c35ec4-5c1b-472e-9e73-991258bb835d-kube-api-access-gj2l8\") pod \"maas-api-e2e-shared-a-w4-029e8c-8c7f458c5-gvzpd\" (UID: \"39c35ec4-5c1b-472e-9e73-991258bb835d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-029e8c-8c7f458c5-gvzpd"
Nov 04 01:19:20.096696 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:20.096669    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-029e8c-574bd8d98-79ssg"
Nov 04 01:19:20.115515 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:20.115487    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gj2l8\" (UniqueName: \"kubernetes.io/projected/39c35ec4-5c1b-472e-9e73-991258bb835d-kube-api-access-gj2l8\") pod \"maas-api-e2e-shared-a-w4-029e8c-8c7f458c5-gvzpd\" (UID: \"39c35ec4-5c1b-472e-9e73-991258bb835d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-029e8c-8c7f458c5-gvzpd"
Nov 04 01:19:20.115760 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:20.115629    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39c35ec4-5c1b-472e-9e73-991258bb835d-metrics-certs\") pod \"maas-api-e2e-shared-a-w4-029e8c-8c7f458c5-gvzpd\" (UID: \"39c35ec4-5c1b-472e-9e73-991258bb835d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-029e8c-8c7f458c5-gvzpd"
Nov 04 01:19:20.115760 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:20.115701    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39c35ec4-5c1b-472e-9e73-991258bb835d-maas-api-tls\") pod \"maas-api-e2e-shared-a-w4-029e8c-8c7f458c5-gvzpd\" (UID: \"39c35ec4-5c1b-472e-9e73-991258bb835d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-029e8c-8c7f458c5-gvzpd"
Nov 04 01:19:20.116247 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:20.115948    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39c35ec4-5c1b-472e-9e73-991258bb835d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w4-029e8c-8c7f458c5-gvzpd\" (UID: \"39c35ec4-5c1b-472e-9e73-991258bb835d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-029e8c-8c7f458c5-gvzpd"
Nov 04 01:19:20.116686 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:20.116661    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39c35ec4-5c1b-472e-9e73-991258bb835d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w4-029e8c-8c7f458c5-gvzpd\" (UID: \"39c35ec4-5c1b-472e-9e73-991258bb835d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-029e8c-8c7f458c5-gvzpd"
Nov 04 01:19:20.118486 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:20.118458    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39c35ec4-5c1b-472e-9e73-991258bb835d-maas-api-tls\") pod \"maas-api-e2e-shared-a-w4-029e8c-8c7f458c5-gvzpd\" (UID: \"39c35ec4-5c1b-472e-9e73-991258bb835d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-029e8c-8c7f458c5-gvzpd"
Nov 04 01:19:20.118704 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:20.118685    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39c35ec4-5c1b-472e-9e73-991258bb835d-metrics-certs\") pod \"maas-api-e2e-shared-a-w4-029e8c-8c7f458c5-gvzpd\" (UID: \"39c35ec4-5c1b-472e-9e73-991258bb835d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-029e8c-8c7f458c5-gvzpd"
Nov 04 01:19:20.132370 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:20.132340    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gj2l8\" (UniqueName: \"kubernetes.io/projected/39c35ec4-5c1b-472e-9e73-991258bb835d-kube-api-access-gj2l8\") pod \"maas-api-e2e-shared-a-w4-029e8c-8c7f458c5-gvzpd\" (UID: \"39c35ec4-5c1b-472e-9e73-991258bb835d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-029e8c-8c7f458c5-gvzpd"
Nov 04 01:19:20.298484 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:20.298425    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-029e8c-8c7f458c5-gvzpd"
Nov 04 01:19:20.606196 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:20.606161    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-029e8c-8c7f458c5-gvzpd"]
Nov 04 01:19:20.608021 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:19:20.607990    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39c35ec4_5c1b_472e_9e73_991258bb835d.slice/crio-c3e7d04184b24934249efb4ab5074b404cb1b3757770532bc2b203ee3af1f8b1 WatchSource:0}: Error finding container c3e7d04184b24934249efb4ab5074b404cb1b3757770532bc2b203ee3af1f8b1: Status 404 returned error can't find the container with id c3e7d04184b24934249efb4ab5074b404cb1b3757770532bc2b203ee3af1f8b1
Nov 04 01:19:21.301766 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:21.301649    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-029e8c-8c7f458c5-gvzpd" event={"ID":"39c35ec4-5c1b-472e-9e73-991258bb835d","Type":"ContainerStarted","Data":"91e13ec9894502a467877006e22026d0f6952c4a721da97d920753bcef6f951c"}
Nov 04 01:19:21.301766 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:21.301749    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-029e8c-8c7f458c5-gvzpd"
Nov 04 01:19:21.302358 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:21.301774    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-029e8c-8c7f458c5-gvzpd" event={"ID":"39c35ec4-5c1b-472e-9e73-991258bb835d","Type":"ContainerStarted","Data":"c3e7d04184b24934249efb4ab5074b404cb1b3757770532bc2b203ee3af1f8b1"}
Nov 04 01:19:21.327051 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:21.326967    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-029e8c-8c7f458c5-gvzpd" podStartSLOduration=1.91468184 podStartE2EDuration="2.326936409s" totalImagesPullingTime="412.254569ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:19:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:19:21.322401655 +0000 UTC m=+1001.368962447" watchObservedRunningTime="2026-11-04 01:19:21.326936409 +0000 UTC m=+1001.373497190"
Nov 04 01:19:27.321719 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:27.321678    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-029e8c-8c7f458c5-gvzpd"
Nov 04 01:19:37.895844 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:37.895804    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w4-3bf4f9-7db74f6687-htzl4"]
Nov 04 01:19:37.900579 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:37.900552    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-3bf4f9-7db74f6687-htzl4"
Nov 04 01:19:37.907859 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:37.907806    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w4-3bf4f9-dockercfg-cvkx2\""
Nov 04 01:19:37.908151 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:37.908128    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w4-3bf4f9\""
Nov 04 01:19:37.913330 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:37.913280    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w4-3bf4f9-7db74f6687-htzl4"]
Nov 04 01:19:37.996550 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:37.996500    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7st4p\" (UniqueName: \"kubernetes.io/projected/2b505e5e-2cb5-47cc-97aa-02b96383d117-kube-api-access-7st4p\") pod \"payload-pre-processing-e2e-shared-b-w4-3bf4f9-7db74f6687-htzl4\" (UID: \"2b505e5e-2cb5-47cc-97aa-02b96383d117\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-3bf4f9-7db74f6687-htzl4"
Nov 04 01:19:37.996818 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:37.996795    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b505e5e-2cb5-47cc-97aa-02b96383d117-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w4-3bf4f9-7db74f6687-htzl4\" (UID: \"2b505e5e-2cb5-47cc-97aa-02b96383d117\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-3bf4f9-7db74f6687-htzl4"
Nov 04 01:19:38.100838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:38.099098    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b505e5e-2cb5-47cc-97aa-02b96383d117-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w4-3bf4f9-7db74f6687-htzl4\" (UID: \"2b505e5e-2cb5-47cc-97aa-02b96383d117\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-3bf4f9-7db74f6687-htzl4"
Nov 04 01:19:38.100838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:38.099352    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7st4p\" (UniqueName: \"kubernetes.io/projected/2b505e5e-2cb5-47cc-97aa-02b96383d117-kube-api-access-7st4p\") pod \"payload-pre-processing-e2e-shared-b-w4-3bf4f9-7db74f6687-htzl4\" (UID: \"2b505e5e-2cb5-47cc-97aa-02b96383d117\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-3bf4f9-7db74f6687-htzl4"
Nov 04 01:19:38.100838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:38.100763    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b505e5e-2cb5-47cc-97aa-02b96383d117-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w4-3bf4f9-7db74f6687-htzl4\" (UID: \"2b505e5e-2cb5-47cc-97aa-02b96383d117\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-3bf4f9-7db74f6687-htzl4"
Nov 04 01:19:38.142706 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:38.139355    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7st4p\" (UniqueName: \"kubernetes.io/projected/2b505e5e-2cb5-47cc-97aa-02b96383d117-kube-api-access-7st4p\") pod \"payload-pre-processing-e2e-shared-b-w4-3bf4f9-7db74f6687-htzl4\" (UID: \"2b505e5e-2cb5-47cc-97aa-02b96383d117\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-3bf4f9-7db74f6687-htzl4"
Nov 04 01:19:38.233249 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:38.232921    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-3bf4f9-7db74f6687-htzl4"
Nov 04 01:19:38.799657 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:38.797522    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w4-3bf4f9-7db74f6687-htzl4"]
Nov 04 01:19:39.602959 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:39.602922    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-3bf4f9-7db74f6687-htzl4" event={"ID":"2b505e5e-2cb5-47cc-97aa-02b96383d117","Type":"ContainerStarted","Data":"86c74b7747d80a8958b97f09f29795e3d8c0da31f7540c2fde4f4fa7bd2bde1c"}
Nov 04 01:19:39.603448 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:39.602986    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-3bf4f9-7db74f6687-htzl4" event={"ID":"2b505e5e-2cb5-47cc-97aa-02b96383d117","Type":"ContainerStarted","Data":"59996fcf51ebcf96311946e73e47c0da16494010da4783be4c8596c12a8d895b"}
Nov 04 01:19:39.603448 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:39.603021    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-3bf4f9-7db74f6687-htzl4"
Nov 04 01:19:39.625657 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:39.625584    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-3bf4f9-7db74f6687-htzl4" podStartSLOduration=2.625566757 podStartE2EDuration="2.625566757s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:19:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:19:39.620584563 +0000 UTC m=+1019.667145352" watchObservedRunningTime="2026-11-04 01:19:39.625566757 +0000 UTC m=+1019.672127546"
Nov 04 01:19:43.454348 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:19:43.454308    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:19:50.614446 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:19:50.614356    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-3bf4f9-7db74f6687-htzl4"
Nov 04 01:20:07.338065 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:07.338029    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:20:13.456146 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:20:13.456102    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:20:29.063349 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:29.063307    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w0-35980d-97689d97f-79zvl"]
Nov 04 01:20:29.069352 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:29.069248    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-35980d-97689d97f-79zvl"
Nov 04 01:20:29.074866 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:29.074766    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w0-35980d\""
Nov 04 01:20:29.097067 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:29.096966    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w0-35980d-97689d97f-79zvl"]
Nov 04 01:20:29.103170 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:29.103133    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-35980d-86c7c666d4-bphw8"]
Nov 04 01:20:29.108901 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:29.108254    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-35980d-86c7c666d4-bphw8"
Nov 04 01:20:29.109877 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:29.109852    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-35980d-86c7c666d4-bphw8"]
Nov 04 01:20:29.112265 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:29.112242    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w0-35980d-dockercfg-8428d\""
Nov 04 01:20:29.138088 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:29.138050    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34c18742-f9bd-4f63-8e99-6a63af9c5999-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w0-35980d-97689d97f-79zvl\" (UID: \"34c18742-f9bd-4f63-8e99-6a63af9c5999\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-35980d-97689d97f-79zvl"
Nov 04 01:20:29.138263 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:29.138163    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6wj8\" (UniqueName: \"kubernetes.io/projected/34c18742-f9bd-4f63-8e99-6a63af9c5999-kube-api-access-q6wj8\") pod \"payload-pre-processing-e2e-shared-b-w0-35980d-97689d97f-79zvl\" (UID: \"34c18742-f9bd-4f63-8e99-6a63af9c5999\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-35980d-97689d97f-79zvl"
Nov 04 01:20:29.138263 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:29.138203    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/788ddd8c-788c-4575-8208-fd949c0bcc38-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w0-35980d-86c7c666d4-bphw8\" (UID: \"788ddd8c-788c-4575-8208-fd949c0bcc38\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-35980d-86c7c666d4-bphw8"
Nov 04 01:20:29.138690 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:29.138661    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrzv9\" (UniqueName: \"kubernetes.io/projected/788ddd8c-788c-4575-8208-fd949c0bcc38-kube-api-access-jrzv9\") pod \"payload-processing-e2e-shared-b-w0-35980d-86c7c666d4-bphw8\" (UID: \"788ddd8c-788c-4575-8208-fd949c0bcc38\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-35980d-86c7c666d4-bphw8"
Nov 04 01:20:29.240287 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:29.240049    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrzv9\" (UniqueName: \"kubernetes.io/projected/788ddd8c-788c-4575-8208-fd949c0bcc38-kube-api-access-jrzv9\") pod \"payload-processing-e2e-shared-b-w0-35980d-86c7c666d4-bphw8\" (UID: \"788ddd8c-788c-4575-8208-fd949c0bcc38\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-35980d-86c7c666d4-bphw8"
Nov 04 01:20:29.240287 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:29.240285    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34c18742-f9bd-4f63-8e99-6a63af9c5999-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w0-35980d-97689d97f-79zvl\" (UID: \"34c18742-f9bd-4f63-8e99-6a63af9c5999\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-35980d-97689d97f-79zvl"
Nov 04 01:20:29.240665 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:29.240380    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6wj8\" (UniqueName: \"kubernetes.io/projected/34c18742-f9bd-4f63-8e99-6a63af9c5999-kube-api-access-q6wj8\") pod \"payload-pre-processing-e2e-shared-b-w0-35980d-97689d97f-79zvl\" (UID: \"34c18742-f9bd-4f63-8e99-6a63af9c5999\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-35980d-97689d97f-79zvl"
Nov 04 01:20:29.240665 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:29.240420    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/788ddd8c-788c-4575-8208-fd949c0bcc38-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w0-35980d-86c7c666d4-bphw8\" (UID: \"788ddd8c-788c-4575-8208-fd949c0bcc38\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-35980d-86c7c666d4-bphw8"
Nov 04 01:20:29.241300 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:29.241276    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34c18742-f9bd-4f63-8e99-6a63af9c5999-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w0-35980d-97689d97f-79zvl\" (UID: \"34c18742-f9bd-4f63-8e99-6a63af9c5999\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-35980d-97689d97f-79zvl"
Nov 04 01:20:29.241433 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:29.241411    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/788ddd8c-788c-4575-8208-fd949c0bcc38-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w0-35980d-86c7c666d4-bphw8\" (UID: \"788ddd8c-788c-4575-8208-fd949c0bcc38\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-35980d-86c7c666d4-bphw8"
Nov 04 01:20:29.254340 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:29.254275    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrzv9\" (UniqueName: \"kubernetes.io/projected/788ddd8c-788c-4575-8208-fd949c0bcc38-kube-api-access-jrzv9\") pod \"payload-processing-e2e-shared-b-w0-35980d-86c7c666d4-bphw8\" (UID: \"788ddd8c-788c-4575-8208-fd949c0bcc38\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-35980d-86c7c666d4-bphw8"
Nov 04 01:20:29.310212 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:29.310139    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6wj8\" (UniqueName: \"kubernetes.io/projected/34c18742-f9bd-4f63-8e99-6a63af9c5999-kube-api-access-q6wj8\") pod \"payload-pre-processing-e2e-shared-b-w0-35980d-97689d97f-79zvl\" (UID: \"34c18742-f9bd-4f63-8e99-6a63af9c5999\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-35980d-97689d97f-79zvl"
Nov 04 01:20:29.424196 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:29.424110    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-35980d-97689d97f-79zvl"
Nov 04 01:20:29.444656 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:29.434727    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-35980d-86c7c666d4-bphw8"
Nov 04 01:20:29.801663 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:29.798208    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w0-35980d-97689d97f-79zvl"]
Nov 04 01:20:29.801663 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:20:29.799358    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34c18742_f9bd_4f63_8e99_6a63af9c5999.slice/crio-85592b8fb5cf2d1a9d87721364945868692a402b755a0411d5795d95ceca791a WatchSource:0}: Error finding container 85592b8fb5cf2d1a9d87721364945868692a402b755a0411d5795d95ceca791a: Status 404 returned error can't find the container with id 85592b8fb5cf2d1a9d87721364945868692a402b755a0411d5795d95ceca791a
Nov 04 01:20:29.804592 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:29.804508    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-35980d-86c7c666d4-bphw8"]
Nov 04 01:20:30.501709 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:30.501669    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-35980d-86c7c666d4-bphw8" event={"ID":"788ddd8c-788c-4575-8208-fd949c0bcc38","Type":"ContainerStarted","Data":"cb04e6904827a62277a15e0cde41bf75fffe9c369d86b99af009e928a14c8070"}
Nov 04 01:20:30.502204 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:30.501727    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-35980d-86c7c666d4-bphw8" event={"ID":"788ddd8c-788c-4575-8208-fd949c0bcc38","Type":"ContainerStarted","Data":"737a101b8059408e2cd997c96145df8c0fb86fd0edb12a656d72100492c7cc36"}
Nov 04 01:20:30.502204 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:30.501757    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-35980d-86c7c666d4-bphw8"
Nov 04 01:20:30.503523 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:30.503493    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-35980d-97689d97f-79zvl" event={"ID":"34c18742-f9bd-4f63-8e99-6a63af9c5999","Type":"ContainerStarted","Data":"14a5bc89d5ef6b9a779cf9e51da432801213a60c157f9edbaf58b3941efdb64f"}
Nov 04 01:20:30.503681 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:30.503544    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-35980d-97689d97f-79zvl" event={"ID":"34c18742-f9bd-4f63-8e99-6a63af9c5999","Type":"ContainerStarted","Data":"85592b8fb5cf2d1a9d87721364945868692a402b755a0411d5795d95ceca791a"}
Nov 04 01:20:30.503681 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:30.503568    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-35980d-97689d97f-79zvl"
Nov 04 01:20:30.520197 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:30.520149    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-35980d-86c7c666d4-bphw8" podStartSLOduration=1.520134569 podStartE2EDuration="1.520134569s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:20:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:20:30.51926621 +0000 UTC m=+1070.565827000" watchObservedRunningTime="2026-11-04 01:20:30.520134569 +0000 UTC m=+1070.566695395"
Nov 04 01:20:30.537738 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:30.537681    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-35980d-97689d97f-79zvl" podStartSLOduration=1.537664927 podStartE2EDuration="1.537664927s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:20:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:20:30.536927768 +0000 UTC m=+1070.583488557" watchObservedRunningTime="2026-11-04 01:20:30.537664927 +0000 UTC m=+1070.584225717"
Nov 04 01:20:36.808348 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:36.808312    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-35980d-64c6d6dc7d-j2s9z"]
Nov 04 01:20:36.813351 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:36.813326    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-35980d-64c6d6dc7d-j2s9z"
Nov 04 01:20:36.826714 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:36.826681    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-35980d-64c6d6dc7d-j2s9z"]
Nov 04 01:20:36.875680 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:36.875601    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w0-35980d-serving-cert\""
Nov 04 01:20:36.911146 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:36.911111    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ac72a77-4112-4df4-ada5-ea100fe5dfb2-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w0-35980d-64c6d6dc7d-j2s9z\" (UID: \"1ac72a77-4112-4df4-ada5-ea100fe5dfb2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-35980d-64c6d6dc7d-j2s9z"
Nov 04 01:20:36.911344 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:36.911305    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ac72a77-4112-4df4-ada5-ea100fe5dfb2-metrics-certs\") pod \"maas-api-e2e-shared-b-w0-35980d-64c6d6dc7d-j2s9z\" (UID: \"1ac72a77-4112-4df4-ada5-ea100fe5dfb2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-35980d-64c6d6dc7d-j2s9z"
Nov 04 01:20:36.911390 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:36.911377    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cszvz\" (UniqueName: \"kubernetes.io/projected/1ac72a77-4112-4df4-ada5-ea100fe5dfb2-kube-api-access-cszvz\") pod \"maas-api-e2e-shared-b-w0-35980d-64c6d6dc7d-j2s9z\" (UID: \"1ac72a77-4112-4df4-ada5-ea100fe5dfb2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-35980d-64c6d6dc7d-j2s9z"
Nov 04 01:20:36.911511 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:36.911477    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ac72a77-4112-4df4-ada5-ea100fe5dfb2-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-35980d-64c6d6dc7d-j2s9z\" (UID: \"1ac72a77-4112-4df4-ada5-ea100fe5dfb2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-35980d-64c6d6dc7d-j2s9z"
Nov 04 01:20:37.012955 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:37.012919    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ac72a77-4112-4df4-ada5-ea100fe5dfb2-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w0-35980d-64c6d6dc7d-j2s9z\" (UID: \"1ac72a77-4112-4df4-ada5-ea100fe5dfb2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-35980d-64c6d6dc7d-j2s9z"
Nov 04 01:20:37.013148 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:37.013049    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ac72a77-4112-4df4-ada5-ea100fe5dfb2-metrics-certs\") pod \"maas-api-e2e-shared-b-w0-35980d-64c6d6dc7d-j2s9z\" (UID: \"1ac72a77-4112-4df4-ada5-ea100fe5dfb2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-35980d-64c6d6dc7d-j2s9z"
Nov 04 01:20:37.013148 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:37.013130    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cszvz\" (UniqueName: \"kubernetes.io/projected/1ac72a77-4112-4df4-ada5-ea100fe5dfb2-kube-api-access-cszvz\") pod \"maas-api-e2e-shared-b-w0-35980d-64c6d6dc7d-j2s9z\" (UID: \"1ac72a77-4112-4df4-ada5-ea100fe5dfb2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-35980d-64c6d6dc7d-j2s9z"
Nov 04 01:20:37.013254 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:37.013197    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ac72a77-4112-4df4-ada5-ea100fe5dfb2-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-35980d-64c6d6dc7d-j2s9z\" (UID: \"1ac72a77-4112-4df4-ada5-ea100fe5dfb2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-35980d-64c6d6dc7d-j2s9z"
Nov 04 01:20:37.013859 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:37.013831    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ac72a77-4112-4df4-ada5-ea100fe5dfb2-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w0-35980d-64c6d6dc7d-j2s9z\" (UID: \"1ac72a77-4112-4df4-ada5-ea100fe5dfb2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-35980d-64c6d6dc7d-j2s9z"
Nov 04 01:20:37.016687 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:37.016660    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ac72a77-4112-4df4-ada5-ea100fe5dfb2-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-35980d-64c6d6dc7d-j2s9z\" (UID: \"1ac72a77-4112-4df4-ada5-ea100fe5dfb2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-35980d-64c6d6dc7d-j2s9z"
Nov 04 01:20:37.018523 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:37.018049    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ac72a77-4112-4df4-ada5-ea100fe5dfb2-metrics-certs\") pod \"maas-api-e2e-shared-b-w0-35980d-64c6d6dc7d-j2s9z\" (UID: \"1ac72a77-4112-4df4-ada5-ea100fe5dfb2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-35980d-64c6d6dc7d-j2s9z"
Nov 04 01:20:37.040596 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:37.040557    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cszvz\" (UniqueName: \"kubernetes.io/projected/1ac72a77-4112-4df4-ada5-ea100fe5dfb2-kube-api-access-cszvz\") pod \"maas-api-e2e-shared-b-w0-35980d-64c6d6dc7d-j2s9z\" (UID: \"1ac72a77-4112-4df4-ada5-ea100fe5dfb2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-35980d-64c6d6dc7d-j2s9z"
Nov 04 01:20:37.135430 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:37.135327    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-35980d-64c6d6dc7d-j2s9z"
Nov 04 01:20:37.558645 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:37.551886    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-35980d-64c6d6dc7d-j2s9z"]
Nov 04 01:20:37.601648 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:37.596176    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-35980d-64c6d6dc7d-j2s9z" event={"ID":"1ac72a77-4112-4df4-ada5-ea100fe5dfb2","Type":"ContainerStarted","Data":"b320bf44a930e7e6b1cdbd5a3a59652cf1a3a5edc72e55b423eea3ccdd087617"}
Nov 04 01:20:38.606323 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:38.606281    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-35980d-64c6d6dc7d-j2s9z" event={"ID":"1ac72a77-4112-4df4-ada5-ea100fe5dfb2","Type":"ContainerStarted","Data":"80a657cbe2cb99374f1212521d12dcf6951caa5b6fb3dfd71a2dadb62d1c3190"}
Nov 04 01:20:38.606756 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:38.606342    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-35980d-64c6d6dc7d-j2s9z"
Nov 04 01:20:38.631942 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:38.631881    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-35980d-64c6d6dc7d-j2s9z" podStartSLOduration=2.224442937 podStartE2EDuration="2.63186573s" totalImagesPullingTime="407.422793ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:20:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:20:38.626448786 +0000 UTC m=+1078.673009577" watchObservedRunningTime="2026-11-04 01:20:38.63186573 +0000 UTC m=+1078.678426576"
Nov 04 01:20:41.063904 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:41.063860    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:20:41.514873 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:41.514844    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-35980d-97689d97f-79zvl"
Nov 04 01:20:41.515049 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:41.514918    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-35980d-86c7c666d4-bphw8"
Nov 04 01:20:41.959220 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:41.959122    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-e7742169/e2e-lifecycle-model-e7742169-kserve-79fc5d5dd5-xdbvv"]
Nov 04 01:20:41.965879 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:41.965843    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e7742169/e2e-lifecycle-model-e7742169-kserve-79fc5d5dd5-xdbvv"
Nov 04 01:20:41.977806 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:41.977777    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-e7742169\"/\"kube-root-ca.crt\""
Nov 04 01:20:41.978373 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:41.978347    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-e7742169\"/\"openshift-service-ca.crt\""
Nov 04 01:20:41.978529 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:41.978472    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-e7742169\"/\"e2e-lifecycle-model-e7742169-kserve-self-signed-certs\""
Nov 04 01:20:41.978672 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:41.978652    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-e7742169\"/\"default-dockercfg-4vmxg\""
Nov 04 01:20:41.979105 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:41.979074    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-e7742169/e2e-lifecycle-model-e7742169-kserve-79fc5d5dd5-xdbvv"]
Nov 04 01:20:42.050706 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:42.050654    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znhtb\" (UniqueName: \"kubernetes.io/projected/55185ce1-87ff-46ab-a8e4-d9f69de69e55-kube-api-access-znhtb\") pod \"e2e-lifecycle-model-e7742169-kserve-79fc5d5dd5-xdbvv\" (UID: \"55185ce1-87ff-46ab-a8e4-d9f69de69e55\") " pod="ai-tenant-e2e-mt-e7742169/e2e-lifecycle-model-e7742169-kserve-79fc5d5dd5-xdbvv"
Nov 04 01:20:42.050916 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:42.050754    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/55185ce1-87ff-46ab-a8e4-d9f69de69e55-tls-certs\") pod \"e2e-lifecycle-model-e7742169-kserve-79fc5d5dd5-xdbvv\" (UID: \"55185ce1-87ff-46ab-a8e4-d9f69de69e55\") " pod="ai-tenant-e2e-mt-e7742169/e2e-lifecycle-model-e7742169-kserve-79fc5d5dd5-xdbvv"
Nov 04 01:20:42.050979 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:42.050932    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55185ce1-87ff-46ab-a8e4-d9f69de69e55-tmp-dir\") pod \"e2e-lifecycle-model-e7742169-kserve-79fc5d5dd5-xdbvv\" (UID: \"55185ce1-87ff-46ab-a8e4-d9f69de69e55\") " pod="ai-tenant-e2e-mt-e7742169/e2e-lifecycle-model-e7742169-kserve-79fc5d5dd5-xdbvv"
Nov 04 01:20:42.051168 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:42.051146    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/55185ce1-87ff-46ab-a8e4-d9f69de69e55-model-cache\") pod \"e2e-lifecycle-model-e7742169-kserve-79fc5d5dd5-xdbvv\" (UID: \"55185ce1-87ff-46ab-a8e4-d9f69de69e55\") " pod="ai-tenant-e2e-mt-e7742169/e2e-lifecycle-model-e7742169-kserve-79fc5d5dd5-xdbvv"
Nov 04 01:20:42.051464 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:42.051444    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/55185ce1-87ff-46ab-a8e4-d9f69de69e55-home\") pod \"e2e-lifecycle-model-e7742169-kserve-79fc5d5dd5-xdbvv\" (UID: \"55185ce1-87ff-46ab-a8e4-d9f69de69e55\") " pod="ai-tenant-e2e-mt-e7742169/e2e-lifecycle-model-e7742169-kserve-79fc5d5dd5-xdbvv"
Nov 04 01:20:42.051524 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:42.051480    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/55185ce1-87ff-46ab-a8e4-d9f69de69e55-dshm\") pod \"e2e-lifecycle-model-e7742169-kserve-79fc5d5dd5-xdbvv\" (UID: \"55185ce1-87ff-46ab-a8e4-d9f69de69e55\") " pod="ai-tenant-e2e-mt-e7742169/e2e-lifecycle-model-e7742169-kserve-79fc5d5dd5-xdbvv"
Nov 04 01:20:42.153532 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:42.153491    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/55185ce1-87ff-46ab-a8e4-d9f69de69e55-home\") pod \"e2e-lifecycle-model-e7742169-kserve-79fc5d5dd5-xdbvv\" (UID: \"55185ce1-87ff-46ab-a8e4-d9f69de69e55\") " pod="ai-tenant-e2e-mt-e7742169/e2e-lifecycle-model-e7742169-kserve-79fc5d5dd5-xdbvv"
Nov 04 01:20:42.153532 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:42.153536    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/55185ce1-87ff-46ab-a8e4-d9f69de69e55-dshm\") pod \"e2e-lifecycle-model-e7742169-kserve-79fc5d5dd5-xdbvv\" (UID: \"55185ce1-87ff-46ab-a8e4-d9f69de69e55\") " pod="ai-tenant-e2e-mt-e7742169/e2e-lifecycle-model-e7742169-kserve-79fc5d5dd5-xdbvv"
Nov 04 01:20:42.154104 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:42.153562    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znhtb\" (UniqueName: \"kubernetes.io/projected/55185ce1-87ff-46ab-a8e4-d9f69de69e55-kube-api-access-znhtb\") pod \"e2e-lifecycle-model-e7742169-kserve-79fc5d5dd5-xdbvv\" (UID: \"55185ce1-87ff-46ab-a8e4-d9f69de69e55\") " pod="ai-tenant-e2e-mt-e7742169/e2e-lifecycle-model-e7742169-kserve-79fc5d5dd5-xdbvv"
Nov 04 01:20:42.154104 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:42.153603    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/55185ce1-87ff-46ab-a8e4-d9f69de69e55-tls-certs\") pod \"e2e-lifecycle-model-e7742169-kserve-79fc5d5dd5-xdbvv\" (UID: \"55185ce1-87ff-46ab-a8e4-d9f69de69e55\") " pod="ai-tenant-e2e-mt-e7742169/e2e-lifecycle-model-e7742169-kserve-79fc5d5dd5-xdbvv"
Nov 04 01:20:42.154104 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:42.153687    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55185ce1-87ff-46ab-a8e4-d9f69de69e55-tmp-dir\") pod \"e2e-lifecycle-model-e7742169-kserve-79fc5d5dd5-xdbvv\" (UID: \"55185ce1-87ff-46ab-a8e4-d9f69de69e55\") " pod="ai-tenant-e2e-mt-e7742169/e2e-lifecycle-model-e7742169-kserve-79fc5d5dd5-xdbvv"
Nov 04 01:20:42.154104 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:42.153802    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/55185ce1-87ff-46ab-a8e4-d9f69de69e55-model-cache\") pod \"e2e-lifecycle-model-e7742169-kserve-79fc5d5dd5-xdbvv\" (UID: \"55185ce1-87ff-46ab-a8e4-d9f69de69e55\") " pod="ai-tenant-e2e-mt-e7742169/e2e-lifecycle-model-e7742169-kserve-79fc5d5dd5-xdbvv"
Nov 04 01:20:42.154697 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:42.154358    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/55185ce1-87ff-46ab-a8e4-d9f69de69e55-home\") pod \"e2e-lifecycle-model-e7742169-kserve-79fc5d5dd5-xdbvv\" (UID: \"55185ce1-87ff-46ab-a8e4-d9f69de69e55\") " pod="ai-tenant-e2e-mt-e7742169/e2e-lifecycle-model-e7742169-kserve-79fc5d5dd5-xdbvv"
Nov 04 01:20:42.154697 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:42.154413    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/55185ce1-87ff-46ab-a8e4-d9f69de69e55-model-cache\") pod \"e2e-lifecycle-model-e7742169-kserve-79fc5d5dd5-xdbvv\" (UID: \"55185ce1-87ff-46ab-a8e4-d9f69de69e55\") " pod="ai-tenant-e2e-mt-e7742169/e2e-lifecycle-model-e7742169-kserve-79fc5d5dd5-xdbvv"
Nov 04 01:20:42.154697 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:42.154657    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55185ce1-87ff-46ab-a8e4-d9f69de69e55-tmp-dir\") pod \"e2e-lifecycle-model-e7742169-kserve-79fc5d5dd5-xdbvv\" (UID: \"55185ce1-87ff-46ab-a8e4-d9f69de69e55\") " pod="ai-tenant-e2e-mt-e7742169/e2e-lifecycle-model-e7742169-kserve-79fc5d5dd5-xdbvv"
Nov 04 01:20:42.157060 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:42.156926    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/55185ce1-87ff-46ab-a8e4-d9f69de69e55-dshm\") pod \"e2e-lifecycle-model-e7742169-kserve-79fc5d5dd5-xdbvv\" (UID: \"55185ce1-87ff-46ab-a8e4-d9f69de69e55\") " pod="ai-tenant-e2e-mt-e7742169/e2e-lifecycle-model-e7742169-kserve-79fc5d5dd5-xdbvv"
Nov 04 01:20:42.158560 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:42.158530    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/55185ce1-87ff-46ab-a8e4-d9f69de69e55-tls-certs\") pod \"e2e-lifecycle-model-e7742169-kserve-79fc5d5dd5-xdbvv\" (UID: \"55185ce1-87ff-46ab-a8e4-d9f69de69e55\") " pod="ai-tenant-e2e-mt-e7742169/e2e-lifecycle-model-e7742169-kserve-79fc5d5dd5-xdbvv"
Nov 04 01:20:42.166509 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:42.166482    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-znhtb\" (UniqueName: \"kubernetes.io/projected/55185ce1-87ff-46ab-a8e4-d9f69de69e55-kube-api-access-znhtb\") pod \"e2e-lifecycle-model-e7742169-kserve-79fc5d5dd5-xdbvv\" (UID: \"55185ce1-87ff-46ab-a8e4-d9f69de69e55\") " pod="ai-tenant-e2e-mt-e7742169/e2e-lifecycle-model-e7742169-kserve-79fc5d5dd5-xdbvv"
Nov 04 01:20:42.309213 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:42.307993    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e7742169/e2e-lifecycle-model-e7742169-kserve-79fc5d5dd5-xdbvv"
Nov 04 01:20:42.935814 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:42.935291    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-e7742169/e2e-lifecycle-model-e7742169-kserve-79fc5d5dd5-xdbvv"]
Nov 04 01:20:42.936262 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:20:42.936213    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod55185ce1_87ff_46ab_a8e4_d9f69de69e55.slice/crio-034504b4df51358eec0e34fb4e142abc07f0aba97badf9a0ec88f72ef7bd5ccb WatchSource:0}: Error finding container 034504b4df51358eec0e34fb4e142abc07f0aba97badf9a0ec88f72ef7bd5ccb: Status 404 returned error can't find the container with id 034504b4df51358eec0e34fb4e142abc07f0aba97badf9a0ec88f72ef7bd5ccb
Nov 04 01:20:43.505295 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:20:43.505243    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:20:43.674791 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:43.674744    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e7742169/e2e-lifecycle-model-e7742169-kserve-79fc5d5dd5-xdbvv" event={"ID":"55185ce1-87ff-46ab-a8e4-d9f69de69e55","Type":"ContainerStarted","Data":"e13830de628d78cfa5ec75f3c70841fd8cf98fd81c3145a201f2af186ce8b369"}
Nov 04 01:20:43.675063 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:43.675043    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e7742169/e2e-lifecycle-model-e7742169-kserve-79fc5d5dd5-xdbvv" event={"ID":"55185ce1-87ff-46ab-a8e4-d9f69de69e55","Type":"ContainerStarted","Data":"034504b4df51358eec0e34fb4e142abc07f0aba97badf9a0ec88f72ef7bd5ccb"}
Nov 04 01:20:43.710414 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:43.708899    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-mt-e7742169/e2e-lifecycle-model-e7742169-kserve-79fc5d5dd5-xdbvv" podStartSLOduration=2.708876403 podStartE2EDuration="2.708876403s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:20:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:20:43.69894058 +0000 UTC m=+1083.745501374" watchObservedRunningTime="2026-11-04 01:20:43.708876403 +0000 UTC m=+1083.755437195"
Nov 04 01:20:44.360802 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:44.360674    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-e7742169/e2e-lifecycle-model-e7742169-kserve-79fc5d5dd5-xdbvv"
Nov 04 01:20:44.642837 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:44.642749    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-35980d-64c6d6dc7d-j2s9z"
Nov 04 01:20:52.311992 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:52.311957    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-e7742169/e2e-lifecycle-model-e7742169-kserve-79fc5d5dd5-xdbvv"
Nov 04 01:20:52.325673 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:52.325646    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-e7742169/e2e-lifecycle-model-e7742169-kserve-79fc5d5dd5-xdbvv"
Nov 04 01:20:52.839325 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:52.839291    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-e7742169/e2e-lifecycle-model-e7742169-kserve-79fc5d5dd5-xdbvv"
Nov 04 01:20:54.249399 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:54.249352    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-6987-a1d191"]
Nov 04 01:20:54.254905 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:54.254881    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6987-a1d191"
Nov 04 01:20:54.262465 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:54.262434    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-6987-a1d191"]
Nov 04 01:20:54.265116 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:54.265089    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-lnpw7\""
Nov 04 01:20:54.418219 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:54.418181    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtmgb\" (UniqueName: \"kubernetes.io/projected/1e2c7811-daf7-4187-a1ad-2a442d206076-kube-api-access-mtmgb\") pod \"test-curl-6987-a1d191\" (UID: \"1e2c7811-daf7-4187-a1ad-2a442d206076\") " pod="opendatahub/test-curl-6987-a1d191"
Nov 04 01:20:54.497478 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:54.497447    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-wljtg"]
Nov 04 01:20:54.502736 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:54.502668    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-wljtg"
Nov 04 01:20:54.518851 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:54.516930    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-wljtg"]
Nov 04 01:20:54.521153 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:54.521120    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtmgb\" (UniqueName: \"kubernetes.io/projected/1e2c7811-daf7-4187-a1ad-2a442d206076-kube-api-access-mtmgb\") pod \"test-curl-6987-a1d191\" (UID: \"1e2c7811-daf7-4187-a1ad-2a442d206076\") " pod="opendatahub/test-curl-6987-a1d191"
Nov 04 01:20:54.533473 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:54.533446    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtmgb\" (UniqueName: \"kubernetes.io/projected/1e2c7811-daf7-4187-a1ad-2a442d206076-kube-api-access-mtmgb\") pod \"test-curl-6987-a1d191\" (UID: \"1e2c7811-daf7-4187-a1ad-2a442d206076\") " pod="opendatahub/test-curl-6987-a1d191"
Nov 04 01:20:54.589679 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:54.589648    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6987-a1d191"
Nov 04 01:20:54.598878 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:54.598851    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-e7742169-openshift-default-dockercfg-dp2rm\""
Nov 04 01:20:54.622283 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:54.622236    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f500082-e92f-4d13-a35b-2b775553b45d-istio-data\") pod \"e2e-mt-e7742169-openshift-default-6986f46949-wljtg\" (UID: \"2f500082-e92f-4d13-a35b-2b775553b45d\") " pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-wljtg"
Nov 04 01:20:54.622542 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:54.622511    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f500082-e92f-4d13-a35b-2b775553b45d-credential-socket\") pod \"e2e-mt-e7742169-openshift-default-6986f46949-wljtg\" (UID: \"2f500082-e92f-4d13-a35b-2b775553b45d\") " pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-wljtg"
Nov 04 01:20:54.622676 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:54.622562    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f500082-e92f-4d13-a35b-2b775553b45d-istio-podinfo\") pod \"e2e-mt-e7742169-openshift-default-6986f46949-wljtg\" (UID: \"2f500082-e92f-4d13-a35b-2b775553b45d\") " pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-wljtg"
Nov 04 01:20:54.622676 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:54.622631    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f500082-e92f-4d13-a35b-2b775553b45d-workload-socket\") pod \"e2e-mt-e7742169-openshift-default-6986f46949-wljtg\" (UID: \"2f500082-e92f-4d13-a35b-2b775553b45d\") " pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-wljtg"
Nov 04 01:20:54.622676 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:54.622660    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7qlz\" (UniqueName: \"kubernetes.io/projected/2f500082-e92f-4d13-a35b-2b775553b45d-kube-api-access-m7qlz\") pod \"e2e-mt-e7742169-openshift-default-6986f46949-wljtg\" (UID: \"2f500082-e92f-4d13-a35b-2b775553b45d\") " pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-wljtg"
Nov 04 01:20:54.622838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:54.622740    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f500082-e92f-4d13-a35b-2b775553b45d-workload-certs\") pod \"e2e-mt-e7742169-openshift-default-6986f46949-wljtg\" (UID: \"2f500082-e92f-4d13-a35b-2b775553b45d\") " pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-wljtg"
Nov 04 01:20:54.622838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:54.622784    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f500082-e92f-4d13-a35b-2b775553b45d-istiod-ca-cert\") pod \"e2e-mt-e7742169-openshift-default-6986f46949-wljtg\" (UID: \"2f500082-e92f-4d13-a35b-2b775553b45d\") " pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-wljtg"
Nov 04 01:20:54.622943 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:54.622837    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f500082-e92f-4d13-a35b-2b775553b45d-istio-token\") pod \"e2e-mt-e7742169-openshift-default-6986f46949-wljtg\" (UID: \"2f500082-e92f-4d13-a35b-2b775553b45d\") " pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-wljtg"
Nov 04 01:20:54.623004 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:54.622952    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f500082-e92f-4d13-a35b-2b775553b45d-istio-envoy\") pod \"e2e-mt-e7742169-openshift-default-6986f46949-wljtg\" (UID: \"2f500082-e92f-4d13-a35b-2b775553b45d\") " pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-wljtg"
Nov 04 01:20:54.726881 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:54.724120    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f500082-e92f-4d13-a35b-2b775553b45d-istio-token\") pod \"e2e-mt-e7742169-openshift-default-6986f46949-wljtg\" (UID: \"2f500082-e92f-4d13-a35b-2b775553b45d\") " pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-wljtg"
Nov 04 01:20:54.726881 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:54.724241    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f500082-e92f-4d13-a35b-2b775553b45d-istio-envoy\") pod \"e2e-mt-e7742169-openshift-default-6986f46949-wljtg\" (UID: \"2f500082-e92f-4d13-a35b-2b775553b45d\") " pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-wljtg"
Nov 04 01:20:54.726881 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:54.724332    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f500082-e92f-4d13-a35b-2b775553b45d-istio-data\") pod \"e2e-mt-e7742169-openshift-default-6986f46949-wljtg\" (UID: \"2f500082-e92f-4d13-a35b-2b775553b45d\") " pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-wljtg"
Nov 04 01:20:54.726881 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:54.724452    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f500082-e92f-4d13-a35b-2b775553b45d-credential-socket\") pod \"e2e-mt-e7742169-openshift-default-6986f46949-wljtg\" (UID: \"2f500082-e92f-4d13-a35b-2b775553b45d\") " pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-wljtg"
Nov 04 01:20:54.726881 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:54.724480    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f500082-e92f-4d13-a35b-2b775553b45d-istio-podinfo\") pod \"e2e-mt-e7742169-openshift-default-6986f46949-wljtg\" (UID: \"2f500082-e92f-4d13-a35b-2b775553b45d\") " pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-wljtg"
Nov 04 01:20:54.726881 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:54.724532    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f500082-e92f-4d13-a35b-2b775553b45d-workload-socket\") pod \"e2e-mt-e7742169-openshift-default-6986f46949-wljtg\" (UID: \"2f500082-e92f-4d13-a35b-2b775553b45d\") " pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-wljtg"
Nov 04 01:20:54.726881 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:54.724562    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7qlz\" (UniqueName: \"kubernetes.io/projected/2f500082-e92f-4d13-a35b-2b775553b45d-kube-api-access-m7qlz\") pod \"e2e-mt-e7742169-openshift-default-6986f46949-wljtg\" (UID: \"2f500082-e92f-4d13-a35b-2b775553b45d\") " pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-wljtg"
Nov 04 01:20:54.726881 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:54.725085    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f500082-e92f-4d13-a35b-2b775553b45d-istio-data\") pod \"e2e-mt-e7742169-openshift-default-6986f46949-wljtg\" (UID: \"2f500082-e92f-4d13-a35b-2b775553b45d\") " pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-wljtg"
Nov 04 01:20:54.726881 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:54.725248    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f500082-e92f-4d13-a35b-2b775553b45d-workload-certs\") pod \"e2e-mt-e7742169-openshift-default-6986f46949-wljtg\" (UID: \"2f500082-e92f-4d13-a35b-2b775553b45d\") " pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-wljtg"
Nov 04 01:20:54.726881 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:54.725296    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f500082-e92f-4d13-a35b-2b775553b45d-istiod-ca-cert\") pod \"e2e-mt-e7742169-openshift-default-6986f46949-wljtg\" (UID: \"2f500082-e92f-4d13-a35b-2b775553b45d\") " pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-wljtg"
Nov 04 01:20:54.726881 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:54.726014    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f500082-e92f-4d13-a35b-2b775553b45d-istiod-ca-cert\") pod \"e2e-mt-e7742169-openshift-default-6986f46949-wljtg\" (UID: \"2f500082-e92f-4d13-a35b-2b775553b45d\") " pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-wljtg"
Nov 04 01:20:54.726881 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:54.726264    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f500082-e92f-4d13-a35b-2b775553b45d-workload-socket\") pod \"e2e-mt-e7742169-openshift-default-6986f46949-wljtg\" (UID: \"2f500082-e92f-4d13-a35b-2b775553b45d\") " pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-wljtg"
Nov 04 01:20:54.726881 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:54.726468    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f500082-e92f-4d13-a35b-2b775553b45d-credential-socket\") pod \"e2e-mt-e7742169-openshift-default-6986f46949-wljtg\" (UID: \"2f500082-e92f-4d13-a35b-2b775553b45d\") " pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-wljtg"
Nov 04 01:20:54.727803 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:54.726956    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f500082-e92f-4d13-a35b-2b775553b45d-workload-certs\") pod \"e2e-mt-e7742169-openshift-default-6986f46949-wljtg\" (UID: \"2f500082-e92f-4d13-a35b-2b775553b45d\") " pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-wljtg"
Nov 04 01:20:54.728627 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:54.727938    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f500082-e92f-4d13-a35b-2b775553b45d-istio-envoy\") pod \"e2e-mt-e7742169-openshift-default-6986f46949-wljtg\" (UID: \"2f500082-e92f-4d13-a35b-2b775553b45d\") " pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-wljtg"
Nov 04 01:20:54.729626 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:54.729593    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f500082-e92f-4d13-a35b-2b775553b45d-istio-podinfo\") pod \"e2e-mt-e7742169-openshift-default-6986f46949-wljtg\" (UID: \"2f500082-e92f-4d13-a35b-2b775553b45d\") " pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-wljtg"
Nov 04 01:20:54.752428 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:54.752390    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7qlz\" (UniqueName: \"kubernetes.io/projected/2f500082-e92f-4d13-a35b-2b775553b45d-kube-api-access-m7qlz\") pod \"e2e-mt-e7742169-openshift-default-6986f46949-wljtg\" (UID: \"2f500082-e92f-4d13-a35b-2b775553b45d\") " pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-wljtg"
Nov 04 01:20:54.774527 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:54.774467    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f500082-e92f-4d13-a35b-2b775553b45d-istio-token\") pod \"e2e-mt-e7742169-openshift-default-6986f46949-wljtg\" (UID: \"2f500082-e92f-4d13-a35b-2b775553b45d\") " pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-wljtg"
Nov 04 01:20:54.824066 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:54.824032    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-wljtg"
Nov 04 01:20:54.949701 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:54.939258    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-6987-a1d191"]
Nov 04 01:20:55.216833 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:55.216799    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-wljtg"]
Nov 04 01:20:55.217218 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:20:55.217193    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f500082_e92f_4d13_a35b_2b775553b45d.slice/crio-244e06a33ead1de1ec677095e84ac0a9d2b58e90ec48e079391da10185065a8b WatchSource:0}: Error finding container 244e06a33ead1de1ec677095e84ac0a9d2b58e90ec48e079391da10185065a8b: Status 404 returned error can't find the container with id 244e06a33ead1de1ec677095e84ac0a9d2b58e90ec48e079391da10185065a8b
Nov 04 01:20:55.220655 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:55.220598    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:20:55.220736 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:55.220695    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:20:55.220736 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:55.220725    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:20:55.894909 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:55.894865    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-6987-a1d191" event={"ID":"1e2c7811-daf7-4187-a1ad-2a442d206076","Type":"ContainerStarted","Data":"572f3b288c58f1ecd4e3e3b10f693c0f2230c991cb81f66d2a93f97ffef2ff8a"}
Nov 04 01:20:55.895503 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:55.894925    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-6987-a1d191" event={"ID":"1e2c7811-daf7-4187-a1ad-2a442d206076","Type":"ContainerStarted","Data":"d6797440df23e94991e2d96975e696bb335087d525f254a269302e5f836c4680"}
Nov 04 01:20:55.898007 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:55.897975    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-wljtg" event={"ID":"2f500082-e92f-4d13-a35b-2b775553b45d","Type":"ContainerStarted","Data":"a7776145535c7f3becaa02d8035503c2bbd04744411ba22cb460afb919307ad1"}
Nov 04 01:20:55.898159 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:55.898064    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-wljtg" event={"ID":"2f500082-e92f-4d13-a35b-2b775553b45d","Type":"ContainerStarted","Data":"244e06a33ead1de1ec677095e84ac0a9d2b58e90ec48e079391da10185065a8b"}
Nov 04 01:20:55.911605 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:55.911560    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-6987-a1d191" podStartSLOduration=1.9115469109999998 podStartE2EDuration="1.911546911s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:20:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:20:55.911098777 +0000 UTC m=+1095.957659568" watchObservedRunningTime="2026-11-04 01:20:55.911546911 +0000 UTC m=+1095.958107700"
Nov 04 01:20:55.937859 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:55.937811    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-wljtg" podStartSLOduration=1.937796845 podStartE2EDuration="1.937796845s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:20:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:20:55.932416601 +0000 UTC m=+1095.978977395" watchObservedRunningTime="2026-11-04 01:20:55.937796845 +0000 UTC m=+1095.984357634"
Nov 04 01:20:56.783413 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:56.783373    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-6987-a1d191"]
Nov 04 01:20:56.814382 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:56.814337    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-6987-a1d191"]
Nov 04 01:20:56.824369 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:56.824339    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-wljtg"
Nov 04 01:20:56.825672 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:56.825631    5068 patch_prober.go:29] interesting pod/e2e-mt-e7742169-openshift-default-6986f46949-wljtg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.108:15021/healthz/ready\": dial tcp 10.132.0.108:15021: connect: connection refused" start-of-body=
Nov 04 01:20:56.826598 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:56.826570    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-wljtg" podUID="2f500082-e92f-4d13-a35b-2b775553b45d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.108:15021/healthz/ready\": dial tcp 10.132.0.108:15021: connect: connection refused"
Nov 04 01:20:56.968634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:56.967699    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-6987-cd29e9"]
Nov 04 01:20:56.969152 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:56.968891    5068 cpu_manager.go:414] "containerMap: removing container" podUID="1e2c7811-daf7-4187-a1ad-2a442d206076" containerName="test-curl-6987-a1d191"
Nov 04 01:20:56.969152 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:56.968913    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1e2c7811-daf7-4187-a1ad-2a442d206076" containerName="test-curl-6987-a1d191"
Nov 04 01:20:56.973640 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:56.972845    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="1e2c7811-daf7-4187-a1ad-2a442d206076" containerName="test-curl-6987-a1d191"
Nov 04 01:20:56.977004 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:56.976979    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6987-cd29e9"
Nov 04 01:20:56.991925 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:56.990024    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-6987-cd29e9"]
Nov 04 01:20:57.164321 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:57.160955    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtfnp\" (UniqueName: \"kubernetes.io/projected/1671a1b1-4b1f-407a-951a-4071d603cd64-kube-api-access-xtfnp\") pod \"test-curl-6987-cd29e9\" (UID: \"1671a1b1-4b1f-407a-951a-4071d603cd64\") " pod="opendatahub/test-curl-6987-cd29e9"
Nov 04 01:20:57.277634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:57.263344    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtfnp\" (UniqueName: \"kubernetes.io/projected/1671a1b1-4b1f-407a-951a-4071d603cd64-kube-api-access-xtfnp\") pod \"test-curl-6987-cd29e9\" (UID: \"1671a1b1-4b1f-407a-951a-4071d603cd64\") " pod="opendatahub/test-curl-6987-cd29e9"
Nov 04 01:20:57.299967 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:57.299931    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtfnp\" (UniqueName: \"kubernetes.io/projected/1671a1b1-4b1f-407a-951a-4071d603cd64-kube-api-access-xtfnp\") pod \"test-curl-6987-cd29e9\" (UID: \"1671a1b1-4b1f-407a-951a-4071d603cd64\") " pod="opendatahub/test-curl-6987-cd29e9"
Nov 04 01:20:57.300204 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:57.300189    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6987-cd29e9"
Nov 04 01:20:57.681264 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:20:57.681177    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1671a1b1_4b1f_407a_951a_4071d603cd64.slice/crio-fc62b4d400ec71eab81b414729e1b78f50bdcef4d1a6e6fa459658294e185988 WatchSource:0}: Error finding container fc62b4d400ec71eab81b414729e1b78f50bdcef4d1a6e6fa459658294e185988: Status 404 returned error can't find the container with id fc62b4d400ec71eab81b414729e1b78f50bdcef4d1a6e6fa459658294e185988
Nov 04 01:20:57.682235 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:57.682208    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-6987-cd29e9"]
Nov 04 01:20:57.824573 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:57.824538    5068 patch_prober.go:29] interesting pod/e2e-mt-e7742169-openshift-default-6986f46949-wljtg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.108:15021/healthz/ready\": dial tcp 10.132.0.108:15021: connect: connection refused" start-of-body=
Nov 04 01:20:57.824750 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:57.824596    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-wljtg" podUID="2f500082-e92f-4d13-a35b-2b775553b45d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.108:15021/healthz/ready\": dial tcp 10.132.0.108:15021: connect: connection refused"
Nov 04 01:20:57.922136 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:57.922100    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-6987-cd29e9" event={"ID":"1671a1b1-4b1f-407a-951a-4071d603cd64","Type":"ContainerStarted","Data":"af68717445c90f33b856cffc3177b4ec67122de888939622f763c533f6b163ff"}
Nov 04 01:20:57.922332 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:57.922158    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-6987-cd29e9" event={"ID":"1671a1b1-4b1f-407a-951a-4071d603cd64","Type":"ContainerStarted","Data":"fc62b4d400ec71eab81b414729e1b78f50bdcef4d1a6e6fa459658294e185988"}
Nov 04 01:20:57.922552 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:57.922526    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-6987-a1d191" podUID="1e2c7811-daf7-4187-a1ad-2a442d206076" containerName="test-curl-6987-a1d191" containerID="cri-o://572f3b288c58f1ecd4e3e3b10f693c0f2230c991cb81f66d2a93f97ffef2ff8a" gracePeriod=2
Nov 04 01:20:57.952695 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:20:57.942815    5068 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-6987-a1d191\" is forbidden: User \"system:node:ip-10-0-132-67.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-67.us-east-2.compute.internal' and this object" podUID="1e2c7811-daf7-4187-a1ad-2a442d206076" pod="opendatahub/test-curl-6987-a1d191"
Nov 04 01:20:57.952695 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:57.947904    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-6987-cd29e9" podStartSLOduration=1.947882763 podStartE2EDuration="1.947882763s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:20:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:20:57.936708442 +0000 UTC m=+1097.983269233" watchObservedRunningTime="2026-11-04 01:20:57.947882763 +0000 UTC m=+1097.994443575"
Nov 04 01:20:58.820637 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:58.817204    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-6987-cd29e9"]
Nov 04 01:20:58.828127 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:58.825278    5068 patch_prober.go:29] interesting pod/e2e-mt-e7742169-openshift-default-6986f46949-wljtg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.108:15021/healthz/ready\": dial tcp 10.132.0.108:15021: connect: connection refused" start-of-body=
Nov 04 01:20:58.828127 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:58.825341    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-wljtg" podUID="2f500082-e92f-4d13-a35b-2b775553b45d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.108:15021/healthz/ready\": dial tcp 10.132.0.108:15021: connect: connection refused"
Nov 04 01:20:58.836343 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:58.836307    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-6987-cd29e9"]
Nov 04 01:20:58.992419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:58.992379    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-6987-5dc600"]
Nov 04 01:20:58.993815 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:58.993789    5068 cpu_manager.go:414] "containerMap: removing container" podUID="1671a1b1-4b1f-407a-951a-4071d603cd64" containerName="test-curl-6987-cd29e9"
Nov 04 01:20:58.994015 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:58.993999    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1671a1b1-4b1f-407a-951a-4071d603cd64" containerName="test-curl-6987-cd29e9"
Nov 04 01:20:58.994303 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:58.994287    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="1671a1b1-4b1f-407a-951a-4071d603cd64" containerName="test-curl-6987-cd29e9"
Nov 04 01:20:58.998791 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:58.998405    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6987-5dc600"
Nov 04 01:20:58.998918 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:58.998898    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-6987-5dc600"]
Nov 04 01:20:59.032854 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:59.032805    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzpqs\" (UniqueName: \"kubernetes.io/projected/3e7d8dbe-f42f-472c-b372-e2bf7a636076-kube-api-access-vzpqs\") pod \"test-curl-6987-5dc600\" (UID: \"3e7d8dbe-f42f-472c-b372-e2bf7a636076\") " pod="opendatahub/test-curl-6987-5dc600"
Nov 04 01:20:59.135405 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:59.135304    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzpqs\" (UniqueName: \"kubernetes.io/projected/3e7d8dbe-f42f-472c-b372-e2bf7a636076-kube-api-access-vzpqs\") pod \"test-curl-6987-5dc600\" (UID: \"3e7d8dbe-f42f-472c-b372-e2bf7a636076\") " pod="opendatahub/test-curl-6987-5dc600"
Nov 04 01:20:59.184942 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:59.181879    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzpqs\" (UniqueName: \"kubernetes.io/projected/3e7d8dbe-f42f-472c-b372-e2bf7a636076-kube-api-access-vzpqs\") pod \"test-curl-6987-5dc600\" (UID: \"3e7d8dbe-f42f-472c-b372-e2bf7a636076\") " pod="opendatahub/test-curl-6987-5dc600"
Nov 04 01:20:59.314013 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:59.313973    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6987-5dc600"
Nov 04 01:20:59.699038 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:59.699006    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-6987-5dc600"]
Nov 04 01:20:59.824913 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:59.824879    5068 patch_prober.go:29] interesting pod/e2e-mt-e7742169-openshift-default-6986f46949-wljtg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.108:15021/healthz/ready\": dial tcp 10.132.0.108:15021: connect: connection refused" start-of-body=
Nov 04 01:20:59.825319 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:59.824943    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-wljtg" podUID="2f500082-e92f-4d13-a35b-2b775553b45d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.108:15021/healthz/ready\": dial tcp 10.132.0.108:15021: connect: connection refused"
Nov 04 01:20:59.978741 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:59.978604    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-6987-5dc600" event={"ID":"3e7d8dbe-f42f-472c-b372-e2bf7a636076","Type":"ContainerStarted","Data":"1e35c836e8335213a875b44dd9d0ae55248042da4d82c69ecc1d04bc4e150061"}
Nov 04 01:20:59.978741 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:59.978681    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-6987-5dc600" event={"ID":"3e7d8dbe-f42f-472c-b372-e2bf7a636076","Type":"ContainerStarted","Data":"ac150d5752dc648098f3751eaebe093eed632ca2a2ab4c543a8e5934618f0c4d"}
Nov 04 01:20:59.978979 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:59.978795    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-6987-cd29e9" podUID="1671a1b1-4b1f-407a-951a-4071d603cd64" containerName="test-curl-6987-cd29e9" containerID="cri-o://af68717445c90f33b856cffc3177b4ec67122de888939622f763c533f6b163ff" gracePeriod=2
Nov 04 01:20:59.996511 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:20:59.996478    5068 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-6987-cd29e9\" is forbidden: User \"system:node:ip-10-0-132-67.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-67.us-east-2.compute.internal' and this object" podUID="1671a1b1-4b1f-407a-951a-4071d603cd64" pod="opendatahub/test-curl-6987-cd29e9"
Nov 04 01:20:59.998365 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:20:59.998315    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-6987-5dc600" podStartSLOduration=1.998300448 podStartE2EDuration="1.998300448s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:20:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:20:59.993971688 +0000 UTC m=+1100.040532497" watchObservedRunningTime="2026-11-04 01:20:59.998300448 +0000 UTC m=+1100.044861238"
Nov 04 01:20:59.998841 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:20:59.998805    5068 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-6987-cd29e9\" is forbidden: User \"system:node:ip-10-0-132-67.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-67.us-east-2.compute.internal' and this object" podUID="1671a1b1-4b1f-407a-951a-4071d603cd64" pod="opendatahub/test-curl-6987-cd29e9"
Nov 04 01:21:00.119585 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:21:00.119548    5068 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-6987-cd29e9\" is forbidden: User \"system:node:ip-10-0-132-67.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-67.us-east-2.compute.internal' and this object" podUID="1671a1b1-4b1f-407a-951a-4071d603cd64" pod="opendatahub/test-curl-6987-cd29e9"
Nov 04 01:21:00.231658 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:00.231551    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6987-a1d191"
Nov 04 01:21:00.234147 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:21:00.234119    5068 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-6987-a1d191\" is forbidden: User \"system:node:ip-10-0-132-67.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-67.us-east-2.compute.internal' and this object" podUID="1e2c7811-daf7-4187-a1ad-2a442d206076" pod="opendatahub/test-curl-6987-a1d191"
Nov 04 01:21:00.288329 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:00.288297    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1e2c7811-daf7-4187-a1ad-2a442d206076-kube-api-access-mtmgb\" (UniqueName: \"kubernetes.io/projected/1e2c7811-daf7-4187-a1ad-2a442d206076-kube-api-access-mtmgb\") pod \"1e2c7811-daf7-4187-a1ad-2a442d206076\" (UID: \"1e2c7811-daf7-4187-a1ad-2a442d206076\") "
Nov 04 01:21:00.290567 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:00.290533    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e2c7811-daf7-4187-a1ad-2a442d206076-kube-api-access-mtmgb" pod "1e2c7811-daf7-4187-a1ad-2a442d206076" (UID: "1e2c7811-daf7-4187-a1ad-2a442d206076"). InnerVolumeSpecName "kube-api-access-mtmgb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:21:00.390629 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:00.390577    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mtmgb\" (UniqueName: \"kubernetes.io/projected/1e2c7811-daf7-4187-a1ad-2a442d206076-kube-api-access-mtmgb\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:21:00.825376 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:00.825340    5068 patch_prober.go:29] interesting pod/e2e-mt-e7742169-openshift-default-6986f46949-wljtg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.108:15021/healthz/ready\": dial tcp 10.132.0.108:15021: connect: connection refused" start-of-body=
Nov 04 01:21:00.825782 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:00.825394    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-wljtg" podUID="2f500082-e92f-4d13-a35b-2b775553b45d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.108:15021/healthz/ready\": dial tcp 10.132.0.108:15021: connect: connection refused"
Nov 04 01:21:00.891768 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:00.891728    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-6987-5dc600"]
Nov 04 01:21:00.917465 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:00.917423    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-6987-5dc600"]
Nov 04 01:21:00.994779 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:00.994736    5068 generic.go:356] "Generic (PLEG): container finished" podID="1e2c7811-daf7-4187-a1ad-2a442d206076" containerID="572f3b288c58f1ecd4e3e3b10f693c0f2230c991cb81f66d2a93f97ffef2ff8a" exitCode=137
Nov 04 01:21:00.994953 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:00.994810    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6987-a1d191"
Nov 04 01:21:00.995496 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:00.995467    5068 scope.go:122] "RemoveContainer" containerID="572f3b288c58f1ecd4e3e3b10f693c0f2230c991cb81f66d2a93f97ffef2ff8a"
Nov 04 01:21:00.998544 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:21:00.998497    5068 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-6987-a1d191\" is forbidden: User \"system:node:ip-10-0-132-67.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-67.us-east-2.compute.internal' and this object" podUID="1e2c7811-daf7-4187-a1ad-2a442d206076" pod="opendatahub/test-curl-6987-a1d191"
Nov 04 01:21:01.008744 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:21:01.008709    5068 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-6987-a1d191\" is forbidden: User \"system:node:ip-10-0-132-67.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-67.us-east-2.compute.internal' and this object" podUID="1e2c7811-daf7-4187-a1ad-2a442d206076" pod="opendatahub/test-curl-6987-a1d191"
Nov 04 01:21:01.010543 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:01.010525    5068 scope.go:122] "RemoveContainer" containerID="572f3b288c58f1ecd4e3e3b10f693c0f2230c991cb81f66d2a93f97ffef2ff8a"
Nov 04 01:21:01.010955 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:21:01.010925    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"572f3b288c58f1ecd4e3e3b10f693c0f2230c991cb81f66d2a93f97ffef2ff8a\": container with ID starting with 572f3b288c58f1ecd4e3e3b10f693c0f2230c991cb81f66d2a93f97ffef2ff8a not found: ID does not exist" containerID="572f3b288c58f1ecd4e3e3b10f693c0f2230c991cb81f66d2a93f97ffef2ff8a"
Nov 04 01:21:01.011067 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:01.010963    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"572f3b288c58f1ecd4e3e3b10f693c0f2230c991cb81f66d2a93f97ffef2ff8a"} err="failed to get container status \"572f3b288c58f1ecd4e3e3b10f693c0f2230c991cb81f66d2a93f97ffef2ff8a\": rpc error: code = NotFound desc = could not find container \"572f3b288c58f1ecd4e3e3b10f693c0f2230c991cb81f66d2a93f97ffef2ff8a\": container with ID starting with 572f3b288c58f1ecd4e3e3b10f693c0f2230c991cb81f66d2a93f97ffef2ff8a not found: ID does not exist"
Nov 04 01:21:01.074284 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:01.074240    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-6987-f1def8"]
Nov 04 01:21:01.079022 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:01.075408    5068 cpu_manager.go:414] "containerMap: removing container" podUID="3e7d8dbe-f42f-472c-b372-e2bf7a636076" containerName="test-curl-6987-5dc600"
Nov 04 01:21:01.079022 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:01.075437    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3e7d8dbe-f42f-472c-b372-e2bf7a636076" containerName="test-curl-6987-5dc600"
Nov 04 01:21:01.079022 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:01.075681    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="3e7d8dbe-f42f-472c-b372-e2bf7a636076" containerName="test-curl-6987-5dc600"
Nov 04 01:21:01.082834 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:01.082461    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6987-f1def8"
Nov 04 01:21:01.082834 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:01.082530    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-6987-f1def8"]
Nov 04 01:21:01.085133 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:21:01.085094    5068 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-6987-a1d191\" is forbidden: User \"system:node:ip-10-0-132-67.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-67.us-east-2.compute.internal' and this object" podUID="1e2c7811-daf7-4187-a1ad-2a442d206076" pod="opendatahub/test-curl-6987-a1d191"
Nov 04 01:21:01.210754 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:01.210719    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmsqn\" (UniqueName: \"kubernetes.io/projected/d7d2df95-72a2-46ff-9f35-32084ce42c5c-kube-api-access-dmsqn\") pod \"test-curl-6987-f1def8\" (UID: \"d7d2df95-72a2-46ff-9f35-32084ce42c5c\") " pod="opendatahub/test-curl-6987-f1def8"
Nov 04 01:21:01.312279 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:01.312247    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmsqn\" (UniqueName: \"kubernetes.io/projected/d7d2df95-72a2-46ff-9f35-32084ce42c5c-kube-api-access-dmsqn\") pod \"test-curl-6987-f1def8\" (UID: \"d7d2df95-72a2-46ff-9f35-32084ce42c5c\") " pod="opendatahub/test-curl-6987-f1def8"
Nov 04 01:21:01.324745 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:01.324717    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmsqn\" (UniqueName: \"kubernetes.io/projected/d7d2df95-72a2-46ff-9f35-32084ce42c5c-kube-api-access-dmsqn\") pod \"test-curl-6987-f1def8\" (UID: \"d7d2df95-72a2-46ff-9f35-32084ce42c5c\") " pod="opendatahub/test-curl-6987-f1def8"
Nov 04 01:21:01.398291 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:01.398209    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6987-f1def8"
Nov 04 01:21:01.718409 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:01.718288    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-6987-f1def8"]
Nov 04 01:21:01.723505 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:21:01.723460    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd7d2df95_72a2_46ff_9f35_32084ce42c5c.slice/crio-4825269973b89498ea632459e638fa9abfa5ff583735d61e4609ed44ea915dfb WatchSource:0}: Error finding container 4825269973b89498ea632459e638fa9abfa5ff583735d61e4609ed44ea915dfb: Status 404 returned error can't find the container with id 4825269973b89498ea632459e638fa9abfa5ff583735d61e4609ed44ea915dfb
Nov 04 01:21:01.825152 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:01.825122    5068 patch_prober.go:29] interesting pod/e2e-mt-e7742169-openshift-default-6986f46949-wljtg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.108:15021/healthz/ready\": dial tcp 10.132.0.108:15021: connect: connection refused" start-of-body=
Nov 04 01:21:01.825304 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:01.825181    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-wljtg" podUID="2f500082-e92f-4d13-a35b-2b775553b45d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.108:15021/healthz/ready\": dial tcp 10.132.0.108:15021: connect: connection refused"
Nov 04 01:21:02.007655 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:02.007571    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-6987-f1def8" event={"ID":"d7d2df95-72a2-46ff-9f35-32084ce42c5c","Type":"ContainerStarted","Data":"a6e3875b9a6bca098d0603fbf128f7b507205c40554dfa1c1210aea4a6ac33f0"}
Nov 04 01:21:02.007655 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:02.007639    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-6987-f1def8" event={"ID":"d7d2df95-72a2-46ff-9f35-32084ce42c5c","Type":"ContainerStarted","Data":"4825269973b89498ea632459e638fa9abfa5ff583735d61e4609ed44ea915dfb"}
Nov 04 01:21:02.008178 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:02.007917    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-6987-5dc600" podUID="3e7d8dbe-f42f-472c-b372-e2bf7a636076" containerName="test-curl-6987-5dc600" containerID="cri-o://1e35c836e8335213a875b44dd9d0ae55248042da4d82c69ecc1d04bc4e150061" gracePeriod=2
Nov 04 01:21:02.025378 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:21:02.025346    5068 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-6987-a1d191\" is forbidden: User \"system:node:ip-10-0-132-67.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-67.us-east-2.compute.internal' and this object" podUID="1e2c7811-daf7-4187-a1ad-2a442d206076" pod="opendatahub/test-curl-6987-a1d191"
Nov 04 01:21:02.026413 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:02.026371    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-6987-f1def8" podStartSLOduration=1.026358779 podStartE2EDuration="1.026358779s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:21:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:21:02.023110359 +0000 UTC m=+1102.069671150" watchObservedRunningTime="2026-11-04 01:21:02.026358779 +0000 UTC m=+1102.072919568"
Nov 04 01:21:02.027681 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:21:02.027653    5068 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-6987-5dc600\" is forbidden: User \"system:node:ip-10-0-132-67.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-67.us-east-2.compute.internal' and this object" podUID="3e7d8dbe-f42f-472c-b372-e2bf7a636076" pod="opendatahub/test-curl-6987-5dc600"
Nov 04 01:21:02.030106 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:21:02.030084    5068 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-6987-a1d191\" is forbidden: User \"system:node:ip-10-0-132-67.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-67.us-east-2.compute.internal' and this object" podUID="1e2c7811-daf7-4187-a1ad-2a442d206076" pod="opendatahub/test-curl-6987-a1d191"
Nov 04 01:21:02.120119 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:02.120089    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1e2c7811-daf7-4187-a1ad-2a442d206076" path="/var/lib/kubelet/pods/1e2c7811-daf7-4187-a1ad-2a442d206076/volumes"
Nov 04 01:21:02.177855 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:02.177823    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6987-cd29e9"
Nov 04 01:21:02.231469 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:02.231438    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1671a1b1-4b1f-407a-951a-4071d603cd64-kube-api-access-xtfnp\" (UniqueName: \"kubernetes.io/projected/1671a1b1-4b1f-407a-951a-4071d603cd64-kube-api-access-xtfnp\") pod \"1671a1b1-4b1f-407a-951a-4071d603cd64\" (UID: \"1671a1b1-4b1f-407a-951a-4071d603cd64\") "
Nov 04 01:21:02.233573 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:02.233518    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1671a1b1-4b1f-407a-951a-4071d603cd64-kube-api-access-xtfnp" pod "1671a1b1-4b1f-407a-951a-4071d603cd64" (UID: "1671a1b1-4b1f-407a-951a-4071d603cd64"). InnerVolumeSpecName "kube-api-access-xtfnp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:21:02.333337 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:02.333302    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xtfnp\" (UniqueName: \"kubernetes.io/projected/1671a1b1-4b1f-407a-951a-4071d603cd64-kube-api-access-xtfnp\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:21:02.830598 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:02.830563    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-wljtg"
Nov 04 01:21:02.830886 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:02.830865    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-wljtg"
Nov 04 01:21:02.831603 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:02.831546    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-wljtg"
Nov 04 01:21:02.836865 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:02.836838    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-6987-f1def8"]
Nov 04 01:21:02.844127 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:02.844100    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-6987-f1def8"]
Nov 04 01:21:03.017591 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:03.017555    5068 generic.go:356] "Generic (PLEG): container finished" podID="1671a1b1-4b1f-407a-951a-4071d603cd64" containerID="af68717445c90f33b856cffc3177b4ec67122de888939622f763c533f6b163ff" exitCode=137
Nov 04 01:21:03.018019 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:03.017596    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6987-cd29e9"
Nov 04 01:21:03.018019 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:03.017657    5068 scope.go:122] "RemoveContainer" containerID="af68717445c90f33b856cffc3177b4ec67122de888939622f763c533f6b163ff"
Nov 04 01:21:03.030227 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:21:03.030198    5068 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-6987-cd29e9\" is forbidden: User \"system:node:ip-10-0-132-67.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-67.us-east-2.compute.internal' and this object" podUID="1671a1b1-4b1f-407a-951a-4071d603cd64" pod="opendatahub/test-curl-6987-cd29e9"
Nov 04 01:21:03.030769 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:03.030750    5068 scope.go:122] "RemoveContainer" containerID="af68717445c90f33b856cffc3177b4ec67122de888939622f763c533f6b163ff"
Nov 04 01:21:03.031019 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:21:03.031002    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af68717445c90f33b856cffc3177b4ec67122de888939622f763c533f6b163ff\": container with ID starting with af68717445c90f33b856cffc3177b4ec67122de888939622f763c533f6b163ff not found: ID does not exist" containerID="af68717445c90f33b856cffc3177b4ec67122de888939622f763c533f6b163ff"
Nov 04 01:21:03.031089 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:03.031025    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af68717445c90f33b856cffc3177b4ec67122de888939622f763c533f6b163ff"} err="failed to get container status \"af68717445c90f33b856cffc3177b4ec67122de888939622f763c533f6b163ff\": rpc error: code = NotFound desc = could not find container \"af68717445c90f33b856cffc3177b4ec67122de888939622f763c533f6b163ff\": container with ID starting with af68717445c90f33b856cffc3177b4ec67122de888939622f763c533f6b163ff not found: ID does not exist"
Nov 04 01:21:04.033037 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:04.033009    5068 generic.go:356] "Generic (PLEG): container finished" podID="3e7d8dbe-f42f-472c-b372-e2bf7a636076" containerID="1e35c836e8335213a875b44dd9d0ae55248042da4d82c69ecc1d04bc4e150061" exitCode=137
Nov 04 01:21:04.033359 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:04.033261    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-6987-f1def8" podUID="d7d2df95-72a2-46ff-9f35-32084ce42c5c" containerName="test-curl-6987-f1def8" containerID="cri-o://a6e3875b9a6bca098d0603fbf128f7b507205c40554dfa1c1210aea4a6ac33f0" gracePeriod=2
Nov 04 01:21:04.035792 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:21:04.035754    5068 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-6987-f1def8\" is forbidden: User \"system:node:ip-10-0-132-67.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-67.us-east-2.compute.internal' and this object" podUID="d7d2df95-72a2-46ff-9f35-32084ce42c5c" pod="opendatahub/test-curl-6987-f1def8"
Nov 04 01:21:04.038049 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:21:04.038022    5068 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-6987-cd29e9\" is forbidden: User \"system:node:ip-10-0-132-67.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-67.us-east-2.compute.internal' and this object" podUID="1671a1b1-4b1f-407a-951a-4071d603cd64" pod="opendatahub/test-curl-6987-cd29e9"
Nov 04 01:21:04.126232 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:04.125457    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1671a1b1-4b1f-407a-951a-4071d603cd64" path="/var/lib/kubelet/pods/1671a1b1-4b1f-407a-951a-4071d603cd64/volumes"
Nov 04 01:21:04.199142 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:04.199109    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6987-5dc600"
Nov 04 01:21:04.273693 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:04.273562    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3e7d8dbe-f42f-472c-b372-e2bf7a636076-kube-api-access-vzpqs\" (UniqueName: \"kubernetes.io/projected/3e7d8dbe-f42f-472c-b372-e2bf7a636076-kube-api-access-vzpqs\") pod \"3e7d8dbe-f42f-472c-b372-e2bf7a636076\" (UID: \"3e7d8dbe-f42f-472c-b372-e2bf7a636076\") "
Nov 04 01:21:04.276368 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:04.276317    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e7d8dbe-f42f-472c-b372-e2bf7a636076-kube-api-access-vzpqs" pod "3e7d8dbe-f42f-472c-b372-e2bf7a636076" (UID: "3e7d8dbe-f42f-472c-b372-e2bf7a636076"). InnerVolumeSpecName "kube-api-access-vzpqs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:21:04.375988 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:04.375950    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vzpqs\" (UniqueName: \"kubernetes.io/projected/3e7d8dbe-f42f-472c-b372-e2bf7a636076-kube-api-access-vzpqs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:21:05.044819 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:05.044779    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6987-5dc600"
Nov 04 01:21:05.045291 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:05.044838    5068 scope.go:122] "RemoveContainer" containerID="1e35c836e8335213a875b44dd9d0ae55248042da4d82c69ecc1d04bc4e150061"
Nov 04 01:21:05.057950 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:21:05.057913    5068 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-6987-5dc600\" is forbidden: User \"system:node:ip-10-0-132-67.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-67.us-east-2.compute.internal' and this object" podUID="3e7d8dbe-f42f-472c-b372-e2bf7a636076" pod="opendatahub/test-curl-6987-5dc600"
Nov 04 01:21:06.117589 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:06.117556    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3e7d8dbe-f42f-472c-b372-e2bf7a636076" path="/var/lib/kubelet/pods/3e7d8dbe-f42f-472c-b372-e2bf7a636076/volumes"
Nov 04 01:21:06.263316 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:06.263289    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6987-f1def8"
Nov 04 01:21:06.327178 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:06.327086    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d7d2df95-72a2-46ff-9f35-32084ce42c5c-kube-api-access-dmsqn\" (UniqueName: \"kubernetes.io/projected/d7d2df95-72a2-46ff-9f35-32084ce42c5c-kube-api-access-dmsqn\") pod \"d7d2df95-72a2-46ff-9f35-32084ce42c5c\" (UID: \"d7d2df95-72a2-46ff-9f35-32084ce42c5c\") "
Nov 04 01:21:06.329704 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:06.329673    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7d2df95-72a2-46ff-9f35-32084ce42c5c-kube-api-access-dmsqn" pod "d7d2df95-72a2-46ff-9f35-32084ce42c5c" (UID: "d7d2df95-72a2-46ff-9f35-32084ce42c5c"). InnerVolumeSpecName "kube-api-access-dmsqn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:21:06.437632 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:06.430921    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dmsqn\" (UniqueName: \"kubernetes.io/projected/d7d2df95-72a2-46ff-9f35-32084ce42c5c-kube-api-access-dmsqn\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:21:07.067090 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:07.067049    5068 generic.go:356] "Generic (PLEG): container finished" podID="d7d2df95-72a2-46ff-9f35-32084ce42c5c" containerID="a6e3875b9a6bca098d0603fbf128f7b507205c40554dfa1c1210aea4a6ac33f0" exitCode=137
Nov 04 01:21:07.067333 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:07.067120    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6987-f1def8"
Nov 04 01:21:07.067333 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:07.067202    5068 scope.go:122] "RemoveContainer" containerID="a6e3875b9a6bca098d0603fbf128f7b507205c40554dfa1c1210aea4a6ac33f0"
Nov 04 01:21:07.086677 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:21:07.086426    5068 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-6987-f1def8\" is forbidden: User \"system:node:ip-10-0-132-67.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-67.us-east-2.compute.internal' and this object" podUID="d7d2df95-72a2-46ff-9f35-32084ce42c5c" pod="opendatahub/test-curl-6987-f1def8"
Nov 04 01:21:07.105239 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:07.105217    5068 scope.go:122] "RemoveContainer" containerID="a6e3875b9a6bca098d0603fbf128f7b507205c40554dfa1c1210aea4a6ac33f0"
Nov 04 01:21:07.105664 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:21:07.105602    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6e3875b9a6bca098d0603fbf128f7b507205c40554dfa1c1210aea4a6ac33f0\": container with ID starting with a6e3875b9a6bca098d0603fbf128f7b507205c40554dfa1c1210aea4a6ac33f0 not found: ID does not exist" containerID="a6e3875b9a6bca098d0603fbf128f7b507205c40554dfa1c1210aea4a6ac33f0"
Nov 04 01:21:07.105721 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:07.105677    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6e3875b9a6bca098d0603fbf128f7b507205c40554dfa1c1210aea4a6ac33f0"} err="failed to get container status \"a6e3875b9a6bca098d0603fbf128f7b507205c40554dfa1c1210aea4a6ac33f0\": rpc error: code = NotFound desc = could not find container \"a6e3875b9a6bca098d0603fbf128f7b507205c40554dfa1c1210aea4a6ac33f0\": container with ID starting with a6e3875b9a6bca098d0603fbf128f7b507205c40554dfa1c1210aea4a6ac33f0 not found: ID does not exist"
Nov 04 01:21:08.114369 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:08.114333    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d7d2df95-72a2-46ff-9f35-32084ce42c5c" path="/var/lib/kubelet/pods/d7d2df95-72a2-46ff-9f35-32084ce42c5c/volumes"
Nov 04 01:21:13.463358 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:21:13.463310    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:21:19.818449 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:19.818258    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0741ef-f87fdfbbb-cqfcq"]
Nov 04 01:21:19.821440 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:19.821410    5068 cpu_manager.go:414] "containerMap: removing container" podUID="d7d2df95-72a2-46ff-9f35-32084ce42c5c" containerName="test-curl-6987-f1def8"
Nov 04 01:21:19.821440 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:19.821436    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d7d2df95-72a2-46ff-9f35-32084ce42c5c" containerName="test-curl-6987-f1def8"
Nov 04 01:21:19.821676 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:19.821601    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="d7d2df95-72a2-46ff-9f35-32084ce42c5c" containerName="test-curl-6987-f1def8"
Nov 04 01:21:19.826206 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:19.826181    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0741ef-f87fdfbbb-cqfcq"
Nov 04 01:21:19.832182 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:19.832157    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0741ef-f87fdfbbb-cqfcq"]
Nov 04 01:21:19.842627 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:19.839459    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w1-0741ef-serving-cert\""
Nov 04 01:21:19.959629 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:19.959295    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f6317cf-be73-46fc-bf2d-b694dc34e258-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-0741ef-f87fdfbbb-cqfcq\" (UID: \"4f6317cf-be73-46fc-bf2d-b694dc34e258\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0741ef-f87fdfbbb-cqfcq"
Nov 04 01:21:19.959629 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:19.959400    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqmxr\" (UniqueName: \"kubernetes.io/projected/4f6317cf-be73-46fc-bf2d-b694dc34e258-kube-api-access-vqmxr\") pod \"maas-api-e2e-shared-a-w1-0741ef-f87fdfbbb-cqfcq\" (UID: \"4f6317cf-be73-46fc-bf2d-b694dc34e258\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0741ef-f87fdfbbb-cqfcq"
Nov 04 01:21:19.959629 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:19.959551    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f6317cf-be73-46fc-bf2d-b694dc34e258-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-0741ef-f87fdfbbb-cqfcq\" (UID: \"4f6317cf-be73-46fc-bf2d-b694dc34e258\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0741ef-f87fdfbbb-cqfcq"
Nov 04 01:21:19.960029 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:19.959725    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f6317cf-be73-46fc-bf2d-b694dc34e258-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-0741ef-f87fdfbbb-cqfcq\" (UID: \"4f6317cf-be73-46fc-bf2d-b694dc34e258\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0741ef-f87fdfbbb-cqfcq"
Nov 04 01:21:20.061363 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:20.061323    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f6317cf-be73-46fc-bf2d-b694dc34e258-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-0741ef-f87fdfbbb-cqfcq\" (UID: \"4f6317cf-be73-46fc-bf2d-b694dc34e258\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0741ef-f87fdfbbb-cqfcq"
Nov 04 01:21:20.061564 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:20.061482    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f6317cf-be73-46fc-bf2d-b694dc34e258-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-0741ef-f87fdfbbb-cqfcq\" (UID: \"4f6317cf-be73-46fc-bf2d-b694dc34e258\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0741ef-f87fdfbbb-cqfcq"
Nov 04 01:21:20.061687 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:20.061670    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f6317cf-be73-46fc-bf2d-b694dc34e258-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-0741ef-f87fdfbbb-cqfcq\" (UID: \"4f6317cf-be73-46fc-bf2d-b694dc34e258\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0741ef-f87fdfbbb-cqfcq"
Nov 04 01:21:20.061782 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:20.061757    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqmxr\" (UniqueName: \"kubernetes.io/projected/4f6317cf-be73-46fc-bf2d-b694dc34e258-kube-api-access-vqmxr\") pod \"maas-api-e2e-shared-a-w1-0741ef-f87fdfbbb-cqfcq\" (UID: \"4f6317cf-be73-46fc-bf2d-b694dc34e258\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0741ef-f87fdfbbb-cqfcq"
Nov 04 01:21:20.062465 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:20.062441    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f6317cf-be73-46fc-bf2d-b694dc34e258-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-0741ef-f87fdfbbb-cqfcq\" (UID: \"4f6317cf-be73-46fc-bf2d-b694dc34e258\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0741ef-f87fdfbbb-cqfcq"
Nov 04 01:21:20.065032 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:20.065005    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f6317cf-be73-46fc-bf2d-b694dc34e258-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-0741ef-f87fdfbbb-cqfcq\" (UID: \"4f6317cf-be73-46fc-bf2d-b694dc34e258\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0741ef-f87fdfbbb-cqfcq"
Nov 04 01:21:20.065378 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:20.065355    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f6317cf-be73-46fc-bf2d-b694dc34e258-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-0741ef-f87fdfbbb-cqfcq\" (UID: \"4f6317cf-be73-46fc-bf2d-b694dc34e258\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0741ef-f87fdfbbb-cqfcq"
Nov 04 01:21:20.074502 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:20.074441    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqmxr\" (UniqueName: \"kubernetes.io/projected/4f6317cf-be73-46fc-bf2d-b694dc34e258-kube-api-access-vqmxr\") pod \"maas-api-e2e-shared-a-w1-0741ef-f87fdfbbb-cqfcq\" (UID: \"4f6317cf-be73-46fc-bf2d-b694dc34e258\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0741ef-f87fdfbbb-cqfcq"
Nov 04 01:21:20.155515 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:20.155469    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0741ef-f87fdfbbb-cqfcq"
Nov 04 01:21:20.508369 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:20.508323    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0741ef-f87fdfbbb-cqfcq"]
Nov 04 01:21:21.244498 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:21.244458    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0741ef-f87fdfbbb-cqfcq" event={"ID":"4f6317cf-be73-46fc-bf2d-b694dc34e258","Type":"ContainerStarted","Data":"89e72ef21de86aa4fdebedc38ce6f3020505fc12f1b6614d0478f86fa31b374d"}
Nov 04 01:21:21.244983 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:21.244511    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0741ef-f87fdfbbb-cqfcq" event={"ID":"4f6317cf-be73-46fc-bf2d-b694dc34e258","Type":"ContainerStarted","Data":"5f46307d415725da580d186e45a25830bba5f3cecfc61f5386502f7158cdc56e"}
Nov 04 01:21:21.244983 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:21.244728    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0741ef-f87fdfbbb-cqfcq"
Nov 04 01:21:21.271255 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:21.271025    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0741ef-f87fdfbbb-cqfcq" podStartSLOduration=1.763683214 podStartE2EDuration="2.271009891s" totalImagesPullingTime="507.326677ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:21:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:21:21.264231328 +0000 UTC m=+1121.310792130" watchObservedRunningTime="2026-11-04 01:21:21.271009891 +0000 UTC m=+1121.317570681"
Nov 04 01:21:24.470236 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:24.470199    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:21:27.275914 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:27.275873    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0741ef-f87fdfbbb-cqfcq"
Nov 04 01:21:42.216446 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:42.216408    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-5ce9e8-5cd7d58c4b-k77tv"]
Nov 04 01:21:42.222390 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:42.222359    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-5ce9e8-7bf85dc8f5-l96mq"]
Nov 04 01:21:42.222541 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:42.222482    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-5ce9e8-5cd7d58c4b-k77tv"
Nov 04 01:21:42.228518 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:42.228487    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-5ce9e8-7bf85dc8f5-l96mq"
Nov 04 01:21:42.230947 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:42.230920    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w1-5ce9e8-dockercfg-m8wl5\""
Nov 04 01:21:42.230947 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:42.230917    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w1-5ce9e8\""
Nov 04 01:21:42.235441 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:42.235410    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-5ce9e8-5cd7d58c4b-k77tv"]
Nov 04 01:21:42.256629 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:42.254719    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-5ce9e8-7bf85dc8f5-l96mq"]
Nov 04 01:21:42.365151 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:42.347460    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7v7t\" (UniqueName: \"kubernetes.io/projected/bda9909c-2480-4f83-9cef-3d14f19568ac-kube-api-access-v7v7t\") pod \"payload-pre-processing-e2e-shared-b-w1-5ce9e8-7bf85dc8f5-l96mq\" (UID: \"bda9909c-2480-4f83-9cef-3d14f19568ac\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-5ce9e8-7bf85dc8f5-l96mq"
Nov 04 01:21:42.365498 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:42.365465    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bda9909c-2480-4f83-9cef-3d14f19568ac-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w1-5ce9e8-7bf85dc8f5-l96mq\" (UID: \"bda9909c-2480-4f83-9cef-3d14f19568ac\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-5ce9e8-7bf85dc8f5-l96mq"
Nov 04 01:21:42.379868 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:42.379828    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khlqs\" (UniqueName: \"kubernetes.io/projected/88233bfe-a2e5-4dc2-b4ab-d6f9772f7219-kube-api-access-khlqs\") pod \"payload-processing-e2e-shared-b-w1-5ce9e8-5cd7d58c4b-k77tv\" (UID: \"88233bfe-a2e5-4dc2-b4ab-d6f9772f7219\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-5ce9e8-5cd7d58c4b-k77tv"
Nov 04 01:21:42.380340 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:42.380314    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88233bfe-a2e5-4dc2-b4ab-d6f9772f7219-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-5ce9e8-5cd7d58c4b-k77tv\" (UID: \"88233bfe-a2e5-4dc2-b4ab-d6f9772f7219\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-5ce9e8-5cd7d58c4b-k77tv"
Nov 04 01:21:42.516689 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:42.490855    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bda9909c-2480-4f83-9cef-3d14f19568ac-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w1-5ce9e8-7bf85dc8f5-l96mq\" (UID: \"bda9909c-2480-4f83-9cef-3d14f19568ac\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-5ce9e8-7bf85dc8f5-l96mq"
Nov 04 01:21:42.517111 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:42.517081    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khlqs\" (UniqueName: \"kubernetes.io/projected/88233bfe-a2e5-4dc2-b4ab-d6f9772f7219-kube-api-access-khlqs\") pod \"payload-processing-e2e-shared-b-w1-5ce9e8-5cd7d58c4b-k77tv\" (UID: \"88233bfe-a2e5-4dc2-b4ab-d6f9772f7219\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-5ce9e8-5cd7d58c4b-k77tv"
Nov 04 01:21:42.517403 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:42.517387    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88233bfe-a2e5-4dc2-b4ab-d6f9772f7219-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-5ce9e8-5cd7d58c4b-k77tv\" (UID: \"88233bfe-a2e5-4dc2-b4ab-d6f9772f7219\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-5ce9e8-5cd7d58c4b-k77tv"
Nov 04 01:21:42.542635 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:42.517788    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7v7t\" (UniqueName: \"kubernetes.io/projected/bda9909c-2480-4f83-9cef-3d14f19568ac-kube-api-access-v7v7t\") pod \"payload-pre-processing-e2e-shared-b-w1-5ce9e8-7bf85dc8f5-l96mq\" (UID: \"bda9909c-2480-4f83-9cef-3d14f19568ac\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-5ce9e8-7bf85dc8f5-l96mq"
Nov 04 01:21:42.543068 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:42.516127    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bda9909c-2480-4f83-9cef-3d14f19568ac-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w1-5ce9e8-7bf85dc8f5-l96mq\" (UID: \"bda9909c-2480-4f83-9cef-3d14f19568ac\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-5ce9e8-7bf85dc8f5-l96mq"
Nov 04 01:21:42.543261 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:42.516641    5068 scope.go:122] "RemoveContainer" containerID="c88d13560c5d07cf79c5137b00b20cc6f33c9e24ee3098678fa161e1aca0f68d"
Nov 04 01:21:42.543457 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:42.525169    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88233bfe-a2e5-4dc2-b4ab-d6f9772f7219-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-5ce9e8-5cd7d58c4b-k77tv\" (UID: \"88233bfe-a2e5-4dc2-b4ab-d6f9772f7219\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-5ce9e8-5cd7d58c4b-k77tv"
Nov 04 01:21:42.543558 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:42.533200    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-khlqs\" (UniqueName: \"kubernetes.io/projected/88233bfe-a2e5-4dc2-b4ab-d6f9772f7219-kube-api-access-khlqs\") pod \"payload-processing-e2e-shared-b-w1-5ce9e8-5cd7d58c4b-k77tv\" (UID: \"88233bfe-a2e5-4dc2-b4ab-d6f9772f7219\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-5ce9e8-5cd7d58c4b-k77tv"
Nov 04 01:21:42.553047 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:42.551567    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7v7t\" (UniqueName: \"kubernetes.io/projected/bda9909c-2480-4f83-9cef-3d14f19568ac-kube-api-access-v7v7t\") pod \"payload-pre-processing-e2e-shared-b-w1-5ce9e8-7bf85dc8f5-l96mq\" (UID: \"bda9909c-2480-4f83-9cef-3d14f19568ac\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-5ce9e8-7bf85dc8f5-l96mq"
Nov 04 01:21:42.577025 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:42.576992    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-5ce9e8-5cd7d58c4b-k77tv"
Nov 04 01:21:42.594690 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:42.594124    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-5ce9e8-7bf85dc8f5-l96mq"
Nov 04 01:21:43.157162 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:43.157076    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-5ce9e8-5cd7d58c4b-k77tv"]
Nov 04 01:21:43.162830 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:21:43.162787    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88233bfe_a2e5_4dc2_b4ab_d6f9772f7219.slice/crio-edf860fa0900e2a898dd51bfb2c03d3a5dbf5b8e2be60b2495b702e68007235d WatchSource:0}: Error finding container edf860fa0900e2a898dd51bfb2c03d3a5dbf5b8e2be60b2495b702e68007235d: Status 404 returned error can't find the container with id edf860fa0900e2a898dd51bfb2c03d3a5dbf5b8e2be60b2495b702e68007235d
Nov 04 01:21:43.228277 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:43.228243    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-5ce9e8-7bf85dc8f5-l96mq"]
Nov 04 01:21:43.456942 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:21:43.456857    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:21:43.658198 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:43.658161    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-5ce9e8-5cd7d58c4b-k77tv" event={"ID":"88233bfe-a2e5-4dc2-b4ab-d6f9772f7219","Type":"ContainerStarted","Data":"21788304f95e42ce848e84d5514d3a7a0ea02d135c1672e694d726baec748e43"}
Nov 04 01:21:43.658495 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:43.658469    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-5ce9e8-5cd7d58c4b-k77tv" event={"ID":"88233bfe-a2e5-4dc2-b4ab-d6f9772f7219","Type":"ContainerStarted","Data":"edf860fa0900e2a898dd51bfb2c03d3a5dbf5b8e2be60b2495b702e68007235d"}
Nov 04 01:21:43.659214 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:43.659144    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-5ce9e8-5cd7d58c4b-k77tv"
Nov 04 01:21:43.660933 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:43.660904    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-5ce9e8-7bf85dc8f5-l96mq" event={"ID":"bda9909c-2480-4f83-9cef-3d14f19568ac","Type":"ContainerStarted","Data":"6264710adb7ce1fa23bdd9c75f6eb6d415d26a5be4862cf4504eaa9bbd4056e0"}
Nov 04 01:21:43.661054 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:43.660964    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-5ce9e8-7bf85dc8f5-l96mq" event={"ID":"bda9909c-2480-4f83-9cef-3d14f19568ac","Type":"ContainerStarted","Data":"013eadc4e241c9027fbdce878623b813e4703745ba06f845a3ce0d3179152203"}
Nov 04 01:21:43.661054 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:43.660995    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-5ce9e8-7bf85dc8f5-l96mq"
Nov 04 01:21:43.686708 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:43.686655    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-5ce9e8-5cd7d58c4b-k77tv" podStartSLOduration=1.6866375260000002 podStartE2EDuration="1.686637526s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:21:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:21:43.679995406 +0000 UTC m=+1143.726556196" watchObservedRunningTime="2026-11-04 01:21:43.686637526 +0000 UTC m=+1143.733198314"
Nov 04 01:21:43.703477 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:43.703422    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-5ce9e8-7bf85dc8f5-l96mq" podStartSLOduration=1.7034057009999999 podStartE2EDuration="1.703405701s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:21:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:21:43.700031114 +0000 UTC m=+1143.746591937" watchObservedRunningTime="2026-11-04 01:21:43.703405701 +0000 UTC m=+1143.749966490"
Nov 04 01:21:54.674818 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:54.674788    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-5ce9e8-7bf85dc8f5-l96mq"
Nov 04 01:21:54.675385 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:54.674871    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-5ce9e8-5cd7d58c4b-k77tv"
Nov 04 01:21:56.609953 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:56.609916    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-27bc4f8b-7ccc8d95f6-cm86k"]
Nov 04 01:21:56.635078 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:56.635039    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-27bc4f8b-7ccc8d95f6-cm86k"
Nov 04 01:21:56.643774 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:56.643714    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-27bc4f8b-serving-cert\""
Nov 04 01:21:56.655463 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:56.655430    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-27bc4f8b-7ccc8d95f6-cm86k"]
Nov 04 01:21:56.664244 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:56.664188    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1799b838-9323-4b50-b630-13f956913a62-maas-api-tls\") pod \"maas-api-e2e-mt-27bc4f8b-7ccc8d95f6-cm86k\" (UID: \"1799b838-9323-4b50-b630-13f956913a62\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-27bc4f8b-7ccc8d95f6-cm86k"
Nov 04 01:21:56.664434 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:56.664416    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1799b838-9323-4b50-b630-13f956913a62-openshift-service-ca\") pod \"maas-api-e2e-mt-27bc4f8b-7ccc8d95f6-cm86k\" (UID: \"1799b838-9323-4b50-b630-13f956913a62\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-27bc4f8b-7ccc8d95f6-cm86k"
Nov 04 01:21:56.664533 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:56.664471    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4bzk\" (UniqueName: \"kubernetes.io/projected/1799b838-9323-4b50-b630-13f956913a62-kube-api-access-f4bzk\") pod \"maas-api-e2e-mt-27bc4f8b-7ccc8d95f6-cm86k\" (UID: \"1799b838-9323-4b50-b630-13f956913a62\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-27bc4f8b-7ccc8d95f6-cm86k"
Nov 04 01:21:56.664619 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:56.664571    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1799b838-9323-4b50-b630-13f956913a62-metrics-certs\") pod \"maas-api-e2e-mt-27bc4f8b-7ccc8d95f6-cm86k\" (UID: \"1799b838-9323-4b50-b630-13f956913a62\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-27bc4f8b-7ccc8d95f6-cm86k"
Nov 04 01:21:56.776504 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:56.769348    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1799b838-9323-4b50-b630-13f956913a62-openshift-service-ca\") pod \"maas-api-e2e-mt-27bc4f8b-7ccc8d95f6-cm86k\" (UID: \"1799b838-9323-4b50-b630-13f956913a62\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-27bc4f8b-7ccc8d95f6-cm86k"
Nov 04 01:21:56.776504 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:56.769417    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4bzk\" (UniqueName: \"kubernetes.io/projected/1799b838-9323-4b50-b630-13f956913a62-kube-api-access-f4bzk\") pod \"maas-api-e2e-mt-27bc4f8b-7ccc8d95f6-cm86k\" (UID: \"1799b838-9323-4b50-b630-13f956913a62\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-27bc4f8b-7ccc8d95f6-cm86k"
Nov 04 01:21:56.776504 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:56.769450    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1799b838-9323-4b50-b630-13f956913a62-metrics-certs\") pod \"maas-api-e2e-mt-27bc4f8b-7ccc8d95f6-cm86k\" (UID: \"1799b838-9323-4b50-b630-13f956913a62\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-27bc4f8b-7ccc8d95f6-cm86k"
Nov 04 01:21:56.776504 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:56.769729    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1799b838-9323-4b50-b630-13f956913a62-maas-api-tls\") pod \"maas-api-e2e-mt-27bc4f8b-7ccc8d95f6-cm86k\" (UID: \"1799b838-9323-4b50-b630-13f956913a62\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-27bc4f8b-7ccc8d95f6-cm86k"
Nov 04 01:21:56.792646 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:56.790184    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1799b838-9323-4b50-b630-13f956913a62-openshift-service-ca\") pod \"maas-api-e2e-mt-27bc4f8b-7ccc8d95f6-cm86k\" (UID: \"1799b838-9323-4b50-b630-13f956913a62\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-27bc4f8b-7ccc8d95f6-cm86k"
Nov 04 01:21:56.797639 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:56.796345    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1799b838-9323-4b50-b630-13f956913a62-metrics-certs\") pod \"maas-api-e2e-mt-27bc4f8b-7ccc8d95f6-cm86k\" (UID: \"1799b838-9323-4b50-b630-13f956913a62\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-27bc4f8b-7ccc8d95f6-cm86k"
Nov 04 01:21:56.832603 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:56.832513    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1799b838-9323-4b50-b630-13f956913a62-maas-api-tls\") pod \"maas-api-e2e-mt-27bc4f8b-7ccc8d95f6-cm86k\" (UID: \"1799b838-9323-4b50-b630-13f956913a62\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-27bc4f8b-7ccc8d95f6-cm86k"
Nov 04 01:21:56.837987 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:56.837953    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4bzk\" (UniqueName: \"kubernetes.io/projected/1799b838-9323-4b50-b630-13f956913a62-kube-api-access-f4bzk\") pod \"maas-api-e2e-mt-27bc4f8b-7ccc8d95f6-cm86k\" (UID: \"1799b838-9323-4b50-b630-13f956913a62\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-27bc4f8b-7ccc8d95f6-cm86k"
Nov 04 01:21:56.962061 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:56.961967    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-27bc4f8b-7ccc8d95f6-cm86k"
Nov 04 01:21:57.259673 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:57.255855    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-27bc4f8b-7ccc8d95f6-cm86k"]
Nov 04 01:21:57.261432 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:21:57.261399    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1799b838_9323_4b50_b630_13f956913a62.slice/crio-20a73b61601e7113dc69fd3283eafcbfa30de6e1635a3914450cb25a8822e6ed WatchSource:0}: Error finding container 20a73b61601e7113dc69fd3283eafcbfa30de6e1635a3914450cb25a8822e6ed: Status 404 returned error can't find the container with id 20a73b61601e7113dc69fd3283eafcbfa30de6e1635a3914450cb25a8822e6ed
Nov 04 01:21:57.881991 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:57.881900    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-27bc4f8b-7ccc8d95f6-cm86k" event={"ID":"1799b838-9323-4b50-b630-13f956913a62","Type":"ContainerStarted","Data":"20a73b61601e7113dc69fd3283eafcbfa30de6e1635a3914450cb25a8822e6ed"}
Nov 04 01:21:58.893756 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:58.893724    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-27bc4f8b-7ccc8d95f6-cm86k" event={"ID":"1799b838-9323-4b50-b630-13f956913a62","Type":"ContainerStarted","Data":"f68b366e31940e058fb718a1124a6667f3fb93424461b0300b137d8ba7490af1"}
Nov 04 01:21:58.894146 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:58.893798    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-27bc4f8b-7ccc8d95f6-cm86k"
Nov 04 01:21:58.918036 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:21:58.917983    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-27bc4f8b-7ccc8d95f6-cm86k" podStartSLOduration=2.42609152 podStartE2EDuration="2.917968864s" totalImagesPullingTime="491.877344ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:21:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:21:58.913677034 +0000 UTC m=+1158.960237826" watchObservedRunningTime="2026-11-04 01:21:58.917968864 +0000 UTC m=+1158.964529653"
Nov 04 01:22:04.980403 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:22:04.980361    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-27bc4f8b-7ccc8d95f6-cm86k"
Nov 04 01:22:13.453793 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:22:13.453740    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:22:37.295853 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:22:37.295810    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:22:43.475549 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:22:43.475467    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:22:44.291705 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:22:44.291673    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tr8b2_21d1d23b-7b6e-4ab5-9593-cf7784fffea2/console-operator/2.log"
Nov 04 01:22:44.387310 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:22:44.387282    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tr8b2_21d1d23b-7b6e-4ab5-9593-cf7784fffea2/console-operator/2.log"
Nov 04 01:22:47.715112 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:22:47.715075    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-6e609c-8496b8f648-d5c9r"]
Nov 04 01:22:47.721117 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:22:47.721095    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-6e609c-8496b8f648-d5c9r"
Nov 04 01:22:47.728667 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:22:47.728635    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-6e609c-8496b8f648-d5c9r"]
Nov 04 01:22:47.729472 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:22:47.729447    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-6e609c\""
Nov 04 01:22:47.732656 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:22:47.732628    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pdv8\" (UniqueName: \"kubernetes.io/projected/27872680-e51b-4785-bd2e-6912cb0fe64f-kube-api-access-4pdv8\") pod \"payload-processing-e2e-api-a-6e609c-8496b8f648-d5c9r\" (UID: \"27872680-e51b-4785-bd2e-6912cb0fe64f\") " pod="openshift-ingress/payload-processing-e2e-api-a-6e609c-8496b8f648-d5c9r"
Nov 04 01:22:47.732790 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:22:47.732706    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27872680-e51b-4785-bd2e-6912cb0fe64f-plugins-config-volume\") pod \"payload-processing-e2e-api-a-6e609c-8496b8f648-d5c9r\" (UID: \"27872680-e51b-4785-bd2e-6912cb0fe64f\") " pod="openshift-ingress/payload-processing-e2e-api-a-6e609c-8496b8f648-d5c9r"
Nov 04 01:22:47.751687 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:22:47.751651    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-6e609c-58d54df979-k4wrw"]
Nov 04 01:22:47.759247 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:22:47.759219    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6e609c-58d54df979-k4wrw"
Nov 04 01:22:47.769072 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:22:47.769048    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-6e609c-dockercfg-2n9kd\""
Nov 04 01:22:47.772400 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:22:47.772374    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-6e609c-58d54df979-k4wrw"]
Nov 04 01:22:47.834464 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:22:47.834430    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c3863ed4-3c10-453c-b11d-c66848ed2900-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-6e609c-58d54df979-k4wrw\" (UID: \"c3863ed4-3c10-453c-b11d-c66848ed2900\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-6e609c-58d54df979-k4wrw"
Nov 04 01:22:47.834766 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:22:47.834569    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pdv8\" (UniqueName: \"kubernetes.io/projected/27872680-e51b-4785-bd2e-6912cb0fe64f-kube-api-access-4pdv8\") pod \"payload-processing-e2e-api-a-6e609c-8496b8f648-d5c9r\" (UID: \"27872680-e51b-4785-bd2e-6912cb0fe64f\") " pod="openshift-ingress/payload-processing-e2e-api-a-6e609c-8496b8f648-d5c9r"
Nov 04 01:22:47.834766 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:22:47.834643    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27872680-e51b-4785-bd2e-6912cb0fe64f-plugins-config-volume\") pod \"payload-processing-e2e-api-a-6e609c-8496b8f648-d5c9r\" (UID: \"27872680-e51b-4785-bd2e-6912cb0fe64f\") " pod="openshift-ingress/payload-processing-e2e-api-a-6e609c-8496b8f648-d5c9r"
Nov 04 01:22:47.834766 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:22:47.834670    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgmjt\" (UniqueName: \"kubernetes.io/projected/c3863ed4-3c10-453c-b11d-c66848ed2900-kube-api-access-lgmjt\") pod \"payload-pre-processing-e2e-api-a-6e609c-58d54df979-k4wrw\" (UID: \"c3863ed4-3c10-453c-b11d-c66848ed2900\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-6e609c-58d54df979-k4wrw"
Nov 04 01:22:47.835577 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:22:47.835556    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27872680-e51b-4785-bd2e-6912cb0fe64f-plugins-config-volume\") pod \"payload-processing-e2e-api-a-6e609c-8496b8f648-d5c9r\" (UID: \"27872680-e51b-4785-bd2e-6912cb0fe64f\") " pod="openshift-ingress/payload-processing-e2e-api-a-6e609c-8496b8f648-d5c9r"
Nov 04 01:22:47.847365 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:22:47.847340    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pdv8\" (UniqueName: \"kubernetes.io/projected/27872680-e51b-4785-bd2e-6912cb0fe64f-kube-api-access-4pdv8\") pod \"payload-processing-e2e-api-a-6e609c-8496b8f648-d5c9r\" (UID: \"27872680-e51b-4785-bd2e-6912cb0fe64f\") " pod="openshift-ingress/payload-processing-e2e-api-a-6e609c-8496b8f648-d5c9r"
Nov 04 01:22:47.936444 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:22:47.936391    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c3863ed4-3c10-453c-b11d-c66848ed2900-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-6e609c-58d54df979-k4wrw\" (UID: \"c3863ed4-3c10-453c-b11d-c66848ed2900\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-6e609c-58d54df979-k4wrw"
Nov 04 01:22:47.936709 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:22:47.936573    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgmjt\" (UniqueName: \"kubernetes.io/projected/c3863ed4-3c10-453c-b11d-c66848ed2900-kube-api-access-lgmjt\") pod \"payload-pre-processing-e2e-api-a-6e609c-58d54df979-k4wrw\" (UID: \"c3863ed4-3c10-453c-b11d-c66848ed2900\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-6e609c-58d54df979-k4wrw"
Nov 04 01:22:47.938441 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:22:47.938412    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c3863ed4-3c10-453c-b11d-c66848ed2900-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-6e609c-58d54df979-k4wrw\" (UID: \"c3863ed4-3c10-453c-b11d-c66848ed2900\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-6e609c-58d54df979-k4wrw"
Nov 04 01:22:47.956633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:22:47.951180    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgmjt\" (UniqueName: \"kubernetes.io/projected/c3863ed4-3c10-453c-b11d-c66848ed2900-kube-api-access-lgmjt\") pod \"payload-pre-processing-e2e-api-a-6e609c-58d54df979-k4wrw\" (UID: \"c3863ed4-3c10-453c-b11d-c66848ed2900\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-6e609c-58d54df979-k4wrw"
Nov 04 01:22:48.040635 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:22:48.040247    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-6e609c-8496b8f648-d5c9r"
Nov 04 01:22:48.084635 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:22:48.083980    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6e609c-58d54df979-k4wrw"
Nov 04 01:22:48.491135 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:22:48.491104    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3863ed4_3c10_453c_b11d_c66848ed2900.slice/crio-0779e617d5df2c63eba2c5d7df29b8088e4aeab73c01cb0b74514ffbe4abfb8a WatchSource:0}: Error finding container 0779e617d5df2c63eba2c5d7df29b8088e4aeab73c01cb0b74514ffbe4abfb8a: Status 404 returned error can't find the container with id 0779e617d5df2c63eba2c5d7df29b8088e4aeab73c01cb0b74514ffbe4abfb8a
Nov 04 01:22:48.493504 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:22:48.493477    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27872680_e51b_4785_bd2e_6912cb0fe64f.slice/crio-7b1bb9723de0fe340eb0e1868b8ec7f4989340ce9db9b859719697575c92f4d9 WatchSource:0}: Error finding container 7b1bb9723de0fe340eb0e1868b8ec7f4989340ce9db9b859719697575c92f4d9: Status 404 returned error can't find the container with id 7b1bb9723de0fe340eb0e1868b8ec7f4989340ce9db9b859719697575c92f4d9
Nov 04 01:22:48.493729 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:22:48.493703    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-6e609c-58d54df979-k4wrw"]
Nov 04 01:22:48.499386 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:22:48.499357    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-6e609c-8496b8f648-d5c9r"]
Nov 04 01:22:48.699843 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:22:48.699799    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-6e609c-8496b8f648-d5c9r" event={"ID":"27872680-e51b-4785-bd2e-6912cb0fe64f","Type":"ContainerStarted","Data":"0efc55a6dfc36ae39ae7d99391dab701b1c3b4aacd70cdca5793c3194110480e"}
Nov 04 01:22:48.700035 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:22:48.699871    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-6e609c-8496b8f648-d5c9r" event={"ID":"27872680-e51b-4785-bd2e-6912cb0fe64f","Type":"ContainerStarted","Data":"7b1bb9723de0fe340eb0e1868b8ec7f4989340ce9db9b859719697575c92f4d9"}
Nov 04 01:22:48.700035 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:22:48.699899    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-6e609c-8496b8f648-d5c9r"
Nov 04 01:22:48.701665 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:22:48.701639    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6e609c-58d54df979-k4wrw" event={"ID":"c3863ed4-3c10-453c-b11d-c66848ed2900","Type":"ContainerStarted","Data":"8bbd24168d1592314f8d30c8d589e104fda45c635cb5fe74bd9d4b7864692745"}
Nov 04 01:22:48.701815 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:22:48.701684    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6e609c-58d54df979-k4wrw" event={"ID":"c3863ed4-3c10-453c-b11d-c66848ed2900","Type":"ContainerStarted","Data":"0779e617d5df2c63eba2c5d7df29b8088e4aeab73c01cb0b74514ffbe4abfb8a"}
Nov 04 01:22:48.701815 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:22:48.701793    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6e609c-58d54df979-k4wrw"
Nov 04 01:22:48.717918 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:22:48.717866    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-6e609c-8496b8f648-d5c9r" podStartSLOduration=1.7178477779999999 podStartE2EDuration="1.717847778s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:22:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:22:48.717010306 +0000 UTC m=+1208.763571097" watchObservedRunningTime="2026-11-04 01:22:48.717847778 +0000 UTC m=+1208.764408568"
Nov 04 01:22:48.739201 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:22:48.739112    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6e609c-58d54df979-k4wrw" podStartSLOduration=1.7390993190000001 podStartE2EDuration="1.739099319s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:22:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:22:48.738254191 +0000 UTC m=+1208.784814982" watchObservedRunningTime="2026-11-04 01:22:48.739099319 +0000 UTC m=+1208.785660109"
Nov 04 01:22:54.489199 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:22:54.489160    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-e7742169/e2e-lifecycle-model-e7742169-kserve-79fc5d5dd5-xdbvv"]
Nov 04 01:22:59.715758 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:22:59.715727    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-6e609c-8496b8f648-d5c9r"
Nov 04 01:22:59.716147 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:22:59.715800    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6e609c-58d54df979-k4wrw"
Nov 04 01:23:09.511917 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:09.511846    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-mt-e7742169/e2e-lifecycle-model-e7742169-kserve-79fc5d5dd5-xdbvv" podUID="55185ce1-87ff-46ab-a8e4-d9f69de69e55" containerName="main" containerID="cri-o://e13830de628d78cfa5ec75f3c70841fd8cf98fd81c3145a201f2af186ce8b369" gracePeriod=45
Nov 04 01:23:09.832424 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:09.832397    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e7742169/e2e-lifecycle-model-e7742169-kserve-79fc5d5dd5-xdbvv"
Nov 04 01:23:09.909669 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:09.906907    5068 generic.go:356] "Generic (PLEG): container finished" podID="55185ce1-87ff-46ab-a8e4-d9f69de69e55" containerID="e13830de628d78cfa5ec75f3c70841fd8cf98fd81c3145a201f2af186ce8b369" exitCode=0
Nov 04 01:23:09.909669 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:09.906965    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e7742169/e2e-lifecycle-model-e7742169-kserve-79fc5d5dd5-xdbvv"
Nov 04 01:23:09.909669 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:09.907038    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e7742169/e2e-lifecycle-model-e7742169-kserve-79fc5d5dd5-xdbvv" event={"ID":"55185ce1-87ff-46ab-a8e4-d9f69de69e55","Type":"ContainerDied","Data":"e13830de628d78cfa5ec75f3c70841fd8cf98fd81c3145a201f2af186ce8b369"}
Nov 04 01:23:09.909669 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:09.907093    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e7742169/e2e-lifecycle-model-e7742169-kserve-79fc5d5dd5-xdbvv" event={"ID":"55185ce1-87ff-46ab-a8e4-d9f69de69e55","Type":"ContainerDied","Data":"034504b4df51358eec0e34fb4e142abc07f0aba97badf9a0ec88f72ef7bd5ccb"}
Nov 04 01:23:09.909669 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:09.907117    5068 scope.go:122] "RemoveContainer" containerID="e13830de628d78cfa5ec75f3c70841fd8cf98fd81c3145a201f2af186ce8b369"
Nov 04 01:23:09.921505 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:09.921478    5068 scope.go:122] "RemoveContainer" containerID="e13830de628d78cfa5ec75f3c70841fd8cf98fd81c3145a201f2af186ce8b369"
Nov 04 01:23:09.921985 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:23:09.921951    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e13830de628d78cfa5ec75f3c70841fd8cf98fd81c3145a201f2af186ce8b369\": container with ID starting with e13830de628d78cfa5ec75f3c70841fd8cf98fd81c3145a201f2af186ce8b369 not found: ID does not exist" containerID="e13830de628d78cfa5ec75f3c70841fd8cf98fd81c3145a201f2af186ce8b369"
Nov 04 01:23:09.922118 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:09.921994    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e13830de628d78cfa5ec75f3c70841fd8cf98fd81c3145a201f2af186ce8b369"} err="failed to get container status \"e13830de628d78cfa5ec75f3c70841fd8cf98fd81c3145a201f2af186ce8b369\": rpc error: code = NotFound desc = could not find container \"e13830de628d78cfa5ec75f3c70841fd8cf98fd81c3145a201f2af186ce8b369\": container with ID starting with e13830de628d78cfa5ec75f3c70841fd8cf98fd81c3145a201f2af186ce8b369 not found: ID does not exist"
Nov 04 01:23:09.938711 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:09.938674    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/55185ce1-87ff-46ab-a8e4-d9f69de69e55-dshm\" (UniqueName: \"kubernetes.io/empty-dir/55185ce1-87ff-46ab-a8e4-d9f69de69e55-dshm\") pod \"55185ce1-87ff-46ab-a8e4-d9f69de69e55\" (UID: \"55185ce1-87ff-46ab-a8e4-d9f69de69e55\") "
Nov 04 01:23:09.938859 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:09.938844    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/55185ce1-87ff-46ab-a8e4-d9f69de69e55-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55185ce1-87ff-46ab-a8e4-d9f69de69e55-tmp-dir\") pod \"55185ce1-87ff-46ab-a8e4-d9f69de69e55\" (UID: \"55185ce1-87ff-46ab-a8e4-d9f69de69e55\") "
Nov 04 01:23:09.938967 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:09.938951    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/55185ce1-87ff-46ab-a8e4-d9f69de69e55-home\" (UniqueName: \"kubernetes.io/empty-dir/55185ce1-87ff-46ab-a8e4-d9f69de69e55-home\") pod \"55185ce1-87ff-46ab-a8e4-d9f69de69e55\" (UID: \"55185ce1-87ff-46ab-a8e4-d9f69de69e55\") "
Nov 04 01:23:09.939052 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:09.939025    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/55185ce1-87ff-46ab-a8e4-d9f69de69e55-tls-certs\" (UniqueName: \"kubernetes.io/secret/55185ce1-87ff-46ab-a8e4-d9f69de69e55-tls-certs\") pod \"55185ce1-87ff-46ab-a8e4-d9f69de69e55\" (UID: \"55185ce1-87ff-46ab-a8e4-d9f69de69e55\") "
Nov 04 01:23:09.939174 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:09.939148    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55185ce1-87ff-46ab-a8e4-d9f69de69e55-tmp-dir" pod "55185ce1-87ff-46ab-a8e4-d9f69de69e55" (UID: "55185ce1-87ff-46ab-a8e4-d9f69de69e55"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:23:09.939231 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:09.939175    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55185ce1-87ff-46ab-a8e4-d9f69de69e55-home" pod "55185ce1-87ff-46ab-a8e4-d9f69de69e55" (UID: "55185ce1-87ff-46ab-a8e4-d9f69de69e55"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:23:09.939231 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:09.939179    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/55185ce1-87ff-46ab-a8e4-d9f69de69e55-kube-api-access-znhtb\" (UniqueName: \"kubernetes.io/projected/55185ce1-87ff-46ab-a8e4-d9f69de69e55-kube-api-access-znhtb\") pod \"55185ce1-87ff-46ab-a8e4-d9f69de69e55\" (UID: \"55185ce1-87ff-46ab-a8e4-d9f69de69e55\") "
Nov 04 01:23:09.939327 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:09.939297    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/55185ce1-87ff-46ab-a8e4-d9f69de69e55-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/55185ce1-87ff-46ab-a8e4-d9f69de69e55-model-cache\") pod \"55185ce1-87ff-46ab-a8e4-d9f69de69e55\" (UID: \"55185ce1-87ff-46ab-a8e4-d9f69de69e55\") "
Nov 04 01:23:09.940200 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:09.940170    5068 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55185ce1-87ff-46ab-a8e4-d9f69de69e55-tmp-dir\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:23:09.940326 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:09.940205    5068 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/55185ce1-87ff-46ab-a8e4-d9f69de69e55-home\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:23:09.940460 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:09.940439    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55185ce1-87ff-46ab-a8e4-d9f69de69e55-model-cache" pod "55185ce1-87ff-46ab-a8e4-d9f69de69e55" (UID: "55185ce1-87ff-46ab-a8e4-d9f69de69e55"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:23:09.941556 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:09.941526    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55185ce1-87ff-46ab-a8e4-d9f69de69e55-kube-api-access-znhtb" pod "55185ce1-87ff-46ab-a8e4-d9f69de69e55" (UID: "55185ce1-87ff-46ab-a8e4-d9f69de69e55"). InnerVolumeSpecName "kube-api-access-znhtb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:23:09.941997 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:09.941975    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55185ce1-87ff-46ab-a8e4-d9f69de69e55-dshm" pod "55185ce1-87ff-46ab-a8e4-d9f69de69e55" (UID: "55185ce1-87ff-46ab-a8e4-d9f69de69e55"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:23:09.942376 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:09.942353    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55185ce1-87ff-46ab-a8e4-d9f69de69e55-tls-certs" pod "55185ce1-87ff-46ab-a8e4-d9f69de69e55" (UID: "55185ce1-87ff-46ab-a8e4-d9f69de69e55"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:23:10.041620 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:10.041574    5068 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/55185ce1-87ff-46ab-a8e4-d9f69de69e55-tls-certs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:23:10.041801 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:10.041606    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-znhtb\" (UniqueName: \"kubernetes.io/projected/55185ce1-87ff-46ab-a8e4-d9f69de69e55-kube-api-access-znhtb\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:23:10.041801 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:10.041665    5068 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/55185ce1-87ff-46ab-a8e4-d9f69de69e55-model-cache\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:23:10.041801 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:10.041678    5068 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/55185ce1-87ff-46ab-a8e4-d9f69de69e55-dshm\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:23:10.227116 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:10.227081    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-e7742169/e2e-lifecycle-model-e7742169-kserve-79fc5d5dd5-xdbvv"]
Nov 04 01:23:10.232239 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:10.232213    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-e7742169/e2e-lifecycle-model-e7742169-kserve-79fc5d5dd5-xdbvv"]
Nov 04 01:23:10.827794 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:10.827753    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-e7742169/e2e-lifecycle-model-e7742169-kserve-79fc5d5dd5-xdbvv" podUID="55185ce1-87ff-46ab-a8e4-d9f69de69e55" containerName="main" probeResult="failure" output="Get \"https://10.132.0.106:8000/ready\": context deadline exceeded"
Nov 04 01:23:12.115839 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:12.115804    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="55185ce1-87ff-46ab-a8e4-d9f69de69e55" path="/var/lib/kubelet/pods/55185ce1-87ff-46ab-a8e4-d9f69de69e55/volumes"
Nov 04 01:23:13.458930 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:23:13.458889    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:23:29.404796 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:29.404755    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-20a250-66d794c856-ql77v"]
Nov 04 01:23:29.405934 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:29.405902    5068 cpu_manager.go:414] "containerMap: removing container" podUID="55185ce1-87ff-46ab-a8e4-d9f69de69e55" containerName="main"
Nov 04 01:23:29.405934 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:29.405925    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="55185ce1-87ff-46ab-a8e4-d9f69de69e55" containerName="main"
Nov 04 01:23:29.406085 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:29.406051    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="55185ce1-87ff-46ab-a8e4-d9f69de69e55" containerName="main"
Nov 04 01:23:29.410017 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:29.410000    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-20a250-66d794c856-ql77v"
Nov 04 01:23:29.414698 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:29.414675    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-20a250-dockercfg-xfhbk\""
Nov 04 01:23:29.415005 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:29.414981    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-20a250\""
Nov 04 01:23:29.420530 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:29.420503    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-20a250-66d794c856-ql77v"]
Nov 04 01:23:29.533975 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:29.533936    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/62dc4132-b26f-4b4f-a4c1-936dfa171b05-plugins-config-volume\") pod \"payload-processing-e2e-api-b-20a250-66d794c856-ql77v\" (UID: \"62dc4132-b26f-4b4f-a4c1-936dfa171b05\") " pod="openshift-ingress/payload-processing-e2e-api-b-20a250-66d794c856-ql77v"
Nov 04 01:23:29.534215 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:29.534195    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bk2bz\" (UniqueName: \"kubernetes.io/projected/62dc4132-b26f-4b4f-a4c1-936dfa171b05-kube-api-access-bk2bz\") pod \"payload-processing-e2e-api-b-20a250-66d794c856-ql77v\" (UID: \"62dc4132-b26f-4b4f-a4c1-936dfa171b05\") " pod="openshift-ingress/payload-processing-e2e-api-b-20a250-66d794c856-ql77v"
Nov 04 01:23:29.636637 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:29.635602    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bk2bz\" (UniqueName: \"kubernetes.io/projected/62dc4132-b26f-4b4f-a4c1-936dfa171b05-kube-api-access-bk2bz\") pod \"payload-processing-e2e-api-b-20a250-66d794c856-ql77v\" (UID: \"62dc4132-b26f-4b4f-a4c1-936dfa171b05\") " pod="openshift-ingress/payload-processing-e2e-api-b-20a250-66d794c856-ql77v"
Nov 04 01:23:29.636637 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:29.635858    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/62dc4132-b26f-4b4f-a4c1-936dfa171b05-plugins-config-volume\") pod \"payload-processing-e2e-api-b-20a250-66d794c856-ql77v\" (UID: \"62dc4132-b26f-4b4f-a4c1-936dfa171b05\") " pod="openshift-ingress/payload-processing-e2e-api-b-20a250-66d794c856-ql77v"
Nov 04 01:23:29.640631 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:29.637133    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/62dc4132-b26f-4b4f-a4c1-936dfa171b05-plugins-config-volume\") pod \"payload-processing-e2e-api-b-20a250-66d794c856-ql77v\" (UID: \"62dc4132-b26f-4b4f-a4c1-936dfa171b05\") " pod="openshift-ingress/payload-processing-e2e-api-b-20a250-66d794c856-ql77v"
Nov 04 01:23:29.662780 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:29.662702    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bk2bz\" (UniqueName: \"kubernetes.io/projected/62dc4132-b26f-4b4f-a4c1-936dfa171b05-kube-api-access-bk2bz\") pod \"payload-processing-e2e-api-b-20a250-66d794c856-ql77v\" (UID: \"62dc4132-b26f-4b4f-a4c1-936dfa171b05\") " pod="openshift-ingress/payload-processing-e2e-api-b-20a250-66d794c856-ql77v"
Nov 04 01:23:29.729384 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:29.729355    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-20a250-66d794c856-ql77v"
Nov 04 01:23:30.116960 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:30.116931    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-20a250-66d794c856-ql77v"]
Nov 04 01:23:30.117125 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:23:30.117105    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62dc4132_b26f_4b4f_a4c1_936dfa171b05.slice/crio-4e2c08ce793effcb9758839e961420d959849918a6e3467c2a77bf69ba14efdc WatchSource:0}: Error finding container 4e2c08ce793effcb9758839e961420d959849918a6e3467c2a77bf69ba14efdc: Status 404 returned error can't find the container with id 4e2c08ce793effcb9758839e961420d959849918a6e3467c2a77bf69ba14efdc
Nov 04 01:23:31.133989 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:31.133893    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-20a250-66d794c856-ql77v" event={"ID":"62dc4132-b26f-4b4f-a4c1-936dfa171b05","Type":"ContainerStarted","Data":"e26652542987ee60abd614fb4f39b4a6608ac0c7eb97fda9d65efa30c5a2bed3"}
Nov 04 01:23:31.133989 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:31.133951    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-20a250-66d794c856-ql77v" event={"ID":"62dc4132-b26f-4b4f-a4c1-936dfa171b05","Type":"ContainerStarted","Data":"4e2c08ce793effcb9758839e961420d959849918a6e3467c2a77bf69ba14efdc"}
Nov 04 01:23:31.133989 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:31.133982    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-20a250-66d794c856-ql77v"
Nov 04 01:23:40.035319 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:40.035242    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-20a250-66d794c856-ql77v" podStartSLOduration=11.035227601 podStartE2EDuration="11.035227601s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:23:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:23:31.17376235 +0000 UTC m=+1251.220323138" watchObservedRunningTime="2026-11-04 01:23:40.035227601 +0000 UTC m=+1260.081788391"
Nov 04 01:23:40.039512 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:40.039481    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:23:42.355638 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:42.351021    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-20a250-66d794c856-ql77v"
Nov 04 01:23:43.564635 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:23:43.561816    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:23:54.765194 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:54.765154    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz"]
Nov 04 01:23:54.772625 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:54.772589    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz"
Nov 04 01:23:54.792448 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:54.792412    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz"]
Nov 04 01:23:54.925652 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:54.914590    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40b656fe-1647-4533-8798-72791dc96d34-istio-podinfo\") pod \"e2e-mt-e7742169-openshift-default-6986f46949-w2wqz\" (UID: \"40b656fe-1647-4533-8798-72791dc96d34\") " pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz"
Nov 04 01:23:54.925652 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:54.914735    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40b656fe-1647-4533-8798-72791dc96d34-istio-token\") pod \"e2e-mt-e7742169-openshift-default-6986f46949-w2wqz\" (UID: \"40b656fe-1647-4533-8798-72791dc96d34\") " pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz"
Nov 04 01:23:54.925652 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:54.914833    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40b656fe-1647-4533-8798-72791dc96d34-istiod-ca-cert\") pod \"e2e-mt-e7742169-openshift-default-6986f46949-w2wqz\" (UID: \"40b656fe-1647-4533-8798-72791dc96d34\") " pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz"
Nov 04 01:23:54.925652 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:54.914951    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcw4w\" (UniqueName: \"kubernetes.io/projected/40b656fe-1647-4533-8798-72791dc96d34-kube-api-access-jcw4w\") pod \"e2e-mt-e7742169-openshift-default-6986f46949-w2wqz\" (UID: \"40b656fe-1647-4533-8798-72791dc96d34\") " pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz"
Nov 04 01:23:54.925652 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:54.915033    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40b656fe-1647-4533-8798-72791dc96d34-workload-socket\") pod \"e2e-mt-e7742169-openshift-default-6986f46949-w2wqz\" (UID: \"40b656fe-1647-4533-8798-72791dc96d34\") " pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz"
Nov 04 01:23:54.925652 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:54.915424    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40b656fe-1647-4533-8798-72791dc96d34-istio-data\") pod \"e2e-mt-e7742169-openshift-default-6986f46949-w2wqz\" (UID: \"40b656fe-1647-4533-8798-72791dc96d34\") " pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz"
Nov 04 01:23:54.925652 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:54.915561    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40b656fe-1647-4533-8798-72791dc96d34-istio-envoy\") pod \"e2e-mt-e7742169-openshift-default-6986f46949-w2wqz\" (UID: \"40b656fe-1647-4533-8798-72791dc96d34\") " pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz"
Nov 04 01:23:54.925652 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:54.915680    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40b656fe-1647-4533-8798-72791dc96d34-credential-socket\") pod \"e2e-mt-e7742169-openshift-default-6986f46949-w2wqz\" (UID: \"40b656fe-1647-4533-8798-72791dc96d34\") " pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz"
Nov 04 01:23:54.925652 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:54.915790    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40b656fe-1647-4533-8798-72791dc96d34-workload-certs\") pod \"e2e-mt-e7742169-openshift-default-6986f46949-w2wqz\" (UID: \"40b656fe-1647-4533-8798-72791dc96d34\") " pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz"
Nov 04 01:23:55.017522 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:55.017413    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40b656fe-1647-4533-8798-72791dc96d34-istio-podinfo\") pod \"e2e-mt-e7742169-openshift-default-6986f46949-w2wqz\" (UID: \"40b656fe-1647-4533-8798-72791dc96d34\") " pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz"
Nov 04 01:23:55.017522 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:55.017476    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40b656fe-1647-4533-8798-72791dc96d34-istio-token\") pod \"e2e-mt-e7742169-openshift-default-6986f46949-w2wqz\" (UID: \"40b656fe-1647-4533-8798-72791dc96d34\") " pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz"
Nov 04 01:23:55.017799 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:55.017549    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40b656fe-1647-4533-8798-72791dc96d34-istiod-ca-cert\") pod \"e2e-mt-e7742169-openshift-default-6986f46949-w2wqz\" (UID: \"40b656fe-1647-4533-8798-72791dc96d34\") " pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz"
Nov 04 01:23:55.017799 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:55.017602    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcw4w\" (UniqueName: \"kubernetes.io/projected/40b656fe-1647-4533-8798-72791dc96d34-kube-api-access-jcw4w\") pod \"e2e-mt-e7742169-openshift-default-6986f46949-w2wqz\" (UID: \"40b656fe-1647-4533-8798-72791dc96d34\") " pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz"
Nov 04 01:23:55.017799 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:55.017722    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40b656fe-1647-4533-8798-72791dc96d34-workload-socket\") pod \"e2e-mt-e7742169-openshift-default-6986f46949-w2wqz\" (UID: \"40b656fe-1647-4533-8798-72791dc96d34\") " pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz"
Nov 04 01:23:55.017914 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:55.017856    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40b656fe-1647-4533-8798-72791dc96d34-istio-data\") pod \"e2e-mt-e7742169-openshift-default-6986f46949-w2wqz\" (UID: \"40b656fe-1647-4533-8798-72791dc96d34\") " pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz"
Nov 04 01:23:55.017914 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:55.017891    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40b656fe-1647-4533-8798-72791dc96d34-istio-envoy\") pod \"e2e-mt-e7742169-openshift-default-6986f46949-w2wqz\" (UID: \"40b656fe-1647-4533-8798-72791dc96d34\") " pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz"
Nov 04 01:23:55.017981 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:55.017919    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40b656fe-1647-4533-8798-72791dc96d34-credential-socket\") pod \"e2e-mt-e7742169-openshift-default-6986f46949-w2wqz\" (UID: \"40b656fe-1647-4533-8798-72791dc96d34\") " pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz"
Nov 04 01:23:55.018040 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:55.018022    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40b656fe-1647-4533-8798-72791dc96d34-workload-certs\") pod \"e2e-mt-e7742169-openshift-default-6986f46949-w2wqz\" (UID: \"40b656fe-1647-4533-8798-72791dc96d34\") " pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz"
Nov 04 01:23:55.018204 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:55.018164    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40b656fe-1647-4533-8798-72791dc96d34-workload-socket\") pod \"e2e-mt-e7742169-openshift-default-6986f46949-w2wqz\" (UID: \"40b656fe-1647-4533-8798-72791dc96d34\") " pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz"
Nov 04 01:23:55.018313 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:55.018285    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40b656fe-1647-4533-8798-72791dc96d34-istio-data\") pod \"e2e-mt-e7742169-openshift-default-6986f46949-w2wqz\" (UID: \"40b656fe-1647-4533-8798-72791dc96d34\") " pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz"
Nov 04 01:23:55.018596 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:55.018578    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40b656fe-1647-4533-8798-72791dc96d34-workload-certs\") pod \"e2e-mt-e7742169-openshift-default-6986f46949-w2wqz\" (UID: \"40b656fe-1647-4533-8798-72791dc96d34\") " pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz"
Nov 04 01:23:55.018727 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:55.018665    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40b656fe-1647-4533-8798-72791dc96d34-credential-socket\") pod \"e2e-mt-e7742169-openshift-default-6986f46949-w2wqz\" (UID: \"40b656fe-1647-4533-8798-72791dc96d34\") " pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz"
Nov 04 01:23:55.018727 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:55.018694    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40b656fe-1647-4533-8798-72791dc96d34-istiod-ca-cert\") pod \"e2e-mt-e7742169-openshift-default-6986f46949-w2wqz\" (UID: \"40b656fe-1647-4533-8798-72791dc96d34\") " pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz"
Nov 04 01:23:55.020172 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:55.020150    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40b656fe-1647-4533-8798-72791dc96d34-istio-envoy\") pod \"e2e-mt-e7742169-openshift-default-6986f46949-w2wqz\" (UID: \"40b656fe-1647-4533-8798-72791dc96d34\") " pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz"
Nov 04 01:23:55.020287 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:55.020269    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40b656fe-1647-4533-8798-72791dc96d34-istio-podinfo\") pod \"e2e-mt-e7742169-openshift-default-6986f46949-w2wqz\" (UID: \"40b656fe-1647-4533-8798-72791dc96d34\") " pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz"
Nov 04 01:23:55.030106 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:55.030082    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40b656fe-1647-4533-8798-72791dc96d34-istio-token\") pod \"e2e-mt-e7742169-openshift-default-6986f46949-w2wqz\" (UID: \"40b656fe-1647-4533-8798-72791dc96d34\") " pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz"
Nov 04 01:23:55.030200 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:55.030084    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcw4w\" (UniqueName: \"kubernetes.io/projected/40b656fe-1647-4533-8798-72791dc96d34-kube-api-access-jcw4w\") pod \"e2e-mt-e7742169-openshift-default-6986f46949-w2wqz\" (UID: \"40b656fe-1647-4533-8798-72791dc96d34\") " pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz"
Nov 04 01:23:55.096750 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:55.096720    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz"
Nov 04 01:23:55.402976 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:55.402930    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz"]
Nov 04 01:23:55.403625 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:23:55.403587    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40b656fe_1647_4533_8798_72791dc96d34.slice/crio-08d57944ab193110f64fd89137295be68255bfd7f3a965b588d5eba8e7915406 WatchSource:0}: Error finding container 08d57944ab193110f64fd89137295be68255bfd7f3a965b588d5eba8e7915406: Status 404 returned error can't find the container with id 08d57944ab193110f64fd89137295be68255bfd7f3a965b588d5eba8e7915406
Nov 04 01:23:55.406749 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:55.406715    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:23:55.406867 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:55.406785    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:23:55.406867 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:55.406817    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:23:55.618207 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:55.617631    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz" event={"ID":"40b656fe-1647-4533-8798-72791dc96d34","Type":"ContainerStarted","Data":"5b3b8686fe6ac228e74d4fea6dc2fcc01b9c91092e5a50b98904666554ae84ca"}
Nov 04 01:23:55.618207 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:55.617720    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz" event={"ID":"40b656fe-1647-4533-8798-72791dc96d34","Type":"ContainerStarted","Data":"08d57944ab193110f64fd89137295be68255bfd7f3a965b588d5eba8e7915406"}
Nov 04 01:23:55.643156 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:55.642180    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz" podStartSLOduration=1.6421611550000002 podStartE2EDuration="1.642161155s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:23:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:23:55.640895377 +0000 UTC m=+1275.687456179" watchObservedRunningTime="2026-11-04 01:23:55.642161155 +0000 UTC m=+1275.688721948"
Nov 04 01:23:56.097035 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:56.096997    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz"
Nov 04 01:23:56.098521 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:56.098491    5068 patch_prober.go:29] interesting pod/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused" start-of-body=
Nov 04 01:23:56.098708 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:56.098549    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz" podUID="40b656fe-1647-4533-8798-72791dc96d34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused"
Nov 04 01:23:57.097519 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:57.097481    5068 patch_prober.go:29] interesting pod/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused" start-of-body=
Nov 04 01:23:57.098038 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:57.097546    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz" podUID="40b656fe-1647-4533-8798-72791dc96d34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused"
Nov 04 01:23:57.488684 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:57.487801    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg"]
Nov 04 01:23:57.496631 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:57.496375    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg"]
Nov 04 01:23:57.496631 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:57.496492    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg"
Nov 04 01:23:57.560635 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:57.550942    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-6e609c-openshift-default-dockercfg-n2474\""
Nov 04 01:23:57.596654 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:57.595416    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d899bb80-977d-4b53-b96c-b354dcbdad2d-istio-data\") pod \"e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg\" (UID: \"d899bb80-977d-4b53-b96c-b354dcbdad2d\") " pod="openshift-ingress/e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg"
Nov 04 01:23:57.596654 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:57.595472    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnd4z\" (UniqueName: \"kubernetes.io/projected/d899bb80-977d-4b53-b96c-b354dcbdad2d-kube-api-access-mnd4z\") pod \"e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg\" (UID: \"d899bb80-977d-4b53-b96c-b354dcbdad2d\") " pod="openshift-ingress/e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg"
Nov 04 01:23:57.596654 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:57.595565    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d899bb80-977d-4b53-b96c-b354dcbdad2d-istio-podinfo\") pod \"e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg\" (UID: \"d899bb80-977d-4b53-b96c-b354dcbdad2d\") " pod="openshift-ingress/e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg"
Nov 04 01:23:57.596654 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:57.595746    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d899bb80-977d-4b53-b96c-b354dcbdad2d-istio-token\") pod \"e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg\" (UID: \"d899bb80-977d-4b53-b96c-b354dcbdad2d\") " pod="openshift-ingress/e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg"
Nov 04 01:23:57.596654 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:57.595791    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d899bb80-977d-4b53-b96c-b354dcbdad2d-workload-certs\") pod \"e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg\" (UID: \"d899bb80-977d-4b53-b96c-b354dcbdad2d\") " pod="openshift-ingress/e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg"
Nov 04 01:23:57.596654 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:57.595820    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d899bb80-977d-4b53-b96c-b354dcbdad2d-istiod-ca-cert\") pod \"e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg\" (UID: \"d899bb80-977d-4b53-b96c-b354dcbdad2d\") " pod="openshift-ingress/e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg"
Nov 04 01:23:57.596654 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:57.595876    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d899bb80-977d-4b53-b96c-b354dcbdad2d-workload-socket\") pod \"e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg\" (UID: \"d899bb80-977d-4b53-b96c-b354dcbdad2d\") " pod="openshift-ingress/e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg"
Nov 04 01:23:57.596654 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:57.595902    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d899bb80-977d-4b53-b96c-b354dcbdad2d-credential-socket\") pod \"e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg\" (UID: \"d899bb80-977d-4b53-b96c-b354dcbdad2d\") " pod="openshift-ingress/e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg"
Nov 04 01:23:57.596654 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:57.596011    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d899bb80-977d-4b53-b96c-b354dcbdad2d-istio-envoy\") pod \"e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg\" (UID: \"d899bb80-977d-4b53-b96c-b354dcbdad2d\") " pod="openshift-ingress/e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg"
Nov 04 01:23:57.697256 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:57.697215    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d899bb80-977d-4b53-b96c-b354dcbdad2d-workload-socket\") pod \"e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg\" (UID: \"d899bb80-977d-4b53-b96c-b354dcbdad2d\") " pod="openshift-ingress/e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg"
Nov 04 01:23:57.697450 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:57.697266    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d899bb80-977d-4b53-b96c-b354dcbdad2d-credential-socket\") pod \"e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg\" (UID: \"d899bb80-977d-4b53-b96c-b354dcbdad2d\") " pod="openshift-ingress/e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg"
Nov 04 01:23:57.697450 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:57.697403    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d899bb80-977d-4b53-b96c-b354dcbdad2d-istio-envoy\") pod \"e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg\" (UID: \"d899bb80-977d-4b53-b96c-b354dcbdad2d\") " pod="openshift-ingress/e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg"
Nov 04 01:23:57.697719 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:57.697677    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d899bb80-977d-4b53-b96c-b354dcbdad2d-credential-socket\") pod \"e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg\" (UID: \"d899bb80-977d-4b53-b96c-b354dcbdad2d\") " pod="openshift-ingress/e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg"
Nov 04 01:23:57.697835 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:57.697692    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d899bb80-977d-4b53-b96c-b354dcbdad2d-istio-data\") pod \"e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg\" (UID: \"d899bb80-977d-4b53-b96c-b354dcbdad2d\") " pod="openshift-ingress/e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg"
Nov 04 01:23:57.697835 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:57.697818    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnd4z\" (UniqueName: \"kubernetes.io/projected/d899bb80-977d-4b53-b96c-b354dcbdad2d-kube-api-access-mnd4z\") pod \"e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg\" (UID: \"d899bb80-977d-4b53-b96c-b354dcbdad2d\") " pod="openshift-ingress/e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg"
Nov 04 01:23:57.697942 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:57.697855    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d899bb80-977d-4b53-b96c-b354dcbdad2d-istio-podinfo\") pod \"e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg\" (UID: \"d899bb80-977d-4b53-b96c-b354dcbdad2d\") " pod="openshift-ingress/e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg"
Nov 04 01:23:57.698155 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:57.698134    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d899bb80-977d-4b53-b96c-b354dcbdad2d-istio-data\") pod \"e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg\" (UID: \"d899bb80-977d-4b53-b96c-b354dcbdad2d\") " pod="openshift-ingress/e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg"
Nov 04 01:23:57.698227 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:57.698175    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d899bb80-977d-4b53-b96c-b354dcbdad2d-istio-token\") pod \"e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg\" (UID: \"d899bb80-977d-4b53-b96c-b354dcbdad2d\") " pod="openshift-ingress/e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg"
Nov 04 01:23:57.698227 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:57.698206    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d899bb80-977d-4b53-b96c-b354dcbdad2d-workload-certs\") pod \"e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg\" (UID: \"d899bb80-977d-4b53-b96c-b354dcbdad2d\") " pod="openshift-ingress/e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg"
Nov 04 01:23:57.698227 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:57.698223    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d899bb80-977d-4b53-b96c-b354dcbdad2d-istiod-ca-cert\") pod \"e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg\" (UID: \"d899bb80-977d-4b53-b96c-b354dcbdad2d\") " pod="openshift-ingress/e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg"
Nov 04 01:23:57.698907 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:57.698883    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d899bb80-977d-4b53-b96c-b354dcbdad2d-workload-socket\") pod \"e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg\" (UID: \"d899bb80-977d-4b53-b96c-b354dcbdad2d\") " pod="openshift-ingress/e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg"
Nov 04 01:23:57.699067 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:57.698919    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d899bb80-977d-4b53-b96c-b354dcbdad2d-istiod-ca-cert\") pod \"e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg\" (UID: \"d899bb80-977d-4b53-b96c-b354dcbdad2d\") " pod="openshift-ingress/e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg"
Nov 04 01:23:57.699762 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:57.699740    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d899bb80-977d-4b53-b96c-b354dcbdad2d-workload-certs\") pod \"e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg\" (UID: \"d899bb80-977d-4b53-b96c-b354dcbdad2d\") " pod="openshift-ingress/e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg"
Nov 04 01:23:57.701756 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:57.701724    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d899bb80-977d-4b53-b96c-b354dcbdad2d-istio-podinfo\") pod \"e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg\" (UID: \"d899bb80-977d-4b53-b96c-b354dcbdad2d\") " pod="openshift-ingress/e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg"
Nov 04 01:23:57.703908 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:57.703890    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d899bb80-977d-4b53-b96c-b354dcbdad2d-istio-envoy\") pod \"e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg\" (UID: \"d899bb80-977d-4b53-b96c-b354dcbdad2d\") " pod="openshift-ingress/e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg"
Nov 04 01:23:57.714660 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:57.713002    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d899bb80-977d-4b53-b96c-b354dcbdad2d-istio-token\") pod \"e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg\" (UID: \"d899bb80-977d-4b53-b96c-b354dcbdad2d\") " pod="openshift-ingress/e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg"
Nov 04 01:23:57.714660 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:57.714298    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnd4z\" (UniqueName: \"kubernetes.io/projected/d899bb80-977d-4b53-b96c-b354dcbdad2d-kube-api-access-mnd4z\") pod \"e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg\" (UID: \"d899bb80-977d-4b53-b96c-b354dcbdad2d\") " pod="openshift-ingress/e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg"
Nov 04 01:23:57.815327 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:57.815286    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg"
Nov 04 01:23:58.097096 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:58.097010    5068 patch_prober.go:29] interesting pod/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused" start-of-body=
Nov 04 01:23:58.097096 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:58.097072    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz" podUID="40b656fe-1647-4533-8798-72791dc96d34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused"
Nov 04 01:23:58.175501 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:23:58.175468    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd899bb80_977d_4b53_b96c_b354dcbdad2d.slice/crio-ddcf6119b2e4eb11dc89cf9529ea82df703e04d44709e86e39a3b7f0e30c97a5 WatchSource:0}: Error finding container ddcf6119b2e4eb11dc89cf9529ea82df703e04d44709e86e39a3b7f0e30c97a5: Status 404 returned error can't find the container with id ddcf6119b2e4eb11dc89cf9529ea82df703e04d44709e86e39a3b7f0e30c97a5
Nov 04 01:23:58.177217 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:58.177192    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg"]
Nov 04 01:23:58.180911 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:58.180870    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:23:58.181006 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:58.180962    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:23:58.181065 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:58.181015    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:23:58.704662 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:58.704587    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg" event={"ID":"d899bb80-977d-4b53-b96c-b354dcbdad2d","Type":"ContainerStarted","Data":"1cda6849311364f1e3fc26d54221e0e4f6f248f3d147559e4c4d1ad38f7b9432"}
Nov 04 01:23:58.704868 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:58.704695    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg" event={"ID":"d899bb80-977d-4b53-b96c-b354dcbdad2d","Type":"ContainerStarted","Data":"ddcf6119b2e4eb11dc89cf9529ea82df703e04d44709e86e39a3b7f0e30c97a5"}
Nov 04 01:23:58.728511 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:58.728447    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg" podStartSLOduration=1.7284278309999999 podStartE2EDuration="1.728427831s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:23:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:23:58.726520475 +0000 UTC m=+1278.773081264" watchObservedRunningTime="2026-11-04 01:23:58.728427831 +0000 UTC m=+1278.774988623"
Nov 04 01:23:58.815893 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:58.815847    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg"
Nov 04 01:23:59.099584 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:59.099543    5068 patch_prober.go:29] interesting pod/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused" start-of-body=
Nov 04 01:23:59.099888 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:59.099646    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz" podUID="40b656fe-1647-4533-8798-72791dc96d34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused"
Nov 04 01:23:59.816733 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:59.816695    5068 patch_prober.go:29] interesting pod/e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.120:15021/healthz/ready\": dial tcp 10.132.0.120:15021: connect: connection refused" start-of-body=
Nov 04 01:23:59.817139 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:23:59.816774    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg" podUID="d899bb80-977d-4b53-b96c-b354dcbdad2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.120:15021/healthz/ready\": dial tcp 10.132.0.120:15021: connect: connection refused"
Nov 04 01:24:00.099235 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:24:00.097289    5068 patch_prober.go:29] interesting pod/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused" start-of-body=
Nov 04 01:24:00.099235 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:24:00.097350    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz" podUID="40b656fe-1647-4533-8798-72791dc96d34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused"
Nov 04 01:24:00.816468 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:24:00.816424    5068 patch_prober.go:29] interesting pod/e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.120:15021/healthz/ready\": dial tcp 10.132.0.120:15021: connect: connection refused" start-of-body=
Nov 04 01:24:00.816769 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:24:00.816503    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg" podUID="d899bb80-977d-4b53-b96c-b354dcbdad2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.120:15021/healthz/ready\": dial tcp 10.132.0.120:15021: connect: connection refused"
Nov 04 01:24:01.106983 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:24:01.106764    5068 patch_prober.go:29] interesting pod/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused" start-of-body=
Nov 04 01:24:01.106983 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:24:01.106902    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz" podUID="40b656fe-1647-4533-8798-72791dc96d34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused"
Nov 04 01:24:01.815988 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:24:01.815950    5068 patch_prober.go:29] interesting pod/e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.120:15021/healthz/ready\": dial tcp 10.132.0.120:15021: connect: connection refused" start-of-body=
Nov 04 01:24:01.816175 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:24:01.816036    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg" podUID="d899bb80-977d-4b53-b96c-b354dcbdad2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.120:15021/healthz/ready\": dial tcp 10.132.0.120:15021: connect: connection refused"
Nov 04 01:24:02.098037 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:24:02.097944    5068 patch_prober.go:29] interesting pod/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused" start-of-body=
Nov 04 01:24:02.098037 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:24:02.098014    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz" podUID="40b656fe-1647-4533-8798-72791dc96d34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused"
Nov 04 01:24:02.816234 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:24:02.816200    5068 patch_prober.go:29] interesting pod/e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.120:15021/healthz/ready\": dial tcp 10.132.0.120:15021: connect: connection refused" start-of-body=
Nov 04 01:24:02.816423 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:24:02.816264    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg" podUID="d899bb80-977d-4b53-b96c-b354dcbdad2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.120:15021/healthz/ready\": dial tcp 10.132.0.120:15021: connect: connection refused"
Nov 04 01:24:03.103240 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:24:03.103153    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz"
Nov 04 01:24:03.103740 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:24:03.103668    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz"
Nov 04 01:24:03.104383 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:24:03.104364    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz"
Nov 04 01:24:03.815951 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:24:03.815910    5068 patch_prober.go:29] interesting pod/e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.120:15021/healthz/ready\": dial tcp 10.132.0.120:15021: connect: connection refused" start-of-body=
Nov 04 01:24:03.816150 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:24:03.815969    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg" podUID="d899bb80-977d-4b53-b96c-b354dcbdad2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.120:15021/healthz/ready\": dial tcp 10.132.0.120:15021: connect: connection refused"
Nov 04 01:24:04.820962 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:24:04.820932    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg"
Nov 04 01:24:04.821454 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:24:04.821310    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg"
Nov 04 01:24:04.821924 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:24:04.821899    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg"
Nov 04 01:24:13.458140 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:24:13.458100    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:24:36.248022 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:24:36.247983    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:24:43.459052 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:24:43.459017    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:25:13.456221 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:25:13.456175    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:25:21.243723 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:25:21.243691    5068 ???:1] "http: TLS handshake error from 10.0.140.186:39949: read tcp 10.0.132.67:10250->10.0.140.186:39949: read: connection reset by peer"
Nov 04 01:25:43.457846 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:25:43.457805    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:26:13.455575 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:26:13.455536    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:26:14.059632 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:14.059071    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp"]
Nov 04 01:26:14.061382 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:14.061344    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp" podUID="73f6a679-d6e5-45d1-a885-ebea2ed4f253" containerName="istio-proxy" containerID="cri-o://e7d6118c5c88d2a64ed465cc3ec0a56bb3879bcf99cc063f749d792e5d7ed748" gracePeriod=30
Nov 04 01:26:14.075119 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:14.075086    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv"]
Nov 04 01:26:14.075452 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:14.075418    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv" podUID="3bde9301-d2f8-4820-9044-463e387b3c3d" containerName="istio-proxy" containerID="cri-o://02d30ff6c58243a35c9e1fc8ab24caa86f435f0a65fd42b4b9d0f9624cc95537" gracePeriod=30
Nov 04 01:26:14.085492 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:14.085459    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7"]
Nov 04 01:26:14.085751 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:14.085712    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7" podUID="7d1dc8f1-3238-4b12-b579-ec8179bc4e2a" containerName="istio-proxy" containerID="cri-o://22a27310c8b2a4b2312973876da0ab42d52308596edfa2bb1a3864033939da24" gracePeriod=30
Nov 04 01:26:14.158454 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:14.158248    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp"]
Nov 04 01:26:14.158702 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:14.158527    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp" podUID="20fba30d-0142-49f8-8489-957bad3db95e" containerName="istio-proxy" containerID="cri-o://40b5f1c0fb186e12538dfe099ccece193a8e0d5aa1ec63ce5734c65849bba32d" gracePeriod=30
Nov 04 01:26:14.176982 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:14.176950    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94"]
Nov 04 01:26:14.177208 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:14.177170    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94" podUID="45b260dc-8b00-4f55-a649-58220ed083bb" containerName="istio-proxy" containerID="cri-o://ab4ccb934f6a263df1ac0d8019a4d7b4957b6b7e4a02be7117c68056ba5131e2" gracePeriod=30
Nov 04 01:26:14.841093 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:14.841050    5068 patch_prober.go:29] interesting pod/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 01:26:14.841598 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:14.841115    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp" podUID="73f6a679-d6e5-45d1-a885-ebea2ed4f253" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 01:26:19.220249 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:26:19.219923    5068 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bde9301_d2f8_4820_9044_463e387b3c3d.slice/crio-conmon-02d30ff6c58243a35c9e1fc8ab24caa86f435f0a65fd42b4b9d0f9624cc95537.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d1dc8f1_3238_4b12_b579_ec8179bc4e2a.slice/crio-22a27310c8b2a4b2312973876da0ab42d52308596edfa2bb1a3864033939da24.scope\": RecentStats: unable to find data in memory cache]"
Nov 04 01:26:19.484505 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.484471    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp"
Nov 04 01:26:19.514872 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.514498    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv"
Nov 04 01:26:19.572948 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.572746    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/73f6a679-d6e5-45d1-a885-ebea2ed4f253-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73f6a679-d6e5-45d1-a885-ebea2ed4f253-credential-socket\") pod \"73f6a679-d6e5-45d1-a885-ebea2ed4f253\" (UID: \"73f6a679-d6e5-45d1-a885-ebea2ed4f253\") "
Nov 04 01:26:19.572948 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.572806    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3bde9301-d2f8-4820-9044-463e387b3c3d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3bde9301-d2f8-4820-9044-463e387b3c3d-istiod-ca-cert\") pod \"3bde9301-d2f8-4820-9044-463e387b3c3d\" (UID: \"3bde9301-d2f8-4820-9044-463e387b3c3d\") "
Nov 04 01:26:19.572948 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.572852    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/73f6a679-d6e5-45d1-a885-ebea2ed4f253-istio-token\" (UniqueName: \"kubernetes.io/projected/73f6a679-d6e5-45d1-a885-ebea2ed4f253-istio-token\") pod \"73f6a679-d6e5-45d1-a885-ebea2ed4f253\" (UID: \"73f6a679-d6e5-45d1-a885-ebea2ed4f253\") "
Nov 04 01:26:19.574651 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.573304    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/73f6a679-d6e5-45d1-a885-ebea2ed4f253-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73f6a679-d6e5-45d1-a885-ebea2ed4f253-istio-data\") pod \"73f6a679-d6e5-45d1-a885-ebea2ed4f253\" (UID: \"73f6a679-d6e5-45d1-a885-ebea2ed4f253\") "
Nov 04 01:26:19.574651 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.573378    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3bde9301-d2f8-4820-9044-463e387b3c3d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3bde9301-d2f8-4820-9044-463e387b3c3d-workload-certs\") pod \"3bde9301-d2f8-4820-9044-463e387b3c3d\" (UID: \"3bde9301-d2f8-4820-9044-463e387b3c3d\") "
Nov 04 01:26:19.574651 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.573421    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3bde9301-d2f8-4820-9044-463e387b3c3d-istio-token\" (UniqueName: \"kubernetes.io/projected/3bde9301-d2f8-4820-9044-463e387b3c3d-istio-token\") pod \"3bde9301-d2f8-4820-9044-463e387b3c3d\" (UID: \"3bde9301-d2f8-4820-9044-463e387b3c3d\") "
Nov 04 01:26:19.574651 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.573469    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3bde9301-d2f8-4820-9044-463e387b3c3d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bde9301-d2f8-4820-9044-463e387b3c3d-credential-socket\") pod \"3bde9301-d2f8-4820-9044-463e387b3c3d\" (UID: \"3bde9301-d2f8-4820-9044-463e387b3c3d\") "
Nov 04 01:26:19.574651 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.573516    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3bde9301-d2f8-4820-9044-463e387b3c3d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3bde9301-d2f8-4820-9044-463e387b3c3d-istio-podinfo\") pod \"3bde9301-d2f8-4820-9044-463e387b3c3d\" (UID: \"3bde9301-d2f8-4820-9044-463e387b3c3d\") "
Nov 04 01:26:19.574651 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.573572    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/73f6a679-d6e5-45d1-a885-ebea2ed4f253-kube-api-access-z828v\" (UniqueName: \"kubernetes.io/projected/73f6a679-d6e5-45d1-a885-ebea2ed4f253-kube-api-access-z828v\") pod \"73f6a679-d6e5-45d1-a885-ebea2ed4f253\" (UID: \"73f6a679-d6e5-45d1-a885-ebea2ed4f253\") "
Nov 04 01:26:19.574651 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.573703    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3bde9301-d2f8-4820-9044-463e387b3c3d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3bde9301-d2f8-4820-9044-463e387b3c3d-istio-data\") pod \"3bde9301-d2f8-4820-9044-463e387b3c3d\" (UID: \"3bde9301-d2f8-4820-9044-463e387b3c3d\") "
Nov 04 01:26:19.574651 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.573738    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3bde9301-d2f8-4820-9044-463e387b3c3d-kube-api-access-bg7bd\" (UniqueName: \"kubernetes.io/projected/3bde9301-d2f8-4820-9044-463e387b3c3d-kube-api-access-bg7bd\") pod \"3bde9301-d2f8-4820-9044-463e387b3c3d\" (UID: \"3bde9301-d2f8-4820-9044-463e387b3c3d\") "
Nov 04 01:26:19.574651 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.573768    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/73f6a679-d6e5-45d1-a885-ebea2ed4f253-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73f6a679-d6e5-45d1-a885-ebea2ed4f253-istio-envoy\") pod \"73f6a679-d6e5-45d1-a885-ebea2ed4f253\" (UID: \"73f6a679-d6e5-45d1-a885-ebea2ed4f253\") "
Nov 04 01:26:19.574651 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.573807    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3bde9301-d2f8-4820-9044-463e387b3c3d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3bde9301-d2f8-4820-9044-463e387b3c3d-istio-envoy\") pod \"3bde9301-d2f8-4820-9044-463e387b3c3d\" (UID: \"3bde9301-d2f8-4820-9044-463e387b3c3d\") "
Nov 04 01:26:19.574651 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.573849    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/73f6a679-d6e5-45d1-a885-ebea2ed4f253-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73f6a679-d6e5-45d1-a885-ebea2ed4f253-workload-socket\") pod \"73f6a679-d6e5-45d1-a885-ebea2ed4f253\" (UID: \"73f6a679-d6e5-45d1-a885-ebea2ed4f253\") "
Nov 04 01:26:19.574651 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.573953    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3bde9301-d2f8-4820-9044-463e387b3c3d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bde9301-d2f8-4820-9044-463e387b3c3d-workload-socket\") pod \"3bde9301-d2f8-4820-9044-463e387b3c3d\" (UID: \"3bde9301-d2f8-4820-9044-463e387b3c3d\") "
Nov 04 01:26:19.574651 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.574045    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/73f6a679-d6e5-45d1-a885-ebea2ed4f253-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73f6a679-d6e5-45d1-a885-ebea2ed4f253-istio-podinfo\") pod \"73f6a679-d6e5-45d1-a885-ebea2ed4f253\" (UID: \"73f6a679-d6e5-45d1-a885-ebea2ed4f253\") "
Nov 04 01:26:19.574651 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.574121    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/73f6a679-d6e5-45d1-a885-ebea2ed4f253-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73f6a679-d6e5-45d1-a885-ebea2ed4f253-workload-certs\") pod \"73f6a679-d6e5-45d1-a885-ebea2ed4f253\" (UID: \"73f6a679-d6e5-45d1-a885-ebea2ed4f253\") "
Nov 04 01:26:19.574651 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.574189    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/73f6a679-d6e5-45d1-a885-ebea2ed4f253-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73f6a679-d6e5-45d1-a885-ebea2ed4f253-istiod-ca-cert\") pod \"73f6a679-d6e5-45d1-a885-ebea2ed4f253\" (UID: \"73f6a679-d6e5-45d1-a885-ebea2ed4f253\") "
Nov 04 01:26:19.576603 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.576363    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73f6a679-d6e5-45d1-a885-ebea2ed4f253-workload-socket" pod "73f6a679-d6e5-45d1-a885-ebea2ed4f253" (UID: "73f6a679-d6e5-45d1-a885-ebea2ed4f253"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:26:19.576603 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.576393    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73f6a679-d6e5-45d1-a885-ebea2ed4f253-istiod-ca-cert" pod "73f6a679-d6e5-45d1-a885-ebea2ed4f253" (UID: "73f6a679-d6e5-45d1-a885-ebea2ed4f253"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:26:19.577076 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.576823    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73f6a679-d6e5-45d1-a885-ebea2ed4f253-workload-certs" pod "73f6a679-d6e5-45d1-a885-ebea2ed4f253" (UID: "73f6a679-d6e5-45d1-a885-ebea2ed4f253"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:26:19.577564 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.577537    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73f6a679-d6e5-45d1-a885-ebea2ed4f253-istio-data" pod "73f6a679-d6e5-45d1-a885-ebea2ed4f253" (UID: "73f6a679-d6e5-45d1-a885-ebea2ed4f253"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:26:19.577796 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.577773    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73f6a679-d6e5-45d1-a885-ebea2ed4f253-credential-socket" pod "73f6a679-d6e5-45d1-a885-ebea2ed4f253" (UID: "73f6a679-d6e5-45d1-a885-ebea2ed4f253"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:26:19.578131 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.578108    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3bde9301-d2f8-4820-9044-463e387b3c3d-istiod-ca-cert" pod "3bde9301-d2f8-4820-9044-463e387b3c3d" (UID: "3bde9301-d2f8-4820-9044-463e387b3c3d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:26:19.578433 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.578409    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bde9301-d2f8-4820-9044-463e387b3c3d-workload-socket" pod "3bde9301-d2f8-4820-9044-463e387b3c3d" (UID: "3bde9301-d2f8-4820-9044-463e387b3c3d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:26:19.578861 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.578833    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bde9301-d2f8-4820-9044-463e387b3c3d-istio-data" pod "3bde9301-d2f8-4820-9044-463e387b3c3d" (UID: "3bde9301-d2f8-4820-9044-463e387b3c3d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:26:19.579041 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.579019    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bde9301-d2f8-4820-9044-463e387b3c3d-workload-certs" pod "3bde9301-d2f8-4820-9044-463e387b3c3d" (UID: "3bde9301-d2f8-4820-9044-463e387b3c3d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:26:19.579219 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.579198    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bde9301-d2f8-4820-9044-463e387b3c3d-credential-socket" pod "3bde9301-d2f8-4820-9044-463e387b3c3d" (UID: "3bde9301-d2f8-4820-9044-463e387b3c3d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:26:19.581164 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.581073    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73f6a679-d6e5-45d1-a885-ebea2ed4f253-kube-api-access-z828v" pod "73f6a679-d6e5-45d1-a885-ebea2ed4f253" (UID: "73f6a679-d6e5-45d1-a885-ebea2ed4f253"). InnerVolumeSpecName "kube-api-access-z828v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:26:19.581164 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.581097    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/73f6a679-d6e5-45d1-a885-ebea2ed4f253-istio-podinfo" pod "73f6a679-d6e5-45d1-a885-ebea2ed4f253" (UID: "73f6a679-d6e5-45d1-a885-ebea2ed4f253"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:26:19.581798 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.581768    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bde9301-d2f8-4820-9044-463e387b3c3d-istio-envoy" pod "3bde9301-d2f8-4820-9044-463e387b3c3d" (UID: "3bde9301-d2f8-4820-9044-463e387b3c3d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:26:19.582717 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.582683    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3bde9301-d2f8-4820-9044-463e387b3c3d-istio-token" pod "3bde9301-d2f8-4820-9044-463e387b3c3d" (UID: "3bde9301-d2f8-4820-9044-463e387b3c3d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:26:19.583952 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.583684    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73f6a679-d6e5-45d1-a885-ebea2ed4f253-istio-token" pod "73f6a679-d6e5-45d1-a885-ebea2ed4f253" (UID: "73f6a679-d6e5-45d1-a885-ebea2ed4f253"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:26:19.583952 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.583777    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73f6a679-d6e5-45d1-a885-ebea2ed4f253-istio-envoy" pod "73f6a679-d6e5-45d1-a885-ebea2ed4f253" (UID: "73f6a679-d6e5-45d1-a885-ebea2ed4f253"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:26:19.583952 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.583841    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3bde9301-d2f8-4820-9044-463e387b3c3d-istio-podinfo" pod "3bde9301-d2f8-4820-9044-463e387b3c3d" (UID: "3bde9301-d2f8-4820-9044-463e387b3c3d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:26:19.584262 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.584235    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3bde9301-d2f8-4820-9044-463e387b3c3d-kube-api-access-bg7bd" pod "3bde9301-d2f8-4820-9044-463e387b3c3d" (UID: "3bde9301-d2f8-4820-9044-463e387b3c3d"). InnerVolumeSpecName "kube-api-access-bg7bd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:26:19.626452 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.626417    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7"
Nov 04 01:26:19.694713 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.676872    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-workload-socket\") pod \"7d1dc8f1-3238-4b12-b579-ec8179bc4e2a\" (UID: \"7d1dc8f1-3238-4b12-b579-ec8179bc4e2a\") "
Nov 04 01:26:19.694713 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.676998    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-istio-token\" (UniqueName: \"kubernetes.io/projected/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-istio-token\") pod \"7d1dc8f1-3238-4b12-b579-ec8179bc4e2a\" (UID: \"7d1dc8f1-3238-4b12-b579-ec8179bc4e2a\") "
Nov 04 01:26:19.694713 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.677052    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-istio-data\") pod \"7d1dc8f1-3238-4b12-b579-ec8179bc4e2a\" (UID: \"7d1dc8f1-3238-4b12-b579-ec8179bc4e2a\") "
Nov 04 01:26:19.694713 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.677140    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-workload-certs\") pod \"7d1dc8f1-3238-4b12-b579-ec8179bc4e2a\" (UID: \"7d1dc8f1-3238-4b12-b579-ec8179bc4e2a\") "
Nov 04 01:26:19.694713 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.677163    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-istio-envoy\") pod \"7d1dc8f1-3238-4b12-b579-ec8179bc4e2a\" (UID: \"7d1dc8f1-3238-4b12-b579-ec8179bc4e2a\") "
Nov 04 01:26:19.694713 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.677196    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-istiod-ca-cert\") pod \"7d1dc8f1-3238-4b12-b579-ec8179bc4e2a\" (UID: \"7d1dc8f1-3238-4b12-b579-ec8179bc4e2a\") "
Nov 04 01:26:19.694713 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.677303    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-istio-podinfo\") pod \"7d1dc8f1-3238-4b12-b579-ec8179bc4e2a\" (UID: \"7d1dc8f1-3238-4b12-b579-ec8179bc4e2a\") "
Nov 04 01:26:19.694713 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.677389    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-credential-socket\") pod \"7d1dc8f1-3238-4b12-b579-ec8179bc4e2a\" (UID: \"7d1dc8f1-3238-4b12-b579-ec8179bc4e2a\") "
Nov 04 01:26:19.694713 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.677503    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-kube-api-access-xgwvl\" (UniqueName: \"kubernetes.io/projected/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-kube-api-access-xgwvl\") pod \"7d1dc8f1-3238-4b12-b579-ec8179bc4e2a\" (UID: \"7d1dc8f1-3238-4b12-b579-ec8179bc4e2a\") "
Nov 04 01:26:19.694713 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.678523    5068 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73f6a679-d6e5-45d1-a885-ebea2ed4f253-istio-podinfo\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:19.694713 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.678552    5068 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73f6a679-d6e5-45d1-a885-ebea2ed4f253-workload-certs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:19.694713 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.678566    5068 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73f6a679-d6e5-45d1-a885-ebea2ed4f253-istiod-ca-cert\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:19.694713 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.678581    5068 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73f6a679-d6e5-45d1-a885-ebea2ed4f253-credential-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:19.694713 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.678596    5068 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3bde9301-d2f8-4820-9044-463e387b3c3d-istiod-ca-cert\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:19.694713 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.678629    5068 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73f6a679-d6e5-45d1-a885-ebea2ed4f253-istio-token\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:19.694713 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.678645    5068 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73f6a679-d6e5-45d1-a885-ebea2ed4f253-istio-data\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:19.694713 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.678659    5068 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3bde9301-d2f8-4820-9044-463e387b3c3d-workload-certs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:19.694713 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.678672    5068 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3bde9301-d2f8-4820-9044-463e387b3c3d-istio-token\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:19.694713 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.678686    5068 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bde9301-d2f8-4820-9044-463e387b3c3d-credential-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:19.694713 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.678700    5068 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3bde9301-d2f8-4820-9044-463e387b3c3d-istio-podinfo\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:19.694713 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.678713    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z828v\" (UniqueName: \"kubernetes.io/projected/73f6a679-d6e5-45d1-a885-ebea2ed4f253-kube-api-access-z828v\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:19.694713 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.678728    5068 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3bde9301-d2f8-4820-9044-463e387b3c3d-istio-data\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:19.694713 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.678745    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bg7bd\" (UniqueName: \"kubernetes.io/projected/3bde9301-d2f8-4820-9044-463e387b3c3d-kube-api-access-bg7bd\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:19.694713 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.678758    5068 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73f6a679-d6e5-45d1-a885-ebea2ed4f253-istio-envoy\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:19.694713 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.678771    5068 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3bde9301-d2f8-4820-9044-463e387b3c3d-istio-envoy\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:19.694713 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.678787    5068 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73f6a679-d6e5-45d1-a885-ebea2ed4f253-workload-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:19.694713 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.678802    5068 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bde9301-d2f8-4820-9044-463e387b3c3d-workload-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:19.694713 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.679453    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-workload-certs" pod "7d1dc8f1-3238-4b12-b579-ec8179bc4e2a" (UID: "7d1dc8f1-3238-4b12-b579-ec8179bc4e2a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:26:19.694713 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.679721    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-workload-socket" pod "7d1dc8f1-3238-4b12-b579-ec8179bc4e2a" (UID: "7d1dc8f1-3238-4b12-b579-ec8179bc4e2a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:26:19.694713 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.682057    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-istio-data" pod "7d1dc8f1-3238-4b12-b579-ec8179bc4e2a" (UID: "7d1dc8f1-3238-4b12-b579-ec8179bc4e2a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:26:19.694713 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.689701    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-credential-socket" pod "7d1dc8f1-3238-4b12-b579-ec8179bc4e2a" (UID: "7d1dc8f1-3238-4b12-b579-ec8179bc4e2a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:26:19.697186 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.695698    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-istiod-ca-cert" pod "7d1dc8f1-3238-4b12-b579-ec8179bc4e2a" (UID: "7d1dc8f1-3238-4b12-b579-ec8179bc4e2a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:26:19.710725 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.709109    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-istio-podinfo" pod "7d1dc8f1-3238-4b12-b579-ec8179bc4e2a" (UID: "7d1dc8f1-3238-4b12-b579-ec8179bc4e2a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:26:19.710725 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.709218    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-istio-token" pod "7d1dc8f1-3238-4b12-b579-ec8179bc4e2a" (UID: "7d1dc8f1-3238-4b12-b579-ec8179bc4e2a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:26:19.710725 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.710458    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-istio-envoy" pod "7d1dc8f1-3238-4b12-b579-ec8179bc4e2a" (UID: "7d1dc8f1-3238-4b12-b579-ec8179bc4e2a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:26:19.711142 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.711122    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-kube-api-access-xgwvl" pod "7d1dc8f1-3238-4b12-b579-ec8179bc4e2a" (UID: "7d1dc8f1-3238-4b12-b579-ec8179bc4e2a"). InnerVolumeSpecName "kube-api-access-xgwvl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:26:19.712921 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.712457    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp"
Nov 04 01:26:19.735859 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.735827    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94"
Nov 04 01:26:19.781428 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.781300    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/20fba30d-0142-49f8-8489-957bad3db95e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20fba30d-0142-49f8-8489-957bad3db95e-workload-socket\") pod \"20fba30d-0142-49f8-8489-957bad3db95e\" (UID: \"20fba30d-0142-49f8-8489-957bad3db95e\") "
Nov 04 01:26:19.782340 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.782290    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20fba30d-0142-49f8-8489-957bad3db95e-workload-socket" pod "20fba30d-0142-49f8-8489-957bad3db95e" (UID: "20fba30d-0142-49f8-8489-957bad3db95e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:26:19.783411 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.783369    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/20fba30d-0142-49f8-8489-957bad3db95e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20fba30d-0142-49f8-8489-957bad3db95e-istio-data\") pod \"20fba30d-0142-49f8-8489-957bad3db95e\" (UID: \"20fba30d-0142-49f8-8489-957bad3db95e\") "
Nov 04 01:26:19.783880 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.783858    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/20fba30d-0142-49f8-8489-957bad3db95e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20fba30d-0142-49f8-8489-957bad3db95e-istio-envoy\") pod \"20fba30d-0142-49f8-8489-957bad3db95e\" (UID: \"20fba30d-0142-49f8-8489-957bad3db95e\") "
Nov 04 01:26:19.791232 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.786005    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/20fba30d-0142-49f8-8489-957bad3db95e-kube-api-access-f94wk\" (UniqueName: \"kubernetes.io/projected/20fba30d-0142-49f8-8489-957bad3db95e-kube-api-access-f94wk\") pod \"20fba30d-0142-49f8-8489-957bad3db95e\" (UID: \"20fba30d-0142-49f8-8489-957bad3db95e\") "
Nov 04 01:26:19.791232 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.786074    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/20fba30d-0142-49f8-8489-957bad3db95e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20fba30d-0142-49f8-8489-957bad3db95e-credential-socket\") pod \"20fba30d-0142-49f8-8489-957bad3db95e\" (UID: \"20fba30d-0142-49f8-8489-957bad3db95e\") "
Nov 04 01:26:19.791232 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.786156    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/20fba30d-0142-49f8-8489-957bad3db95e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20fba30d-0142-49f8-8489-957bad3db95e-istio-podinfo\") pod \"20fba30d-0142-49f8-8489-957bad3db95e\" (UID: \"20fba30d-0142-49f8-8489-957bad3db95e\") "
Nov 04 01:26:19.791232 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.786228    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/20fba30d-0142-49f8-8489-957bad3db95e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20fba30d-0142-49f8-8489-957bad3db95e-istiod-ca-cert\") pod \"20fba30d-0142-49f8-8489-957bad3db95e\" (UID: \"20fba30d-0142-49f8-8489-957bad3db95e\") "
Nov 04 01:26:19.791232 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.786256    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/20fba30d-0142-49f8-8489-957bad3db95e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20fba30d-0142-49f8-8489-957bad3db95e-workload-certs\") pod \"20fba30d-0142-49f8-8489-957bad3db95e\" (UID: \"20fba30d-0142-49f8-8489-957bad3db95e\") "
Nov 04 01:26:19.791232 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.786347    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/20fba30d-0142-49f8-8489-957bad3db95e-istio-token\" (UniqueName: \"kubernetes.io/projected/20fba30d-0142-49f8-8489-957bad3db95e-istio-token\") pod \"20fba30d-0142-49f8-8489-957bad3db95e\" (UID: \"20fba30d-0142-49f8-8489-957bad3db95e\") "
Nov 04 01:26:19.791232 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.783802    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20fba30d-0142-49f8-8489-957bad3db95e-istio-data" pod "20fba30d-0142-49f8-8489-957bad3db95e" (UID: "20fba30d-0142-49f8-8489-957bad3db95e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:26:19.791232 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.787438    5068 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20fba30d-0142-49f8-8489-957bad3db95e-workload-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:19.791232 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.787462    5068 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-workload-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:19.791232 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.787478    5068 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-istio-token\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:19.791232 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.787493    5068 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20fba30d-0142-49f8-8489-957bad3db95e-istio-data\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:19.791232 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.787507    5068 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-istio-data\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:19.791232 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.787524    5068 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-workload-certs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:19.791232 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.787538    5068 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-istio-envoy\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:19.791232 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.787552    5068 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-istiod-ca-cert\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:19.791232 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.787565    5068 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-istio-podinfo\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:19.791232 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.787579    5068 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-credential-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:19.791232 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.787592    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xgwvl\" (UniqueName: \"kubernetes.io/projected/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a-kube-api-access-xgwvl\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:19.794944 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.794913    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20fba30d-0142-49f8-8489-957bad3db95e-credential-socket" pod "20fba30d-0142-49f8-8489-957bad3db95e" (UID: "20fba30d-0142-49f8-8489-957bad3db95e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:26:19.795234 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.795205    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20fba30d-0142-49f8-8489-957bad3db95e-istiod-ca-cert" pod "20fba30d-0142-49f8-8489-957bad3db95e" (UID: "20fba30d-0142-49f8-8489-957bad3db95e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:26:19.795468 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.795448    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20fba30d-0142-49f8-8489-957bad3db95e-workload-certs" pod "20fba30d-0142-49f8-8489-957bad3db95e" (UID: "20fba30d-0142-49f8-8489-957bad3db95e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:26:19.795922 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.795880    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20fba30d-0142-49f8-8489-957bad3db95e-istio-envoy" pod "20fba30d-0142-49f8-8489-957bad3db95e" (UID: "20fba30d-0142-49f8-8489-957bad3db95e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:26:19.795922 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.795912    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20fba30d-0142-49f8-8489-957bad3db95e-kube-api-access-f94wk" pod "20fba30d-0142-49f8-8489-957bad3db95e" (UID: "20fba30d-0142-49f8-8489-957bad3db95e"). InnerVolumeSpecName "kube-api-access-f94wk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:26:19.796102 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.796000    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20fba30d-0142-49f8-8489-957bad3db95e-istio-token" pod "20fba30d-0142-49f8-8489-957bad3db95e" (UID: "20fba30d-0142-49f8-8489-957bad3db95e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:26:19.800726 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.800697    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/20fba30d-0142-49f8-8489-957bad3db95e-istio-podinfo" pod "20fba30d-0142-49f8-8489-957bad3db95e" (UID: "20fba30d-0142-49f8-8489-957bad3db95e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:26:19.891229 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.891130    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/45b260dc-8b00-4f55-a649-58220ed083bb-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45b260dc-8b00-4f55-a649-58220ed083bb-workload-certs\") pod \"45b260dc-8b00-4f55-a649-58220ed083bb\" (UID: \"45b260dc-8b00-4f55-a649-58220ed083bb\") "
Nov 04 01:26:19.891229 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.891180    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/45b260dc-8b00-4f55-a649-58220ed083bb-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45b260dc-8b00-4f55-a649-58220ed083bb-istio-envoy\") pod \"45b260dc-8b00-4f55-a649-58220ed083bb\" (UID: \"45b260dc-8b00-4f55-a649-58220ed083bb\") "
Nov 04 01:26:19.891229 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.891216    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/45b260dc-8b00-4f55-a649-58220ed083bb-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45b260dc-8b00-4f55-a649-58220ed083bb-credential-socket\") pod \"45b260dc-8b00-4f55-a649-58220ed083bb\" (UID: \"45b260dc-8b00-4f55-a649-58220ed083bb\") "
Nov 04 01:26:19.891535 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.891295    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/45b260dc-8b00-4f55-a649-58220ed083bb-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45b260dc-8b00-4f55-a649-58220ed083bb-istio-data\") pod \"45b260dc-8b00-4f55-a649-58220ed083bb\" (UID: \"45b260dc-8b00-4f55-a649-58220ed083bb\") "
Nov 04 01:26:19.891535 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.891366    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/45b260dc-8b00-4f55-a649-58220ed083bb-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45b260dc-8b00-4f55-a649-58220ed083bb-istiod-ca-cert\") pod \"45b260dc-8b00-4f55-a649-58220ed083bb\" (UID: \"45b260dc-8b00-4f55-a649-58220ed083bb\") "
Nov 04 01:26:19.891535 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.891465    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/45b260dc-8b00-4f55-a649-58220ed083bb-istio-token\" (UniqueName: \"kubernetes.io/projected/45b260dc-8b00-4f55-a649-58220ed083bb-istio-token\") pod \"45b260dc-8b00-4f55-a649-58220ed083bb\" (UID: \"45b260dc-8b00-4f55-a649-58220ed083bb\") "
Nov 04 01:26:19.891759 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.891586    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45b260dc-8b00-4f55-a649-58220ed083bb-istio-data" pod "45b260dc-8b00-4f55-a649-58220ed083bb" (UID: "45b260dc-8b00-4f55-a649-58220ed083bb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:26:19.891759 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.891604    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/45b260dc-8b00-4f55-a649-58220ed083bb-kube-api-access-l9tdk\" (UniqueName: \"kubernetes.io/projected/45b260dc-8b00-4f55-a649-58220ed083bb-kube-api-access-l9tdk\") pod \"45b260dc-8b00-4f55-a649-58220ed083bb\" (UID: \"45b260dc-8b00-4f55-a649-58220ed083bb\") "
Nov 04 01:26:19.891759 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.891681    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45b260dc-8b00-4f55-a649-58220ed083bb-credential-socket" pod "45b260dc-8b00-4f55-a649-58220ed083bb" (UID: "45b260dc-8b00-4f55-a649-58220ed083bb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:26:19.891759 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.891707    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/45b260dc-8b00-4f55-a649-58220ed083bb-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45b260dc-8b00-4f55-a649-58220ed083bb-workload-socket\") pod \"45b260dc-8b00-4f55-a649-58220ed083bb\" (UID: \"45b260dc-8b00-4f55-a649-58220ed083bb\") "
Nov 04 01:26:19.892018 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.891823    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/45b260dc-8b00-4f55-a649-58220ed083bb-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45b260dc-8b00-4f55-a649-58220ed083bb-istio-podinfo\") pod \"45b260dc-8b00-4f55-a649-58220ed083bb\" (UID: \"45b260dc-8b00-4f55-a649-58220ed083bb\") "
Nov 04 01:26:19.892074 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.892012    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45b260dc-8b00-4f55-a649-58220ed083bb-istiod-ca-cert" pod "45b260dc-8b00-4f55-a649-58220ed083bb" (UID: "45b260dc-8b00-4f55-a649-58220ed083bb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:26:19.892129 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.892112    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45b260dc-8b00-4f55-a649-58220ed083bb-workload-certs" pod "45b260dc-8b00-4f55-a649-58220ed083bb" (UID: "45b260dc-8b00-4f55-a649-58220ed083bb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:26:19.892726 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.892701    5068 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45b260dc-8b00-4f55-a649-58220ed083bb-workload-certs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:19.892866 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.892729    5068 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45b260dc-8b00-4f55-a649-58220ed083bb-credential-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:19.892866 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.892745    5068 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45b260dc-8b00-4f55-a649-58220ed083bb-istio-data\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:19.892866 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.892758    5068 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20fba30d-0142-49f8-8489-957bad3db95e-istio-envoy\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:19.892866 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.892772    5068 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45b260dc-8b00-4f55-a649-58220ed083bb-istiod-ca-cert\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:19.892866 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.892791    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f94wk\" (UniqueName: \"kubernetes.io/projected/20fba30d-0142-49f8-8489-957bad3db95e-kube-api-access-f94wk\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:19.892866 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.892804    5068 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20fba30d-0142-49f8-8489-957bad3db95e-credential-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:19.892866 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.892817    5068 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20fba30d-0142-49f8-8489-957bad3db95e-istio-podinfo\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:19.892866 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.892831    5068 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20fba30d-0142-49f8-8489-957bad3db95e-istiod-ca-cert\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:19.892866 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.892581    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45b260dc-8b00-4f55-a649-58220ed083bb-workload-socket" pod "45b260dc-8b00-4f55-a649-58220ed083bb" (UID: "45b260dc-8b00-4f55-a649-58220ed083bb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:26:19.892866 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.892844    5068 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20fba30d-0142-49f8-8489-957bad3db95e-workload-certs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:19.892866 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.892858    5068 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20fba30d-0142-49f8-8489-957bad3db95e-istio-token\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:19.895166 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.895138    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45b260dc-8b00-4f55-a649-58220ed083bb-istio-envoy" pod "45b260dc-8b00-4f55-a649-58220ed083bb" (UID: "45b260dc-8b00-4f55-a649-58220ed083bb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:26:19.895258 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.895208    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45b260dc-8b00-4f55-a649-58220ed083bb-kube-api-access-l9tdk" pod "45b260dc-8b00-4f55-a649-58220ed083bb" (UID: "45b260dc-8b00-4f55-a649-58220ed083bb"). InnerVolumeSpecName "kube-api-access-l9tdk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:26:19.895623 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.895568    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/45b260dc-8b00-4f55-a649-58220ed083bb-istio-podinfo" pod "45b260dc-8b00-4f55-a649-58220ed083bb" (UID: "45b260dc-8b00-4f55-a649-58220ed083bb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:26:19.896357 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.896334    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45b260dc-8b00-4f55-a649-58220ed083bb-istio-token" pod "45b260dc-8b00-4f55-a649-58220ed083bb" (UID: "45b260dc-8b00-4f55-a649-58220ed083bb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:26:19.994383 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.994347    5068 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45b260dc-8b00-4f55-a649-58220ed083bb-istio-envoy\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:19.994383 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.994376    5068 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45b260dc-8b00-4f55-a649-58220ed083bb-istio-token\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:19.994383 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.994387    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l9tdk\" (UniqueName: \"kubernetes.io/projected/45b260dc-8b00-4f55-a649-58220ed083bb-kube-api-access-l9tdk\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:19.994725 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.994396    5068 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45b260dc-8b00-4f55-a649-58220ed083bb-workload-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:19.994725 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:19.994406    5068 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45b260dc-8b00-4f55-a649-58220ed083bb-istio-podinfo\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:20.067670 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:20.067559    5068 generic.go:356] "Generic (PLEG): container finished" podID="20fba30d-0142-49f8-8489-957bad3db95e" containerID="40b5f1c0fb186e12538dfe099ccece193a8e0d5aa1ec63ce5734c65849bba32d" exitCode=0
Nov 04 01:26:20.067670 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:20.067659    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp"
Nov 04 01:26:20.067934 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:20.067742    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp" event={"ID":"20fba30d-0142-49f8-8489-957bad3db95e","Type":"ContainerDied","Data":"40b5f1c0fb186e12538dfe099ccece193a8e0d5aa1ec63ce5734c65849bba32d"}
Nov 04 01:26:20.067934 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:20.067795    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp" event={"ID":"20fba30d-0142-49f8-8489-957bad3db95e","Type":"ContainerDied","Data":"e657c85e63f817eabb701068b7d9bced58da25d0dc1874fcfeee0d84494083bf"}
Nov 04 01:26:20.067934 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:20.067824    5068 scope.go:122] "RemoveContainer" containerID="40b5f1c0fb186e12538dfe099ccece193a8e0d5aa1ec63ce5734c65849bba32d"
Nov 04 01:26:20.071743 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:20.070967    5068 generic.go:356] "Generic (PLEG): container finished" podID="73f6a679-d6e5-45d1-a885-ebea2ed4f253" containerID="e7d6118c5c88d2a64ed465cc3ec0a56bb3879bcf99cc063f749d792e5d7ed748" exitCode=0
Nov 04 01:26:20.071743 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:20.071072    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp" event={"ID":"73f6a679-d6e5-45d1-a885-ebea2ed4f253","Type":"ContainerDied","Data":"e7d6118c5c88d2a64ed465cc3ec0a56bb3879bcf99cc063f749d792e5d7ed748"}
Nov 04 01:26:20.071743 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:20.071117    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp" event={"ID":"73f6a679-d6e5-45d1-a885-ebea2ed4f253","Type":"ContainerDied","Data":"4021509b22dd20d8bb5caffbbfd20d43a59130465050dfeb0f3eae5c4245131a"}
Nov 04 01:26:20.071743 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:20.071195    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp"
Nov 04 01:26:20.086125 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:20.085589    5068 generic.go:356] "Generic (PLEG): container finished" podID="3bde9301-d2f8-4820-9044-463e387b3c3d" containerID="02d30ff6c58243a35c9e1fc8ab24caa86f435f0a65fd42b4b9d0f9624cc95537" exitCode=0
Nov 04 01:26:20.086125 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:20.086091    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv"
Nov 04 01:26:20.086400 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:20.086087    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv" event={"ID":"3bde9301-d2f8-4820-9044-463e387b3c3d","Type":"ContainerDied","Data":"02d30ff6c58243a35c9e1fc8ab24caa86f435f0a65fd42b4b9d0f9624cc95537"}
Nov 04 01:26:20.086526 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:20.086440    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv" event={"ID":"3bde9301-d2f8-4820-9044-463e387b3c3d","Type":"ContainerDied","Data":"52875d83d3168be23e945f667fced37b28e2ad23c02dd6d4f8984aa03a4959a9"}
Nov 04 01:26:20.088864 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:20.088836    5068 generic.go:356] "Generic (PLEG): container finished" podID="7d1dc8f1-3238-4b12-b579-ec8179bc4e2a" containerID="22a27310c8b2a4b2312973876da0ab42d52308596edfa2bb1a3864033939da24" exitCode=0
Nov 04 01:26:20.089072 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:20.089050    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7" event={"ID":"7d1dc8f1-3238-4b12-b579-ec8179bc4e2a","Type":"ContainerDied","Data":"22a27310c8b2a4b2312973876da0ab42d52308596edfa2bb1a3864033939da24"}
Nov 04 01:26:20.089150 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:20.089105    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7" event={"ID":"7d1dc8f1-3238-4b12-b579-ec8179bc4e2a","Type":"ContainerDied","Data":"f499661d4f10b5e4546f357483e519f23d6eda941d34bd01aca47dfa15dff956"}
Nov 04 01:26:20.089211 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:20.089195    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7"
Nov 04 01:26:20.091980 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:20.091007    5068 scope.go:122] "RemoveContainer" containerID="40b5f1c0fb186e12538dfe099ccece193a8e0d5aa1ec63ce5734c65849bba32d"
Nov 04 01:26:20.091980 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:26:20.091277    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40b5f1c0fb186e12538dfe099ccece193a8e0d5aa1ec63ce5734c65849bba32d\": container with ID starting with 40b5f1c0fb186e12538dfe099ccece193a8e0d5aa1ec63ce5734c65849bba32d not found: ID does not exist" containerID="40b5f1c0fb186e12538dfe099ccece193a8e0d5aa1ec63ce5734c65849bba32d"
Nov 04 01:26:20.091980 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:20.091304    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40b5f1c0fb186e12538dfe099ccece193a8e0d5aa1ec63ce5734c65849bba32d"} err="failed to get container status \"40b5f1c0fb186e12538dfe099ccece193a8e0d5aa1ec63ce5734c65849bba32d\": rpc error: code = NotFound desc = could not find container \"40b5f1c0fb186e12538dfe099ccece193a8e0d5aa1ec63ce5734c65849bba32d\": container with ID starting with 40b5f1c0fb186e12538dfe099ccece193a8e0d5aa1ec63ce5734c65849bba32d not found: ID does not exist"
Nov 04 01:26:20.091980 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:20.091321    5068 scope.go:122] "RemoveContainer" containerID="e7d6118c5c88d2a64ed465cc3ec0a56bb3879bcf99cc063f749d792e5d7ed748"
Nov 04 01:26:20.096657 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:20.094347    5068 generic.go:356] "Generic (PLEG): container finished" podID="45b260dc-8b00-4f55-a649-58220ed083bb" containerID="ab4ccb934f6a263df1ac0d8019a4d7b4957b6b7e4a02be7117c68056ba5131e2" exitCode=0
Nov 04 01:26:20.096657 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:20.094419    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94" event={"ID":"45b260dc-8b00-4f55-a649-58220ed083bb","Type":"ContainerDied","Data":"ab4ccb934f6a263df1ac0d8019a4d7b4957b6b7e4a02be7117c68056ba5131e2"}
Nov 04 01:26:20.096657 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:20.094453    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94" event={"ID":"45b260dc-8b00-4f55-a649-58220ed083bb","Type":"ContainerDied","Data":"039dbbe110ece46360f374831722f2898778f87bc0e1fedf01b2c60f02f80306"}
Nov 04 01:26:20.096657 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:20.094511    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94"
Nov 04 01:26:20.177942 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:20.171992    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp"]
Nov 04 01:26:20.182261 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:20.181802    5068 scope.go:122] "RemoveContainer" containerID="e7d6118c5c88d2a64ed465cc3ec0a56bb3879bcf99cc063f749d792e5d7ed748"
Nov 04 01:26:20.194596 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:26:20.182362    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7d6118c5c88d2a64ed465cc3ec0a56bb3879bcf99cc063f749d792e5d7ed748\": container with ID starting with e7d6118c5c88d2a64ed465cc3ec0a56bb3879bcf99cc063f749d792e5d7ed748 not found: ID does not exist" containerID="e7d6118c5c88d2a64ed465cc3ec0a56bb3879bcf99cc063f749d792e5d7ed748"
Nov 04 01:26:20.194596 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:20.194347    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7d6118c5c88d2a64ed465cc3ec0a56bb3879bcf99cc063f749d792e5d7ed748"} err="failed to get container status \"e7d6118c5c88d2a64ed465cc3ec0a56bb3879bcf99cc063f749d792e5d7ed748\": rpc error: code = NotFound desc = could not find container \"e7d6118c5c88d2a64ed465cc3ec0a56bb3879bcf99cc063f749d792e5d7ed748\": container with ID starting with e7d6118c5c88d2a64ed465cc3ec0a56bb3879bcf99cc063f749d792e5d7ed748 not found: ID does not exist"
Nov 04 01:26:20.194596 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:20.194376    5068 scope.go:122] "RemoveContainer" containerID="02d30ff6c58243a35c9e1fc8ab24caa86f435f0a65fd42b4b9d0f9624cc95537"
Nov 04 01:26:20.204183 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:20.204139    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-gjqrp"]
Nov 04 01:26:20.216759 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:20.216718    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp"]
Nov 04 01:26:20.222520 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:20.222495    5068 scope.go:122] "RemoveContainer" containerID="02d30ff6c58243a35c9e1fc8ab24caa86f435f0a65fd42b4b9d0f9624cc95537"
Nov 04 01:26:20.224054 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:26:20.224028    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02d30ff6c58243a35c9e1fc8ab24caa86f435f0a65fd42b4b9d0f9624cc95537\": container with ID starting with 02d30ff6c58243a35c9e1fc8ab24caa86f435f0a65fd42b4b9d0f9624cc95537 not found: ID does not exist" containerID="02d30ff6c58243a35c9e1fc8ab24caa86f435f0a65fd42b4b9d0f9624cc95537"
Nov 04 01:26:20.224156 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:20.224062    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02d30ff6c58243a35c9e1fc8ab24caa86f435f0a65fd42b4b9d0f9624cc95537"} err="failed to get container status \"02d30ff6c58243a35c9e1fc8ab24caa86f435f0a65fd42b4b9d0f9624cc95537\": rpc error: code = NotFound desc = could not find container \"02d30ff6c58243a35c9e1fc8ab24caa86f435f0a65fd42b4b9d0f9624cc95537\": container with ID starting with 02d30ff6c58243a35c9e1fc8ab24caa86f435f0a65fd42b4b9d0f9624cc95537 not found: ID does not exist"
Nov 04 01:26:20.224156 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:20.224082    5068 scope.go:122] "RemoveContainer" containerID="22a27310c8b2a4b2312973876da0ab42d52308596edfa2bb1a3864033939da24"
Nov 04 01:26:20.226879 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:20.226850    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-zndcp"]
Nov 04 01:26:20.236737 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:20.236708    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94"]
Nov 04 01:26:20.237300 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:20.237284    5068 scope.go:122] "RemoveContainer" containerID="22a27310c8b2a4b2312973876da0ab42d52308596edfa2bb1a3864033939da24"
Nov 04 01:26:20.237644 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:26:20.237605    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22a27310c8b2a4b2312973876da0ab42d52308596edfa2bb1a3864033939da24\": container with ID starting with 22a27310c8b2a4b2312973876da0ab42d52308596edfa2bb1a3864033939da24 not found: ID does not exist" containerID="22a27310c8b2a4b2312973876da0ab42d52308596edfa2bb1a3864033939da24"
Nov 04 01:26:20.237752 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:20.237652    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22a27310c8b2a4b2312973876da0ab42d52308596edfa2bb1a3864033939da24"} err="failed to get container status \"22a27310c8b2a4b2312973876da0ab42d52308596edfa2bb1a3864033939da24\": rpc error: code = NotFound desc = could not find container \"22a27310c8b2a4b2312973876da0ab42d52308596edfa2bb1a3864033939da24\": container with ID starting with 22a27310c8b2a4b2312973876da0ab42d52308596edfa2bb1a3864033939da24 not found: ID does not exist"
Nov 04 01:26:20.237752 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:20.237670    5068 scope.go:122] "RemoveContainer" containerID="ab4ccb934f6a263df1ac0d8019a4d7b4957b6b7e4a02be7117c68056ba5131e2"
Nov 04 01:26:20.243265 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:20.243243    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-g9x94"]
Nov 04 01:26:20.266637 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:20.266462    5068 scope.go:122] "RemoveContainer" containerID="ab4ccb934f6a263df1ac0d8019a4d7b4957b6b7e4a02be7117c68056ba5131e2"
Nov 04 01:26:20.266985 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:26:20.266906    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab4ccb934f6a263df1ac0d8019a4d7b4957b6b7e4a02be7117c68056ba5131e2\": container with ID starting with ab4ccb934f6a263df1ac0d8019a4d7b4957b6b7e4a02be7117c68056ba5131e2 not found: ID does not exist" containerID="ab4ccb934f6a263df1ac0d8019a4d7b4957b6b7e4a02be7117c68056ba5131e2"
Nov 04 01:26:20.266985 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:20.266946    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab4ccb934f6a263df1ac0d8019a4d7b4957b6b7e4a02be7117c68056ba5131e2"} err="failed to get container status \"ab4ccb934f6a263df1ac0d8019a4d7b4957b6b7e4a02be7117c68056ba5131e2\": rpc error: code = NotFound desc = could not find container \"ab4ccb934f6a263df1ac0d8019a4d7b4957b6b7e4a02be7117c68056ba5131e2\": container with ID starting with ab4ccb934f6a263df1ac0d8019a4d7b4957b6b7e4a02be7117c68056ba5131e2 not found: ID does not exist"
Nov 04 01:26:20.267620 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:20.267587    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7"]
Nov 04 01:26:20.277084 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:20.277050    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-b87d7"]
Nov 04 01:26:20.294600 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:20.294572    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv"]
Nov 04 01:26:20.300080 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:20.300057    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-tddpv"]
Nov 04 01:26:22.114963 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:22.114926    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="20fba30d-0142-49f8-8489-957bad3db95e" path="/var/lib/kubelet/pods/20fba30d-0142-49f8-8489-957bad3db95e/volumes"
Nov 04 01:26:22.115733 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:22.115704    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3bde9301-d2f8-4820-9044-463e387b3c3d" path="/var/lib/kubelet/pods/3bde9301-d2f8-4820-9044-463e387b3c3d/volumes"
Nov 04 01:26:22.116235 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:22.116215    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="45b260dc-8b00-4f55-a649-58220ed083bb" path="/var/lib/kubelet/pods/45b260dc-8b00-4f55-a649-58220ed083bb/volumes"
Nov 04 01:26:22.117056 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:22.117035    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="73f6a679-d6e5-45d1-a885-ebea2ed4f253" path="/var/lib/kubelet/pods/73f6a679-d6e5-45d1-a885-ebea2ed4f253/volumes"
Nov 04 01:26:22.117532 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:22.117514    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7d1dc8f1-3238-4b12-b579-ec8179bc4e2a" path="/var/lib/kubelet/pods/7d1dc8f1-3238-4b12-b579-ec8179bc4e2a/volumes"
Nov 04 01:26:24.482164 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:24.482129    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq"]
Nov 04 01:26:24.489639 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:24.489596    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx"]
Nov 04 01:26:24.504597 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:24.504563    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n"]
Nov 04 01:26:35.088687 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:35.088642    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-6e609c-8496b8f648-d5c9r"]
Nov 04 01:26:35.102306 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:35.102274    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-6e609c-58d54df979-k4wrw"]
Nov 04 01:26:39.506248 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:39.506181    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq" podUID="b87aafac-9ab8-40e5-8d08-72eafc4187dd" containerName="main" containerID="cri-o://9c26cad512c9c16f8fe19b6bd028eeedd6857d76eafda5e5011abd2ccabf090d" gracePeriod=45
Nov 04 01:26:39.515284 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:39.515247    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx" podUID="46d4ec7b-558d-4fb5-8011-177760a3291c" containerName="main" containerID="cri-o://7b08752aa3a0af5715424d861acaf320decde640a2d39d3d63ab1725ee78098b" gracePeriod=45
Nov 04 01:26:39.526848 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:39.526790    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n" podUID="b3223811-a857-4279-a671-189e65383481" containerName="main" containerID="cri-o://b0a18eed313cc80f07f5c24837dfced89714fe177b1cc708ed77b03144073ffc" gracePeriod=45
Nov 04 01:26:39.878428 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:39.878395    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n"
Nov 04 01:26:39.893732 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:39.893701    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx"
Nov 04 01:26:39.917345 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:39.917315    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq"
Nov 04 01:26:39.935200 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:39.935169    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b3223811-a857-4279-a671-189e65383481-home\" (UniqueName: \"kubernetes.io/empty-dir/b3223811-a857-4279-a671-189e65383481-home\") pod \"b3223811-a857-4279-a671-189e65383481\" (UID: \"b3223811-a857-4279-a671-189e65383481\") "
Nov 04 01:26:39.935805 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:39.935780    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b3223811-a857-4279-a671-189e65383481-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3223811-a857-4279-a671-189e65383481-model-cache\") pod \"b3223811-a857-4279-a671-189e65383481\" (UID: \"b3223811-a857-4279-a671-189e65383481\") "
Nov 04 01:26:39.936343 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:39.936325    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/46d4ec7b-558d-4fb5-8011-177760a3291c-tls-certs\" (UniqueName: \"kubernetes.io/secret/46d4ec7b-558d-4fb5-8011-177760a3291c-tls-certs\") pod \"46d4ec7b-558d-4fb5-8011-177760a3291c\" (UID: \"46d4ec7b-558d-4fb5-8011-177760a3291c\") "
Nov 04 01:26:39.936918 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:39.936900    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/46d4ec7b-558d-4fb5-8011-177760a3291c-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46d4ec7b-558d-4fb5-8011-177760a3291c-tmp-dir\") pod \"46d4ec7b-558d-4fb5-8011-177760a3291c\" (UID: \"46d4ec7b-558d-4fb5-8011-177760a3291c\") "
Nov 04 01:26:39.937067 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:39.937053    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/46d4ec7b-558d-4fb5-8011-177760a3291c-kube-api-access-qc267\" (UniqueName: \"kubernetes.io/projected/46d4ec7b-558d-4fb5-8011-177760a3291c-kube-api-access-qc267\") pod \"46d4ec7b-558d-4fb5-8011-177760a3291c\" (UID: \"46d4ec7b-558d-4fb5-8011-177760a3291c\") "
Nov 04 01:26:39.937230 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:39.937216    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/46d4ec7b-558d-4fb5-8011-177760a3291c-dshm\" (UniqueName: \"kubernetes.io/empty-dir/46d4ec7b-558d-4fb5-8011-177760a3291c-dshm\") pod \"46d4ec7b-558d-4fb5-8011-177760a3291c\" (UID: \"46d4ec7b-558d-4fb5-8011-177760a3291c\") "
Nov 04 01:26:39.937359 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:39.937343    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/46d4ec7b-558d-4fb5-8011-177760a3291c-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46d4ec7b-558d-4fb5-8011-177760a3291c-model-cache\") pod \"46d4ec7b-558d-4fb5-8011-177760a3291c\" (UID: \"46d4ec7b-558d-4fb5-8011-177760a3291c\") "
Nov 04 01:26:39.937637 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:39.937570    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b3223811-a857-4279-a671-189e65383481-dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3223811-a857-4279-a671-189e65383481-dshm\") pod \"b3223811-a857-4279-a671-189e65383481\" (UID: \"b3223811-a857-4279-a671-189e65383481\") "
Nov 04 01:26:39.937985 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:39.937967    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b3223811-a857-4279-a671-189e65383481-tls-certs\" (UniqueName: \"kubernetes.io/secret/b3223811-a857-4279-a671-189e65383481-tls-certs\") pod \"b3223811-a857-4279-a671-189e65383481\" (UID: \"b3223811-a857-4279-a671-189e65383481\") "
Nov 04 01:26:39.938108 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:39.938094    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b3223811-a857-4279-a671-189e65383481-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3223811-a857-4279-a671-189e65383481-tmp-dir\") pod \"b3223811-a857-4279-a671-189e65383481\" (UID: \"b3223811-a857-4279-a671-189e65383481\") "
Nov 04 01:26:39.938238 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:39.938226    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/46d4ec7b-558d-4fb5-8011-177760a3291c-home\" (UniqueName: \"kubernetes.io/empty-dir/46d4ec7b-558d-4fb5-8011-177760a3291c-home\") pod \"46d4ec7b-558d-4fb5-8011-177760a3291c\" (UID: \"46d4ec7b-558d-4fb5-8011-177760a3291c\") "
Nov 04 01:26:39.938417 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:39.938400    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b3223811-a857-4279-a671-189e65383481-kube-api-access-4m8n6\" (UniqueName: \"kubernetes.io/projected/b3223811-a857-4279-a671-189e65383481-kube-api-access-4m8n6\") pod \"b3223811-a857-4279-a671-189e65383481\" (UID: \"b3223811-a857-4279-a671-189e65383481\") "
Nov 04 01:26:39.945338 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:39.935697    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3223811-a857-4279-a671-189e65383481-home" pod "b3223811-a857-4279-a671-189e65383481" (UID: "b3223811-a857-4279-a671-189e65383481"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:26:39.945445 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:39.936221    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3223811-a857-4279-a671-189e65383481-model-cache" pod "b3223811-a857-4279-a671-189e65383481" (UID: "b3223811-a857-4279-a671-189e65383481"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:26:39.945445 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:39.945015    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46d4ec7b-558d-4fb5-8011-177760a3291c-tmp-dir" pod "46d4ec7b-558d-4fb5-8011-177760a3291c" (UID: "46d4ec7b-558d-4fb5-8011-177760a3291c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:26:39.945445 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:39.945148    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3223811-a857-4279-a671-189e65383481-tmp-dir" pod "b3223811-a857-4279-a671-189e65383481" (UID: "b3223811-a857-4279-a671-189e65383481"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:26:39.945676 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:39.945624    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46d4ec7b-558d-4fb5-8011-177760a3291c-model-cache" pod "46d4ec7b-558d-4fb5-8011-177760a3291c" (UID: "46d4ec7b-558d-4fb5-8011-177760a3291c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:26:39.945819 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:39.945800    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46d4ec7b-558d-4fb5-8011-177760a3291c-home" pod "46d4ec7b-558d-4fb5-8011-177760a3291c" (UID: "46d4ec7b-558d-4fb5-8011-177760a3291c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:26:39.966965 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:39.966907    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b3223811-a857-4279-a671-189e65383481-tls-certs" pod "b3223811-a857-4279-a671-189e65383481" (UID: "b3223811-a857-4279-a671-189e65383481"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:26:39.966965 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:39.966957    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3223811-a857-4279-a671-189e65383481-dshm" pod "b3223811-a857-4279-a671-189e65383481" (UID: "b3223811-a857-4279-a671-189e65383481"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:26:39.967164 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:39.967038    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46d4ec7b-558d-4fb5-8011-177760a3291c-kube-api-access-qc267" pod "46d4ec7b-558d-4fb5-8011-177760a3291c" (UID: "46d4ec7b-558d-4fb5-8011-177760a3291c"). InnerVolumeSpecName "kube-api-access-qc267". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:26:39.967164 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:39.967046    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46d4ec7b-558d-4fb5-8011-177760a3291c-tls-certs" pod "46d4ec7b-558d-4fb5-8011-177760a3291c" (UID: "46d4ec7b-558d-4fb5-8011-177760a3291c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:26:39.967246 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:39.967196    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46d4ec7b-558d-4fb5-8011-177760a3291c-dshm" pod "46d4ec7b-558d-4fb5-8011-177760a3291c" (UID: "46d4ec7b-558d-4fb5-8011-177760a3291c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:26:39.969108 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:39.969078    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3223811-a857-4279-a671-189e65383481-kube-api-access-4m8n6" pod "b3223811-a857-4279-a671-189e65383481" (UID: "b3223811-a857-4279-a671-189e65383481"). InnerVolumeSpecName "kube-api-access-4m8n6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:26:40.046777 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.046684    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b87aafac-9ab8-40e5-8d08-72eafc4187dd-tls-certs\" (UniqueName: \"kubernetes.io/secret/b87aafac-9ab8-40e5-8d08-72eafc4187dd-tls-certs\") pod \"b87aafac-9ab8-40e5-8d08-72eafc4187dd\" (UID: \"b87aafac-9ab8-40e5-8d08-72eafc4187dd\") "
Nov 04 01:26:40.047060 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.047035    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b87aafac-9ab8-40e5-8d08-72eafc4187dd-home\" (UniqueName: \"kubernetes.io/empty-dir/b87aafac-9ab8-40e5-8d08-72eafc4187dd-home\") pod \"b87aafac-9ab8-40e5-8d08-72eafc4187dd\" (UID: \"b87aafac-9ab8-40e5-8d08-72eafc4187dd\") "
Nov 04 01:26:40.047298 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.047277    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b87aafac-9ab8-40e5-8d08-72eafc4187dd-kube-api-access-mpsrj\" (UniqueName: \"kubernetes.io/projected/b87aafac-9ab8-40e5-8d08-72eafc4187dd-kube-api-access-mpsrj\") pod \"b87aafac-9ab8-40e5-8d08-72eafc4187dd\" (UID: \"b87aafac-9ab8-40e5-8d08-72eafc4187dd\") "
Nov 04 01:26:40.047452 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.047439    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b87aafac-9ab8-40e5-8d08-72eafc4187dd-dshm\" (UniqueName: \"kubernetes.io/empty-dir/b87aafac-9ab8-40e5-8d08-72eafc4187dd-dshm\") pod \"b87aafac-9ab8-40e5-8d08-72eafc4187dd\" (UID: \"b87aafac-9ab8-40e5-8d08-72eafc4187dd\") "
Nov 04 01:26:40.047658 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.047629    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b87aafac-9ab8-40e5-8d08-72eafc4187dd-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b87aafac-9ab8-40e5-8d08-72eafc4187dd-model-cache\") pod \"b87aafac-9ab8-40e5-8d08-72eafc4187dd\" (UID: \"b87aafac-9ab8-40e5-8d08-72eafc4187dd\") "
Nov 04 01:26:40.047799 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.047723    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b87aafac-9ab8-40e5-8d08-72eafc4187dd-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b87aafac-9ab8-40e5-8d08-72eafc4187dd-tmp-dir\") pod \"b87aafac-9ab8-40e5-8d08-72eafc4187dd\" (UID: \"b87aafac-9ab8-40e5-8d08-72eafc4187dd\") "
Nov 04 01:26:40.047962 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.047931    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b87aafac-9ab8-40e5-8d08-72eafc4187dd-home" pod "b87aafac-9ab8-40e5-8d08-72eafc4187dd" (UID: "b87aafac-9ab8-40e5-8d08-72eafc4187dd"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:26:40.048109 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.048090    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b87aafac-9ab8-40e5-8d08-72eafc4187dd-tmp-dir" pod "b87aafac-9ab8-40e5-8d08-72eafc4187dd" (UID: "b87aafac-9ab8-40e5-8d08-72eafc4187dd"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:26:40.048178 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.048112    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b87aafac-9ab8-40e5-8d08-72eafc4187dd-model-cache" pod "b87aafac-9ab8-40e5-8d08-72eafc4187dd" (UID: "b87aafac-9ab8-40e5-8d08-72eafc4187dd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:26:40.051417 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.051216    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b87aafac-9ab8-40e5-8d08-72eafc4187dd-dshm" pod "b87aafac-9ab8-40e5-8d08-72eafc4187dd" (UID: "b87aafac-9ab8-40e5-8d08-72eafc4187dd"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:26:40.051595 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.051563    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b87aafac-9ab8-40e5-8d08-72eafc4187dd-tls-certs" pod "b87aafac-9ab8-40e5-8d08-72eafc4187dd" (UID: "b87aafac-9ab8-40e5-8d08-72eafc4187dd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:26:40.052184 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.052162    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b87aafac-9ab8-40e5-8d08-72eafc4187dd-kube-api-access-mpsrj" pod "b87aafac-9ab8-40e5-8d08-72eafc4187dd" (UID: "b87aafac-9ab8-40e5-8d08-72eafc4187dd"). InnerVolumeSpecName "kube-api-access-mpsrj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:26:40.052297 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.052273    5068 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/46d4ec7b-558d-4fb5-8011-177760a3291c-home\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:40.052361 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.052302    5068 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b87aafac-9ab8-40e5-8d08-72eafc4187dd-dshm\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:40.052361 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.052339    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4m8n6\" (UniqueName: \"kubernetes.io/projected/b3223811-a857-4279-a671-189e65383481-kube-api-access-4m8n6\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:40.052361 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.052356    5068 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3223811-a857-4279-a671-189e65383481-home\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:40.052504 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.052374    5068 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3223811-a857-4279-a671-189e65383481-model-cache\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:40.052504 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.052387    5068 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46d4ec7b-558d-4fb5-8011-177760a3291c-tls-certs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:40.052504 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.052400    5068 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b87aafac-9ab8-40e5-8d08-72eafc4187dd-model-cache\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:40.052504 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.052412    5068 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46d4ec7b-558d-4fb5-8011-177760a3291c-tmp-dir\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:40.052504 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.052424    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qc267\" (UniqueName: \"kubernetes.io/projected/46d4ec7b-558d-4fb5-8011-177760a3291c-kube-api-access-qc267\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:40.052504 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.052436    5068 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b87aafac-9ab8-40e5-8d08-72eafc4187dd-tmp-dir\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:40.052504 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.052448    5068 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/46d4ec7b-558d-4fb5-8011-177760a3291c-dshm\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:40.052504 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.052459    5068 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46d4ec7b-558d-4fb5-8011-177760a3291c-model-cache\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:40.052504 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.052470    5068 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3223811-a857-4279-a671-189e65383481-dshm\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:40.052504 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.052482    5068 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b87aafac-9ab8-40e5-8d08-72eafc4187dd-tls-certs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:40.052504 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.052493    5068 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b87aafac-9ab8-40e5-8d08-72eafc4187dd-home\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:40.052504 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.052504    5068 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3223811-a857-4279-a671-189e65383481-tls-certs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:40.053093 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.052516    5068 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3223811-a857-4279-a671-189e65383481-tmp-dir\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:40.089362 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.089322    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-6e609c-8496b8f648-d5c9r" podUID="27872680-e51b-4785-bd2e-6912cb0fe64f" containerName="payload-processing" containerID="cri-o://0efc55a6dfc36ae39ae7d99391dab701b1c3b4aacd70cdca5793c3194110480e" gracePeriod=25
Nov 04 01:26:40.102672 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.102640    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6e609c-58d54df979-k4wrw" podUID="c3863ed4-3c10-453c-b11d-c66848ed2900" containerName="payload-pre-processing" containerID="cri-o://8bbd24168d1592314f8d30c8d589e104fda45c635cb5fe74bd9d4b7864692745" gracePeriod=25
Nov 04 01:26:40.153681 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.153645    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mpsrj\" (UniqueName: \"kubernetes.io/projected/b87aafac-9ab8-40e5-8d08-72eafc4187dd-kube-api-access-mpsrj\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:40.322536 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.319120    5068 generic.go:356] "Generic (PLEG): container finished" podID="b87aafac-9ab8-40e5-8d08-72eafc4187dd" containerID="9c26cad512c9c16f8fe19b6bd028eeedd6857d76eafda5e5011abd2ccabf090d" exitCode=0
Nov 04 01:26:40.322536 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.319237    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq" event={"ID":"b87aafac-9ab8-40e5-8d08-72eafc4187dd","Type":"ContainerDied","Data":"9c26cad512c9c16f8fe19b6bd028eeedd6857d76eafda5e5011abd2ccabf090d"}
Nov 04 01:26:40.322536 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.319269    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq" event={"ID":"b87aafac-9ab8-40e5-8d08-72eafc4187dd","Type":"ContainerDied","Data":"eb600abab916071dee0555f16c69cdfe4b7b87825df79f8cd412851f33cbc374"}
Nov 04 01:26:40.322536 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.319285    5068 scope.go:122] "RemoveContainer" containerID="9c26cad512c9c16f8fe19b6bd028eeedd6857d76eafda5e5011abd2ccabf090d"
Nov 04 01:26:40.322536 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.319419    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq"
Nov 04 01:26:40.331574 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.327641    5068 generic.go:356] "Generic (PLEG): container finished" podID="27872680-e51b-4785-bd2e-6912cb0fe64f" containerID="0efc55a6dfc36ae39ae7d99391dab701b1c3b4aacd70cdca5793c3194110480e" exitCode=0
Nov 04 01:26:40.331574 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.327733    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-6e609c-8496b8f648-d5c9r" event={"ID":"27872680-e51b-4785-bd2e-6912cb0fe64f","Type":"ContainerDied","Data":"0efc55a6dfc36ae39ae7d99391dab701b1c3b4aacd70cdca5793c3194110480e"}
Nov 04 01:26:40.331574 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.329962    5068 generic.go:356] "Generic (PLEG): container finished" podID="b3223811-a857-4279-a671-189e65383481" containerID="b0a18eed313cc80f07f5c24837dfced89714fe177b1cc708ed77b03144073ffc" exitCode=0
Nov 04 01:26:40.331574 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.330119    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n"
Nov 04 01:26:40.331574 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.330663    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n" event={"ID":"b3223811-a857-4279-a671-189e65383481","Type":"ContainerDied","Data":"b0a18eed313cc80f07f5c24837dfced89714fe177b1cc708ed77b03144073ffc"}
Nov 04 01:26:40.331574 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.330697    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n" event={"ID":"b3223811-a857-4279-a671-189e65383481","Type":"ContainerDied","Data":"e85236436c194bff4dd1ad9d0fdaac7c82750a5b5fd027a00244fc89cf093df9"}
Nov 04 01:26:40.333533 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.333285    5068 generic.go:356] "Generic (PLEG): container finished" podID="c3863ed4-3c10-453c-b11d-c66848ed2900" containerID="8bbd24168d1592314f8d30c8d589e104fda45c635cb5fe74bd9d4b7864692745" exitCode=0
Nov 04 01:26:40.333533 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.333450    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6e609c-58d54df979-k4wrw" event={"ID":"c3863ed4-3c10-453c-b11d-c66848ed2900","Type":"ContainerDied","Data":"8bbd24168d1592314f8d30c8d589e104fda45c635cb5fe74bd9d4b7864692745"}
Nov 04 01:26:40.337277 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.336549    5068 generic.go:356] "Generic (PLEG): container finished" podID="46d4ec7b-558d-4fb5-8011-177760a3291c" containerID="7b08752aa3a0af5715424d861acaf320decde640a2d39d3d63ab1725ee78098b" exitCode=0
Nov 04 01:26:40.337277 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.336672    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx" event={"ID":"46d4ec7b-558d-4fb5-8011-177760a3291c","Type":"ContainerDied","Data":"7b08752aa3a0af5715424d861acaf320decde640a2d39d3d63ab1725ee78098b"}
Nov 04 01:26:40.337277 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.336697    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx" event={"ID":"46d4ec7b-558d-4fb5-8011-177760a3291c","Type":"ContainerDied","Data":"0acb30231e0b0d146753d3b5f0326775a2cbf065de94e648c7121247308cf043"}
Nov 04 01:26:40.337277 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.336773    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx"
Nov 04 01:26:40.360124 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.359767    5068 scope.go:122] "RemoveContainer" containerID="9c26cad512c9c16f8fe19b6bd028eeedd6857d76eafda5e5011abd2ccabf090d"
Nov 04 01:26:40.361107 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:26:40.360594    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c26cad512c9c16f8fe19b6bd028eeedd6857d76eafda5e5011abd2ccabf090d\": container with ID starting with 9c26cad512c9c16f8fe19b6bd028eeedd6857d76eafda5e5011abd2ccabf090d not found: ID does not exist" containerID="9c26cad512c9c16f8fe19b6bd028eeedd6857d76eafda5e5011abd2ccabf090d"
Nov 04 01:26:40.361107 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.360651    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c26cad512c9c16f8fe19b6bd028eeedd6857d76eafda5e5011abd2ccabf090d"} err="failed to get container status \"9c26cad512c9c16f8fe19b6bd028eeedd6857d76eafda5e5011abd2ccabf090d\": rpc error: code = NotFound desc = could not find container \"9c26cad512c9c16f8fe19b6bd028eeedd6857d76eafda5e5011abd2ccabf090d\": container with ID starting with 9c26cad512c9c16f8fe19b6bd028eeedd6857d76eafda5e5011abd2ccabf090d not found: ID does not exist"
Nov 04 01:26:40.361107 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.360675    5068 scope.go:122] "RemoveContainer" containerID="b0a18eed313cc80f07f5c24837dfced89714fe177b1cc708ed77b03144073ffc"
Nov 04 01:26:40.367800 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.367748    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq"]
Nov 04 01:26:40.375410 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.375386    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-9902c3/distinct-w6-9902c3-kserve-5c9894f7df-zg5mq"]
Nov 04 01:26:40.386292 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.385816    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx"]
Nov 04 01:26:40.396551 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.396527    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-9902c3/premium-w6-9902c3-kserve-6c485f796b-p7cpx"]
Nov 04 01:26:40.414550 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.414517    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-6e609c-8496b8f648-d5c9r"
Nov 04 01:26:40.416898 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.416873    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n"]
Nov 04 01:26:40.421233 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.421210    5068 scope.go:122] "RemoveContainer" containerID="b0a18eed313cc80f07f5c24837dfced89714fe177b1cc708ed77b03144073ffc"
Nov 04 01:26:40.421628 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:26:40.421587    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0a18eed313cc80f07f5c24837dfced89714fe177b1cc708ed77b03144073ffc\": container with ID starting with b0a18eed313cc80f07f5c24837dfced89714fe177b1cc708ed77b03144073ffc not found: ID does not exist" containerID="b0a18eed313cc80f07f5c24837dfced89714fe177b1cc708ed77b03144073ffc"
Nov 04 01:26:40.421779 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.421752    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0a18eed313cc80f07f5c24837dfced89714fe177b1cc708ed77b03144073ffc"} err="failed to get container status \"b0a18eed313cc80f07f5c24837dfced89714fe177b1cc708ed77b03144073ffc\": rpc error: code = NotFound desc = could not find container \"b0a18eed313cc80f07f5c24837dfced89714fe177b1cc708ed77b03144073ffc\": container with ID starting with b0a18eed313cc80f07f5c24837dfced89714fe177b1cc708ed77b03144073ffc not found: ID does not exist"
Nov 04 01:26:40.421876 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.421866    5068 scope.go:122] "RemoveContainer" containerID="7b08752aa3a0af5715424d861acaf320decde640a2d39d3d63ab1725ee78098b"
Nov 04 01:26:40.423815 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.423793    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-9902c3/simulator-w6-9902c3-kserve-6bdb88785f-7tf6n"]
Nov 04 01:26:40.456159 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.456138    5068 scope.go:122] "RemoveContainer" containerID="7b08752aa3a0af5715424d861acaf320decde640a2d39d3d63ab1725ee78098b"
Nov 04 01:26:40.457097 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:26:40.457071    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b08752aa3a0af5715424d861acaf320decde640a2d39d3d63ab1725ee78098b\": container with ID starting with 7b08752aa3a0af5715424d861acaf320decde640a2d39d3d63ab1725ee78098b not found: ID does not exist" containerID="7b08752aa3a0af5715424d861acaf320decde640a2d39d3d63ab1725ee78098b"
Nov 04 01:26:40.457256 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.457228    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b08752aa3a0af5715424d861acaf320decde640a2d39d3d63ab1725ee78098b"} err="failed to get container status \"7b08752aa3a0af5715424d861acaf320decde640a2d39d3d63ab1725ee78098b\": rpc error: code = NotFound desc = could not find container \"7b08752aa3a0af5715424d861acaf320decde640a2d39d3d63ab1725ee78098b\": container with ID starting with 7b08752aa3a0af5715424d861acaf320decde640a2d39d3d63ab1725ee78098b not found: ID does not exist"
Nov 04 01:26:40.463140 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.463104    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/27872680-e51b-4785-bd2e-6912cb0fe64f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27872680-e51b-4785-bd2e-6912cb0fe64f-plugins-config-volume\") pod \"27872680-e51b-4785-bd2e-6912cb0fe64f\" (UID: \"27872680-e51b-4785-bd2e-6912cb0fe64f\") "
Nov 04 01:26:40.463471 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.463450    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/27872680-e51b-4785-bd2e-6912cb0fe64f-kube-api-access-4pdv8\" (UniqueName: \"kubernetes.io/projected/27872680-e51b-4785-bd2e-6912cb0fe64f-kube-api-access-4pdv8\") pod \"27872680-e51b-4785-bd2e-6912cb0fe64f\" (UID: \"27872680-e51b-4785-bd2e-6912cb0fe64f\") "
Nov 04 01:26:40.465327 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.465301    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27872680-e51b-4785-bd2e-6912cb0fe64f-plugins-config-volume" pod "27872680-e51b-4785-bd2e-6912cb0fe64f" (UID: "27872680-e51b-4785-bd2e-6912cb0fe64f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:26:40.477088 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.477054    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27872680-e51b-4785-bd2e-6912cb0fe64f-kube-api-access-4pdv8" pod "27872680-e51b-4785-bd2e-6912cb0fe64f" (UID: "27872680-e51b-4785-bd2e-6912cb0fe64f"). InnerVolumeSpecName "kube-api-access-4pdv8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:26:40.511277 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.511252    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6e609c-58d54df979-k4wrw"
Nov 04 01:26:40.564983 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.564951    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c3863ed4-3c10-453c-b11d-c66848ed2900-kube-api-access-lgmjt\" (UniqueName: \"kubernetes.io/projected/c3863ed4-3c10-453c-b11d-c66848ed2900-kube-api-access-lgmjt\") pod \"c3863ed4-3c10-453c-b11d-c66848ed2900\" (UID: \"c3863ed4-3c10-453c-b11d-c66848ed2900\") "
Nov 04 01:26:40.565149 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.565039    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c3863ed4-3c10-453c-b11d-c66848ed2900-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c3863ed4-3c10-453c-b11d-c66848ed2900-plugins-config-volume\") pod \"c3863ed4-3c10-453c-b11d-c66848ed2900\" (UID: \"c3863ed4-3c10-453c-b11d-c66848ed2900\") "
Nov 04 01:26:40.565779 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.565745    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3863ed4-3c10-453c-b11d-c66848ed2900-plugins-config-volume" pod "c3863ed4-3c10-453c-b11d-c66848ed2900" (UID: "c3863ed4-3c10-453c-b11d-c66848ed2900"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:26:40.565935 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.565912    5068 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27872680-e51b-4785-bd2e-6912cb0fe64f-plugins-config-volume\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:40.565999 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.565939    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4pdv8\" (UniqueName: \"kubernetes.io/projected/27872680-e51b-4785-bd2e-6912cb0fe64f-kube-api-access-4pdv8\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:40.570300 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.570172    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3863ed4-3c10-453c-b11d-c66848ed2900-kube-api-access-lgmjt" pod "c3863ed4-3c10-453c-b11d-c66848ed2900" (UID: "c3863ed4-3c10-453c-b11d-c66848ed2900"). InnerVolumeSpecName "kube-api-access-lgmjt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:26:40.667391 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.667340    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lgmjt\" (UniqueName: \"kubernetes.io/projected/c3863ed4-3c10-453c-b11d-c66848ed2900-kube-api-access-lgmjt\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:40.667391 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:40.667383    5068 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c3863ed4-3c10-453c-b11d-c66848ed2900-plugins-config-volume\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:41.360558 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:41.360527    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-6e609c-8496b8f648-d5c9r"
Nov 04 01:26:41.360558 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:41.360535    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-6e609c-8496b8f648-d5c9r" event={"ID":"27872680-e51b-4785-bd2e-6912cb0fe64f","Type":"ContainerDied","Data":"7b1bb9723de0fe340eb0e1868b8ec7f4989340ce9db9b859719697575c92f4d9"}
Nov 04 01:26:41.360850 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:41.360594    5068 scope.go:122] "RemoveContainer" containerID="0efc55a6dfc36ae39ae7d99391dab701b1c3b4aacd70cdca5793c3194110480e"
Nov 04 01:26:41.363677 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:41.363640    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6e609c-58d54df979-k4wrw" event={"ID":"c3863ed4-3c10-453c-b11d-c66848ed2900","Type":"ContainerDied","Data":"0779e617d5df2c63eba2c5d7df29b8088e4aeab73c01cb0b74514ffbe4abfb8a"}
Nov 04 01:26:41.363815 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:41.363663    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6e609c-58d54df979-k4wrw"
Nov 04 01:26:41.376268 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:41.376247    5068 scope.go:122] "RemoveContainer" containerID="8bbd24168d1592314f8d30c8d589e104fda45c635cb5fe74bd9d4b7864692745"
Nov 04 01:26:41.397174 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:41.397147    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-6e609c-8496b8f648-d5c9r"]
Nov 04 01:26:41.402386 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:41.402361    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-6e609c-8496b8f648-d5c9r"]
Nov 04 01:26:41.410936 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:41.410905    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-6e609c-58d54df979-k4wrw"]
Nov 04 01:26:41.440528 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:41.440499    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-6e609c-58d54df979-k4wrw"]
Nov 04 01:26:42.115904 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:42.115870    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="27872680-e51b-4785-bd2e-6912cb0fe64f" path="/var/lib/kubelet/pods/27872680-e51b-4785-bd2e-6912cb0fe64f/volumes"
Nov 04 01:26:42.116532 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:42.116516    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="46d4ec7b-558d-4fb5-8011-177760a3291c" path="/var/lib/kubelet/pods/46d4ec7b-558d-4fb5-8011-177760a3291c/volumes"
Nov 04 01:26:42.117030 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:42.117015    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b3223811-a857-4279-a671-189e65383481" path="/var/lib/kubelet/pods/b3223811-a857-4279-a671-189e65383481/volumes"
Nov 04 01:26:42.117464 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:42.117452    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b87aafac-9ab8-40e5-8d08-72eafc4187dd" path="/var/lib/kubelet/pods/b87aafac-9ab8-40e5-8d08-72eafc4187dd/volumes"
Nov 04 01:26:42.118217 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:42.118195    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c3863ed4-3c10-453c-b11d-c66848ed2900" path="/var/lib/kubelet/pods/c3863ed4-3c10-453c-b11d-c66848ed2900/volumes"
Nov 04 01:26:43.452731 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:26:43.452629    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:26:49.156459 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:49.156333    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-wljtg"]
Nov 04 01:26:49.157269 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:49.157233    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-wljtg" podUID="2f500082-e92f-4d13-a35b-2b775553b45d" containerName="istio-proxy" containerID="cri-o://a7776145535c7f3becaa02d8035503c2bbd04744411ba22cb460afb919307ad1" gracePeriod=30
Nov 04 01:26:49.165176 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:49.163507    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz"]
Nov 04 01:26:49.165176 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:49.163747    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz" podUID="40b656fe-1647-4533-8798-72791dc96d34" containerName="istio-proxy" containerID="cri-o://5b3b8686fe6ac228e74d4fea6dc2fcc01b9c91092e5a50b98904666554ae84ca" gracePeriod=30
Nov 04 01:26:54.525195 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:54.525154    5068 generic.go:356] "Generic (PLEG): container finished" podID="40b656fe-1647-4533-8798-72791dc96d34" containerID="5b3b8686fe6ac228e74d4fea6dc2fcc01b9c91092e5a50b98904666554ae84ca" exitCode=0
Nov 04 01:26:54.525661 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:54.525257    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz" event={"ID":"40b656fe-1647-4533-8798-72791dc96d34","Type":"ContainerDied","Data":"5b3b8686fe6ac228e74d4fea6dc2fcc01b9c91092e5a50b98904666554ae84ca"}
Nov 04 01:26:54.527562 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:54.527515    5068 generic.go:356] "Generic (PLEG): container finished" podID="2f500082-e92f-4d13-a35b-2b775553b45d" containerID="a7776145535c7f3becaa02d8035503c2bbd04744411ba22cb460afb919307ad1" exitCode=0
Nov 04 01:26:54.527737 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:54.527565    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-wljtg" event={"ID":"2f500082-e92f-4d13-a35b-2b775553b45d","Type":"ContainerDied","Data":"a7776145535c7f3becaa02d8035503c2bbd04744411ba22cb460afb919307ad1"}
Nov 04 01:26:54.570340 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:54.570307    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz"
Nov 04 01:26:54.609773 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:54.609733    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/40b656fe-1647-4533-8798-72791dc96d34-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40b656fe-1647-4533-8798-72791dc96d34-workload-certs\") pod \"40b656fe-1647-4533-8798-72791dc96d34\" (UID: \"40b656fe-1647-4533-8798-72791dc96d34\") "
Nov 04 01:26:54.609944 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:54.609879    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/40b656fe-1647-4533-8798-72791dc96d34-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40b656fe-1647-4533-8798-72791dc96d34-istio-data\") pod \"40b656fe-1647-4533-8798-72791dc96d34\" (UID: \"40b656fe-1647-4533-8798-72791dc96d34\") "
Nov 04 01:26:54.610086 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:54.610053    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40b656fe-1647-4533-8798-72791dc96d34-workload-certs" pod "40b656fe-1647-4533-8798-72791dc96d34" (UID: "40b656fe-1647-4533-8798-72791dc96d34"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:26:54.610366 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:54.610333    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40b656fe-1647-4533-8798-72791dc96d34-istio-data" pod "40b656fe-1647-4533-8798-72791dc96d34" (UID: "40b656fe-1647-4533-8798-72791dc96d34"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:26:54.612799 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:54.612484    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/40b656fe-1647-4533-8798-72791dc96d34-istio-token\" (UniqueName: \"kubernetes.io/projected/40b656fe-1647-4533-8798-72791dc96d34-istio-token\") pod \"40b656fe-1647-4533-8798-72791dc96d34\" (UID: \"40b656fe-1647-4533-8798-72791dc96d34\") "
Nov 04 01:26:54.612799 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:54.612564    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/40b656fe-1647-4533-8798-72791dc96d34-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40b656fe-1647-4533-8798-72791dc96d34-credential-socket\") pod \"40b656fe-1647-4533-8798-72791dc96d34\" (UID: \"40b656fe-1647-4533-8798-72791dc96d34\") "
Nov 04 01:26:54.612799 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:54.612746    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/40b656fe-1647-4533-8798-72791dc96d34-kube-api-access-jcw4w\" (UniqueName: \"kubernetes.io/projected/40b656fe-1647-4533-8798-72791dc96d34-kube-api-access-jcw4w\") pod \"40b656fe-1647-4533-8798-72791dc96d34\" (UID: \"40b656fe-1647-4533-8798-72791dc96d34\") "
Nov 04 01:26:54.618535 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:54.615492    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/40b656fe-1647-4533-8798-72791dc96d34-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40b656fe-1647-4533-8798-72791dc96d34-istio-podinfo\") pod \"40b656fe-1647-4533-8798-72791dc96d34\" (UID: \"40b656fe-1647-4533-8798-72791dc96d34\") "
Nov 04 01:26:54.618535 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:54.615546    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/40b656fe-1647-4533-8798-72791dc96d34-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40b656fe-1647-4533-8798-72791dc96d34-workload-socket\") pod \"40b656fe-1647-4533-8798-72791dc96d34\" (UID: \"40b656fe-1647-4533-8798-72791dc96d34\") "
Nov 04 01:26:54.618535 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:54.615634    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/40b656fe-1647-4533-8798-72791dc96d34-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40b656fe-1647-4533-8798-72791dc96d34-istio-envoy\") pod \"40b656fe-1647-4533-8798-72791dc96d34\" (UID: \"40b656fe-1647-4533-8798-72791dc96d34\") "
Nov 04 01:26:54.618535 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:54.615706    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/40b656fe-1647-4533-8798-72791dc96d34-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40b656fe-1647-4533-8798-72791dc96d34-istiod-ca-cert\") pod \"40b656fe-1647-4533-8798-72791dc96d34\" (UID: \"40b656fe-1647-4533-8798-72791dc96d34\") "
Nov 04 01:26:54.618535 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:54.616713    5068 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40b656fe-1647-4533-8798-72791dc96d34-workload-certs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:54.618535 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:54.616736    5068 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40b656fe-1647-4533-8798-72791dc96d34-istio-data\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:54.618535 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:54.617577    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40b656fe-1647-4533-8798-72791dc96d34-workload-socket" pod "40b656fe-1647-4533-8798-72791dc96d34" (UID: "40b656fe-1647-4533-8798-72791dc96d34"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:26:54.618535 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:54.617657    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40b656fe-1647-4533-8798-72791dc96d34-credential-socket" pod "40b656fe-1647-4533-8798-72791dc96d34" (UID: "40b656fe-1647-4533-8798-72791dc96d34"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:26:54.618535 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:54.617798    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40b656fe-1647-4533-8798-72791dc96d34-istiod-ca-cert" pod "40b656fe-1647-4533-8798-72791dc96d34" (UID: "40b656fe-1647-4533-8798-72791dc96d34"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:26:54.628012 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:54.627970    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/40b656fe-1647-4533-8798-72791dc96d34-istio-podinfo" pod "40b656fe-1647-4533-8798-72791dc96d34" (UID: "40b656fe-1647-4533-8798-72791dc96d34"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:26:54.628187 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:54.628069    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40b656fe-1647-4533-8798-72791dc96d34-kube-api-access-jcw4w" pod "40b656fe-1647-4533-8798-72791dc96d34" (UID: "40b656fe-1647-4533-8798-72791dc96d34"). InnerVolumeSpecName "kube-api-access-jcw4w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:26:54.628266 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:54.628197    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40b656fe-1647-4533-8798-72791dc96d34-istio-token" pod "40b656fe-1647-4533-8798-72791dc96d34" (UID: "40b656fe-1647-4533-8798-72791dc96d34"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:26:54.630010 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:54.629977    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40b656fe-1647-4533-8798-72791dc96d34-istio-envoy" pod "40b656fe-1647-4533-8798-72791dc96d34" (UID: "40b656fe-1647-4533-8798-72791dc96d34"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:26:54.718943 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:54.718141    5068 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40b656fe-1647-4533-8798-72791dc96d34-istio-token\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:54.718943 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:54.718181    5068 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40b656fe-1647-4533-8798-72791dc96d34-credential-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:54.718943 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:54.718196    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jcw4w\" (UniqueName: \"kubernetes.io/projected/40b656fe-1647-4533-8798-72791dc96d34-kube-api-access-jcw4w\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:54.718943 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:54.718212    5068 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40b656fe-1647-4533-8798-72791dc96d34-istio-podinfo\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:54.718943 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:54.718228    5068 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40b656fe-1647-4533-8798-72791dc96d34-workload-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:54.718943 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:54.718243    5068 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40b656fe-1647-4533-8798-72791dc96d34-istio-envoy\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:54.718943 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:54.718258    5068 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40b656fe-1647-4533-8798-72791dc96d34-istiod-ca-cert\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:54.862247 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:54.862218    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-wljtg"
Nov 04 01:26:54.920128 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:54.920094    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f500082-e92f-4d13-a35b-2b775553b45d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f500082-e92f-4d13-a35b-2b775553b45d-workload-certs\") pod \"2f500082-e92f-4d13-a35b-2b775553b45d\" (UID: \"2f500082-e92f-4d13-a35b-2b775553b45d\") "
Nov 04 01:26:54.920297 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:54.920145    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2f500082-e92f-4d13-a35b-2b775553b45d-istio-token\" (UniqueName: \"kubernetes.io/projected/2f500082-e92f-4d13-a35b-2b775553b45d-istio-token\") pod \"2f500082-e92f-4d13-a35b-2b775553b45d\" (UID: \"2f500082-e92f-4d13-a35b-2b775553b45d\") "
Nov 04 01:26:54.920297 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:54.920207    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2f500082-e92f-4d13-a35b-2b775553b45d-kube-api-access-m7qlz\" (UniqueName: \"kubernetes.io/projected/2f500082-e92f-4d13-a35b-2b775553b45d-kube-api-access-m7qlz\") pod \"2f500082-e92f-4d13-a35b-2b775553b45d\" (UID: \"2f500082-e92f-4d13-a35b-2b775553b45d\") "
Nov 04 01:26:54.920446 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:54.920357    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2f500082-e92f-4d13-a35b-2b775553b45d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f500082-e92f-4d13-a35b-2b775553b45d-istio-podinfo\") pod \"2f500082-e92f-4d13-a35b-2b775553b45d\" (UID: \"2f500082-e92f-4d13-a35b-2b775553b45d\") "
Nov 04 01:26:54.920446 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:54.920411    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f500082-e92f-4d13-a35b-2b775553b45d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f500082-e92f-4d13-a35b-2b775553b45d-workload-socket\") pod \"2f500082-e92f-4d13-a35b-2b775553b45d\" (UID: \"2f500082-e92f-4d13-a35b-2b775553b45d\") "
Nov 04 01:26:54.920446 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:54.920430    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f500082-e92f-4d13-a35b-2b775553b45d-workload-certs" pod "2f500082-e92f-4d13-a35b-2b775553b45d" (UID: "2f500082-e92f-4d13-a35b-2b775553b45d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:26:54.920744 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:54.920703    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f500082-e92f-4d13-a35b-2b775553b45d-workload-socket" pod "2f500082-e92f-4d13-a35b-2b775553b45d" (UID: "2f500082-e92f-4d13-a35b-2b775553b45d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:26:54.920894 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:54.920816    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f500082-e92f-4d13-a35b-2b775553b45d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f500082-e92f-4d13-a35b-2b775553b45d-istio-envoy\") pod \"2f500082-e92f-4d13-a35b-2b775553b45d\" (UID: \"2f500082-e92f-4d13-a35b-2b775553b45d\") "
Nov 04 01:26:54.920894 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:54.920868    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f500082-e92f-4d13-a35b-2b775553b45d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f500082-e92f-4d13-a35b-2b775553b45d-istio-data\") pod \"2f500082-e92f-4d13-a35b-2b775553b45d\" (UID: \"2f500082-e92f-4d13-a35b-2b775553b45d\") "
Nov 04 01:26:54.921010 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:54.920922    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2f500082-e92f-4d13-a35b-2b775553b45d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f500082-e92f-4d13-a35b-2b775553b45d-istiod-ca-cert\") pod \"2f500082-e92f-4d13-a35b-2b775553b45d\" (UID: \"2f500082-e92f-4d13-a35b-2b775553b45d\") "
Nov 04 01:26:54.921010 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:54.921001    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f500082-e92f-4d13-a35b-2b775553b45d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f500082-e92f-4d13-a35b-2b775553b45d-credential-socket\") pod \"2f500082-e92f-4d13-a35b-2b775553b45d\" (UID: \"2f500082-e92f-4d13-a35b-2b775553b45d\") "
Nov 04 01:26:54.921965 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:54.921604    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f500082-e92f-4d13-a35b-2b775553b45d-istiod-ca-cert" pod "2f500082-e92f-4d13-a35b-2b775553b45d" (UID: "2f500082-e92f-4d13-a35b-2b775553b45d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:26:54.921965 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:54.921725    5068 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f500082-e92f-4d13-a35b-2b775553b45d-workload-certs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:54.921965 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:54.921749    5068 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f500082-e92f-4d13-a35b-2b775553b45d-workload-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:54.921965 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:54.921926    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f500082-e92f-4d13-a35b-2b775553b45d-credential-socket" pod "2f500082-e92f-4d13-a35b-2b775553b45d" (UID: "2f500082-e92f-4d13-a35b-2b775553b45d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:26:54.923284 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:54.923251    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f500082-e92f-4d13-a35b-2b775553b45d-istio-data" pod "2f500082-e92f-4d13-a35b-2b775553b45d" (UID: "2f500082-e92f-4d13-a35b-2b775553b45d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:26:54.924121 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:54.924098    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2f500082-e92f-4d13-a35b-2b775553b45d-istio-podinfo" pod "2f500082-e92f-4d13-a35b-2b775553b45d" (UID: "2f500082-e92f-4d13-a35b-2b775553b45d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:26:54.924959 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:54.924933    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f500082-e92f-4d13-a35b-2b775553b45d-istio-token" pod "2f500082-e92f-4d13-a35b-2b775553b45d" (UID: "2f500082-e92f-4d13-a35b-2b775553b45d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:26:54.925298 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:54.925275    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f500082-e92f-4d13-a35b-2b775553b45d-istio-envoy" pod "2f500082-e92f-4d13-a35b-2b775553b45d" (UID: "2f500082-e92f-4d13-a35b-2b775553b45d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:26:54.926214 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:54.926192    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f500082-e92f-4d13-a35b-2b775553b45d-kube-api-access-m7qlz" pod "2f500082-e92f-4d13-a35b-2b775553b45d" (UID: "2f500082-e92f-4d13-a35b-2b775553b45d"). InnerVolumeSpecName "kube-api-access-m7qlz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:26:55.032424 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:55.032383    5068 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f500082-e92f-4d13-a35b-2b775553b45d-istio-envoy\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:55.032424 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:55.032423    5068 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f500082-e92f-4d13-a35b-2b775553b45d-istio-data\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:55.032717 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:55.032439    5068 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f500082-e92f-4d13-a35b-2b775553b45d-istiod-ca-cert\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:55.032717 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:55.032453    5068 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f500082-e92f-4d13-a35b-2b775553b45d-credential-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:55.032717 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:55.032465    5068 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f500082-e92f-4d13-a35b-2b775553b45d-istio-token\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:55.032717 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:55.032479    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m7qlz\" (UniqueName: \"kubernetes.io/projected/2f500082-e92f-4d13-a35b-2b775553b45d-kube-api-access-m7qlz\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:55.032717 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:55.032492    5068 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f500082-e92f-4d13-a35b-2b775553b45d-istio-podinfo\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:26:55.536733 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:55.536702    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz"
Nov 04 01:26:55.537265 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:55.536708    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz" event={"ID":"40b656fe-1647-4533-8798-72791dc96d34","Type":"ContainerDied","Data":"08d57944ab193110f64fd89137295be68255bfd7f3a965b588d5eba8e7915406"}
Nov 04 01:26:55.537396 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:55.537300    5068 scope.go:122] "RemoveContainer" containerID="5b3b8686fe6ac228e74d4fea6dc2fcc01b9c91092e5a50b98904666554ae84ca"
Nov 04 01:26:55.538790 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:55.538681    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-wljtg" event={"ID":"2f500082-e92f-4d13-a35b-2b775553b45d","Type":"ContainerDied","Data":"244e06a33ead1de1ec677095e84ac0a9d2b58e90ec48e079391da10185065a8b"}
Nov 04 01:26:55.538790 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:55.538770    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-wljtg"
Nov 04 01:26:55.551661 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:55.551548    5068 scope.go:122] "RemoveContainer" containerID="a7776145535c7f3becaa02d8035503c2bbd04744411ba22cb460afb919307ad1"
Nov 04 01:26:55.574150 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:55.574117    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz"]
Nov 04 01:26:55.588946 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:55.588916    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-w2wqz"]
Nov 04 01:26:55.594205 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:55.594171    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-wljtg"]
Nov 04 01:26:55.598833 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:55.598810    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-e7742169-openshift-default-6986f46949-wljtg"]
Nov 04 01:26:56.115152 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:56.115119    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2f500082-e92f-4d13-a35b-2b775553b45d" path="/var/lib/kubelet/pods/2f500082-e92f-4d13-a35b-2b775553b45d/volumes"
Nov 04 01:26:56.115780 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:56.115760    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="40b656fe-1647-4533-8798-72791dc96d34" path="/var/lib/kubelet/pods/40b656fe-1647-4533-8798-72791dc96d34/volumes"
Nov 04 01:26:57.025534 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.025486    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg"]
Nov 04 01:26:57.026316 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.026279    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg" podUID="d899bb80-977d-4b53-b96c-b354dcbdad2d" containerName="istio-proxy" containerID="cri-o://1cda6849311364f1e3fc26d54221e0e4f6f248f3d147559e4c4d1ad38f7b9432" gracePeriod=30
Nov 04 01:26:57.564791 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.564757    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs"]
Nov 04 01:26:57.566660 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.565741    5068 cpu_manager.go:414] "containerMap: removing container" podUID="45b260dc-8b00-4f55-a649-58220ed083bb" containerName="istio-proxy"
Nov 04 01:26:57.566660 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.565765    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="45b260dc-8b00-4f55-a649-58220ed083bb" containerName="istio-proxy"
Nov 04 01:26:57.566660 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.565782    5068 cpu_manager.go:414] "containerMap: removing container" podUID="46d4ec7b-558d-4fb5-8011-177760a3291c" containerName="main"
Nov 04 01:26:57.566660 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.565789    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="46d4ec7b-558d-4fb5-8011-177760a3291c" containerName="main"
Nov 04 01:26:57.566660 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.565817    5068 cpu_manager.go:414] "containerMap: removing container" podUID="b3223811-a857-4279-a671-189e65383481" containerName="main"
Nov 04 01:26:57.566660 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.565823    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b3223811-a857-4279-a671-189e65383481" containerName="main"
Nov 04 01:26:57.566660 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.565834    5068 cpu_manager.go:414] "containerMap: removing container" podUID="3bde9301-d2f8-4820-9044-463e387b3c3d" containerName="istio-proxy"
Nov 04 01:26:57.566660 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.565840    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3bde9301-d2f8-4820-9044-463e387b3c3d" containerName="istio-proxy"
Nov 04 01:26:57.566660 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.565867    5068 cpu_manager.go:414] "containerMap: removing container" podUID="27872680-e51b-4785-bd2e-6912cb0fe64f" containerName="payload-processing"
Nov 04 01:26:57.566660 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.565875    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="27872680-e51b-4785-bd2e-6912cb0fe64f" containerName="payload-processing"
Nov 04 01:26:57.566660 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.565900    5068 cpu_manager.go:414] "containerMap: removing container" podUID="2f500082-e92f-4d13-a35b-2b775553b45d" containerName="istio-proxy"
Nov 04 01:26:57.566660 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.565908    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2f500082-e92f-4d13-a35b-2b775553b45d" containerName="istio-proxy"
Nov 04 01:26:57.566660 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.565923    5068 cpu_manager.go:414] "containerMap: removing container" podUID="b87aafac-9ab8-40e5-8d08-72eafc4187dd" containerName="main"
Nov 04 01:26:57.566660 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.565930    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b87aafac-9ab8-40e5-8d08-72eafc4187dd" containerName="main"
Nov 04 01:26:57.566660 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.565967    5068 cpu_manager.go:414] "containerMap: removing container" podUID="40b656fe-1647-4533-8798-72791dc96d34" containerName="istio-proxy"
Nov 04 01:26:57.566660 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.565976    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="40b656fe-1647-4533-8798-72791dc96d34" containerName="istio-proxy"
Nov 04 01:26:57.566660 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.565989    5068 cpu_manager.go:414] "containerMap: removing container" podUID="73f6a679-d6e5-45d1-a885-ebea2ed4f253" containerName="istio-proxy"
Nov 04 01:26:57.566660 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.565996    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="73f6a679-d6e5-45d1-a885-ebea2ed4f253" containerName="istio-proxy"
Nov 04 01:26:57.566660 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.566005    5068 cpu_manager.go:414] "containerMap: removing container" podUID="7d1dc8f1-3238-4b12-b579-ec8179bc4e2a" containerName="istio-proxy"
Nov 04 01:26:57.566660 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.566011    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7d1dc8f1-3238-4b12-b579-ec8179bc4e2a" containerName="istio-proxy"
Nov 04 01:26:57.566660 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.566025    5068 cpu_manager.go:414] "containerMap: removing container" podUID="20fba30d-0142-49f8-8489-957bad3db95e" containerName="istio-proxy"
Nov 04 01:26:57.566660 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.566032    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="20fba30d-0142-49f8-8489-957bad3db95e" containerName="istio-proxy"
Nov 04 01:26:57.566660 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.566045    5068 cpu_manager.go:414] "containerMap: removing container" podUID="c3863ed4-3c10-453c-b11d-c66848ed2900" containerName="payload-pre-processing"
Nov 04 01:26:57.566660 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.566054    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c3863ed4-3c10-453c-b11d-c66848ed2900" containerName="payload-pre-processing"
Nov 04 01:26:57.566660 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.566245    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="40b656fe-1647-4533-8798-72791dc96d34" containerName="istio-proxy"
Nov 04 01:26:57.566660 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.566262    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="73f6a679-d6e5-45d1-a885-ebea2ed4f253" containerName="istio-proxy"
Nov 04 01:26:57.566660 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.566274    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="7d1dc8f1-3238-4b12-b579-ec8179bc4e2a" containerName="istio-proxy"
Nov 04 01:26:57.566660 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.566284    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="20fba30d-0142-49f8-8489-957bad3db95e" containerName="istio-proxy"
Nov 04 01:26:57.566660 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.566294    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="c3863ed4-3c10-453c-b11d-c66848ed2900" containerName="payload-pre-processing"
Nov 04 01:26:57.566660 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.566306    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="45b260dc-8b00-4f55-a649-58220ed083bb" containerName="istio-proxy"
Nov 04 01:26:57.566660 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.566317    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="46d4ec7b-558d-4fb5-8011-177760a3291c" containerName="main"
Nov 04 01:26:57.566660 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.566328    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="b3223811-a857-4279-a671-189e65383481" containerName="main"
Nov 04 01:26:57.566660 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.566335    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="3bde9301-d2f8-4820-9044-463e387b3c3d" containerName="istio-proxy"
Nov 04 01:26:57.566660 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.566345    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="27872680-e51b-4785-bd2e-6912cb0fe64f" containerName="payload-processing"
Nov 04 01:26:57.566660 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.566358    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="2f500082-e92f-4d13-a35b-2b775553b45d" containerName="istio-proxy"
Nov 04 01:26:57.566660 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.566367    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="b87aafac-9ab8-40e5-8d08-72eafc4187dd" containerName="main"
Nov 04 01:26:57.572538 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.572515    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs"
Nov 04 01:26:57.592942 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.592886    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs"]
Nov 04 01:26:57.661782 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.661748    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-27bc4f8b-openshift-default-dockercfg-6dxmm\""
Nov 04 01:26:57.692492 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.692456    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31ede3d2-d690-4178-90a7-69c82caa9e2c-istio-token\") pod \"e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs\" (UID: \"31ede3d2-d690-4178-90a7-69c82caa9e2c\") " pod="openshift-ingress/e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs"
Nov 04 01:26:57.692722 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.692523    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31ede3d2-d690-4178-90a7-69c82caa9e2c-istiod-ca-cert\") pod \"e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs\" (UID: \"31ede3d2-d690-4178-90a7-69c82caa9e2c\") " pod="openshift-ingress/e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs"
Nov 04 01:26:57.692722 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.692602    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31ede3d2-d690-4178-90a7-69c82caa9e2c-credential-socket\") pod \"e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs\" (UID: \"31ede3d2-d690-4178-90a7-69c82caa9e2c\") " pod="openshift-ingress/e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs"
Nov 04 01:26:57.692872 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.692791    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31ede3d2-d690-4178-90a7-69c82caa9e2c-workload-certs\") pod \"e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs\" (UID: \"31ede3d2-d690-4178-90a7-69c82caa9e2c\") " pod="openshift-ingress/e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs"
Nov 04 01:26:57.692943 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.692927    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31ede3d2-d690-4178-90a7-69c82caa9e2c-istio-podinfo\") pod \"e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs\" (UID: \"31ede3d2-d690-4178-90a7-69c82caa9e2c\") " pod="openshift-ingress/e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs"
Nov 04 01:26:57.693000 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.692972    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31ede3d2-d690-4178-90a7-69c82caa9e2c-workload-socket\") pod \"e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs\" (UID: \"31ede3d2-d690-4178-90a7-69c82caa9e2c\") " pod="openshift-ingress/e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs"
Nov 04 01:26:57.693058 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.693043    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31ede3d2-d690-4178-90a7-69c82caa9e2c-istio-envoy\") pod \"e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs\" (UID: \"31ede3d2-d690-4178-90a7-69c82caa9e2c\") " pod="openshift-ingress/e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs"
Nov 04 01:26:57.693164 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.693151    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2ssj\" (UniqueName: \"kubernetes.io/projected/31ede3d2-d690-4178-90a7-69c82caa9e2c-kube-api-access-q2ssj\") pod \"e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs\" (UID: \"31ede3d2-d690-4178-90a7-69c82caa9e2c\") " pod="openshift-ingress/e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs"
Nov 04 01:26:57.693205 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.693181    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31ede3d2-d690-4178-90a7-69c82caa9e2c-istio-data\") pod \"e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs\" (UID: \"31ede3d2-d690-4178-90a7-69c82caa9e2c\") " pod="openshift-ingress/e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs"
Nov 04 01:26:57.794380 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.794340    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31ede3d2-d690-4178-90a7-69c82caa9e2c-workload-socket\") pod \"e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs\" (UID: \"31ede3d2-d690-4178-90a7-69c82caa9e2c\") " pod="openshift-ingress/e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs"
Nov 04 01:26:57.794579 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.794403    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31ede3d2-d690-4178-90a7-69c82caa9e2c-istio-envoy\") pod \"e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs\" (UID: \"31ede3d2-d690-4178-90a7-69c82caa9e2c\") " pod="openshift-ingress/e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs"
Nov 04 01:26:57.794579 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.794524    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2ssj\" (UniqueName: \"kubernetes.io/projected/31ede3d2-d690-4178-90a7-69c82caa9e2c-kube-api-access-q2ssj\") pod \"e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs\" (UID: \"31ede3d2-d690-4178-90a7-69c82caa9e2c\") " pod="openshift-ingress/e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs"
Nov 04 01:26:57.794579 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.794565    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31ede3d2-d690-4178-90a7-69c82caa9e2c-istio-data\") pod \"e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs\" (UID: \"31ede3d2-d690-4178-90a7-69c82caa9e2c\") " pod="openshift-ingress/e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs"
Nov 04 01:26:57.794801 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.794637    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31ede3d2-d690-4178-90a7-69c82caa9e2c-istio-token\") pod \"e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs\" (UID: \"31ede3d2-d690-4178-90a7-69c82caa9e2c\") " pod="openshift-ingress/e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs"
Nov 04 01:26:57.794801 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.794708    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31ede3d2-d690-4178-90a7-69c82caa9e2c-istiod-ca-cert\") pod \"e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs\" (UID: \"31ede3d2-d690-4178-90a7-69c82caa9e2c\") " pod="openshift-ingress/e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs"
Nov 04 01:26:57.794801 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.794742    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31ede3d2-d690-4178-90a7-69c82caa9e2c-credential-socket\") pod \"e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs\" (UID: \"31ede3d2-d690-4178-90a7-69c82caa9e2c\") " pod="openshift-ingress/e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs"
Nov 04 01:26:57.794948 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.794808    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31ede3d2-d690-4178-90a7-69c82caa9e2c-workload-socket\") pod \"e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs\" (UID: \"31ede3d2-d690-4178-90a7-69c82caa9e2c\") " pod="openshift-ingress/e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs"
Nov 04 01:26:57.794948 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.794825    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31ede3d2-d690-4178-90a7-69c82caa9e2c-workload-certs\") pod \"e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs\" (UID: \"31ede3d2-d690-4178-90a7-69c82caa9e2c\") " pod="openshift-ingress/e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs"
Nov 04 01:26:57.794948 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.794922    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31ede3d2-d690-4178-90a7-69c82caa9e2c-istio-data\") pod \"e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs\" (UID: \"31ede3d2-d690-4178-90a7-69c82caa9e2c\") " pod="openshift-ingress/e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs"
Nov 04 01:26:57.795112 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.794973    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31ede3d2-d690-4178-90a7-69c82caa9e2c-istio-podinfo\") pod \"e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs\" (UID: \"31ede3d2-d690-4178-90a7-69c82caa9e2c\") " pod="openshift-ingress/e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs"
Nov 04 01:26:57.795167 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.795110    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31ede3d2-d690-4178-90a7-69c82caa9e2c-workload-certs\") pod \"e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs\" (UID: \"31ede3d2-d690-4178-90a7-69c82caa9e2c\") " pod="openshift-ingress/e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs"
Nov 04 01:26:57.795221 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.795197    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31ede3d2-d690-4178-90a7-69c82caa9e2c-credential-socket\") pod \"e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs\" (UID: \"31ede3d2-d690-4178-90a7-69c82caa9e2c\") " pod="openshift-ingress/e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs"
Nov 04 01:26:57.795468 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.795446    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31ede3d2-d690-4178-90a7-69c82caa9e2c-istiod-ca-cert\") pod \"e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs\" (UID: \"31ede3d2-d690-4178-90a7-69c82caa9e2c\") " pod="openshift-ingress/e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs"
Nov 04 01:26:57.797006 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.796983    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31ede3d2-d690-4178-90a7-69c82caa9e2c-istio-envoy\") pod \"e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs\" (UID: \"31ede3d2-d690-4178-90a7-69c82caa9e2c\") " pod="openshift-ingress/e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs"
Nov 04 01:26:57.797492 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.797469    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31ede3d2-d690-4178-90a7-69c82caa9e2c-istio-podinfo\") pod \"e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs\" (UID: \"31ede3d2-d690-4178-90a7-69c82caa9e2c\") " pod="openshift-ingress/e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs"
Nov 04 01:26:57.807281 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.807218    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31ede3d2-d690-4178-90a7-69c82caa9e2c-istio-token\") pod \"e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs\" (UID: \"31ede3d2-d690-4178-90a7-69c82caa9e2c\") " pod="openshift-ingress/e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs"
Nov 04 01:26:57.807381 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.807293    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2ssj\" (UniqueName: \"kubernetes.io/projected/31ede3d2-d690-4178-90a7-69c82caa9e2c-kube-api-access-q2ssj\") pod \"e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs\" (UID: \"31ede3d2-d690-4178-90a7-69c82caa9e2c\") " pod="openshift-ingress/e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs"
Nov 04 01:26:57.901001 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:57.900899    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs"
Nov 04 01:26:58.272055 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:58.272028    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs"]
Nov 04 01:26:58.273752 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:26:58.273722    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31ede3d2_d690_4178_90a7_69c82caa9e2c.slice/crio-41efb186d1ffdb66fc8ffb674afcbfbec3d9fe89c650182f957c40c56a2cf53d WatchSource:0}: Error finding container 41efb186d1ffdb66fc8ffb674afcbfbec3d9fe89c650182f957c40c56a2cf53d: Status 404 returned error can't find the container with id 41efb186d1ffdb66fc8ffb674afcbfbec3d9fe89c650182f957c40c56a2cf53d
Nov 04 01:26:58.276583 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:58.276549    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:26:58.276751 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:58.276680    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:26:58.276751 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:58.276720    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:26:58.585825 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:58.584576    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs" event={"ID":"31ede3d2-d690-4178-90a7-69c82caa9e2c","Type":"ContainerStarted","Data":"e7fc1527cb18c7e59ab1cb7a6412b2d875def441fcad4d82b7822571f6e0df4e"}
Nov 04 01:26:58.585825 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:58.584665    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs" event={"ID":"31ede3d2-d690-4178-90a7-69c82caa9e2c","Type":"ContainerStarted","Data":"41efb186d1ffdb66fc8ffb674afcbfbec3d9fe89c650182f957c40c56a2cf53d"}
Nov 04 01:26:58.661643 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:58.653170    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs" podStartSLOduration=1.653147723 podStartE2EDuration="1.653147723s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:26:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:26:58.626557681 +0000 UTC m=+1458.673118476" watchObservedRunningTime="2026-11-04 01:26:58.653147723 +0000 UTC m=+1458.699708514"
Nov 04 01:26:58.901996 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:58.901840    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs"
Nov 04 01:26:59.903071 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:59.903033    5068 patch_prober.go:29] interesting pod/e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused" start-of-body=
Nov 04 01:26:59.903479 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:26:59.903110    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs" podUID="31ede3d2-d690-4178-90a7-69c82caa9e2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused"
Nov 04 01:27:00.902118 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:00.902086    5068 patch_prober.go:29] interesting pod/e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused" start-of-body=
Nov 04 01:27:00.902296 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:00.902137    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs" podUID="31ede3d2-d690-4178-90a7-69c82caa9e2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused"
Nov 04 01:27:01.901661 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:01.901590    5068 patch_prober.go:29] interesting pod/e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused" start-of-body=
Nov 04 01:27:01.902082 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:01.901687    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs" podUID="31ede3d2-d690-4178-90a7-69c82caa9e2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused"
Nov 04 01:27:02.505626 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:02.505525    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg"
Nov 04 01:27:02.602162 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:02.597903    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d899bb80-977d-4b53-b96c-b354dcbdad2d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d899bb80-977d-4b53-b96c-b354dcbdad2d-istio-podinfo\") pod \"d899bb80-977d-4b53-b96c-b354dcbdad2d\" (UID: \"d899bb80-977d-4b53-b96c-b354dcbdad2d\") "
Nov 04 01:27:02.602162 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:02.597981    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d899bb80-977d-4b53-b96c-b354dcbdad2d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d899bb80-977d-4b53-b96c-b354dcbdad2d-workload-certs\") pod \"d899bb80-977d-4b53-b96c-b354dcbdad2d\" (UID: \"d899bb80-977d-4b53-b96c-b354dcbdad2d\") "
Nov 04 01:27:02.602162 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:02.598015    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d899bb80-977d-4b53-b96c-b354dcbdad2d-istio-token\" (UniqueName: \"kubernetes.io/projected/d899bb80-977d-4b53-b96c-b354dcbdad2d-istio-token\") pod \"d899bb80-977d-4b53-b96c-b354dcbdad2d\" (UID: \"d899bb80-977d-4b53-b96c-b354dcbdad2d\") "
Nov 04 01:27:02.602162 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:02.598058    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d899bb80-977d-4b53-b96c-b354dcbdad2d-kube-api-access-mnd4z\" (UniqueName: \"kubernetes.io/projected/d899bb80-977d-4b53-b96c-b354dcbdad2d-kube-api-access-mnd4z\") pod \"d899bb80-977d-4b53-b96c-b354dcbdad2d\" (UID: \"d899bb80-977d-4b53-b96c-b354dcbdad2d\") "
Nov 04 01:27:02.602162 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:02.598116    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d899bb80-977d-4b53-b96c-b354dcbdad2d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d899bb80-977d-4b53-b96c-b354dcbdad2d-workload-socket\") pod \"d899bb80-977d-4b53-b96c-b354dcbdad2d\" (UID: \"d899bb80-977d-4b53-b96c-b354dcbdad2d\") "
Nov 04 01:27:02.602162 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:02.598244    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d899bb80-977d-4b53-b96c-b354dcbdad2d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d899bb80-977d-4b53-b96c-b354dcbdad2d-istio-data\") pod \"d899bb80-977d-4b53-b96c-b354dcbdad2d\" (UID: \"d899bb80-977d-4b53-b96c-b354dcbdad2d\") "
Nov 04 01:27:02.602162 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:02.598307    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d899bb80-977d-4b53-b96c-b354dcbdad2d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d899bb80-977d-4b53-b96c-b354dcbdad2d-istiod-ca-cert\") pod \"d899bb80-977d-4b53-b96c-b354dcbdad2d\" (UID: \"d899bb80-977d-4b53-b96c-b354dcbdad2d\") "
Nov 04 01:27:02.602162 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:02.598442    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d899bb80-977d-4b53-b96c-b354dcbdad2d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d899bb80-977d-4b53-b96c-b354dcbdad2d-credential-socket\") pod \"d899bb80-977d-4b53-b96c-b354dcbdad2d\" (UID: \"d899bb80-977d-4b53-b96c-b354dcbdad2d\") "
Nov 04 01:27:02.602162 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:02.598477    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d899bb80-977d-4b53-b96c-b354dcbdad2d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d899bb80-977d-4b53-b96c-b354dcbdad2d-istio-envoy\") pod \"d899bb80-977d-4b53-b96c-b354dcbdad2d\" (UID: \"d899bb80-977d-4b53-b96c-b354dcbdad2d\") "
Nov 04 01:27:02.602162 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:02.598796    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d899bb80-977d-4b53-b96c-b354dcbdad2d-workload-certs" pod "d899bb80-977d-4b53-b96c-b354dcbdad2d" (UID: "d899bb80-977d-4b53-b96c-b354dcbdad2d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:27:02.602162 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:02.599178    5068 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d899bb80-977d-4b53-b96c-b354dcbdad2d-workload-certs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:02.602162 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:02.599369    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d899bb80-977d-4b53-b96c-b354dcbdad2d-istio-data" pod "d899bb80-977d-4b53-b96c-b354dcbdad2d" (UID: "d899bb80-977d-4b53-b96c-b354dcbdad2d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:27:02.602162 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:02.599506    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d899bb80-977d-4b53-b96c-b354dcbdad2d-credential-socket" pod "d899bb80-977d-4b53-b96c-b354dcbdad2d" (UID: "d899bb80-977d-4b53-b96c-b354dcbdad2d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:27:02.602162 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:02.599566    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d899bb80-977d-4b53-b96c-b354dcbdad2d-workload-socket" pod "d899bb80-977d-4b53-b96c-b354dcbdad2d" (UID: "d899bb80-977d-4b53-b96c-b354dcbdad2d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:27:02.602162 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:02.599730    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d899bb80-977d-4b53-b96c-b354dcbdad2d-istiod-ca-cert" pod "d899bb80-977d-4b53-b96c-b354dcbdad2d" (UID: "d899bb80-977d-4b53-b96c-b354dcbdad2d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:27:02.602162 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:02.601776    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d899bb80-977d-4b53-b96c-b354dcbdad2d-istio-token" pod "d899bb80-977d-4b53-b96c-b354dcbdad2d" (UID: "d899bb80-977d-4b53-b96c-b354dcbdad2d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:27:02.602162 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:02.601881    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d899bb80-977d-4b53-b96c-b354dcbdad2d-istio-envoy" pod "d899bb80-977d-4b53-b96c-b354dcbdad2d" (UID: "d899bb80-977d-4b53-b96c-b354dcbdad2d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:27:02.605183 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:02.605151    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d899bb80-977d-4b53-b96c-b354dcbdad2d-istio-podinfo" pod "d899bb80-977d-4b53-b96c-b354dcbdad2d" (UID: "d899bb80-977d-4b53-b96c-b354dcbdad2d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:27:02.605294 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:02.605197    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d899bb80-977d-4b53-b96c-b354dcbdad2d-kube-api-access-mnd4z" pod "d899bb80-977d-4b53-b96c-b354dcbdad2d" (UID: "d899bb80-977d-4b53-b96c-b354dcbdad2d"). InnerVolumeSpecName "kube-api-access-mnd4z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:27:02.627604 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:02.627569    5068 generic.go:356] "Generic (PLEG): container finished" podID="d899bb80-977d-4b53-b96c-b354dcbdad2d" containerID="1cda6849311364f1e3fc26d54221e0e4f6f248f3d147559e4c4d1ad38f7b9432" exitCode=0
Nov 04 01:27:02.627775 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:02.627655    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg"
Nov 04 01:27:02.627775 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:02.627760    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg" event={"ID":"d899bb80-977d-4b53-b96c-b354dcbdad2d","Type":"ContainerDied","Data":"1cda6849311364f1e3fc26d54221e0e4f6f248f3d147559e4c4d1ad38f7b9432"}
Nov 04 01:27:02.627898 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:02.627803    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg" event={"ID":"d899bb80-977d-4b53-b96c-b354dcbdad2d","Type":"ContainerDied","Data":"ddcf6119b2e4eb11dc89cf9529ea82df703e04d44709e86e39a3b7f0e30c97a5"}
Nov 04 01:27:02.627898 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:02.627828    5068 scope.go:122] "RemoveContainer" containerID="1cda6849311364f1e3fc26d54221e0e4f6f248f3d147559e4c4d1ad38f7b9432"
Nov 04 01:27:02.688107 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:02.688078    5068 scope.go:122] "RemoveContainer" containerID="1cda6849311364f1e3fc26d54221e0e4f6f248f3d147559e4c4d1ad38f7b9432"
Nov 04 01:27:02.688574 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:27:02.688546    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1cda6849311364f1e3fc26d54221e0e4f6f248f3d147559e4c4d1ad38f7b9432\": container with ID starting with 1cda6849311364f1e3fc26d54221e0e4f6f248f3d147559e4c4d1ad38f7b9432 not found: ID does not exist" containerID="1cda6849311364f1e3fc26d54221e0e4f6f248f3d147559e4c4d1ad38f7b9432"
Nov 04 01:27:02.688657 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:02.688593    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1cda6849311364f1e3fc26d54221e0e4f6f248f3d147559e4c4d1ad38f7b9432"} err="failed to get container status \"1cda6849311364f1e3fc26d54221e0e4f6f248f3d147559e4c4d1ad38f7b9432\": rpc error: code = NotFound desc = could not find container \"1cda6849311364f1e3fc26d54221e0e4f6f248f3d147559e4c4d1ad38f7b9432\": container with ID starting with 1cda6849311364f1e3fc26d54221e0e4f6f248f3d147559e4c4d1ad38f7b9432 not found: ID does not exist"
Nov 04 01:27:02.695478 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:02.695456    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg"]
Nov 04 01:27:02.699688 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:02.699665    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-6e609c-openshift-default-7dbbb69bf9-2nncg"]
Nov 04 01:27:02.700504 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:02.700484    5068 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d899bb80-977d-4b53-b96c-b354dcbdad2d-credential-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:02.700551 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:02.700508    5068 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d899bb80-977d-4b53-b96c-b354dcbdad2d-istio-envoy\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:02.700551 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:02.700518    5068 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d899bb80-977d-4b53-b96c-b354dcbdad2d-istio-podinfo\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:02.700551 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:02.700528    5068 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d899bb80-977d-4b53-b96c-b354dcbdad2d-istio-token\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:02.700551 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:02.700537    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mnd4z\" (UniqueName: \"kubernetes.io/projected/d899bb80-977d-4b53-b96c-b354dcbdad2d-kube-api-access-mnd4z\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:02.700551 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:02.700547    5068 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d899bb80-977d-4b53-b96c-b354dcbdad2d-workload-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:02.700729 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:02.700555    5068 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d899bb80-977d-4b53-b96c-b354dcbdad2d-istio-data\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:02.700729 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:02.700563    5068 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d899bb80-977d-4b53-b96c-b354dcbdad2d-istiod-ca-cert\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:02.901791 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:02.901756    5068 patch_prober.go:29] interesting pod/e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused" start-of-body=
Nov 04 01:27:02.902246 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:02.901818    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs" podUID="31ede3d2-d690-4178-90a7-69c82caa9e2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused"
Nov 04 01:27:03.799946 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:03.799914    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-20a250-66d794c856-ql77v"]
Nov 04 01:27:03.901897 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:03.901856    5068 patch_prober.go:29] interesting pod/e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused" start-of-body=
Nov 04 01:27:03.902375 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:03.901926    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs" podUID="31ede3d2-d690-4178-90a7-69c82caa9e2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused"
Nov 04 01:27:04.117907 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:04.117821    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d899bb80-977d-4b53-b96c-b354dcbdad2d" path="/var/lib/kubelet/pods/d899bb80-977d-4b53-b96c-b354dcbdad2d/volumes"
Nov 04 01:27:04.952890 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:04.952854    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs"
Nov 04 01:27:04.953422 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:04.953296    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs"
Nov 04 01:27:04.954266 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:04.954246    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs"
Nov 04 01:27:08.255771 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:08.255736    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-27bc4f8b-7ccc8d95f6-cm86k"]
Nov 04 01:27:08.256644 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:08.256599    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-27bc4f8b-7ccc8d95f6-cm86k" podUID="1799b838-9323-4b50-b630-13f956913a62" containerName="maas-api" containerID="cri-o://f68b366e31940e058fb718a1124a6667f3fb93424461b0300b137d8ba7490af1" gracePeriod=30
Nov 04 01:27:08.814934 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:08.813369    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs"]
Nov 04 01:27:08.815544 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:08.815508    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs" podUID="31ede3d2-d690-4178-90a7-69c82caa9e2c" containerName="istio-proxy" containerID="cri-o://e7fc1527cb18c7e59ab1cb7a6412b2d875def441fcad4d82b7822571f6e0df4e" gracePeriod=30
Nov 04 01:27:08.828340 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:08.816926    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-20a250-66d794c856-ql77v" podUID="62dc4132-b26f-4b4f-a4c1-936dfa171b05" containerName="payload-processing" containerID="cri-o://e26652542987ee60abd614fb4f39b4a6608ac0c7eb97fda9d65efa30c5a2bed3" gracePeriod=25
Nov 04 01:27:09.057203 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:09.056946    5068 generic.go:356] "Generic (PLEG): container finished" podID="62dc4132-b26f-4b4f-a4c1-936dfa171b05" containerID="e26652542987ee60abd614fb4f39b4a6608ac0c7eb97fda9d65efa30c5a2bed3" exitCode=0
Nov 04 01:27:09.057203 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:09.057122    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-20a250-66d794c856-ql77v" event={"ID":"62dc4132-b26f-4b4f-a4c1-936dfa171b05","Type":"ContainerDied","Data":"e26652542987ee60abd614fb4f39b4a6608ac0c7eb97fda9d65efa30c5a2bed3"}
Nov 04 01:27:09.066485 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:09.066203    5068 generic.go:356] "Generic (PLEG): container finished" podID="1799b838-9323-4b50-b630-13f956913a62" containerID="f68b366e31940e058fb718a1124a6667f3fb93424461b0300b137d8ba7490af1" exitCode=0
Nov 04 01:27:09.066485 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:09.066374    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-27bc4f8b-7ccc8d95f6-cm86k" event={"ID":"1799b838-9323-4b50-b630-13f956913a62","Type":"ContainerDied","Data":"f68b366e31940e058fb718a1124a6667f3fb93424461b0300b137d8ba7490af1"}
Nov 04 01:27:09.206503 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:09.202160    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-27bc4f8b-7ccc8d95f6-cm86k"
Nov 04 01:27:09.288546 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:09.288504    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1799b838-9323-4b50-b630-13f956913a62-kube-api-access-f4bzk\" (UniqueName: \"kubernetes.io/projected/1799b838-9323-4b50-b630-13f956913a62-kube-api-access-f4bzk\") pod \"1799b838-9323-4b50-b630-13f956913a62\" (UID: \"1799b838-9323-4b50-b630-13f956913a62\") "
Nov 04 01:27:09.288969 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:09.288568    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1799b838-9323-4b50-b630-13f956913a62-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1799b838-9323-4b50-b630-13f956913a62-maas-api-tls\") pod \"1799b838-9323-4b50-b630-13f956913a62\" (UID: \"1799b838-9323-4b50-b630-13f956913a62\") "
Nov 04 01:27:09.288969 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:09.288641    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1799b838-9323-4b50-b630-13f956913a62-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1799b838-9323-4b50-b630-13f956913a62-openshift-service-ca\") pod \"1799b838-9323-4b50-b630-13f956913a62\" (UID: \"1799b838-9323-4b50-b630-13f956913a62\") "
Nov 04 01:27:09.288969 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:09.288777    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1799b838-9323-4b50-b630-13f956913a62-metrics-certs\" (UniqueName: \"kubernetes.io/secret/1799b838-9323-4b50-b630-13f956913a62-metrics-certs\") pod \"1799b838-9323-4b50-b630-13f956913a62\" (UID: \"1799b838-9323-4b50-b630-13f956913a62\") "
Nov 04 01:27:09.291096 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:09.291056    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1799b838-9323-4b50-b630-13f956913a62-openshift-service-ca" pod "1799b838-9323-4b50-b630-13f956913a62" (UID: "1799b838-9323-4b50-b630-13f956913a62"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:27:09.297164 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:09.297126    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1799b838-9323-4b50-b630-13f956913a62-metrics-certs" pod "1799b838-9323-4b50-b630-13f956913a62" (UID: "1799b838-9323-4b50-b630-13f956913a62"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:27:09.298505 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:09.298468    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1799b838-9323-4b50-b630-13f956913a62-kube-api-access-f4bzk" pod "1799b838-9323-4b50-b630-13f956913a62" (UID: "1799b838-9323-4b50-b630-13f956913a62"). InnerVolumeSpecName "kube-api-access-f4bzk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:27:09.299987 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:09.299961    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1799b838-9323-4b50-b630-13f956913a62-maas-api-tls" pod "1799b838-9323-4b50-b630-13f956913a62" (UID: "1799b838-9323-4b50-b630-13f956913a62"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:27:09.303518 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:09.303490    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-20a250-66d794c856-ql77v"
Nov 04 01:27:09.392655 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:09.390997    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/62dc4132-b26f-4b4f-a4c1-936dfa171b05-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/62dc4132-b26f-4b4f-a4c1-936dfa171b05-plugins-config-volume\") pod \"62dc4132-b26f-4b4f-a4c1-936dfa171b05\" (UID: \"62dc4132-b26f-4b4f-a4c1-936dfa171b05\") "
Nov 04 01:27:09.392655 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:09.391210    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/62dc4132-b26f-4b4f-a4c1-936dfa171b05-kube-api-access-bk2bz\" (UniqueName: \"kubernetes.io/projected/62dc4132-b26f-4b4f-a4c1-936dfa171b05-kube-api-access-bk2bz\") pod \"62dc4132-b26f-4b4f-a4c1-936dfa171b05\" (UID: \"62dc4132-b26f-4b4f-a4c1-936dfa171b05\") "
Nov 04 01:27:09.392655 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:09.391350    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62dc4132-b26f-4b4f-a4c1-936dfa171b05-plugins-config-volume" pod "62dc4132-b26f-4b4f-a4c1-936dfa171b05" (UID: "62dc4132-b26f-4b4f-a4c1-936dfa171b05"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:27:09.392655 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:09.391807    5068 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/62dc4132-b26f-4b4f-a4c1-936dfa171b05-plugins-config-volume\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:09.392655 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:09.391827    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f4bzk\" (UniqueName: \"kubernetes.io/projected/1799b838-9323-4b50-b630-13f956913a62-kube-api-access-f4bzk\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:09.392655 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:09.391842    5068 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1799b838-9323-4b50-b630-13f956913a62-maas-api-tls\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:09.392655 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:09.391856    5068 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1799b838-9323-4b50-b630-13f956913a62-openshift-service-ca\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:09.392655 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:09.391869    5068 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1799b838-9323-4b50-b630-13f956913a62-metrics-certs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:09.394778 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:09.394747    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62dc4132-b26f-4b4f-a4c1-936dfa171b05-kube-api-access-bk2bz" pod "62dc4132-b26f-4b4f-a4c1-936dfa171b05" (UID: "62dc4132-b26f-4b4f-a4c1-936dfa171b05"). InnerVolumeSpecName "kube-api-access-bk2bz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:27:09.494998 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:09.494952    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bk2bz\" (UniqueName: \"kubernetes.io/projected/62dc4132-b26f-4b4f-a4c1-936dfa171b05-kube-api-access-bk2bz\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:10.078634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:10.078219    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-20a250-66d794c856-ql77v" event={"ID":"62dc4132-b26f-4b4f-a4c1-936dfa171b05","Type":"ContainerDied","Data":"4e2c08ce793effcb9758839e961420d959849918a6e3467c2a77bf69ba14efdc"}
Nov 04 01:27:10.078634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:10.078303    5068 scope.go:122] "RemoveContainer" containerID="e26652542987ee60abd614fb4f39b4a6608ac0c7eb97fda9d65efa30c5a2bed3"
Nov 04 01:27:10.078922 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:10.078704    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-20a250-66d794c856-ql77v"
Nov 04 01:27:10.085037 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:10.081591    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-27bc4f8b-7ccc8d95f6-cm86k" event={"ID":"1799b838-9323-4b50-b630-13f956913a62","Type":"ContainerDied","Data":"20a73b61601e7113dc69fd3283eafcbfa30de6e1635a3914450cb25a8822e6ed"}
Nov 04 01:27:10.085037 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:10.081719    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-27bc4f8b-7ccc8d95f6-cm86k"
Nov 04 01:27:10.098392 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:10.098369    5068 scope.go:122] "RemoveContainer" containerID="f68b366e31940e058fb718a1124a6667f3fb93424461b0300b137d8ba7490af1"
Nov 04 01:27:10.136126 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:10.136083    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-27bc4f8b-7ccc8d95f6-cm86k"]
Nov 04 01:27:10.200127 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:10.200052    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-27bc4f8b-7ccc8d95f6-cm86k"]
Nov 04 01:27:10.207032 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:10.207003    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-20a250-66d794c856-ql77v"]
Nov 04 01:27:10.213712 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:10.213669    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-20a250-66d794c856-ql77v"]
Nov 04 01:27:12.114792 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:12.114752    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1799b838-9323-4b50-b630-13f956913a62" path="/var/lib/kubelet/pods/1799b838-9323-4b50-b630-13f956913a62/volumes"
Nov 04 01:27:12.115379 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:12.115357    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="62dc4132-b26f-4b4f-a4c1-936dfa171b05" path="/var/lib/kubelet/pods/62dc4132-b26f-4b4f-a4c1-936dfa171b05/volumes"
Nov 04 01:27:13.458838 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:27:13.458795    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:27:14.117603 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:14.117573    5068 generic.go:356] "Generic (PLEG): container finished" podID="31ede3d2-d690-4178-90a7-69c82caa9e2c" containerID="e7fc1527cb18c7e59ab1cb7a6412b2d875def441fcad4d82b7822571f6e0df4e" exitCode=0
Nov 04 01:27:14.117878 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:14.117643    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs" event={"ID":"31ede3d2-d690-4178-90a7-69c82caa9e2c","Type":"ContainerDied","Data":"e7fc1527cb18c7e59ab1cb7a6412b2d875def441fcad4d82b7822571f6e0df4e"}
Nov 04 01:27:14.780826 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:14.780795    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs"
Nov 04 01:27:14.912242 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:14.912210    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/31ede3d2-d690-4178-90a7-69c82caa9e2c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31ede3d2-d690-4178-90a7-69c82caa9e2c-istio-envoy\") pod \"31ede3d2-d690-4178-90a7-69c82caa9e2c\" (UID: \"31ede3d2-d690-4178-90a7-69c82caa9e2c\") "
Nov 04 01:27:14.912429 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:14.912256    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/31ede3d2-d690-4178-90a7-69c82caa9e2c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31ede3d2-d690-4178-90a7-69c82caa9e2c-credential-socket\") pod \"31ede3d2-d690-4178-90a7-69c82caa9e2c\" (UID: \"31ede3d2-d690-4178-90a7-69c82caa9e2c\") "
Nov 04 01:27:14.912429 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:14.912355    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/31ede3d2-d690-4178-90a7-69c82caa9e2c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31ede3d2-d690-4178-90a7-69c82caa9e2c-istiod-ca-cert\") pod \"31ede3d2-d690-4178-90a7-69c82caa9e2c\" (UID: \"31ede3d2-d690-4178-90a7-69c82caa9e2c\") "
Nov 04 01:27:14.912429 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:14.912415    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/31ede3d2-d690-4178-90a7-69c82caa9e2c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31ede3d2-d690-4178-90a7-69c82caa9e2c-istio-podinfo\") pod \"31ede3d2-d690-4178-90a7-69c82caa9e2c\" (UID: \"31ede3d2-d690-4178-90a7-69c82caa9e2c\") "
Nov 04 01:27:14.912620 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:14.912471    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/31ede3d2-d690-4178-90a7-69c82caa9e2c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31ede3d2-d690-4178-90a7-69c82caa9e2c-workload-certs\") pod \"31ede3d2-d690-4178-90a7-69c82caa9e2c\" (UID: \"31ede3d2-d690-4178-90a7-69c82caa9e2c\") "
Nov 04 01:27:14.912620 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:14.912547    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/31ede3d2-d690-4178-90a7-69c82caa9e2c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31ede3d2-d690-4178-90a7-69c82caa9e2c-workload-socket\") pod \"31ede3d2-d690-4178-90a7-69c82caa9e2c\" (UID: \"31ede3d2-d690-4178-90a7-69c82caa9e2c\") "
Nov 04 01:27:14.912729 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:14.912660    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31ede3d2-d690-4178-90a7-69c82caa9e2c-credential-socket" pod "31ede3d2-d690-4178-90a7-69c82caa9e2c" (UID: "31ede3d2-d690-4178-90a7-69c82caa9e2c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:27:14.912729 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:14.912684    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/31ede3d2-d690-4178-90a7-69c82caa9e2c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31ede3d2-d690-4178-90a7-69c82caa9e2c-istio-data\") pod \"31ede3d2-d690-4178-90a7-69c82caa9e2c\" (UID: \"31ede3d2-d690-4178-90a7-69c82caa9e2c\") "
Nov 04 01:27:14.912840 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:14.912786    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/31ede3d2-d690-4178-90a7-69c82caa9e2c-istio-token\" (UniqueName: \"kubernetes.io/projected/31ede3d2-d690-4178-90a7-69c82caa9e2c-istio-token\") pod \"31ede3d2-d690-4178-90a7-69c82caa9e2c\" (UID: \"31ede3d2-d690-4178-90a7-69c82caa9e2c\") "
Nov 04 01:27:14.912898 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:14.912802    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31ede3d2-d690-4178-90a7-69c82caa9e2c-workload-certs" pod "31ede3d2-d690-4178-90a7-69c82caa9e2c" (UID: "31ede3d2-d690-4178-90a7-69c82caa9e2c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:27:14.912898 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:14.912860    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31ede3d2-d690-4178-90a7-69c82caa9e2c-istiod-ca-cert" pod "31ede3d2-d690-4178-90a7-69c82caa9e2c" (UID: "31ede3d2-d690-4178-90a7-69c82caa9e2c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:27:14.912993 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:14.912867    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31ede3d2-d690-4178-90a7-69c82caa9e2c-workload-socket" pod "31ede3d2-d690-4178-90a7-69c82caa9e2c" (UID: "31ede3d2-d690-4178-90a7-69c82caa9e2c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:27:14.912993 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:14.912922    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31ede3d2-d690-4178-90a7-69c82caa9e2c-istio-data" pod "31ede3d2-d690-4178-90a7-69c82caa9e2c" (UID: "31ede3d2-d690-4178-90a7-69c82caa9e2c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:27:14.913147 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:14.913129    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/31ede3d2-d690-4178-90a7-69c82caa9e2c-kube-api-access-q2ssj\" (UniqueName: \"kubernetes.io/projected/31ede3d2-d690-4178-90a7-69c82caa9e2c-kube-api-access-q2ssj\") pod \"31ede3d2-d690-4178-90a7-69c82caa9e2c\" (UID: \"31ede3d2-d690-4178-90a7-69c82caa9e2c\") "
Nov 04 01:27:14.913895 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:14.913867    5068 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31ede3d2-d690-4178-90a7-69c82caa9e2c-credential-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:14.914029 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:14.913897    5068 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31ede3d2-d690-4178-90a7-69c82caa9e2c-istiod-ca-cert\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:14.914029 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:14.913911    5068 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31ede3d2-d690-4178-90a7-69c82caa9e2c-workload-certs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:14.914029 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:14.913925    5068 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31ede3d2-d690-4178-90a7-69c82caa9e2c-workload-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:14.914029 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:14.913939    5068 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31ede3d2-d690-4178-90a7-69c82caa9e2c-istio-data\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:14.914711 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:14.914681    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/31ede3d2-d690-4178-90a7-69c82caa9e2c-istio-podinfo" pod "31ede3d2-d690-4178-90a7-69c82caa9e2c" (UID: "31ede3d2-d690-4178-90a7-69c82caa9e2c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:27:14.914812 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:14.914757    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31ede3d2-d690-4178-90a7-69c82caa9e2c-istio-envoy" pod "31ede3d2-d690-4178-90a7-69c82caa9e2c" (UID: "31ede3d2-d690-4178-90a7-69c82caa9e2c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:27:14.914886 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:14.914866    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31ede3d2-d690-4178-90a7-69c82caa9e2c-istio-token" pod "31ede3d2-d690-4178-90a7-69c82caa9e2c" (UID: "31ede3d2-d690-4178-90a7-69c82caa9e2c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:27:14.915255 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:14.915241    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31ede3d2-d690-4178-90a7-69c82caa9e2c-kube-api-access-q2ssj" pod "31ede3d2-d690-4178-90a7-69c82caa9e2c" (UID: "31ede3d2-d690-4178-90a7-69c82caa9e2c"). InnerVolumeSpecName "kube-api-access-q2ssj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:27:15.015002 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:15.014966    5068 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31ede3d2-d690-4178-90a7-69c82caa9e2c-istio-envoy\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:15.015002 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:15.014999    5068 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31ede3d2-d690-4178-90a7-69c82caa9e2c-istio-podinfo\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:15.015002 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:15.015010    5068 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31ede3d2-d690-4178-90a7-69c82caa9e2c-istio-token\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:15.015233 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:15.015019    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q2ssj\" (UniqueName: \"kubernetes.io/projected/31ede3d2-d690-4178-90a7-69c82caa9e2c-kube-api-access-q2ssj\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:15.126386 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:15.126359    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs"
Nov 04 01:27:15.126563 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:15.126389    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs" event={"ID":"31ede3d2-d690-4178-90a7-69c82caa9e2c","Type":"ContainerDied","Data":"41efb186d1ffdb66fc8ffb674afcbfbec3d9fe89c650182f957c40c56a2cf53d"}
Nov 04 01:27:15.126563 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:15.126455    5068 scope.go:122] "RemoveContainer" containerID="e7fc1527cb18c7e59ab1cb7a6412b2d875def441fcad4d82b7822571f6e0df4e"
Nov 04 01:27:15.157975 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:15.157949    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs"]
Nov 04 01:27:15.165185 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:15.165158    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-27bc4f8b-openshift-default-948b689b9-ltrcs"]
Nov 04 01:27:16.118877 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:16.118843    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="31ede3d2-d690-4178-90a7-69c82caa9e2c" path="/var/lib/kubelet/pods/31ede3d2-d690-4178-90a7-69c82caa9e2c/volumes"
Nov 04 01:27:17.639648 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:17.636677    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89"]
Nov 04 01:27:18.154817 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:18.154783    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:27:18.983194 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:18.983148    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p"]
Nov 04 01:27:18.983718 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:18.983404    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p" podUID="fa8debd2-b803-412c-9e4c-53917a748a59" containerName="istio-proxy" containerID="cri-o://17f6a25c0747c6f62e4c96a5682b89b9994e762152c2f2bad4e9f77a4b92b634" gracePeriod=30
Nov 04 01:27:24.289036 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:24.289005    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p"
Nov 04 01:27:24.317579 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:24.317534    5068 generic.go:356] "Generic (PLEG): container finished" podID="fa8debd2-b803-412c-9e4c-53917a748a59" containerID="17f6a25c0747c6f62e4c96a5682b89b9994e762152c2f2bad4e9f77a4b92b634" exitCode=0
Nov 04 01:27:24.317800 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:24.317755    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p" event={"ID":"fa8debd2-b803-412c-9e4c-53917a748a59","Type":"ContainerDied","Data":"17f6a25c0747c6f62e4c96a5682b89b9994e762152c2f2bad4e9f77a4b92b634"}
Nov 04 01:27:24.317878 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:24.317813    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p" event={"ID":"fa8debd2-b803-412c-9e4c-53917a748a59","Type":"ContainerDied","Data":"9733fc232e7cf4befebbb534865e91f1934772ee9b9e6794c7affb8bec6213a7"}
Nov 04 01:27:24.317878 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:24.317842    5068 scope.go:122] "RemoveContainer" containerID="17f6a25c0747c6f62e4c96a5682b89b9994e762152c2f2bad4e9f77a4b92b634"
Nov 04 01:27:24.318021 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:24.318002    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p"
Nov 04 01:27:24.333023 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:24.332980    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fa8debd2-b803-412c-9e4c-53917a748a59-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa8debd2-b803-412c-9e4c-53917a748a59-credential-socket\") pod \"fa8debd2-b803-412c-9e4c-53917a748a59\" (UID: \"fa8debd2-b803-412c-9e4c-53917a748a59\") "
Nov 04 01:27:24.333170 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:24.333026    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fa8debd2-b803-412c-9e4c-53917a748a59-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa8debd2-b803-412c-9e4c-53917a748a59-istio-envoy\") pod \"fa8debd2-b803-412c-9e4c-53917a748a59\" (UID: \"fa8debd2-b803-412c-9e4c-53917a748a59\") "
Nov 04 01:27:24.333170 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:24.333111    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fa8debd2-b803-412c-9e4c-53917a748a59-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa8debd2-b803-412c-9e4c-53917a748a59-istio-data\") pod \"fa8debd2-b803-412c-9e4c-53917a748a59\" (UID: \"fa8debd2-b803-412c-9e4c-53917a748a59\") "
Nov 04 01:27:24.333170 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:24.333141    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fa8debd2-b803-412c-9e4c-53917a748a59-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa8debd2-b803-412c-9e4c-53917a748a59-workload-socket\") pod \"fa8debd2-b803-412c-9e4c-53917a748a59\" (UID: \"fa8debd2-b803-412c-9e4c-53917a748a59\") "
Nov 04 01:27:24.333348 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:24.333213    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fa8debd2-b803-412c-9e4c-53917a748a59-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa8debd2-b803-412c-9e4c-53917a748a59-istio-podinfo\") pod \"fa8debd2-b803-412c-9e4c-53917a748a59\" (UID: \"fa8debd2-b803-412c-9e4c-53917a748a59\") "
Nov 04 01:27:24.333348 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:24.333269    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fa8debd2-b803-412c-9e4c-53917a748a59-istio-token\" (UniqueName: \"kubernetes.io/projected/fa8debd2-b803-412c-9e4c-53917a748a59-istio-token\") pod \"fa8debd2-b803-412c-9e4c-53917a748a59\" (UID: \"fa8debd2-b803-412c-9e4c-53917a748a59\") "
Nov 04 01:27:24.333348 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:24.333316    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fa8debd2-b803-412c-9e4c-53917a748a59-kube-api-access-t45c2\" (UniqueName: \"kubernetes.io/projected/fa8debd2-b803-412c-9e4c-53917a748a59-kube-api-access-t45c2\") pod \"fa8debd2-b803-412c-9e4c-53917a748a59\" (UID: \"fa8debd2-b803-412c-9e4c-53917a748a59\") "
Nov 04 01:27:24.333516 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:24.333411    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fa8debd2-b803-412c-9e4c-53917a748a59-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa8debd2-b803-412c-9e4c-53917a748a59-workload-certs\") pod \"fa8debd2-b803-412c-9e4c-53917a748a59\" (UID: \"fa8debd2-b803-412c-9e4c-53917a748a59\") "
Nov 04 01:27:24.333516 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:24.333467    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa8debd2-b803-412c-9e4c-53917a748a59-credential-socket" pod "fa8debd2-b803-412c-9e4c-53917a748a59" (UID: "fa8debd2-b803-412c-9e4c-53917a748a59"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:27:24.333669 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:24.333586    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fa8debd2-b803-412c-9e4c-53917a748a59-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa8debd2-b803-412c-9e4c-53917a748a59-istiod-ca-cert\") pod \"fa8debd2-b803-412c-9e4c-53917a748a59\" (UID: \"fa8debd2-b803-412c-9e4c-53917a748a59\") "
Nov 04 01:27:24.333916 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:24.333803    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa8debd2-b803-412c-9e4c-53917a748a59-workload-certs" pod "fa8debd2-b803-412c-9e4c-53917a748a59" (UID: "fa8debd2-b803-412c-9e4c-53917a748a59"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:27:24.334187 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:24.334048    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa8debd2-b803-412c-9e4c-53917a748a59-workload-socket" pod "fa8debd2-b803-412c-9e4c-53917a748a59" (UID: "fa8debd2-b803-412c-9e4c-53917a748a59"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:27:24.334364 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:24.334223    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa8debd2-b803-412c-9e4c-53917a748a59-istio-data" pod "fa8debd2-b803-412c-9e4c-53917a748a59" (UID: "fa8debd2-b803-412c-9e4c-53917a748a59"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:27:24.334825 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:24.334805    5068 scope.go:122] "RemoveContainer" containerID="17f6a25c0747c6f62e4c96a5682b89b9994e762152c2f2bad4e9f77a4b92b634"
Nov 04 01:27:24.335045 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:24.334851    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa8debd2-b803-412c-9e4c-53917a748a59-istiod-ca-cert" pod "fa8debd2-b803-412c-9e4c-53917a748a59" (UID: "fa8debd2-b803-412c-9e4c-53917a748a59"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:27:24.335195 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:24.335178    5068 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa8debd2-b803-412c-9e4c-53917a748a59-istiod-ca-cert\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:24.335269 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:24.335201    5068 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa8debd2-b803-412c-9e4c-53917a748a59-credential-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:24.335568 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:24.335217    5068 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa8debd2-b803-412c-9e4c-53917a748a59-istio-data\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:24.335718 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:24.335582    5068 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa8debd2-b803-412c-9e4c-53917a748a59-workload-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:24.335718 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:24.335597    5068 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa8debd2-b803-412c-9e4c-53917a748a59-workload-certs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:24.335718 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:27:24.335512    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17f6a25c0747c6f62e4c96a5682b89b9994e762152c2f2bad4e9f77a4b92b634\": container with ID starting with 17f6a25c0747c6f62e4c96a5682b89b9994e762152c2f2bad4e9f77a4b92b634 not found: ID does not exist" containerID="17f6a25c0747c6f62e4c96a5682b89b9994e762152c2f2bad4e9f77a4b92b634"
Nov 04 01:27:24.335718 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:24.335651    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17f6a25c0747c6f62e4c96a5682b89b9994e762152c2f2bad4e9f77a4b92b634"} err="failed to get container status \"17f6a25c0747c6f62e4c96a5682b89b9994e762152c2f2bad4e9f77a4b92b634\": rpc error: code = NotFound desc = could not find container \"17f6a25c0747c6f62e4c96a5682b89b9994e762152c2f2bad4e9f77a4b92b634\": container with ID starting with 17f6a25c0747c6f62e4c96a5682b89b9994e762152c2f2bad4e9f77a4b92b634 not found: ID does not exist"
Nov 04 01:27:24.337712 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:24.337680    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fa8debd2-b803-412c-9e4c-53917a748a59-istio-podinfo" pod "fa8debd2-b803-412c-9e4c-53917a748a59" (UID: "fa8debd2-b803-412c-9e4c-53917a748a59"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:27:24.338090 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:24.338067    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa8debd2-b803-412c-9e4c-53917a748a59-istio-token" pod "fa8debd2-b803-412c-9e4c-53917a748a59" (UID: "fa8debd2-b803-412c-9e4c-53917a748a59"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:27:24.339341 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:24.339313    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa8debd2-b803-412c-9e4c-53917a748a59-kube-api-access-t45c2" pod "fa8debd2-b803-412c-9e4c-53917a748a59" (UID: "fa8debd2-b803-412c-9e4c-53917a748a59"). InnerVolumeSpecName "kube-api-access-t45c2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:27:24.340043 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:24.340020    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa8debd2-b803-412c-9e4c-53917a748a59-istio-envoy" pod "fa8debd2-b803-412c-9e4c-53917a748a59" (UID: "fa8debd2-b803-412c-9e4c-53917a748a59"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:27:24.437175 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:24.437142    5068 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa8debd2-b803-412c-9e4c-53917a748a59-istio-envoy\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:24.437175 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:24.437171    5068 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa8debd2-b803-412c-9e4c-53917a748a59-istio-podinfo\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:24.437175 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:24.437181    5068 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa8debd2-b803-412c-9e4c-53917a748a59-istio-token\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:24.437396 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:24.437190    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t45c2\" (UniqueName: \"kubernetes.io/projected/fa8debd2-b803-412c-9e4c-53917a748a59-kube-api-access-t45c2\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:24.645515 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:24.645468    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p"]
Nov 04 01:27:24.650301 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:24.650269    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-9902c3-openshift-default-58c9d957d5-lvf6p"]
Nov 04 01:27:26.089956 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:26.089921    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8"]
Nov 04 01:27:26.117015 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:26.116983    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fa8debd2-b803-412c-9e4c-53917a748a59" path="/var/lib/kubelet/pods/fa8debd2-b803-412c-9e4c-53917a748a59/volumes"
Nov 04 01:27:31.853707 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:31.853655    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-cffa80-564fd64f66-qhhzq"]
Nov 04 01:27:31.865006 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:31.864963    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-cffa80-9b645ff4c-2pj8n"]
Nov 04 01:27:32.799640 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:32.797155    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89" podUID="6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d" containerName="main" containerID="cri-o://4d663e0a2b93d554eeac27ea1992e39fd5390fbca572ce885849dd2d09a9d98c" gracePeriod=45
Nov 04 01:27:33.401928 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:33.401886    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89" podUID="6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": dial tcp 10.132.0.78:8000: connect: connection refused"
Nov 04 01:27:33.524543 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:33.524500    5068 generic.go:356] "Generic (PLEG): container finished" podID="6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d" containerID="4d663e0a2b93d554eeac27ea1992e39fd5390fbca572ce885849dd2d09a9d98c" exitCode=0
Nov 04 01:27:33.524787 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:33.524636    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89" event={"ID":"6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d","Type":"ContainerDied","Data":"4d663e0a2b93d554eeac27ea1992e39fd5390fbca572ce885849dd2d09a9d98c"}
Nov 04 01:27:33.740515 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:33.740490    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89"
Nov 04 01:27:33.894945 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:33.894916    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d-dshm\" (UniqueName: \"kubernetes.io/empty-dir/6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d-dshm\") pod \"6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d\" (UID: \"6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d\") "
Nov 04 01:27:33.895137 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:33.894985    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d-home\" (UniqueName: \"kubernetes.io/empty-dir/6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d-home\") pod \"6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d\" (UID: \"6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d\") "
Nov 04 01:27:33.895137 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:33.895061    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d-tmp-dir\") pod \"6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d\" (UID: \"6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d\") "
Nov 04 01:27:33.895137 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:33.895093    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d-kube-api-access-xc45k\" (UniqueName: \"kubernetes.io/projected/6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d-kube-api-access-xc45k\") pod \"6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d\" (UID: \"6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d\") "
Nov 04 01:27:33.895137 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:33.895120    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d-tls-certs\" (UniqueName: \"kubernetes.io/secret/6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d-tls-certs\") pod \"6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d\" (UID: \"6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d\") "
Nov 04 01:27:33.895365 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:33.895283    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d-model-cache\") pod \"6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d\" (UID: \"6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d\") "
Nov 04 01:27:33.895365 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:33.895303    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d-home" pod "6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d" (UID: "6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:27:33.895476 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:33.895454    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d-tmp-dir" pod "6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d" (UID: "6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:27:33.895703 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:33.895685    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d-model-cache" pod "6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d" (UID: "6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:27:33.895947 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:33.895928    5068 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d-model-cache\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:33.896010 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:33.895955    5068 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d-home\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:33.896010 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:33.895970    5068 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d-tmp-dir\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:33.897900 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:33.897861    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d-kube-api-access-xc45k" pod "6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d" (UID: "6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d"). InnerVolumeSpecName "kube-api-access-xc45k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:27:33.898109 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:33.898062    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d-tls-certs" pod "6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d" (UID: "6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:27:33.898109 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:33.898083    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d-dshm" pod "6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d" (UID: "6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:27:34.001944 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:34.001805    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xc45k\" (UniqueName: \"kubernetes.io/projected/6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d-kube-api-access-xc45k\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:34.001944 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:34.001840    5068 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d-tls-certs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:34.001944 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:34.001861    5068 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d-dshm\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:34.560632 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:34.559765    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89"
Nov 04 01:27:34.560632 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:34.559776    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89" event={"ID":"6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d","Type":"ContainerDied","Data":"4847263e8d8805b13ed0e2bcdba5d2bcb0880fbc41f8d5925b2d13bfc4efd4f5"}
Nov 04 01:27:34.560632 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:34.559831    5068 scope.go:122] "RemoveContainer" containerID="4d663e0a2b93d554eeac27ea1992e39fd5390fbca572ce885849dd2d09a9d98c"
Nov 04 01:27:34.614630 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:34.607680    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89"]
Nov 04 01:27:34.614630 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:34.608535    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-cffa80/distinct-w2-cffa80-kserve-64996cfcbc-z7w89"]
Nov 04 01:27:36.116925 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:36.116892    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d" path="/var/lib/kubelet/pods/6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d/volumes"
Nov 04 01:27:36.854203 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:36.854145    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w2-cffa80-564fd64f66-qhhzq" podUID="1b4a1b43-a286-436d-bafe-a10d9a57a4d2" containerName="payload-processing" containerID="cri-o://3abb2e93264e4e70655b9bbadb6fae705629758c380b56c46258f3da781ab15d" gracePeriod=25
Nov 04 01:27:36.865495 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:36.865446    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-cffa80-9b645ff4c-2pj8n" podUID="841e9fef-46d6-4625-9b72-14b4e0eeb7dd" containerName="payload-pre-processing" containerID="cri-o://586e192813d95194bcadb83bf3a3cd34721c99f089734ca29eff252dbb718dab" gracePeriod=25
Nov 04 01:27:38.422716 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:38.422681    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w4-029e8c-574bd8d98-79ssg"]
Nov 04 01:27:41.113417 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:41.113352    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8" podUID="eb9132e5-b90b-46fb-b86c-ca713419906d" containerName="main" containerID="cri-o://13561f0b11491d1e4e982cc0706fd0cbd8391a6375e8139f534e65f151e709a8" gracePeriod=45
Nov 04 01:27:41.422601 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:41.422567    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8"
Nov 04 01:27:41.472568 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:41.472519    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb9132e5-b90b-46fb-b86c-ca713419906d-dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb9132e5-b90b-46fb-b86c-ca713419906d-dshm\") pod \"eb9132e5-b90b-46fb-b86c-ca713419906d\" (UID: \"eb9132e5-b90b-46fb-b86c-ca713419906d\") "
Nov 04 01:27:41.472751 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:41.472579    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb9132e5-b90b-46fb-b86c-ca713419906d-home\" (UniqueName: \"kubernetes.io/empty-dir/eb9132e5-b90b-46fb-b86c-ca713419906d-home\") pod \"eb9132e5-b90b-46fb-b86c-ca713419906d\" (UID: \"eb9132e5-b90b-46fb-b86c-ca713419906d\") "
Nov 04 01:27:41.472751 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:41.472719    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb9132e5-b90b-46fb-b86c-ca713419906d-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb9132e5-b90b-46fb-b86c-ca713419906d-model-cache\") pod \"eb9132e5-b90b-46fb-b86c-ca713419906d\" (UID: \"eb9132e5-b90b-46fb-b86c-ca713419906d\") "
Nov 04 01:27:41.472904 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:41.472778    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb9132e5-b90b-46fb-b86c-ca713419906d-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb9132e5-b90b-46fb-b86c-ca713419906d-tmp-dir\") pod \"eb9132e5-b90b-46fb-b86c-ca713419906d\" (UID: \"eb9132e5-b90b-46fb-b86c-ca713419906d\") "
Nov 04 01:27:41.472904 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:41.472865    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/eb9132e5-b90b-46fb-b86c-ca713419906d-tls-certs\" (UniqueName: \"kubernetes.io/secret/eb9132e5-b90b-46fb-b86c-ca713419906d-tls-certs\") pod \"eb9132e5-b90b-46fb-b86c-ca713419906d\" (UID: \"eb9132e5-b90b-46fb-b86c-ca713419906d\") "
Nov 04 01:27:41.473011 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:41.472903    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eb9132e5-b90b-46fb-b86c-ca713419906d-kube-api-access-5vddw\" (UniqueName: \"kubernetes.io/projected/eb9132e5-b90b-46fb-b86c-ca713419906d-kube-api-access-5vddw\") pod \"eb9132e5-b90b-46fb-b86c-ca713419906d\" (UID: \"eb9132e5-b90b-46fb-b86c-ca713419906d\") "
Nov 04 01:27:41.474984 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:41.474802    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb9132e5-b90b-46fb-b86c-ca713419906d-model-cache" pod "eb9132e5-b90b-46fb-b86c-ca713419906d" (UID: "eb9132e5-b90b-46fb-b86c-ca713419906d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:27:41.475321 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:41.475156    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb9132e5-b90b-46fb-b86c-ca713419906d-home" pod "eb9132e5-b90b-46fb-b86c-ca713419906d" (UID: "eb9132e5-b90b-46fb-b86c-ca713419906d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:27:41.475503 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:41.475468    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb9132e5-b90b-46fb-b86c-ca713419906d-tmp-dir" pod "eb9132e5-b90b-46fb-b86c-ca713419906d" (UID: "eb9132e5-b90b-46fb-b86c-ca713419906d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:27:41.477879 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:41.477849    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb9132e5-b90b-46fb-b86c-ca713419906d-kube-api-access-5vddw" pod "eb9132e5-b90b-46fb-b86c-ca713419906d" (UID: "eb9132e5-b90b-46fb-b86c-ca713419906d"). InnerVolumeSpecName "kube-api-access-5vddw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:27:41.487325 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:41.482188    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb9132e5-b90b-46fb-b86c-ca713419906d-dshm" pod "eb9132e5-b90b-46fb-b86c-ca713419906d" (UID: "eb9132e5-b90b-46fb-b86c-ca713419906d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:27:41.487325 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:41.482452    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb9132e5-b90b-46fb-b86c-ca713419906d-tls-certs" pod "eb9132e5-b90b-46fb-b86c-ca713419906d" (UID: "eb9132e5-b90b-46fb-b86c-ca713419906d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:27:41.578249 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:41.578125    5068 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb9132e5-b90b-46fb-b86c-ca713419906d-dshm\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:41.578249 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:41.578162    5068 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb9132e5-b90b-46fb-b86c-ca713419906d-home\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:41.578249 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:41.578177    5068 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb9132e5-b90b-46fb-b86c-ca713419906d-model-cache\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:41.578249 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:41.578191    5068 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb9132e5-b90b-46fb-b86c-ca713419906d-tmp-dir\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:41.578249 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:41.578206    5068 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb9132e5-b90b-46fb-b86c-ca713419906d-tls-certs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:41.578249 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:41.578220    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5vddw\" (UniqueName: \"kubernetes.io/projected/eb9132e5-b90b-46fb-b86c-ca713419906d-kube-api-access-5vddw\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:41.622113 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:41.622074    5068 generic.go:356] "Generic (PLEG): container finished" podID="eb9132e5-b90b-46fb-b86c-ca713419906d" containerID="13561f0b11491d1e4e982cc0706fd0cbd8391a6375e8139f534e65f151e709a8" exitCode=0
Nov 04 01:27:41.622309 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:41.622133    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8"
Nov 04 01:27:41.622309 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:41.622126    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8" event={"ID":"eb9132e5-b90b-46fb-b86c-ca713419906d","Type":"ContainerDied","Data":"13561f0b11491d1e4e982cc0706fd0cbd8391a6375e8139f534e65f151e709a8"}
Nov 04 01:27:41.622309 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:41.622250    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8" event={"ID":"eb9132e5-b90b-46fb-b86c-ca713419906d","Type":"ContainerDied","Data":"000556ba2d7d1aa6d188040a4b6a38ead1078b6e3f6825d30b74f875e447ac43"}
Nov 04 01:27:41.622309 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:41.622285    5068 scope.go:122] "RemoveContainer" containerID="13561f0b11491d1e4e982cc0706fd0cbd8391a6375e8139f534e65f151e709a8"
Nov 04 01:27:41.636000 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:41.635980    5068 scope.go:122] "RemoveContainer" containerID="13561f0b11491d1e4e982cc0706fd0cbd8391a6375e8139f534e65f151e709a8"
Nov 04 01:27:41.636265 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:27:41.636243    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13561f0b11491d1e4e982cc0706fd0cbd8391a6375e8139f534e65f151e709a8\": container with ID starting with 13561f0b11491d1e4e982cc0706fd0cbd8391a6375e8139f534e65f151e709a8 not found: ID does not exist" containerID="13561f0b11491d1e4e982cc0706fd0cbd8391a6375e8139f534e65f151e709a8"
Nov 04 01:27:41.636352 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:41.636270    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13561f0b11491d1e4e982cc0706fd0cbd8391a6375e8139f534e65f151e709a8"} err="failed to get container status \"13561f0b11491d1e4e982cc0706fd0cbd8391a6375e8139f534e65f151e709a8\": rpc error: code = NotFound desc = could not find container \"13561f0b11491d1e4e982cc0706fd0cbd8391a6375e8139f534e65f151e709a8\": container with ID starting with 13561f0b11491d1e4e982cc0706fd0cbd8391a6375e8139f534e65f151e709a8 not found: ID does not exist"
Nov 04 01:27:41.652309 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:41.652234    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8"]
Nov 04 01:27:41.659578 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:41.658955    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8"]
Nov 04 01:27:42.114619 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:42.114587    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eb9132e5-b90b-46fb-b86c-ca713419906d" path="/var/lib/kubelet/pods/eb9132e5-b90b-46fb-b86c-ca713419906d/volumes"
Nov 04 01:27:42.367642 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:42.367515    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-875528/distinct-2-w5-875528-kserve-c595b45c4-rg6h8" podUID="eb9132e5-b90b-46fb-b86c-ca713419906d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:27:42.415313 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:42.415278    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w2-cffa80-564fd64f66-qhhzq" podUID="1b4a1b43-a286-436d-bafe-a10d9a57a4d2" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:27:42.415513 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:42.415411    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-cffa80-9b645ff4c-2pj8n" podUID="841e9fef-46d6-4625-9b72-14b4e0eeb7dd" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:27:43.423097 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:43.422983    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-029e8c-574bd8d98-79ssg" podUID="1f176eb7-f53c-48c8-a7dc-0e9212448556" containerName="payload-processing" containerID="cri-o://3d73c9ce8d931148f2f166487090aa05731180187966087300f707241c65887a" gracePeriod=25
Nov 04 01:27:43.454861 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:27:43.454832    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:27:43.643758 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:43.643508    5068 generic.go:356] "Generic (PLEG): container finished" podID="1f176eb7-f53c-48c8-a7dc-0e9212448556" containerID="3d73c9ce8d931148f2f166487090aa05731180187966087300f707241c65887a" exitCode=0
Nov 04 01:27:43.644095 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:43.644022    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-029e8c-574bd8d98-79ssg" event={"ID":"1f176eb7-f53c-48c8-a7dc-0e9212448556","Type":"ContainerDied","Data":"3d73c9ce8d931148f2f166487090aa05731180187966087300f707241c65887a"}
Nov 04 01:27:43.764368 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:43.764338    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-029e8c-574bd8d98-79ssg"
Nov 04 01:27:43.927634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:43.918065    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1f176eb7-f53c-48c8-a7dc-0e9212448556-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f176eb7-f53c-48c8-a7dc-0e9212448556-plugins-config-volume\") pod \"1f176eb7-f53c-48c8-a7dc-0e9212448556\" (UID: \"1f176eb7-f53c-48c8-a7dc-0e9212448556\") "
Nov 04 01:27:43.927634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:43.918336    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1f176eb7-f53c-48c8-a7dc-0e9212448556-kube-api-access-fsqhh\" (UniqueName: \"kubernetes.io/projected/1f176eb7-f53c-48c8-a7dc-0e9212448556-kube-api-access-fsqhh\") pod \"1f176eb7-f53c-48c8-a7dc-0e9212448556\" (UID: \"1f176eb7-f53c-48c8-a7dc-0e9212448556\") "
Nov 04 01:27:43.938843 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:43.938801    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f176eb7-f53c-48c8-a7dc-0e9212448556-plugins-config-volume" pod "1f176eb7-f53c-48c8-a7dc-0e9212448556" (UID: "1f176eb7-f53c-48c8-a7dc-0e9212448556"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:27:44.025664 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:44.025573    5068 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f176eb7-f53c-48c8-a7dc-0e9212448556-plugins-config-volume\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:44.049632 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:44.029354    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f176eb7-f53c-48c8-a7dc-0e9212448556-kube-api-access-fsqhh" pod "1f176eb7-f53c-48c8-a7dc-0e9212448556" (UID: "1f176eb7-f53c-48c8-a7dc-0e9212448556"). InnerVolumeSpecName "kube-api-access-fsqhh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:27:44.136762 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:44.136732    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fsqhh\" (UniqueName: \"kubernetes.io/projected/1f176eb7-f53c-48c8-a7dc-0e9212448556-kube-api-access-fsqhh\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:44.349362 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:44.349322    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tr8b2_21d1d23b-7b6e-4ab5-9593-cf7784fffea2/console-operator/2.log"
Nov 04 01:27:44.452813 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:44.452785    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tr8b2_21d1d23b-7b6e-4ab5-9593-cf7784fffea2/console-operator/2.log"
Nov 04 01:27:44.653284 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:44.653192    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-029e8c-574bd8d98-79ssg"
Nov 04 01:27:44.653535 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:44.653199    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-029e8c-574bd8d98-79ssg" event={"ID":"1f176eb7-f53c-48c8-a7dc-0e9212448556","Type":"ContainerDied","Data":"fbc146bdffc9e198c0c8eb4f95979e607a5060a0fae4340279dbf2fd5522094a"}
Nov 04 01:27:44.653535 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:44.653370    5068 scope.go:122] "RemoveContainer" containerID="3d73c9ce8d931148f2f166487090aa05731180187966087300f707241c65887a"
Nov 04 01:27:44.676167 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:44.676045    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w4-029e8c-574bd8d98-79ssg"]
Nov 04 01:27:44.681264 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:44.681241    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w4-029e8c-574bd8d98-79ssg"]
Nov 04 01:27:46.114654 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:46.114595    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1f176eb7-f53c-48c8-a7dc-0e9212448556" path="/var/lib/kubelet/pods/1f176eb7-f53c-48c8-a7dc-0e9212448556/volumes"
Nov 04 01:27:49.511793 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:49.511751    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-cffa80-8dcbb5c89-jx699"]
Nov 04 01:27:49.512279 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:49.512099    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-cffa80-8dcbb5c89-jx699" podUID="c4f8ccdf-b225-453b-87a9-9c6212d77802" containerName="maas-api" containerID="cri-o://3f715d388c4658cde8f49e563a68f41146c83409b5beefaa31b184d321ee5611" gracePeriod=30
Nov 04 01:27:49.634144 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:49.634101    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-029e8c-8c7f458c5-gvzpd"]
Nov 04 01:27:49.635956 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:49.635894    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-029e8c-8c7f458c5-gvzpd" podUID="39c35ec4-5c1b-472e-9e73-991258bb835d" containerName="maas-api" containerID="cri-o://91e13ec9894502a467877006e22026d0f6952c4a721da97d920753bcef6f951c" gracePeriod=30
Nov 04 01:27:49.753604 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:49.753563    5068 generic.go:356] "Generic (PLEG): container finished" podID="c4f8ccdf-b225-453b-87a9-9c6212d77802" containerID="3f715d388c4658cde8f49e563a68f41146c83409b5beefaa31b184d321ee5611" exitCode=0
Nov 04 01:27:49.753851 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:49.753831    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-cffa80-8dcbb5c89-jx699" event={"ID":"c4f8ccdf-b225-453b-87a9-9c6212d77802","Type":"ContainerDied","Data":"3f715d388c4658cde8f49e563a68f41146c83409b5beefaa31b184d321ee5611"}
Nov 04 01:27:49.963774 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:49.963742    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-cffa80-8dcbb5c89-jx699"
Nov 04 01:27:50.064692 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:50.059586    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-029e8c-8c7f458c5-gvzpd"
Nov 04 01:27:50.092007 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:50.091970    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c4f8ccdf-b225-453b-87a9-9c6212d77802-metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4f8ccdf-b225-453b-87a9-9c6212d77802-metrics-certs\") pod \"c4f8ccdf-b225-453b-87a9-9c6212d77802\" (UID: \"c4f8ccdf-b225-453b-87a9-9c6212d77802\") "
Nov 04 01:27:50.094749 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:50.094716    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c4f8ccdf-b225-453b-87a9-9c6212d77802-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4f8ccdf-b225-453b-87a9-9c6212d77802-maas-api-tls\") pod \"c4f8ccdf-b225-453b-87a9-9c6212d77802\" (UID: \"c4f8ccdf-b225-453b-87a9-9c6212d77802\") "
Nov 04 01:27:50.094921 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:50.094767    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c4f8ccdf-b225-453b-87a9-9c6212d77802-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4f8ccdf-b225-453b-87a9-9c6212d77802-openshift-service-ca\") pod \"c4f8ccdf-b225-453b-87a9-9c6212d77802\" (UID: \"c4f8ccdf-b225-453b-87a9-9c6212d77802\") "
Nov 04 01:27:50.095010 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:50.094989    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c4f8ccdf-b225-453b-87a9-9c6212d77802-kube-api-access-d9csg\" (UniqueName: \"kubernetes.io/projected/c4f8ccdf-b225-453b-87a9-9c6212d77802-kube-api-access-d9csg\") pod \"c4f8ccdf-b225-453b-87a9-9c6212d77802\" (UID: \"c4f8ccdf-b225-453b-87a9-9c6212d77802\") "
Nov 04 01:27:50.097372 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:50.097337    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4f8ccdf-b225-453b-87a9-9c6212d77802-openshift-service-ca" pod "c4f8ccdf-b225-453b-87a9-9c6212d77802" (UID: "c4f8ccdf-b225-453b-87a9-9c6212d77802"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:27:50.097971 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:50.097943    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4f8ccdf-b225-453b-87a9-9c6212d77802-maas-api-tls" pod "c4f8ccdf-b225-453b-87a9-9c6212d77802" (UID: "c4f8ccdf-b225-453b-87a9-9c6212d77802"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:27:50.098357 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:50.098333    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4f8ccdf-b225-453b-87a9-9c6212d77802-kube-api-access-d9csg" pod "c4f8ccdf-b225-453b-87a9-9c6212d77802" (UID: "c4f8ccdf-b225-453b-87a9-9c6212d77802"). InnerVolumeSpecName "kube-api-access-d9csg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:27:50.098437 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:50.098404    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4f8ccdf-b225-453b-87a9-9c6212d77802-metrics-certs" pod "c4f8ccdf-b225-453b-87a9-9c6212d77802" (UID: "c4f8ccdf-b225-453b-87a9-9c6212d77802"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:27:50.196813 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:50.196770    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/39c35ec4-5c1b-472e-9e73-991258bb835d-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39c35ec4-5c1b-472e-9e73-991258bb835d-openshift-service-ca\") pod \"39c35ec4-5c1b-472e-9e73-991258bb835d\" (UID: \"39c35ec4-5c1b-472e-9e73-991258bb835d\") "
Nov 04 01:27:50.197009 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:50.196857    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/39c35ec4-5c1b-472e-9e73-991258bb835d-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39c35ec4-5c1b-472e-9e73-991258bb835d-maas-api-tls\") pod \"39c35ec4-5c1b-472e-9e73-991258bb835d\" (UID: \"39c35ec4-5c1b-472e-9e73-991258bb835d\") "
Nov 04 01:27:50.197009 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:50.196928    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/39c35ec4-5c1b-472e-9e73-991258bb835d-metrics-certs\" (UniqueName: \"kubernetes.io/secret/39c35ec4-5c1b-472e-9e73-991258bb835d-metrics-certs\") pod \"39c35ec4-5c1b-472e-9e73-991258bb835d\" (UID: \"39c35ec4-5c1b-472e-9e73-991258bb835d\") "
Nov 04 01:27:50.197132 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:50.197047    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/39c35ec4-5c1b-472e-9e73-991258bb835d-kube-api-access-gj2l8\" (UniqueName: \"kubernetes.io/projected/39c35ec4-5c1b-472e-9e73-991258bb835d-kube-api-access-gj2l8\") pod \"39c35ec4-5c1b-472e-9e73-991258bb835d\" (UID: \"39c35ec4-5c1b-472e-9e73-991258bb835d\") "
Nov 04 01:27:50.197810 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:50.197784    5068 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4f8ccdf-b225-453b-87a9-9c6212d77802-metrics-certs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:50.203681 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:50.197821    5068 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4f8ccdf-b225-453b-87a9-9c6212d77802-maas-api-tls\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:50.203681 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:50.197836    5068 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4f8ccdf-b225-453b-87a9-9c6212d77802-openshift-service-ca\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:50.203681 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:50.197850    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d9csg\" (UniqueName: \"kubernetes.io/projected/c4f8ccdf-b225-453b-87a9-9c6212d77802-kube-api-access-d9csg\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:50.203681 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:50.199283    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39c35ec4-5c1b-472e-9e73-991258bb835d-openshift-service-ca" pod "39c35ec4-5c1b-472e-9e73-991258bb835d" (UID: "39c35ec4-5c1b-472e-9e73-991258bb835d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:27:50.203681 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:27:50.202298    5068 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4f8ccdf_b225_453b_87a9_9c6212d77802.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4f8ccdf_b225_453b_87a9_9c6212d77802.slice/crio-72e4981da1ff1806edbe90aacb9eacaecc0f0b7eede600dfd7ec83577d0df71e\": RecentStats: unable to find data in memory cache]"
Nov 04 01:27:50.208895 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:50.208855    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39c35ec4-5c1b-472e-9e73-991258bb835d-metrics-certs" pod "39c35ec4-5c1b-472e-9e73-991258bb835d" (UID: "39c35ec4-5c1b-472e-9e73-991258bb835d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:27:50.210523 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:50.209510    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39c35ec4-5c1b-472e-9e73-991258bb835d-maas-api-tls" pod "39c35ec4-5c1b-472e-9e73-991258bb835d" (UID: "39c35ec4-5c1b-472e-9e73-991258bb835d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:27:50.212843 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:50.212815    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39c35ec4-5c1b-472e-9e73-991258bb835d-kube-api-access-gj2l8" pod "39c35ec4-5c1b-472e-9e73-991258bb835d" (UID: "39c35ec4-5c1b-472e-9e73-991258bb835d"). InnerVolumeSpecName "kube-api-access-gj2l8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:27:50.303690 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:50.300926    5068 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39c35ec4-5c1b-472e-9e73-991258bb835d-maas-api-tls\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:50.303690 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:50.300964    5068 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39c35ec4-5c1b-472e-9e73-991258bb835d-metrics-certs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:50.303690 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:50.300980    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gj2l8\" (UniqueName: \"kubernetes.io/projected/39c35ec4-5c1b-472e-9e73-991258bb835d-kube-api-access-gj2l8\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:50.303690 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:50.300996    5068 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39c35ec4-5c1b-472e-9e73-991258bb835d-openshift-service-ca\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:50.763391 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:50.763352    5068 generic.go:356] "Generic (PLEG): container finished" podID="39c35ec4-5c1b-472e-9e73-991258bb835d" containerID="91e13ec9894502a467877006e22026d0f6952c4a721da97d920753bcef6f951c" exitCode=0
Nov 04 01:27:50.763936 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:50.763401    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-029e8c-8c7f458c5-gvzpd"
Nov 04 01:27:50.763936 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:50.763438    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-029e8c-8c7f458c5-gvzpd" event={"ID":"39c35ec4-5c1b-472e-9e73-991258bb835d","Type":"ContainerDied","Data":"91e13ec9894502a467877006e22026d0f6952c4a721da97d920753bcef6f951c"}
Nov 04 01:27:50.763936 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:50.763495    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-029e8c-8c7f458c5-gvzpd" event={"ID":"39c35ec4-5c1b-472e-9e73-991258bb835d","Type":"ContainerDied","Data":"c3e7d04184b24934249efb4ab5074b404cb1b3757770532bc2b203ee3af1f8b1"}
Nov 04 01:27:50.763936 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:50.763526    5068 scope.go:122] "RemoveContainer" containerID="91e13ec9894502a467877006e22026d0f6952c4a721da97d920753bcef6f951c"
Nov 04 01:27:50.766191 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:50.766161    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-cffa80-8dcbb5c89-jx699" event={"ID":"c4f8ccdf-b225-453b-87a9-9c6212d77802","Type":"ContainerDied","Data":"72e4981da1ff1806edbe90aacb9eacaecc0f0b7eede600dfd7ec83577d0df71e"}
Nov 04 01:27:50.766328 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:50.766261    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-cffa80-8dcbb5c89-jx699"
Nov 04 01:27:50.778505 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:50.778486    5068 scope.go:122] "RemoveContainer" containerID="91e13ec9894502a467877006e22026d0f6952c4a721da97d920753bcef6f951c"
Nov 04 01:27:50.778803 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:27:50.778782    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91e13ec9894502a467877006e22026d0f6952c4a721da97d920753bcef6f951c\": container with ID starting with 91e13ec9894502a467877006e22026d0f6952c4a721da97d920753bcef6f951c not found: ID does not exist" containerID="91e13ec9894502a467877006e22026d0f6952c4a721da97d920753bcef6f951c"
Nov 04 01:27:50.778898 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:50.778812    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91e13ec9894502a467877006e22026d0f6952c4a721da97d920753bcef6f951c"} err="failed to get container status \"91e13ec9894502a467877006e22026d0f6952c4a721da97d920753bcef6f951c\": rpc error: code = NotFound desc = could not find container \"91e13ec9894502a467877006e22026d0f6952c4a721da97d920753bcef6f951c\": container with ID starting with 91e13ec9894502a467877006e22026d0f6952c4a721da97d920753bcef6f951c not found: ID does not exist"
Nov 04 01:27:50.778898 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:50.778836    5068 scope.go:122] "RemoveContainer" containerID="3f715d388c4658cde8f49e563a68f41146c83409b5beefaa31b184d321ee5611"
Nov 04 01:27:50.842578 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:50.841682    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-cffa80-8dcbb5c89-jx699"]
Nov 04 01:27:50.922677 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:50.922550    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-cffa80-8dcbb5c89-jx699"]
Nov 04 01:27:50.949038 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:50.949003    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-029e8c-8c7f458c5-gvzpd"]
Nov 04 01:27:50.956718 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:50.956683    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-029e8c-8c7f458c5-gvzpd"]
Nov 04 01:27:51.782675 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:51.782639    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:27:52.120695 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:52.119432    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="39c35ec4-5c1b-472e-9e73-991258bb835d" path="/var/lib/kubelet/pods/39c35ec4-5c1b-472e-9e73-991258bb835d/volumes"
Nov 04 01:27:52.120695 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:52.120336    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c4f8ccdf-b225-453b-87a9-9c6212d77802" path="/var/lib/kubelet/pods/c4f8ccdf-b225-453b-87a9-9c6212d77802/volumes"
Nov 04 01:27:52.415823 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:52.415738    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w2-cffa80-564fd64f66-qhhzq" podUID="1b4a1b43-a286-436d-bafe-a10d9a57a4d2" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:27:52.415969 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:52.415738    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-cffa80-9b645ff4c-2pj8n" podUID="841e9fef-46d6-4625-9b72-14b4e0eeb7dd" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:27:57.260596 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:57.260213    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-cffa80-564fd64f66-qhhzq"
Nov 04 01:27:57.317748 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:57.317397    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-cffa80-9b645ff4c-2pj8n"
Nov 04 01:27:57.318073 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:57.318037    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1b4a1b43-a286-436d-bafe-a10d9a57a4d2-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b4a1b43-a286-436d-bafe-a10d9a57a4d2-plugins-config-volume\") pod \"1b4a1b43-a286-436d-bafe-a10d9a57a4d2\" (UID: \"1b4a1b43-a286-436d-bafe-a10d9a57a4d2\") "
Nov 04 01:27:57.318201 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:57.318140    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1b4a1b43-a286-436d-bafe-a10d9a57a4d2-kube-api-access-ftcfz\" (UniqueName: \"kubernetes.io/projected/1b4a1b43-a286-436d-bafe-a10d9a57a4d2-kube-api-access-ftcfz\") pod \"1b4a1b43-a286-436d-bafe-a10d9a57a4d2\" (UID: \"1b4a1b43-a286-436d-bafe-a10d9a57a4d2\") "
Nov 04 01:27:57.320384 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:57.320352    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b4a1b43-a286-436d-bafe-a10d9a57a4d2-plugins-config-volume" pod "1b4a1b43-a286-436d-bafe-a10d9a57a4d2" (UID: "1b4a1b43-a286-436d-bafe-a10d9a57a4d2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:27:57.322904 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:57.322875    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b4a1b43-a286-436d-bafe-a10d9a57a4d2-kube-api-access-ftcfz" pod "1b4a1b43-a286-436d-bafe-a10d9a57a4d2" (UID: "1b4a1b43-a286-436d-bafe-a10d9a57a4d2"). InnerVolumeSpecName "kube-api-access-ftcfz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:27:57.425680 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:57.421180    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/841e9fef-46d6-4625-9b72-14b4e0eeb7dd-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/841e9fef-46d6-4625-9b72-14b4e0eeb7dd-plugins-config-volume\") pod \"841e9fef-46d6-4625-9b72-14b4e0eeb7dd\" (UID: \"841e9fef-46d6-4625-9b72-14b4e0eeb7dd\") "
Nov 04 01:27:57.425680 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:57.421261    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/841e9fef-46d6-4625-9b72-14b4e0eeb7dd-kube-api-access-cq8n5\" (UniqueName: \"kubernetes.io/projected/841e9fef-46d6-4625-9b72-14b4e0eeb7dd-kube-api-access-cq8n5\") pod \"841e9fef-46d6-4625-9b72-14b4e0eeb7dd\" (UID: \"841e9fef-46d6-4625-9b72-14b4e0eeb7dd\") "
Nov 04 01:27:57.425680 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:57.422043    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ftcfz\" (UniqueName: \"kubernetes.io/projected/1b4a1b43-a286-436d-bafe-a10d9a57a4d2-kube-api-access-ftcfz\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:57.425680 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:57.422068    5068 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b4a1b43-a286-436d-bafe-a10d9a57a4d2-plugins-config-volume\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:57.425680 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:57.424155    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/841e9fef-46d6-4625-9b72-14b4e0eeb7dd-plugins-config-volume" pod "841e9fef-46d6-4625-9b72-14b4e0eeb7dd" (UID: "841e9fef-46d6-4625-9b72-14b4e0eeb7dd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:27:57.453633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:57.442767    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/841e9fef-46d6-4625-9b72-14b4e0eeb7dd-kube-api-access-cq8n5" pod "841e9fef-46d6-4625-9b72-14b4e0eeb7dd" (UID: "841e9fef-46d6-4625-9b72-14b4e0eeb7dd"). InnerVolumeSpecName "kube-api-access-cq8n5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:27:57.530636 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:57.523390    5068 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/841e9fef-46d6-4625-9b72-14b4e0eeb7dd-plugins-config-volume\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:57.530636 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:57.523430    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cq8n5\" (UniqueName: \"kubernetes.io/projected/841e9fef-46d6-4625-9b72-14b4e0eeb7dd-kube-api-access-cq8n5\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:57.913565 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:57.913521    5068 generic.go:356] "Generic (PLEG): container finished" podID="1b4a1b43-a286-436d-bafe-a10d9a57a4d2" containerID="3abb2e93264e4e70655b9bbadb6fae705629758c380b56c46258f3da781ab15d" exitCode=0
Nov 04 01:27:57.913786 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:57.913584    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-cffa80-564fd64f66-qhhzq"
Nov 04 01:27:57.913786 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:57.913619    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-cffa80-564fd64f66-qhhzq" event={"ID":"1b4a1b43-a286-436d-bafe-a10d9a57a4d2","Type":"ContainerDied","Data":"3abb2e93264e4e70655b9bbadb6fae705629758c380b56c46258f3da781ab15d"}
Nov 04 01:27:57.913786 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:57.913726    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-cffa80-564fd64f66-qhhzq" event={"ID":"1b4a1b43-a286-436d-bafe-a10d9a57a4d2","Type":"ContainerDied","Data":"039ffb7fc2d5f3101f7045667017b696fc55e5adfe632451f197d15c4976c542"}
Nov 04 01:27:57.913786 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:57.913760    5068 scope.go:122] "RemoveContainer" containerID="3abb2e93264e4e70655b9bbadb6fae705629758c380b56c46258f3da781ab15d"
Nov 04 01:27:57.915805 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:57.915777    5068 generic.go:356] "Generic (PLEG): container finished" podID="841e9fef-46d6-4625-9b72-14b4e0eeb7dd" containerID="586e192813d95194bcadb83bf3a3cd34721c99f089734ca29eff252dbb718dab" exitCode=0
Nov 04 01:27:57.916015 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:57.915991    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-cffa80-9b645ff4c-2pj8n"
Nov 04 01:27:57.916162 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:57.916026    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-cffa80-9b645ff4c-2pj8n" event={"ID":"841e9fef-46d6-4625-9b72-14b4e0eeb7dd","Type":"ContainerDied","Data":"586e192813d95194bcadb83bf3a3cd34721c99f089734ca29eff252dbb718dab"}
Nov 04 01:27:57.916234 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:57.916196    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-cffa80-9b645ff4c-2pj8n" event={"ID":"841e9fef-46d6-4625-9b72-14b4e0eeb7dd","Type":"ContainerDied","Data":"a8db9090b9e68247abcb72e39c6839e64feb8d5a6948623f5194ba2dd1c8a2be"}
Nov 04 01:27:57.973795 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:57.973763    5068 scope.go:122] "RemoveContainer" containerID="3abb2e93264e4e70655b9bbadb6fae705629758c380b56c46258f3da781ab15d"
Nov 04 01:27:57.978503 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:57.978475    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-cffa80-564fd64f66-qhhzq"]
Nov 04 01:27:57.979250 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:27:57.979216    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3abb2e93264e4e70655b9bbadb6fae705629758c380b56c46258f3da781ab15d\": container with ID starting with 3abb2e93264e4e70655b9bbadb6fae705629758c380b56c46258f3da781ab15d not found: ID does not exist" containerID="3abb2e93264e4e70655b9bbadb6fae705629758c380b56c46258f3da781ab15d"
Nov 04 01:27:57.979416 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:57.979391    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3abb2e93264e4e70655b9bbadb6fae705629758c380b56c46258f3da781ab15d"} err="failed to get container status \"3abb2e93264e4e70655b9bbadb6fae705629758c380b56c46258f3da781ab15d\": rpc error: code = NotFound desc = could not find container \"3abb2e93264e4e70655b9bbadb6fae705629758c380b56c46258f3da781ab15d\": container with ID starting with 3abb2e93264e4e70655b9bbadb6fae705629758c380b56c46258f3da781ab15d not found: ID does not exist"
Nov 04 01:27:57.979542 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:57.979528    5068 scope.go:122] "RemoveContainer" containerID="586e192813d95194bcadb83bf3a3cd34721c99f089734ca29eff252dbb718dab"
Nov 04 01:27:57.999103 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:57.999075    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-cffa80-564fd64f66-qhhzq"]
Nov 04 01:27:58.002141 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:58.002105    5068 scope.go:122] "RemoveContainer" containerID="586e192813d95194bcadb83bf3a3cd34721c99f089734ca29eff252dbb718dab"
Nov 04 01:27:58.002647 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:27:58.002605    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"586e192813d95194bcadb83bf3a3cd34721c99f089734ca29eff252dbb718dab\": container with ID starting with 586e192813d95194bcadb83bf3a3cd34721c99f089734ca29eff252dbb718dab not found: ID does not exist" containerID="586e192813d95194bcadb83bf3a3cd34721c99f089734ca29eff252dbb718dab"
Nov 04 01:27:58.002823 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:58.002797    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"586e192813d95194bcadb83bf3a3cd34721c99f089734ca29eff252dbb718dab"} err="failed to get container status \"586e192813d95194bcadb83bf3a3cd34721c99f089734ca29eff252dbb718dab\": rpc error: code = NotFound desc = could not find container \"586e192813d95194bcadb83bf3a3cd34721c99f089734ca29eff252dbb718dab\": container with ID starting with 586e192813d95194bcadb83bf3a3cd34721c99f089734ca29eff252dbb718dab not found: ID does not exist"
Nov 04 01:27:58.004295 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:58.004277    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-cffa80-9b645ff4c-2pj8n"]
Nov 04 01:27:58.022660 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:58.017190    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-cffa80-9b645ff4c-2pj8n"]
Nov 04 01:27:58.142724 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:58.132449    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1b4a1b43-a286-436d-bafe-a10d9a57a4d2" path="/var/lib/kubelet/pods/1b4a1b43-a286-436d-bafe-a10d9a57a4d2/volumes"
Nov 04 01:27:58.142724 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:58.133118    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="841e9fef-46d6-4625-9b72-14b4e0eeb7dd" path="/var/lib/kubelet/pods/841e9fef-46d6-4625-9b72-14b4e0eeb7dd/volumes"
Nov 04 01:27:58.142724 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:58.133545    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf"]
Nov 04 01:27:58.142724 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:58.133604    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f"]
Nov 04 01:27:58.142724 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:58.133686    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8"]
Nov 04 01:27:58.142724 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:58.133964    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8" podUID="43eaae59-c37c-4553-a856-fc8aa9506c00" containerName="istio-proxy" containerID="cri-o://845dd23dc57bc3672fcca20302134c048016c9ff6715f965bbdf3998c69c9764" gracePeriod=30
Nov 04 01:27:58.142724 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:58.134531    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf" podUID="c3d5c7b1-d7fe-4a70-aba1-1131171f55c1" containerName="istio-proxy" containerID="cri-o://7cd84fdd31600ea3f367d1c9b49b1a4bdc85f0d9a894fc11d6104fd55766f601" gracePeriod=30
Nov 04 01:27:58.142724 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:27:58.135103    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f" podUID="7ca2a53a-ada2-4565-bc5a-66c7eefef997" containerName="istio-proxy" containerID="cri-o://c442438a5f770f4b9326055ad3c246e0146284879f99db5e5d1d1865b1e22f3a" gracePeriod=30
Nov 04 01:28:01.366902 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:01.366869    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w4-3bf4f9-7db74f6687-htzl4"]
Nov 04 01:28:03.598678 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.598646    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8"
Nov 04 01:28:03.651812 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.651783    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/43eaae59-c37c-4553-a856-fc8aa9506c00-istio-token\" (UniqueName: \"kubernetes.io/projected/43eaae59-c37c-4553-a856-fc8aa9506c00-istio-token\") pod \"43eaae59-c37c-4553-a856-fc8aa9506c00\" (UID: \"43eaae59-c37c-4553-a856-fc8aa9506c00\") "
Nov 04 01:28:03.651959 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.651851    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/43eaae59-c37c-4553-a856-fc8aa9506c00-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43eaae59-c37c-4553-a856-fc8aa9506c00-workload-socket\") pod \"43eaae59-c37c-4553-a856-fc8aa9506c00\" (UID: \"43eaae59-c37c-4553-a856-fc8aa9506c00\") "
Nov 04 01:28:03.651959 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.651925    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/43eaae59-c37c-4553-a856-fc8aa9506c00-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43eaae59-c37c-4553-a856-fc8aa9506c00-istio-data\") pod \"43eaae59-c37c-4553-a856-fc8aa9506c00\" (UID: \"43eaae59-c37c-4553-a856-fc8aa9506c00\") "
Nov 04 01:28:03.652083 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.651992    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/43eaae59-c37c-4553-a856-fc8aa9506c00-kube-api-access-8jnvd\" (UniqueName: \"kubernetes.io/projected/43eaae59-c37c-4553-a856-fc8aa9506c00-kube-api-access-8jnvd\") pod \"43eaae59-c37c-4553-a856-fc8aa9506c00\" (UID: \"43eaae59-c37c-4553-a856-fc8aa9506c00\") "
Nov 04 01:28:03.652083 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.652024    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/43eaae59-c37c-4553-a856-fc8aa9506c00-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43eaae59-c37c-4553-a856-fc8aa9506c00-istiod-ca-cert\") pod \"43eaae59-c37c-4553-a856-fc8aa9506c00\" (UID: \"43eaae59-c37c-4553-a856-fc8aa9506c00\") "
Nov 04 01:28:03.652183 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.652127    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/43eaae59-c37c-4553-a856-fc8aa9506c00-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43eaae59-c37c-4553-a856-fc8aa9506c00-credential-socket\") pod \"43eaae59-c37c-4553-a856-fc8aa9506c00\" (UID: \"43eaae59-c37c-4553-a856-fc8aa9506c00\") "
Nov 04 01:28:03.652232 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.652192    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/43eaae59-c37c-4553-a856-fc8aa9506c00-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43eaae59-c37c-4553-a856-fc8aa9506c00-istio-podinfo\") pod \"43eaae59-c37c-4553-a856-fc8aa9506c00\" (UID: \"43eaae59-c37c-4553-a856-fc8aa9506c00\") "
Nov 04 01:28:03.652287 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.652230    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/43eaae59-c37c-4553-a856-fc8aa9506c00-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43eaae59-c37c-4553-a856-fc8aa9506c00-istio-envoy\") pod \"43eaae59-c37c-4553-a856-fc8aa9506c00\" (UID: \"43eaae59-c37c-4553-a856-fc8aa9506c00\") "
Nov 04 01:28:03.652287 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.652268    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/43eaae59-c37c-4553-a856-fc8aa9506c00-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43eaae59-c37c-4553-a856-fc8aa9506c00-workload-certs\") pod \"43eaae59-c37c-4553-a856-fc8aa9506c00\" (UID: \"43eaae59-c37c-4553-a856-fc8aa9506c00\") "
Nov 04 01:28:03.653002 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.652979    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f"
Nov 04 01:28:03.653786 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.653760    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43eaae59-c37c-4553-a856-fc8aa9506c00-workload-certs" pod "43eaae59-c37c-4553-a856-fc8aa9506c00" (UID: "43eaae59-c37c-4553-a856-fc8aa9506c00"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:03.654069 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.654051    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43eaae59-c37c-4553-a856-fc8aa9506c00-istiod-ca-cert" pod "43eaae59-c37c-4553-a856-fc8aa9506c00" (UID: "43eaae59-c37c-4553-a856-fc8aa9506c00"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:28:03.654260 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.654244    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43eaae59-c37c-4553-a856-fc8aa9506c00-credential-socket" pod "43eaae59-c37c-4553-a856-fc8aa9506c00" (UID: "43eaae59-c37c-4553-a856-fc8aa9506c00"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:03.655483 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.655450    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43eaae59-c37c-4553-a856-fc8aa9506c00-istio-data" pod "43eaae59-c37c-4553-a856-fc8aa9506c00" (UID: "43eaae59-c37c-4553-a856-fc8aa9506c00"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:03.657725 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.657703    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43eaae59-c37c-4553-a856-fc8aa9506c00-workload-socket" pod "43eaae59-c37c-4553-a856-fc8aa9506c00" (UID: "43eaae59-c37c-4553-a856-fc8aa9506c00"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:03.659980 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.659958    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/43eaae59-c37c-4553-a856-fc8aa9506c00-istio-podinfo" pod "43eaae59-c37c-4553-a856-fc8aa9506c00" (UID: "43eaae59-c37c-4553-a856-fc8aa9506c00"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:28:03.660274 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.660254    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43eaae59-c37c-4553-a856-fc8aa9506c00-istio-envoy" pod "43eaae59-c37c-4553-a856-fc8aa9506c00" (UID: "43eaae59-c37c-4553-a856-fc8aa9506c00"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:03.661023 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.661001    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf"
Nov 04 01:28:03.661601 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.661571    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43eaae59-c37c-4553-a856-fc8aa9506c00-istio-token" pod "43eaae59-c37c-4553-a856-fc8aa9506c00" (UID: "43eaae59-c37c-4553-a856-fc8aa9506c00"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:28:03.661856 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.661836    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43eaae59-c37c-4553-a856-fc8aa9506c00-kube-api-access-8jnvd" pod "43eaae59-c37c-4553-a856-fc8aa9506c00" (UID: "43eaae59-c37c-4553-a856-fc8aa9506c00"). InnerVolumeSpecName "kube-api-access-8jnvd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:28:03.753831 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.753737    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7ca2a53a-ada2-4565-bc5a-66c7eefef997-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ca2a53a-ada2-4565-bc5a-66c7eefef997-istio-envoy\") pod \"7ca2a53a-ada2-4565-bc5a-66c7eefef997\" (UID: \"7ca2a53a-ada2-4565-bc5a-66c7eefef997\") "
Nov 04 01:28:03.753831 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.753779    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7ca2a53a-ada2-4565-bc5a-66c7eefef997-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ca2a53a-ada2-4565-bc5a-66c7eefef997-istio-podinfo\") pod \"7ca2a53a-ada2-4565-bc5a-66c7eefef997\" (UID: \"7ca2a53a-ada2-4565-bc5a-66c7eefef997\") "
Nov 04 01:28:03.753831 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.753807    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7ca2a53a-ada2-4565-bc5a-66c7eefef997-kube-api-access-dbsxw\" (UniqueName: \"kubernetes.io/projected/7ca2a53a-ada2-4565-bc5a-66c7eefef997-kube-api-access-dbsxw\") pod \"7ca2a53a-ada2-4565-bc5a-66c7eefef997\" (UID: \"7ca2a53a-ada2-4565-bc5a-66c7eefef997\") "
Nov 04 01:28:03.754140 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.753924    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-istiod-ca-cert\") pod \"c3d5c7b1-d7fe-4a70-aba1-1131171f55c1\" (UID: \"c3d5c7b1-d7fe-4a70-aba1-1131171f55c1\") "
Nov 04 01:28:03.754140 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.754007    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-workload-socket\") pod \"c3d5c7b1-d7fe-4a70-aba1-1131171f55c1\" (UID: \"c3d5c7b1-d7fe-4a70-aba1-1131171f55c1\") "
Nov 04 01:28:03.754140 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.754036    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7ca2a53a-ada2-4565-bc5a-66c7eefef997-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ca2a53a-ada2-4565-bc5a-66c7eefef997-istio-data\") pod \"7ca2a53a-ada2-4565-bc5a-66c7eefef997\" (UID: \"7ca2a53a-ada2-4565-bc5a-66c7eefef997\") "
Nov 04 01:28:03.754382 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.754352    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-istiod-ca-cert" pod "c3d5c7b1-d7fe-4a70-aba1-1131171f55c1" (UID: "c3d5c7b1-d7fe-4a70-aba1-1131171f55c1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:28:03.754507 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.754479    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ca2a53a-ada2-4565-bc5a-66c7eefef997-istio-data" pod "7ca2a53a-ada2-4565-bc5a-66c7eefef997" (UID: "7ca2a53a-ada2-4565-bc5a-66c7eefef997"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:03.754574 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.754553    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-workload-socket" pod "c3d5c7b1-d7fe-4a70-aba1-1131171f55c1" (UID: "c3d5c7b1-d7fe-4a70-aba1-1131171f55c1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:03.754692 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.754670    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-credential-socket\") pod \"c3d5c7b1-d7fe-4a70-aba1-1131171f55c1\" (UID: \"c3d5c7b1-d7fe-4a70-aba1-1131171f55c1\") "
Nov 04 01:28:03.754767 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.754737    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-workload-certs\") pod \"c3d5c7b1-d7fe-4a70-aba1-1131171f55c1\" (UID: \"c3d5c7b1-d7fe-4a70-aba1-1131171f55c1\") "
Nov 04 01:28:03.754836 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.754769    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7ca2a53a-ada2-4565-bc5a-66c7eefef997-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ca2a53a-ada2-4565-bc5a-66c7eefef997-workload-socket\") pod \"7ca2a53a-ada2-4565-bc5a-66c7eefef997\" (UID: \"7ca2a53a-ada2-4565-bc5a-66c7eefef997\") "
Nov 04 01:28:03.754836 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.754808    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7ca2a53a-ada2-4565-bc5a-66c7eefef997-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ca2a53a-ada2-4565-bc5a-66c7eefef997-istiod-ca-cert\") pod \"7ca2a53a-ada2-4565-bc5a-66c7eefef997\" (UID: \"7ca2a53a-ada2-4565-bc5a-66c7eefef997\") "
Nov 04 01:28:03.754948 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.754834    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7ca2a53a-ada2-4565-bc5a-66c7eefef997-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ca2a53a-ada2-4565-bc5a-66c7eefef997-credential-socket\") pod \"7ca2a53a-ada2-4565-bc5a-66c7eefef997\" (UID: \"7ca2a53a-ada2-4565-bc5a-66c7eefef997\") "
Nov 04 01:28:03.754948 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.754873    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-istio-data\") pod \"c3d5c7b1-d7fe-4a70-aba1-1131171f55c1\" (UID: \"c3d5c7b1-d7fe-4a70-aba1-1131171f55c1\") "
Nov 04 01:28:03.755060 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.755009    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-istio-podinfo\") pod \"c3d5c7b1-d7fe-4a70-aba1-1131171f55c1\" (UID: \"c3d5c7b1-d7fe-4a70-aba1-1131171f55c1\") "
Nov 04 01:28:03.755115 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.755076    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7ca2a53a-ada2-4565-bc5a-66c7eefef997-istio-token\" (UniqueName: \"kubernetes.io/projected/7ca2a53a-ada2-4565-bc5a-66c7eefef997-istio-token\") pod \"7ca2a53a-ada2-4565-bc5a-66c7eefef997\" (UID: \"7ca2a53a-ada2-4565-bc5a-66c7eefef997\") "
Nov 04 01:28:03.755175 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.755149    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-kube-api-access-xwk77\" (UniqueName: \"kubernetes.io/projected/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-kube-api-access-xwk77\") pod \"c3d5c7b1-d7fe-4a70-aba1-1131171f55c1\" (UID: \"c3d5c7b1-d7fe-4a70-aba1-1131171f55c1\") "
Nov 04 01:28:03.755228 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.755180    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7ca2a53a-ada2-4565-bc5a-66c7eefef997-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ca2a53a-ada2-4565-bc5a-66c7eefef997-workload-certs\") pod \"7ca2a53a-ada2-4565-bc5a-66c7eefef997\" (UID: \"7ca2a53a-ada2-4565-bc5a-66c7eefef997\") "
Nov 04 01:28:03.755228 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.755218    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-istio-envoy\") pod \"c3d5c7b1-d7fe-4a70-aba1-1131171f55c1\" (UID: \"c3d5c7b1-d7fe-4a70-aba1-1131171f55c1\") "
Nov 04 01:28:03.755330 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.755246    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-istio-token\" (UniqueName: \"kubernetes.io/projected/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-istio-token\") pod \"c3d5c7b1-d7fe-4a70-aba1-1131171f55c1\" (UID: \"c3d5c7b1-d7fe-4a70-aba1-1131171f55c1\") "
Nov 04 01:28:03.756725 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.755474    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-credential-socket" pod "c3d5c7b1-d7fe-4a70-aba1-1131171f55c1" (UID: "c3d5c7b1-d7fe-4a70-aba1-1131171f55c1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:03.756725 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.755965    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ca2a53a-ada2-4565-bc5a-66c7eefef997-istiod-ca-cert" pod "7ca2a53a-ada2-4565-bc5a-66c7eefef997" (UID: "7ca2a53a-ada2-4565-bc5a-66c7eefef997"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:28:03.756725 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.755989    5068 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43eaae59-c37c-4553-a856-fc8aa9506c00-workload-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:03.756725 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.756012    5068 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-istiod-ca-cert\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:03.756725 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.756028    5068 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-workload-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:03.756725 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.756041    5068 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ca2a53a-ada2-4565-bc5a-66c7eefef997-istio-data\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:03.756725 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.756053    5068 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43eaae59-c37c-4553-a856-fc8aa9506c00-istio-data\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:03.756725 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.756069    5068 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-credential-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:03.756725 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.756257    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-workload-certs" pod "c3d5c7b1-d7fe-4a70-aba1-1131171f55c1" (UID: "c3d5c7b1-d7fe-4a70-aba1-1131171f55c1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:03.756725 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.756278    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8jnvd\" (UniqueName: \"kubernetes.io/projected/43eaae59-c37c-4553-a856-fc8aa9506c00-kube-api-access-8jnvd\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:03.756725 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.756305    5068 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43eaae59-c37c-4553-a856-fc8aa9506c00-istiod-ca-cert\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:03.756725 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.756321    5068 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43eaae59-c37c-4553-a856-fc8aa9506c00-credential-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:03.756725 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.756335    5068 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43eaae59-c37c-4553-a856-fc8aa9506c00-istio-podinfo\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:03.756725 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.756353    5068 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43eaae59-c37c-4553-a856-fc8aa9506c00-istio-envoy\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:03.756725 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.756369    5068 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43eaae59-c37c-4553-a856-fc8aa9506c00-workload-certs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:03.756725 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.756383    5068 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43eaae59-c37c-4553-a856-fc8aa9506c00-istio-token\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:03.756725 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.756381    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ca2a53a-ada2-4565-bc5a-66c7eefef997-credential-socket" pod "7ca2a53a-ada2-4565-bc5a-66c7eefef997" (UID: "7ca2a53a-ada2-4565-bc5a-66c7eefef997"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:03.756725 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.756449    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ca2a53a-ada2-4565-bc5a-66c7eefef997-workload-socket" pod "7ca2a53a-ada2-4565-bc5a-66c7eefef997" (UID: "7ca2a53a-ada2-4565-bc5a-66c7eefef997"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:03.756725 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.756571    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-istio-data" pod "c3d5c7b1-d7fe-4a70-aba1-1131171f55c1" (UID: "c3d5c7b1-d7fe-4a70-aba1-1131171f55c1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:03.756725 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.756656    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ca2a53a-ada2-4565-bc5a-66c7eefef997-istio-envoy" pod "7ca2a53a-ada2-4565-bc5a-66c7eefef997" (UID: "7ca2a53a-ada2-4565-bc5a-66c7eefef997"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:03.756725 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.756680    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7ca2a53a-ada2-4565-bc5a-66c7eefef997-istio-podinfo" pod "7ca2a53a-ada2-4565-bc5a-66c7eefef997" (UID: "7ca2a53a-ada2-4565-bc5a-66c7eefef997"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:28:03.756725 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.756685    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ca2a53a-ada2-4565-bc5a-66c7eefef997-workload-certs" pod "7ca2a53a-ada2-4565-bc5a-66c7eefef997" (UID: "7ca2a53a-ada2-4565-bc5a-66c7eefef997"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:03.758291 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.758068    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ca2a53a-ada2-4565-bc5a-66c7eefef997-kube-api-access-dbsxw" pod "7ca2a53a-ada2-4565-bc5a-66c7eefef997" (UID: "7ca2a53a-ada2-4565-bc5a-66c7eefef997"). InnerVolumeSpecName "kube-api-access-dbsxw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:28:03.758291 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.758113    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-istio-podinfo" pod "c3d5c7b1-d7fe-4a70-aba1-1131171f55c1" (UID: "c3d5c7b1-d7fe-4a70-aba1-1131171f55c1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:28:03.758527 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.758507    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-istio-token" pod "c3d5c7b1-d7fe-4a70-aba1-1131171f55c1" (UID: "c3d5c7b1-d7fe-4a70-aba1-1131171f55c1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:28:03.759269 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.759249    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ca2a53a-ada2-4565-bc5a-66c7eefef997-istio-token" pod "7ca2a53a-ada2-4565-bc5a-66c7eefef997" (UID: "7ca2a53a-ada2-4565-bc5a-66c7eefef997"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:28:03.759399 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.759377    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-istio-envoy" pod "c3d5c7b1-d7fe-4a70-aba1-1131171f55c1" (UID: "c3d5c7b1-d7fe-4a70-aba1-1131171f55c1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:03.759468 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.759399    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-kube-api-access-xwk77" pod "c3d5c7b1-d7fe-4a70-aba1-1131171f55c1" (UID: "c3d5c7b1-d7fe-4a70-aba1-1131171f55c1"). InnerVolumeSpecName "kube-api-access-xwk77". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:28:03.857799 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.857755    5068 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-workload-certs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:03.857799 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.857798    5068 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ca2a53a-ada2-4565-bc5a-66c7eefef997-workload-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:03.858015 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.857815    5068 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ca2a53a-ada2-4565-bc5a-66c7eefef997-istiod-ca-cert\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:03.858015 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.857830    5068 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ca2a53a-ada2-4565-bc5a-66c7eefef997-credential-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:03.858015 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.857843    5068 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-istio-data\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:03.858015 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.857858    5068 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-istio-podinfo\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:03.858015 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.857873    5068 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ca2a53a-ada2-4565-bc5a-66c7eefef997-istio-token\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:03.858015 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.857887    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xwk77\" (UniqueName: \"kubernetes.io/projected/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-kube-api-access-xwk77\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:03.858015 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.857902    5068 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ca2a53a-ada2-4565-bc5a-66c7eefef997-workload-certs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:03.858015 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.857915    5068 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-istio-envoy\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:03.858015 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.857928    5068 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1-istio-token\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:03.858015 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.857941    5068 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ca2a53a-ada2-4565-bc5a-66c7eefef997-istio-envoy\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:03.858015 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.857955    5068 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ca2a53a-ada2-4565-bc5a-66c7eefef997-istio-podinfo\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:03.858015 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.857968    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dbsxw\" (UniqueName: \"kubernetes.io/projected/7ca2a53a-ada2-4565-bc5a-66c7eefef997-kube-api-access-dbsxw\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:03.976788 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.976751    5068 generic.go:356] "Generic (PLEG): container finished" podID="43eaae59-c37c-4553-a856-fc8aa9506c00" containerID="845dd23dc57bc3672fcca20302134c048016c9ff6715f965bbdf3998c69c9764" exitCode=0
Nov 04 01:28:03.976983 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.976825    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8"
Nov 04 01:28:03.976983 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.976843    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8" event={"ID":"43eaae59-c37c-4553-a856-fc8aa9506c00","Type":"ContainerDied","Data":"845dd23dc57bc3672fcca20302134c048016c9ff6715f965bbdf3998c69c9764"}
Nov 04 01:28:03.976983 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.976912    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8" event={"ID":"43eaae59-c37c-4553-a856-fc8aa9506c00","Type":"ContainerDied","Data":"34666a567819f0bc165e1e61c58a88aee038ecc9d26c3d65e198a9f10196b6b4"}
Nov 04 01:28:03.976983 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.976944    5068 scope.go:122] "RemoveContainer" containerID="845dd23dc57bc3672fcca20302134c048016c9ff6715f965bbdf3998c69c9764"
Nov 04 01:28:03.979201 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.979177    5068 generic.go:356] "Generic (PLEG): container finished" podID="c3d5c7b1-d7fe-4a70-aba1-1131171f55c1" containerID="7cd84fdd31600ea3f367d1c9b49b1a4bdc85f0d9a894fc11d6104fd55766f601" exitCode=0
Nov 04 01:28:03.979403 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.979387    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf"
Nov 04 01:28:03.980025 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.979996    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf" event={"ID":"c3d5c7b1-d7fe-4a70-aba1-1131171f55c1","Type":"ContainerDied","Data":"7cd84fdd31600ea3f367d1c9b49b1a4bdc85f0d9a894fc11d6104fd55766f601"}
Nov 04 01:28:03.980117 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.980057    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf" event={"ID":"c3d5c7b1-d7fe-4a70-aba1-1131171f55c1","Type":"ContainerDied","Data":"117915303c947da9f6c28c47d5aaa6c92d14ede62c50facf06ad933da5babdc7"}
Nov 04 01:28:03.982759 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.982733    5068 generic.go:356] "Generic (PLEG): container finished" podID="7ca2a53a-ada2-4565-bc5a-66c7eefef997" containerID="c442438a5f770f4b9326055ad3c246e0146284879f99db5e5d1d1865b1e22f3a" exitCode=0
Nov 04 01:28:03.982906 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.982888    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f" event={"ID":"7ca2a53a-ada2-4565-bc5a-66c7eefef997","Type":"ContainerDied","Data":"c442438a5f770f4b9326055ad3c246e0146284879f99db5e5d1d1865b1e22f3a"}
Nov 04 01:28:03.982946 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.982934    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f" event={"ID":"7ca2a53a-ada2-4565-bc5a-66c7eefef997","Type":"ContainerDied","Data":"b4f393fd7c4336b02f80c0fe04cf92066cd9310a4b8c2700b959eef122a69a64"}
Nov 04 01:28:03.983018 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.983007    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f"
Nov 04 01:28:03.991066 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.991044    5068 scope.go:122] "RemoveContainer" containerID="845dd23dc57bc3672fcca20302134c048016c9ff6715f965bbdf3998c69c9764"
Nov 04 01:28:03.991592 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:28:03.991560    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"845dd23dc57bc3672fcca20302134c048016c9ff6715f965bbdf3998c69c9764\": container with ID starting with 845dd23dc57bc3672fcca20302134c048016c9ff6715f965bbdf3998c69c9764 not found: ID does not exist" containerID="845dd23dc57bc3672fcca20302134c048016c9ff6715f965bbdf3998c69c9764"
Nov 04 01:28:03.991808 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.991770    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"845dd23dc57bc3672fcca20302134c048016c9ff6715f965bbdf3998c69c9764"} err="failed to get container status \"845dd23dc57bc3672fcca20302134c048016c9ff6715f965bbdf3998c69c9764\": rpc error: code = NotFound desc = could not find container \"845dd23dc57bc3672fcca20302134c048016c9ff6715f965bbdf3998c69c9764\": container with ID starting with 845dd23dc57bc3672fcca20302134c048016c9ff6715f965bbdf3998c69c9764 not found: ID does not exist"
Nov 04 01:28:03.991927 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:03.991907    5068 scope.go:122] "RemoveContainer" containerID="7cd84fdd31600ea3f367d1c9b49b1a4bdc85f0d9a894fc11d6104fd55766f601"
Nov 04 01:28:04.004255 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:04.004233    5068 scope.go:122] "RemoveContainer" containerID="7cd84fdd31600ea3f367d1c9b49b1a4bdc85f0d9a894fc11d6104fd55766f601"
Nov 04 01:28:04.004555 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:28:04.004536    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7cd84fdd31600ea3f367d1c9b49b1a4bdc85f0d9a894fc11d6104fd55766f601\": container with ID starting with 7cd84fdd31600ea3f367d1c9b49b1a4bdc85f0d9a894fc11d6104fd55766f601 not found: ID does not exist" containerID="7cd84fdd31600ea3f367d1c9b49b1a4bdc85f0d9a894fc11d6104fd55766f601"
Nov 04 01:28:04.004634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:04.004564    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7cd84fdd31600ea3f367d1c9b49b1a4bdc85f0d9a894fc11d6104fd55766f601"} err="failed to get container status \"7cd84fdd31600ea3f367d1c9b49b1a4bdc85f0d9a894fc11d6104fd55766f601\": rpc error: code = NotFound desc = could not find container \"7cd84fdd31600ea3f367d1c9b49b1a4bdc85f0d9a894fc11d6104fd55766f601\": container with ID starting with 7cd84fdd31600ea3f367d1c9b49b1a4bdc85f0d9a894fc11d6104fd55766f601 not found: ID does not exist"
Nov 04 01:28:04.004634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:04.004586    5068 scope.go:122] "RemoveContainer" containerID="c442438a5f770f4b9326055ad3c246e0146284879f99db5e5d1d1865b1e22f3a"
Nov 04 01:28:04.012436 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:04.012410    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8"]
Nov 04 01:28:04.015815 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:04.015760    5068 scope.go:122] "RemoveContainer" containerID="c442438a5f770f4b9326055ad3c246e0146284879f99db5e5d1d1865b1e22f3a"
Nov 04 01:28:04.017720 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:04.017695    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-8tlh8"]
Nov 04 01:28:04.017856 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:28:04.017757    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c442438a5f770f4b9326055ad3c246e0146284879f99db5e5d1d1865b1e22f3a\": container with ID starting with c442438a5f770f4b9326055ad3c246e0146284879f99db5e5d1d1865b1e22f3a not found: ID does not exist" containerID="c442438a5f770f4b9326055ad3c246e0146284879f99db5e5d1d1865b1e22f3a"
Nov 04 01:28:04.017856 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:04.017791    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c442438a5f770f4b9326055ad3c246e0146284879f99db5e5d1d1865b1e22f3a"} err="failed to get container status \"c442438a5f770f4b9326055ad3c246e0146284879f99db5e5d1d1865b1e22f3a\": rpc error: code = NotFound desc = could not find container \"c442438a5f770f4b9326055ad3c246e0146284879f99db5e5d1d1865b1e22f3a\": container with ID starting with c442438a5f770f4b9326055ad3c246e0146284879f99db5e5d1d1865b1e22f3a not found: ID does not exist"
Nov 04 01:28:04.029935 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:04.029902    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf"]
Nov 04 01:28:04.036162 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:04.036136    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-4fflf"]
Nov 04 01:28:04.049698 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:04.049642    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f"]
Nov 04 01:28:04.282666 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:04.282602    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="43eaae59-c37c-4553-a856-fc8aa9506c00" path="/var/lib/kubelet/pods/43eaae59-c37c-4553-a856-fc8aa9506c00/volumes"
Nov 04 01:28:04.283619 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:04.283581    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c3d5c7b1-d7fe-4a70-aba1-1131171f55c1" path="/var/lib/kubelet/pods/c3d5c7b1-d7fe-4a70-aba1-1131171f55c1/volumes"
Nov 04 01:28:04.284290 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:04.284178    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-cffa80-openshift-default-69c4c7fd8f-vxm7f"]
Nov 04 01:28:05.969744 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:05.969569    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-875528-74f64c5467-26b7b"]
Nov 04 01:28:05.970559 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:05.970521    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-875528-74f64c5467-26b7b" podUID="bb1ea6b5-c573-4bfe-8339-caf72acbfc36" containerName="maas-api" containerID="cri-o://4b869b3553258c78432ea706d0f1043ad490d1dc2d9a0d0562dcd012671a5c58" gracePeriod=30
Nov 04 01:28:06.119727 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:06.119680    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7ca2a53a-ada2-4565-bc5a-66c7eefef997" path="/var/lib/kubelet/pods/7ca2a53a-ada2-4565-bc5a-66c7eefef997/volumes"
Nov 04 01:28:06.417697 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:06.416998    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-3bf4f9-7db74f6687-htzl4" podUID="2b505e5e-2cb5-47cc-97aa-02b96383d117" containerName="payload-pre-processing" containerID="cri-o://86c74b7747d80a8958b97f09f29795e3d8c0da31f7540c2fde4f4fa7bd2bde1c" gracePeriod=25
Nov 04 01:28:06.685109 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:06.685032    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-875528-74f64c5467-26b7b"
Nov 04 01:28:06.730967 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:06.728361    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bb1ea6b5-c573-4bfe-8339-caf72acbfc36-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb1ea6b5-c573-4bfe-8339-caf72acbfc36-openshift-service-ca\") pod \"bb1ea6b5-c573-4bfe-8339-caf72acbfc36\" (UID: \"bb1ea6b5-c573-4bfe-8339-caf72acbfc36\") "
Nov 04 01:28:06.730967 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:06.728806    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb1ea6b5-c573-4bfe-8339-caf72acbfc36-openshift-service-ca" pod "bb1ea6b5-c573-4bfe-8339-caf72acbfc36" (UID: "bb1ea6b5-c573-4bfe-8339-caf72acbfc36"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:28:06.730967 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:06.728906    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bb1ea6b5-c573-4bfe-8339-caf72acbfc36-metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb1ea6b5-c573-4bfe-8339-caf72acbfc36-metrics-certs\") pod \"bb1ea6b5-c573-4bfe-8339-caf72acbfc36\" (UID: \"bb1ea6b5-c573-4bfe-8339-caf72acbfc36\") "
Nov 04 01:28:06.730967 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:06.729375    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bb1ea6b5-c573-4bfe-8339-caf72acbfc36-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb1ea6b5-c573-4bfe-8339-caf72acbfc36-maas-api-tls\") pod \"bb1ea6b5-c573-4bfe-8339-caf72acbfc36\" (UID: \"bb1ea6b5-c573-4bfe-8339-caf72acbfc36\") "
Nov 04 01:28:06.730967 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:06.729955    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bb1ea6b5-c573-4bfe-8339-caf72acbfc36-kube-api-access-tb6kq\" (UniqueName: \"kubernetes.io/projected/bb1ea6b5-c573-4bfe-8339-caf72acbfc36-kube-api-access-tb6kq\") pod \"bb1ea6b5-c573-4bfe-8339-caf72acbfc36\" (UID: \"bb1ea6b5-c573-4bfe-8339-caf72acbfc36\") "
Nov 04 01:28:06.731267 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:06.731082    5068 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb1ea6b5-c573-4bfe-8339-caf72acbfc36-openshift-service-ca\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:06.733043 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:06.732220    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb1ea6b5-c573-4bfe-8339-caf72acbfc36-metrics-certs" pod "bb1ea6b5-c573-4bfe-8339-caf72acbfc36" (UID: "bb1ea6b5-c573-4bfe-8339-caf72acbfc36"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:28:06.733043 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:06.732282    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb1ea6b5-c573-4bfe-8339-caf72acbfc36-maas-api-tls" pod "bb1ea6b5-c573-4bfe-8339-caf72acbfc36" (UID: "bb1ea6b5-c573-4bfe-8339-caf72acbfc36"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:28:06.733182 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:06.733102    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb1ea6b5-c573-4bfe-8339-caf72acbfc36-kube-api-access-tb6kq" pod "bb1ea6b5-c573-4bfe-8339-caf72acbfc36" (UID: "bb1ea6b5-c573-4bfe-8339-caf72acbfc36"). InnerVolumeSpecName "kube-api-access-tb6kq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:28:06.856634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:06.835651    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tb6kq\" (UniqueName: \"kubernetes.io/projected/bb1ea6b5-c573-4bfe-8339-caf72acbfc36-kube-api-access-tb6kq\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:06.856634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:06.835691    5068 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb1ea6b5-c573-4bfe-8339-caf72acbfc36-metrics-certs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:06.856634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:06.835705    5068 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb1ea6b5-c573-4bfe-8339-caf72acbfc36-maas-api-tls\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:06.892426 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:06.892395    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-3bf4f9-7db74f6687-htzl4"
Nov 04 01:28:06.937154 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:06.937077    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2b505e5e-2cb5-47cc-97aa-02b96383d117-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b505e5e-2cb5-47cc-97aa-02b96383d117-plugins-config-volume\") pod \"2b505e5e-2cb5-47cc-97aa-02b96383d117\" (UID: \"2b505e5e-2cb5-47cc-97aa-02b96383d117\") "
Nov 04 01:28:06.937340 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:06.937326    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2b505e5e-2cb5-47cc-97aa-02b96383d117-kube-api-access-7st4p\" (UniqueName: \"kubernetes.io/projected/2b505e5e-2cb5-47cc-97aa-02b96383d117-kube-api-access-7st4p\") pod \"2b505e5e-2cb5-47cc-97aa-02b96383d117\" (UID: \"2b505e5e-2cb5-47cc-97aa-02b96383d117\") "
Nov 04 01:28:06.937468 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:06.937445    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b505e5e-2cb5-47cc-97aa-02b96383d117-plugins-config-volume" pod "2b505e5e-2cb5-47cc-97aa-02b96383d117" (UID: "2b505e5e-2cb5-47cc-97aa-02b96383d117"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:28:06.937762 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:06.937746    5068 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b505e5e-2cb5-47cc-97aa-02b96383d117-plugins-config-volume\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:06.939561 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:06.939534    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b505e5e-2cb5-47cc-97aa-02b96383d117-kube-api-access-7st4p" pod "2b505e5e-2cb5-47cc-97aa-02b96383d117" (UID: "2b505e5e-2cb5-47cc-97aa-02b96383d117"). InnerVolumeSpecName "kube-api-access-7st4p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:28:07.021286 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:07.021243    5068 generic.go:356] "Generic (PLEG): container finished" podID="2b505e5e-2cb5-47cc-97aa-02b96383d117" containerID="86c74b7747d80a8958b97f09f29795e3d8c0da31f7540c2fde4f4fa7bd2bde1c" exitCode=0
Nov 04 01:28:07.021759 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:07.021294    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-3bf4f9-7db74f6687-htzl4"
Nov 04 01:28:07.021759 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:07.021328    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-3bf4f9-7db74f6687-htzl4" event={"ID":"2b505e5e-2cb5-47cc-97aa-02b96383d117","Type":"ContainerDied","Data":"86c74b7747d80a8958b97f09f29795e3d8c0da31f7540c2fde4f4fa7bd2bde1c"}
Nov 04 01:28:07.021759 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:07.021388    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-3bf4f9-7db74f6687-htzl4" event={"ID":"2b505e5e-2cb5-47cc-97aa-02b96383d117","Type":"ContainerDied","Data":"59996fcf51ebcf96311946e73e47c0da16494010da4783be4c8596c12a8d895b"}
Nov 04 01:28:07.021759 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:07.021418    5068 scope.go:122] "RemoveContainer" containerID="86c74b7747d80a8958b97f09f29795e3d8c0da31f7540c2fde4f4fa7bd2bde1c"
Nov 04 01:28:07.023488 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:07.023459    5068 generic.go:356] "Generic (PLEG): container finished" podID="bb1ea6b5-c573-4bfe-8339-caf72acbfc36" containerID="4b869b3553258c78432ea706d0f1043ad490d1dc2d9a0d0562dcd012671a5c58" exitCode=0
Nov 04 01:28:07.023634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:07.023501    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-875528-74f64c5467-26b7b" event={"ID":"bb1ea6b5-c573-4bfe-8339-caf72acbfc36","Type":"ContainerDied","Data":"4b869b3553258c78432ea706d0f1043ad490d1dc2d9a0d0562dcd012671a5c58"}
Nov 04 01:28:07.023634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:07.023522    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-875528-74f64c5467-26b7b"
Nov 04 01:28:07.023634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:07.023545    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-875528-74f64c5467-26b7b" event={"ID":"bb1ea6b5-c573-4bfe-8339-caf72acbfc36","Type":"ContainerDied","Data":"0927c3cf607baee4352f528a970b13ae70d54676155defbdf8789f95e28f2079"}
Nov 04 01:28:07.034347 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:07.034323    5068 scope.go:122] "RemoveContainer" containerID="86c74b7747d80a8958b97f09f29795e3d8c0da31f7540c2fde4f4fa7bd2bde1c"
Nov 04 01:28:07.034658 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:28:07.034628    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86c74b7747d80a8958b97f09f29795e3d8c0da31f7540c2fde4f4fa7bd2bde1c\": container with ID starting with 86c74b7747d80a8958b97f09f29795e3d8c0da31f7540c2fde4f4fa7bd2bde1c not found: ID does not exist" containerID="86c74b7747d80a8958b97f09f29795e3d8c0da31f7540c2fde4f4fa7bd2bde1c"
Nov 04 01:28:07.034743 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:07.034666    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86c74b7747d80a8958b97f09f29795e3d8c0da31f7540c2fde4f4fa7bd2bde1c"} err="failed to get container status \"86c74b7747d80a8958b97f09f29795e3d8c0da31f7540c2fde4f4fa7bd2bde1c\": rpc error: code = NotFound desc = could not find container \"86c74b7747d80a8958b97f09f29795e3d8c0da31f7540c2fde4f4fa7bd2bde1c\": container with ID starting with 86c74b7747d80a8958b97f09f29795e3d8c0da31f7540c2fde4f4fa7bd2bde1c not found: ID does not exist"
Nov 04 01:28:07.034743 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:07.034683    5068 scope.go:122] "RemoveContainer" containerID="4b869b3553258c78432ea706d0f1043ad490d1dc2d9a0d0562dcd012671a5c58"
Nov 04 01:28:07.039072 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:07.039050    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7st4p\" (UniqueName: \"kubernetes.io/projected/2b505e5e-2cb5-47cc-97aa-02b96383d117-kube-api-access-7st4p\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:07.052259 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:07.052202    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-875528-74f64c5467-26b7b"]
Nov 04 01:28:07.056470 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:07.056449    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-875528-74f64c5467-26b7b"]
Nov 04 01:28:07.072335 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:07.072315    5068 scope.go:122] "RemoveContainer" containerID="4b869b3553258c78432ea706d0f1043ad490d1dc2d9a0d0562dcd012671a5c58"
Nov 04 01:28:07.072429 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:07.072421    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w4-3bf4f9-7db74f6687-htzl4"]
Nov 04 01:28:07.072731 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:28:07.072711    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b869b3553258c78432ea706d0f1043ad490d1dc2d9a0d0562dcd012671a5c58\": container with ID starting with 4b869b3553258c78432ea706d0f1043ad490d1dc2d9a0d0562dcd012671a5c58 not found: ID does not exist" containerID="4b869b3553258c78432ea706d0f1043ad490d1dc2d9a0d0562dcd012671a5c58"
Nov 04 01:28:07.072823 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:07.072740    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b869b3553258c78432ea706d0f1043ad490d1dc2d9a0d0562dcd012671a5c58"} err="failed to get container status \"4b869b3553258c78432ea706d0f1043ad490d1dc2d9a0d0562dcd012671a5c58\": rpc error: code = NotFound desc = could not find container \"4b869b3553258c78432ea706d0f1043ad490d1dc2d9a0d0562dcd012671a5c58\": container with ID starting with 4b869b3553258c78432ea706d0f1043ad490d1dc2d9a0d0562dcd012671a5c58 not found: ID does not exist"
Nov 04 01:28:07.076545 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:07.076526    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w4-3bf4f9-7db74f6687-htzl4"]
Nov 04 01:28:08.114943 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:08.114901    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2b505e5e-2cb5-47cc-97aa-02b96383d117" path="/var/lib/kubelet/pods/2b505e5e-2cb5-47cc-97aa-02b96383d117/volumes"
Nov 04 01:28:08.115371 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:08.115356    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bb1ea6b5-c573-4bfe-8339-caf72acbfc36" path="/var/lib/kubelet/pods/bb1ea6b5-c573-4bfe-8339-caf72acbfc36/volumes"
Nov 04 01:28:09.474474 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:09.474428    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-875528-74f64c5467-26b7b" podUID="bb1ea6b5-c573-4bfe-8339-caf72acbfc36" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.57:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:28:10.554292 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.554248    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6ecd1f-857447647c-54bjb"]
Nov 04 01:28:10.555666 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.555625    5068 cpu_manager.go:414] "containerMap: removing container" podUID="31ede3d2-d690-4178-90a7-69c82caa9e2c" containerName="istio-proxy"
Nov 04 01:28:10.555872 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.555856    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="31ede3d2-d690-4178-90a7-69c82caa9e2c" containerName="istio-proxy"
Nov 04 01:28:10.555987 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.555976    5068 cpu_manager.go:414] "containerMap: removing container" podUID="1f176eb7-f53c-48c8-a7dc-0e9212448556" containerName="payload-processing"
Nov 04 01:28:10.556067 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.556055    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1f176eb7-f53c-48c8-a7dc-0e9212448556" containerName="payload-processing"
Nov 04 01:28:10.556147 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.556138    5068 cpu_manager.go:414] "containerMap: removing container" podUID="eb9132e5-b90b-46fb-b86c-ca713419906d" containerName="main"
Nov 04 01:28:10.556213 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.556204    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eb9132e5-b90b-46fb-b86c-ca713419906d" containerName="main"
Nov 04 01:28:10.556298 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.556288    5068 cpu_manager.go:414] "containerMap: removing container" podUID="c4f8ccdf-b225-453b-87a9-9c6212d77802" containerName="maas-api"
Nov 04 01:28:10.556361 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.556353    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c4f8ccdf-b225-453b-87a9-9c6212d77802" containerName="maas-api"
Nov 04 01:28:10.556431 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.556422    5068 cpu_manager.go:414] "containerMap: removing container" podUID="7ca2a53a-ada2-4565-bc5a-66c7eefef997" containerName="istio-proxy"
Nov 04 01:28:10.556500 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.556490    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7ca2a53a-ada2-4565-bc5a-66c7eefef997" containerName="istio-proxy"
Nov 04 01:28:10.556571 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.556562    5068 cpu_manager.go:414] "containerMap: removing container" podUID="fa8debd2-b803-412c-9e4c-53917a748a59" containerName="istio-proxy"
Nov 04 01:28:10.556659 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.556647    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fa8debd2-b803-412c-9e4c-53917a748a59" containerName="istio-proxy"
Nov 04 01:28:10.556752 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.556742    5068 cpu_manager.go:414] "containerMap: removing container" podUID="bb1ea6b5-c573-4bfe-8339-caf72acbfc36" containerName="maas-api"
Nov 04 01:28:10.556821 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.556812    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bb1ea6b5-c573-4bfe-8339-caf72acbfc36" containerName="maas-api"
Nov 04 01:28:10.556889 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.556881    5068 cpu_manager.go:414] "containerMap: removing container" podUID="c3d5c7b1-d7fe-4a70-aba1-1131171f55c1" containerName="istio-proxy"
Nov 04 01:28:10.556956 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.556947    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c3d5c7b1-d7fe-4a70-aba1-1131171f55c1" containerName="istio-proxy"
Nov 04 01:28:10.557031 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.557022    5068 cpu_manager.go:414] "containerMap: removing container" podUID="2b505e5e-2cb5-47cc-97aa-02b96383d117" containerName="payload-pre-processing"
Nov 04 01:28:10.557096 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.557086    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2b505e5e-2cb5-47cc-97aa-02b96383d117" containerName="payload-pre-processing"
Nov 04 01:28:10.557166 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.557157    5068 cpu_manager.go:414] "containerMap: removing container" podUID="43eaae59-c37c-4553-a856-fc8aa9506c00" containerName="istio-proxy"
Nov 04 01:28:10.557236 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.557227    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="43eaae59-c37c-4553-a856-fc8aa9506c00" containerName="istio-proxy"
Nov 04 01:28:10.557323 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.557314    5068 cpu_manager.go:414] "containerMap: removing container" podUID="6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d" containerName="main"
Nov 04 01:28:10.557393 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.557383    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d" containerName="main"
Nov 04 01:28:10.557466 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.557458    5068 cpu_manager.go:414] "containerMap: removing container" podUID="1b4a1b43-a286-436d-bafe-a10d9a57a4d2" containerName="payload-processing"
Nov 04 01:28:10.557530 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.557521    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1b4a1b43-a286-436d-bafe-a10d9a57a4d2" containerName="payload-processing"
Nov 04 01:28:10.557598 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.557589    5068 cpu_manager.go:414] "containerMap: removing container" podUID="841e9fef-46d6-4625-9b72-14b4e0eeb7dd" containerName="payload-pre-processing"
Nov 04 01:28:10.557695 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.557682    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="841e9fef-46d6-4625-9b72-14b4e0eeb7dd" containerName="payload-pre-processing"
Nov 04 01:28:10.557776 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.557767    5068 cpu_manager.go:414] "containerMap: removing container" podUID="62dc4132-b26f-4b4f-a4c1-936dfa171b05" containerName="payload-processing"
Nov 04 01:28:10.557848 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.557839    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="62dc4132-b26f-4b4f-a4c1-936dfa171b05" containerName="payload-processing"
Nov 04 01:28:10.557920 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.557911    5068 cpu_manager.go:414] "containerMap: removing container" podUID="39c35ec4-5c1b-472e-9e73-991258bb835d" containerName="maas-api"
Nov 04 01:28:10.557996 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.557987    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="39c35ec4-5c1b-472e-9e73-991258bb835d" containerName="maas-api"
Nov 04 01:28:10.558068 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.558060    5068 cpu_manager.go:414] "containerMap: removing container" podUID="1799b838-9323-4b50-b630-13f956913a62" containerName="maas-api"
Nov 04 01:28:10.558144 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.558135    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1799b838-9323-4b50-b630-13f956913a62" containerName="maas-api"
Nov 04 01:28:10.558220 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.558211    5068 cpu_manager.go:414] "containerMap: removing container" podUID="d899bb80-977d-4b53-b96c-b354dcbdad2d" containerName="istio-proxy"
Nov 04 01:28:10.558282 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.558273    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d899bb80-977d-4b53-b96c-b354dcbdad2d" containerName="istio-proxy"
Nov 04 01:28:10.558516 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.558503    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="6cff90d2-d1f7-4dc1-b2be-5ca5c71c125d" containerName="main"
Nov 04 01:28:10.558604 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.558593    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="1b4a1b43-a286-436d-bafe-a10d9a57a4d2" containerName="payload-processing"
Nov 04 01:28:10.558707 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.558697    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="841e9fef-46d6-4625-9b72-14b4e0eeb7dd" containerName="payload-pre-processing"
Nov 04 01:28:10.558823 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.558811    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="62dc4132-b26f-4b4f-a4c1-936dfa171b05" containerName="payload-processing"
Nov 04 01:28:10.558922 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.558912    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="39c35ec4-5c1b-472e-9e73-991258bb835d" containerName="maas-api"
Nov 04 01:28:10.559012 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.559003    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="1799b838-9323-4b50-b630-13f956913a62" containerName="maas-api"
Nov 04 01:28:10.559111 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.559100    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="d899bb80-977d-4b53-b96c-b354dcbdad2d" containerName="istio-proxy"
Nov 04 01:28:10.559201 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.559192    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="31ede3d2-d690-4178-90a7-69c82caa9e2c" containerName="istio-proxy"
Nov 04 01:28:10.559302 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.559292    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="1f176eb7-f53c-48c8-a7dc-0e9212448556" containerName="payload-processing"
Nov 04 01:28:10.559389 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.559379    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="eb9132e5-b90b-46fb-b86c-ca713419906d" containerName="main"
Nov 04 01:28:10.559478 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.559469    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="c4f8ccdf-b225-453b-87a9-9c6212d77802" containerName="maas-api"
Nov 04 01:28:10.559564 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.559555    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="7ca2a53a-ada2-4565-bc5a-66c7eefef997" containerName="istio-proxy"
Nov 04 01:28:10.559666 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.559656    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="fa8debd2-b803-412c-9e4c-53917a748a59" containerName="istio-proxy"
Nov 04 01:28:10.559803 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.559792    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="bb1ea6b5-c573-4bfe-8339-caf72acbfc36" containerName="maas-api"
Nov 04 01:28:10.559903 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.559893    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="c3d5c7b1-d7fe-4a70-aba1-1131171f55c1" containerName="istio-proxy"
Nov 04 01:28:10.560024 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.559989    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="2b505e5e-2cb5-47cc-97aa-02b96383d117" containerName="payload-pre-processing"
Nov 04 01:28:10.560157 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.560146    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="43eaae59-c37c-4553-a856-fc8aa9506c00" containerName="istio-proxy"
Nov 04 01:28:10.581891 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.574750    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ecd1f-857447647c-54bjb"
Nov 04 01:28:10.581891 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.576132    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6ecd1f-857447647c-54bjb"]
Nov 04 01:28:10.581891 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.581397    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-6ecd1f-serving-cert\""
Nov 04 01:28:10.626504 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.626447    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9dfccbc-dc8d-4569-bac9-6847e6658b8c-maas-api-tls\") pod \"maas-api-e2e-api-b-6ecd1f-857447647c-54bjb\" (UID: \"c9dfccbc-dc8d-4569-bac9-6847e6658b8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ecd1f-857447647c-54bjb"
Nov 04 01:28:10.626686 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.626550    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpwd8\" (UniqueName: \"kubernetes.io/projected/c9dfccbc-dc8d-4569-bac9-6847e6658b8c-kube-api-access-kpwd8\") pod \"maas-api-e2e-api-b-6ecd1f-857447647c-54bjb\" (UID: \"c9dfccbc-dc8d-4569-bac9-6847e6658b8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ecd1f-857447647c-54bjb"
Nov 04 01:28:10.626686 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.626598    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9dfccbc-dc8d-4569-bac9-6847e6658b8c-openshift-service-ca\") pod \"maas-api-e2e-api-b-6ecd1f-857447647c-54bjb\" (UID: \"c9dfccbc-dc8d-4569-bac9-6847e6658b8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ecd1f-857447647c-54bjb"
Nov 04 01:28:10.626819 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.626748    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9dfccbc-dc8d-4569-bac9-6847e6658b8c-metrics-certs\") pod \"maas-api-e2e-api-b-6ecd1f-857447647c-54bjb\" (UID: \"c9dfccbc-dc8d-4569-bac9-6847e6658b8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ecd1f-857447647c-54bjb"
Nov 04 01:28:10.728446 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.728402    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9dfccbc-dc8d-4569-bac9-6847e6658b8c-metrics-certs\") pod \"maas-api-e2e-api-b-6ecd1f-857447647c-54bjb\" (UID: \"c9dfccbc-dc8d-4569-bac9-6847e6658b8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ecd1f-857447647c-54bjb"
Nov 04 01:28:10.728637 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.728599    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9dfccbc-dc8d-4569-bac9-6847e6658b8c-maas-api-tls\") pod \"maas-api-e2e-api-b-6ecd1f-857447647c-54bjb\" (UID: \"c9dfccbc-dc8d-4569-bac9-6847e6658b8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ecd1f-857447647c-54bjb"
Nov 04 01:28:10.728793 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.728657    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpwd8\" (UniqueName: \"kubernetes.io/projected/c9dfccbc-dc8d-4569-bac9-6847e6658b8c-kube-api-access-kpwd8\") pod \"maas-api-e2e-api-b-6ecd1f-857447647c-54bjb\" (UID: \"c9dfccbc-dc8d-4569-bac9-6847e6658b8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ecd1f-857447647c-54bjb"
Nov 04 01:28:10.728793 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.728696    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9dfccbc-dc8d-4569-bac9-6847e6658b8c-openshift-service-ca\") pod \"maas-api-e2e-api-b-6ecd1f-857447647c-54bjb\" (UID: \"c9dfccbc-dc8d-4569-bac9-6847e6658b8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ecd1f-857447647c-54bjb"
Nov 04 01:28:10.730343 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.730286    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9dfccbc-dc8d-4569-bac9-6847e6658b8c-openshift-service-ca\") pod \"maas-api-e2e-api-b-6ecd1f-857447647c-54bjb\" (UID: \"c9dfccbc-dc8d-4569-bac9-6847e6658b8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ecd1f-857447647c-54bjb"
Nov 04 01:28:10.756726 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.755222    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9dfccbc-dc8d-4569-bac9-6847e6658b8c-maas-api-tls\") pod \"maas-api-e2e-api-b-6ecd1f-857447647c-54bjb\" (UID: \"c9dfccbc-dc8d-4569-bac9-6847e6658b8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ecd1f-857447647c-54bjb"
Nov 04 01:28:10.769653 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.767924    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpwd8\" (UniqueName: \"kubernetes.io/projected/c9dfccbc-dc8d-4569-bac9-6847e6658b8c-kube-api-access-kpwd8\") pod \"maas-api-e2e-api-b-6ecd1f-857447647c-54bjb\" (UID: \"c9dfccbc-dc8d-4569-bac9-6847e6658b8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ecd1f-857447647c-54bjb"
Nov 04 01:28:10.769653 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.768171    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9dfccbc-dc8d-4569-bac9-6847e6658b8c-metrics-certs\") pod \"maas-api-e2e-api-b-6ecd1f-857447647c-54bjb\" (UID: \"c9dfccbc-dc8d-4569-bac9-6847e6658b8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ecd1f-857447647c-54bjb"
Nov 04 01:28:10.897858 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:10.897782    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ecd1f-857447647c-54bjb"
Nov 04 01:28:11.190952 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:28:11.190922    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9dfccbc_dc8d_4569_bac9_6847e6658b8c.slice/crio-37947f6b07314ae714194af4aaf8a66024ea0f907edb42a4294eba7051a65c95 WatchSource:0}: Error finding container 37947f6b07314ae714194af4aaf8a66024ea0f907edb42a4294eba7051a65c95: Status 404 returned error can't find the container with id 37947f6b07314ae714194af4aaf8a66024ea0f907edb42a4294eba7051a65c95
Nov 04 01:28:11.191880 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:11.191852    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6ecd1f-857447647c-54bjb"]
Nov 04 01:28:11.192518 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:11.192500    5068 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 01:28:12.104648 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:12.102273    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ecd1f-857447647c-54bjb" event={"ID":"c9dfccbc-dc8d-4569-bac9-6847e6658b8c","Type":"ContainerStarted","Data":"71a4fea5e3f5d50da81a6657560c471be4842ed3101e12454279f5f3d958ad72"}
Nov 04 01:28:12.104648 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:12.102341    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ecd1f-857447647c-54bjb" event={"ID":"c9dfccbc-dc8d-4569-bac9-6847e6658b8c","Type":"ContainerStarted","Data":"37947f6b07314ae714194af4aaf8a66024ea0f907edb42a4294eba7051a65c95"}
Nov 04 01:28:12.104648 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:12.103404    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ecd1f-857447647c-54bjb"
Nov 04 01:28:12.147422 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:12.147360    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ecd1f-857447647c-54bjb" podStartSLOduration=1.700262859 podStartE2EDuration="2.14733903s" totalImagesPullingTime="447.076171ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:28:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:28:12.145673973 +0000 UTC m=+1532.192234764" watchObservedRunningTime="2026-11-04 01:28:12.14733903 +0000 UTC m=+1532.193899821"
Nov 04 01:28:12.420687 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:12.412498    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc"]
Nov 04 01:28:12.489816 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:12.489557    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc"]
Nov 04 01:28:12.489816 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:12.489707    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc"
Nov 04 01:28:12.582687 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:12.582592    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2697761-a360-4c3c-a5d5-c96f06998bb3-workload-socket\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc\" (UID: \"c2697761-a360-4c3c-a5d5-c96f06998bb3\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc"
Nov 04 01:28:12.582932 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:12.582734    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdbjp\" (UniqueName: \"kubernetes.io/projected/c2697761-a360-4c3c-a5d5-c96f06998bb3-kube-api-access-gdbjp\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc\" (UID: \"c2697761-a360-4c3c-a5d5-c96f06998bb3\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc"
Nov 04 01:28:12.582932 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:12.582827    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2697761-a360-4c3c-a5d5-c96f06998bb3-istio-podinfo\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc\" (UID: \"c2697761-a360-4c3c-a5d5-c96f06998bb3\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc"
Nov 04 01:28:12.582932 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:12.582880    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2697761-a360-4c3c-a5d5-c96f06998bb3-istiod-ca-cert\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc\" (UID: \"c2697761-a360-4c3c-a5d5-c96f06998bb3\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc"
Nov 04 01:28:12.582932 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:12.582921    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2697761-a360-4c3c-a5d5-c96f06998bb3-credential-socket\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc\" (UID: \"c2697761-a360-4c3c-a5d5-c96f06998bb3\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc"
Nov 04 01:28:12.583229 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:12.582970    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2697761-a360-4c3c-a5d5-c96f06998bb3-workload-certs\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc\" (UID: \"c2697761-a360-4c3c-a5d5-c96f06998bb3\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc"
Nov 04 01:28:12.583229 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:12.583017    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2697761-a360-4c3c-a5d5-c96f06998bb3-istio-token\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc\" (UID: \"c2697761-a360-4c3c-a5d5-c96f06998bb3\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc"
Nov 04 01:28:12.583229 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:12.583048    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2697761-a360-4c3c-a5d5-c96f06998bb3-istio-envoy\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc\" (UID: \"c2697761-a360-4c3c-a5d5-c96f06998bb3\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc"
Nov 04 01:28:12.583229 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:12.583098    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2697761-a360-4c3c-a5d5-c96f06998bb3-istio-data\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc\" (UID: \"c2697761-a360-4c3c-a5d5-c96f06998bb3\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc"
Nov 04 01:28:12.684432 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:12.684342    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdbjp\" (UniqueName: \"kubernetes.io/projected/c2697761-a360-4c3c-a5d5-c96f06998bb3-kube-api-access-gdbjp\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc\" (UID: \"c2697761-a360-4c3c-a5d5-c96f06998bb3\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc"
Nov 04 01:28:12.684595 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:12.684434    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2697761-a360-4c3c-a5d5-c96f06998bb3-istio-podinfo\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc\" (UID: \"c2697761-a360-4c3c-a5d5-c96f06998bb3\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc"
Nov 04 01:28:12.684595 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:12.684465    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2697761-a360-4c3c-a5d5-c96f06998bb3-istiod-ca-cert\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc\" (UID: \"c2697761-a360-4c3c-a5d5-c96f06998bb3\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc"
Nov 04 01:28:12.684595 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:12.684506    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2697761-a360-4c3c-a5d5-c96f06998bb3-credential-socket\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc\" (UID: \"c2697761-a360-4c3c-a5d5-c96f06998bb3\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc"
Nov 04 01:28:12.684595 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:12.684550    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2697761-a360-4c3c-a5d5-c96f06998bb3-workload-certs\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc\" (UID: \"c2697761-a360-4c3c-a5d5-c96f06998bb3\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc"
Nov 04 01:28:12.684860 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:12.684599    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2697761-a360-4c3c-a5d5-c96f06998bb3-istio-token\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc\" (UID: \"c2697761-a360-4c3c-a5d5-c96f06998bb3\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc"
Nov 04 01:28:12.684860 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:12.684663    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2697761-a360-4c3c-a5d5-c96f06998bb3-istio-envoy\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc\" (UID: \"c2697761-a360-4c3c-a5d5-c96f06998bb3\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc"
Nov 04 01:28:12.684860 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:12.684704    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2697761-a360-4c3c-a5d5-c96f06998bb3-istio-data\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc\" (UID: \"c2697761-a360-4c3c-a5d5-c96f06998bb3\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc"
Nov 04 01:28:12.684860 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:12.684800    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2697761-a360-4c3c-a5d5-c96f06998bb3-workload-socket\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc\" (UID: \"c2697761-a360-4c3c-a5d5-c96f06998bb3\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc"
Nov 04 01:28:12.685338 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:12.685309    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2697761-a360-4c3c-a5d5-c96f06998bb3-workload-certs\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc\" (UID: \"c2697761-a360-4c3c-a5d5-c96f06998bb3\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc"
Nov 04 01:28:12.685494 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:12.685472    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2697761-a360-4c3c-a5d5-c96f06998bb3-istiod-ca-cert\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc\" (UID: \"c2697761-a360-4c3c-a5d5-c96f06998bb3\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc"
Nov 04 01:28:12.685571 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:12.685472    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2697761-a360-4c3c-a5d5-c96f06998bb3-istio-data\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc\" (UID: \"c2697761-a360-4c3c-a5d5-c96f06998bb3\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc"
Nov 04 01:28:12.686577 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:12.686019    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2697761-a360-4c3c-a5d5-c96f06998bb3-credential-socket\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc\" (UID: \"c2697761-a360-4c3c-a5d5-c96f06998bb3\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc"
Nov 04 01:28:12.686577 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:12.686133    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2697761-a360-4c3c-a5d5-c96f06998bb3-workload-socket\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc\" (UID: \"c2697761-a360-4c3c-a5d5-c96f06998bb3\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc"
Nov 04 01:28:12.688099 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:12.688077    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2697761-a360-4c3c-a5d5-c96f06998bb3-istio-podinfo\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc\" (UID: \"c2697761-a360-4c3c-a5d5-c96f06998bb3\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc"
Nov 04 01:28:12.688371 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:12.688351    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2697761-a360-4c3c-a5d5-c96f06998bb3-istio-envoy\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc\" (UID: \"c2697761-a360-4c3c-a5d5-c96f06998bb3\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc"
Nov 04 01:28:12.696972 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:12.696944    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2697761-a360-4c3c-a5d5-c96f06998bb3-istio-token\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc\" (UID: \"c2697761-a360-4c3c-a5d5-c96f06998bb3\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc"
Nov 04 01:28:12.697136 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:12.697113    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdbjp\" (UniqueName: \"kubernetes.io/projected/c2697761-a360-4c3c-a5d5-c96f06998bb3-kube-api-access-gdbjp\") pod \"e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc\" (UID: \"c2697761-a360-4c3c-a5d5-c96f06998bb3\") " pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc"
Nov 04 01:28:12.806032 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:12.806001    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc"
Nov 04 01:28:13.113361 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:28:13.113330    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2697761_a360_4c3c_a5d5_c96f06998bb3.slice/crio-dc60c08b7e72fa4cca15658416a983a302b6deabf782c3570df3d08b5b364815 WatchSource:0}: Error finding container dc60c08b7e72fa4cca15658416a983a302b6deabf782c3570df3d08b5b364815: Status 404 returned error can't find the container with id dc60c08b7e72fa4cca15658416a983a302b6deabf782c3570df3d08b5b364815
Nov 04 01:28:13.114196 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:13.114170    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc"]
Nov 04 01:28:13.116700 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:13.116664    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:28:13.116796 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:13.116743    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:28:13.116796 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:13.116772    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:28:13.584487 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:28:13.584451    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:28:14.136380 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:14.136336    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc" event={"ID":"c2697761-a360-4c3c-a5d5-c96f06998bb3","Type":"ContainerStarted","Data":"33372768c1e40b5058e82519f32767016312d6ddf1759497cd14862b7611ec20"}
Nov 04 01:28:14.136827 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:14.136388    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc" event={"ID":"c2697761-a360-4c3c-a5d5-c96f06998bb3","Type":"ContainerStarted","Data":"dc60c08b7e72fa4cca15658416a983a302b6deabf782c3570df3d08b5b364815"}
Nov 04 01:28:14.212632 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:14.212142    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc" podStartSLOduration=2.2121249179999998 podStartE2EDuration="2.212124918s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:28:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:28:14.206381422 +0000 UTC m=+1534.252942214" watchObservedRunningTime="2026-11-04 01:28:14.212124918 +0000 UTC m=+1534.258685711"
Nov 04 01:28:14.807206 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:14.807174    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc"
Nov 04 01:28:14.808528 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:14.808500    5068 patch_prober.go:29] interesting pod/e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.123:15021/healthz/ready\": dial tcp 10.132.0.123:15021: connect: connection refused" start-of-body=
Nov 04 01:28:14.808686 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:14.808548    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc" podUID="c2697761-a360-4c3c-a5d5-c96f06998bb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.123:15021/healthz/ready\": dial tcp 10.132.0.123:15021: connect: connection refused"
Nov 04 01:28:15.606665 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:15.606624    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82"]
Nov 04 01:28:15.610684 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:15.610660    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm"]
Nov 04 01:28:15.623492 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:15.623465    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz"]
Nov 04 01:28:15.634653 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:15.633734    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9"]
Nov 04 01:28:15.806798 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:15.806758    5068 patch_prober.go:29] interesting pod/e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.123:15021/healthz/ready\": dial tcp 10.132.0.123:15021: connect: connection refused" start-of-body=
Nov 04 01:28:15.807022 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:15.806995    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc" podUID="c2697761-a360-4c3c-a5d5-c96f06998bb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.123:15021/healthz/ready\": dial tcp 10.132.0.123:15021: connect: connection refused"
Nov 04 01:28:16.807210 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:16.807165    5068 patch_prober.go:29] interesting pod/e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.123:15021/healthz/ready\": dial tcp 10.132.0.123:15021: connect: connection refused" start-of-body=
Nov 04 01:28:16.807710 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:16.807226    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc" podUID="c2697761-a360-4c3c-a5d5-c96f06998bb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.123:15021/healthz/ready\": dial tcp 10.132.0.123:15021: connect: connection refused"
Nov 04 01:28:17.806712 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:17.806681    5068 patch_prober.go:29] interesting pod/e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.123:15021/healthz/ready\": dial tcp 10.132.0.123:15021: connect: connection refused" start-of-body=
Nov 04 01:28:17.807019 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:17.806736    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc" podUID="c2697761-a360-4c3c-a5d5-c96f06998bb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.123:15021/healthz/ready\": dial tcp 10.132.0.123:15021: connect: connection refused"
Nov 04 01:28:18.806839 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:18.806807    5068 patch_prober.go:29] interesting pod/e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.123:15021/healthz/ready\": dial tcp 10.132.0.123:15021: connect: connection refused" start-of-body=
Nov 04 01:28:18.807218 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:18.806861    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc" podUID="c2697761-a360-4c3c-a5d5-c96f06998bb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.123:15021/healthz/ready\": dial tcp 10.132.0.123:15021: connect: connection refused"
Nov 04 01:28:19.190746 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:19.187819    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ecd1f-857447647c-54bjb"
Nov 04 01:28:19.502139 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:19.502054    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc"]
Nov 04 01:28:19.502649 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:19.502597    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc" podUID="c2697761-a360-4c3c-a5d5-c96f06998bb3" containerName="istio-proxy" containerID="cri-o://33372768c1e40b5058e82519f32767016312d6ddf1759497cd14862b7611ec20" gracePeriod=30
Nov 04 01:28:19.516967 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:19.515854    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64"]
Nov 04 01:28:19.516967 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:19.516082    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64" podUID="aae6d603-4415-436c-917f-448a09092e29" containerName="istio-proxy" containerID="cri-o://1a3e56562a94d0f7be583e9a7c03fc053544819101fa7a4d16b5180779e38379" gracePeriod=30
Nov 04 01:28:19.523179 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:19.523158    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n"]
Nov 04 01:28:19.524801 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:19.524592    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n" podUID="d8725463-9c1f-4cee-b5ec-28fb3ca9f939" containerName="istio-proxy" containerID="cri-o://e23b484dd69e1e7c555dffaaf47a76ee4845e1a6536e714f6a576db96dc795c5" gracePeriod=30
Nov 04 01:28:24.827298 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:24.827268    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64"
Nov 04 01:28:24.936472 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:24.936439    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/aae6d603-4415-436c-917f-448a09092e29-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aae6d603-4415-436c-917f-448a09092e29-istio-podinfo\") pod \"aae6d603-4415-436c-917f-448a09092e29\" (UID: \"aae6d603-4415-436c-917f-448a09092e29\") "
Nov 04 01:28:24.936670 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:24.936486    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aae6d603-4415-436c-917f-448a09092e29-istio-token\" (UniqueName: \"kubernetes.io/projected/aae6d603-4415-436c-917f-448a09092e29-istio-token\") pod \"aae6d603-4415-436c-917f-448a09092e29\" (UID: \"aae6d603-4415-436c-917f-448a09092e29\") "
Nov 04 01:28:24.936670 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:24.936588    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aae6d603-4415-436c-917f-448a09092e29-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aae6d603-4415-436c-917f-448a09092e29-workload-socket\") pod \"aae6d603-4415-436c-917f-448a09092e29\" (UID: \"aae6d603-4415-436c-917f-448a09092e29\") "
Nov 04 01:28:24.936785 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:24.936670    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aae6d603-4415-436c-917f-448a09092e29-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aae6d603-4415-436c-917f-448a09092e29-credential-socket\") pod \"aae6d603-4415-436c-917f-448a09092e29\" (UID: \"aae6d603-4415-436c-917f-448a09092e29\") "
Nov 04 01:28:24.936785 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:24.936702    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aae6d603-4415-436c-917f-448a09092e29-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aae6d603-4415-436c-917f-448a09092e29-istio-envoy\") pod \"aae6d603-4415-436c-917f-448a09092e29\" (UID: \"aae6d603-4415-436c-917f-448a09092e29\") "
Nov 04 01:28:24.936884 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:24.936800    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aae6d603-4415-436c-917f-448a09092e29-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aae6d603-4415-436c-917f-448a09092e29-workload-certs\") pod \"aae6d603-4415-436c-917f-448a09092e29\" (UID: \"aae6d603-4415-436c-917f-448a09092e29\") "
Nov 04 01:28:24.936942 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:24.936862    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aae6d603-4415-436c-917f-448a09092e29-workload-socket" pod "aae6d603-4415-436c-917f-448a09092e29" (UID: "aae6d603-4415-436c-917f-448a09092e29"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:24.937067 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:24.936932    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aae6d603-4415-436c-917f-448a09092e29-credential-socket" pod "aae6d603-4415-436c-917f-448a09092e29" (UID: "aae6d603-4415-436c-917f-448a09092e29"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:24.937067 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:24.936998    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aae6d603-4415-436c-917f-448a09092e29-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aae6d603-4415-436c-917f-448a09092e29-istio-data\") pod \"aae6d603-4415-436c-917f-448a09092e29\" (UID: \"aae6d603-4415-436c-917f-448a09092e29\") "
Nov 04 01:28:24.937163 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:24.937060    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aae6d603-4415-436c-917f-448a09092e29-workload-certs" pod "aae6d603-4415-436c-917f-448a09092e29" (UID: "aae6d603-4415-436c-917f-448a09092e29"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:24.937163 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:24.937076    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aae6d603-4415-436c-917f-448a09092e29-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aae6d603-4415-436c-917f-448a09092e29-istiod-ca-cert\") pod \"aae6d603-4415-436c-917f-448a09092e29\" (UID: \"aae6d603-4415-436c-917f-448a09092e29\") "
Nov 04 01:28:24.937163 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:24.937143    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aae6d603-4415-436c-917f-448a09092e29-kube-api-access-xfw9r\" (UniqueName: \"kubernetes.io/projected/aae6d603-4415-436c-917f-448a09092e29-kube-api-access-xfw9r\") pod \"aae6d603-4415-436c-917f-448a09092e29\" (UID: \"aae6d603-4415-436c-917f-448a09092e29\") "
Nov 04 01:28:24.937472 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:24.937444    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aae6d603-4415-436c-917f-448a09092e29-istiod-ca-cert" pod "aae6d603-4415-436c-917f-448a09092e29" (UID: "aae6d603-4415-436c-917f-448a09092e29"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:28:24.937914 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:24.937867    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aae6d603-4415-436c-917f-448a09092e29-istio-data" pod "aae6d603-4415-436c-917f-448a09092e29" (UID: "aae6d603-4415-436c-917f-448a09092e29"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:24.938766 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:24.938677    5068 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aae6d603-4415-436c-917f-448a09092e29-istio-data\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:24.938766 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:24.938702    5068 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aae6d603-4415-436c-917f-448a09092e29-istiod-ca-cert\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:24.938766 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:24.938718    5068 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aae6d603-4415-436c-917f-448a09092e29-workload-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:24.938766 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:24.938731    5068 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aae6d603-4415-436c-917f-448a09092e29-credential-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:24.938766 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:24.938745    5068 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aae6d603-4415-436c-917f-448a09092e29-workload-certs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:24.939231 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:24.939208    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/aae6d603-4415-436c-917f-448a09092e29-istio-podinfo" pod "aae6d603-4415-436c-917f-448a09092e29" (UID: "aae6d603-4415-436c-917f-448a09092e29"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:28:24.939509 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:24.939487    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aae6d603-4415-436c-917f-448a09092e29-istio-token" pod "aae6d603-4415-436c-917f-448a09092e29" (UID: "aae6d603-4415-436c-917f-448a09092e29"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:28:24.939746 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:24.939712    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aae6d603-4415-436c-917f-448a09092e29-kube-api-access-xfw9r" pod "aae6d603-4415-436c-917f-448a09092e29" (UID: "aae6d603-4415-436c-917f-448a09092e29"). InnerVolumeSpecName "kube-api-access-xfw9r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:28:24.939746 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:24.939732    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aae6d603-4415-436c-917f-448a09092e29-istio-envoy" pod "aae6d603-4415-436c-917f-448a09092e29" (UID: "aae6d603-4415-436c-917f-448a09092e29"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:25.064211 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:25.064121    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xfw9r\" (UniqueName: \"kubernetes.io/projected/aae6d603-4415-436c-917f-448a09092e29-kube-api-access-xfw9r\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:25.064211 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:25.064159    5068 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aae6d603-4415-436c-917f-448a09092e29-istio-podinfo\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:25.064211 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:25.064173    5068 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aae6d603-4415-436c-917f-448a09092e29-istio-token\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:25.064211 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:25.064185    5068 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aae6d603-4415-436c-917f-448a09092e29-istio-envoy\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:25.240301 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:25.240272    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n"
Nov 04 01:28:25.267936 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:25.267901    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-istio-token\" (UniqueName: \"kubernetes.io/projected/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-istio-token\") pod \"d8725463-9c1f-4cee-b5ec-28fb3ca9f939\" (UID: \"d8725463-9c1f-4cee-b5ec-28fb3ca9f939\") "
Nov 04 01:28:25.268116 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:25.267996    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-credential-socket\") pod \"d8725463-9c1f-4cee-b5ec-28fb3ca9f939\" (UID: \"d8725463-9c1f-4cee-b5ec-28fb3ca9f939\") "
Nov 04 01:28:25.268116 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:25.268058    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-kube-api-access-g9ssf\" (UniqueName: \"kubernetes.io/projected/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-kube-api-access-g9ssf\") pod \"d8725463-9c1f-4cee-b5ec-28fb3ca9f939\" (UID: \"d8725463-9c1f-4cee-b5ec-28fb3ca9f939\") "
Nov 04 01:28:25.268248 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:25.268125    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-workload-socket\") pod \"d8725463-9c1f-4cee-b5ec-28fb3ca9f939\" (UID: \"d8725463-9c1f-4cee-b5ec-28fb3ca9f939\") "
Nov 04 01:28:25.268248 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:25.268193    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-istio-data\") pod \"d8725463-9c1f-4cee-b5ec-28fb3ca9f939\" (UID: \"d8725463-9c1f-4cee-b5ec-28fb3ca9f939\") "
Nov 04 01:28:25.268355 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:25.268285    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-istio-envoy\") pod \"d8725463-9c1f-4cee-b5ec-28fb3ca9f939\" (UID: \"d8725463-9c1f-4cee-b5ec-28fb3ca9f939\") "
Nov 04 01:28:25.268420 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:25.268365    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-istio-podinfo\") pod \"d8725463-9c1f-4cee-b5ec-28fb3ca9f939\" (UID: \"d8725463-9c1f-4cee-b5ec-28fb3ca9f939\") "
Nov 04 01:28:25.268420 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:25.268394    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-workload-certs\") pod \"d8725463-9c1f-4cee-b5ec-28fb3ca9f939\" (UID: \"d8725463-9c1f-4cee-b5ec-28fb3ca9f939\") "
Nov 04 01:28:25.268524 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:25.268434    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-istiod-ca-cert\") pod \"d8725463-9c1f-4cee-b5ec-28fb3ca9f939\" (UID: \"d8725463-9c1f-4cee-b5ec-28fb3ca9f939\") "
Nov 04 01:28:25.270699 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:25.270670    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-istiod-ca-cert" pod "d8725463-9c1f-4cee-b5ec-28fb3ca9f939" (UID: "d8725463-9c1f-4cee-b5ec-28fb3ca9f939"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:28:25.270968 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:25.270944    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-credential-socket" pod "d8725463-9c1f-4cee-b5ec-28fb3ca9f939" (UID: "d8725463-9c1f-4cee-b5ec-28fb3ca9f939"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:25.272243 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:25.271824    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-workload-socket" pod "d8725463-9c1f-4cee-b5ec-28fb3ca9f939" (UID: "d8725463-9c1f-4cee-b5ec-28fb3ca9f939"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:25.272243 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:25.272006    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-istio-data" pod "d8725463-9c1f-4cee-b5ec-28fb3ca9f939" (UID: "d8725463-9c1f-4cee-b5ec-28fb3ca9f939"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:25.273559 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:25.273531    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-workload-certs" pod "d8725463-9c1f-4cee-b5ec-28fb3ca9f939" (UID: "d8725463-9c1f-4cee-b5ec-28fb3ca9f939"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:25.278861 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:25.278834    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-istio-podinfo" pod "d8725463-9c1f-4cee-b5ec-28fb3ca9f939" (UID: "d8725463-9c1f-4cee-b5ec-28fb3ca9f939"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:28:25.281294 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:25.281269    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-kube-api-access-g9ssf" pod "d8725463-9c1f-4cee-b5ec-28fb3ca9f939" (UID: "d8725463-9c1f-4cee-b5ec-28fb3ca9f939"). InnerVolumeSpecName "kube-api-access-g9ssf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:28:25.281400 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:25.281295    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-istio-token" pod "d8725463-9c1f-4cee-b5ec-28fb3ca9f939" (UID: "d8725463-9c1f-4cee-b5ec-28fb3ca9f939"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:28:25.282744 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:25.282713    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-istio-envoy" pod "d8725463-9c1f-4cee-b5ec-28fb3ca9f939" (UID: "d8725463-9c1f-4cee-b5ec-28fb3ca9f939"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:25.348337 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:25.348245    5068 generic.go:356] "Generic (PLEG): container finished" podID="d8725463-9c1f-4cee-b5ec-28fb3ca9f939" containerID="e23b484dd69e1e7c555dffaaf47a76ee4845e1a6536e714f6a576db96dc795c5" exitCode=0
Nov 04 01:28:25.348337 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:25.348310    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n"
Nov 04 01:28:25.348548 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:25.348352    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n" event={"ID":"d8725463-9c1f-4cee-b5ec-28fb3ca9f939","Type":"ContainerDied","Data":"e23b484dd69e1e7c555dffaaf47a76ee4845e1a6536e714f6a576db96dc795c5"}
Nov 04 01:28:25.348548 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:25.348431    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n" event={"ID":"d8725463-9c1f-4cee-b5ec-28fb3ca9f939","Type":"ContainerDied","Data":"c322f62513767f5c00d63808a84f068ed77464d630670da916bceddbc49bb003"}
Nov 04 01:28:25.348548 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:25.348476    5068 scope.go:122] "RemoveContainer" containerID="e23b484dd69e1e7c555dffaaf47a76ee4845e1a6536e714f6a576db96dc795c5"
Nov 04 01:28:25.357985 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:25.357777    5068 generic.go:356] "Generic (PLEG): container finished" podID="aae6d603-4415-436c-917f-448a09092e29" containerID="1a3e56562a94d0f7be583e9a7c03fc053544819101fa7a4d16b5180779e38379" exitCode=0
Nov 04 01:28:25.357985 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:25.357838    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64"
Nov 04 01:28:25.357985 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:25.357934    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64" event={"ID":"aae6d603-4415-436c-917f-448a09092e29","Type":"ContainerDied","Data":"1a3e56562a94d0f7be583e9a7c03fc053544819101fa7a4d16b5180779e38379"}
Nov 04 01:28:25.357985 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:25.357979    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64" event={"ID":"aae6d603-4415-436c-917f-448a09092e29","Type":"ContainerDied","Data":"f0a93a037d23befb62a29b6089dd2c194fa23f282a092e22cbdf7e2015d0cb23"}
Nov 04 01:28:25.369758 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:25.369736    5068 scope.go:122] "RemoveContainer" containerID="e23b484dd69e1e7c555dffaaf47a76ee4845e1a6536e714f6a576db96dc795c5"
Nov 04 01:28:25.370086 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:28:25.370060    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e23b484dd69e1e7c555dffaaf47a76ee4845e1a6536e714f6a576db96dc795c5\": container with ID starting with e23b484dd69e1e7c555dffaaf47a76ee4845e1a6536e714f6a576db96dc795c5 not found: ID does not exist" containerID="e23b484dd69e1e7c555dffaaf47a76ee4845e1a6536e714f6a576db96dc795c5"
Nov 04 01:28:25.370162 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:25.370101    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e23b484dd69e1e7c555dffaaf47a76ee4845e1a6536e714f6a576db96dc795c5"} err="failed to get container status \"e23b484dd69e1e7c555dffaaf47a76ee4845e1a6536e714f6a576db96dc795c5\": rpc error: code = NotFound desc = could not find container \"e23b484dd69e1e7c555dffaaf47a76ee4845e1a6536e714f6a576db96dc795c5\": container with ID starting with e23b484dd69e1e7c555dffaaf47a76ee4845e1a6536e714f6a576db96dc795c5 not found: ID does not exist"
Nov 04 01:28:25.370162 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:25.370125    5068 scope.go:122] "RemoveContainer" containerID="1a3e56562a94d0f7be583e9a7c03fc053544819101fa7a4d16b5180779e38379"
Nov 04 01:28:25.370559 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:25.370537    5068 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-istio-envoy\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:25.370694 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:25.370565    5068 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-istio-podinfo\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:25.370694 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:25.370581    5068 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-workload-certs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:25.370694 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:25.370595    5068 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-istiod-ca-cert\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:25.370837 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:25.370695    5068 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-istio-token\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:25.370837 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:25.370712    5068 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-credential-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:25.370837 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:25.370726    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g9ssf\" (UniqueName: \"kubernetes.io/projected/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-kube-api-access-g9ssf\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:25.370837 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:25.370745    5068 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-workload-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:25.370837 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:25.370771    5068 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d8725463-9c1f-4cee-b5ec-28fb3ca9f939-istio-data\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:25.381084 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:25.381035    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n"]
Nov 04 01:28:25.383552 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:25.383459    5068 scope.go:122] "RemoveContainer" containerID="1a3e56562a94d0f7be583e9a7c03fc053544819101fa7a4d16b5180779e38379"
Nov 04 01:28:25.383926 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:28:25.383904    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a3e56562a94d0f7be583e9a7c03fc053544819101fa7a4d16b5180779e38379\": container with ID starting with 1a3e56562a94d0f7be583e9a7c03fc053544819101fa7a4d16b5180779e38379 not found: ID does not exist" containerID="1a3e56562a94d0f7be583e9a7c03fc053544819101fa7a4d16b5180779e38379"
Nov 04 01:28:25.384038 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:25.383936    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a3e56562a94d0f7be583e9a7c03fc053544819101fa7a4d16b5180779e38379"} err="failed to get container status \"1a3e56562a94d0f7be583e9a7c03fc053544819101fa7a4d16b5180779e38379\": rpc error: code = NotFound desc = could not find container \"1a3e56562a94d0f7be583e9a7c03fc053544819101fa7a4d16b5180779e38379\": container with ID starting with 1a3e56562a94d0f7be583e9a7c03fc053544819101fa7a4d16b5180779e38379 not found: ID does not exist"
Nov 04 01:28:25.385300 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:25.385276    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-d4f9n"]
Nov 04 01:28:25.397383 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:25.397357    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64"]
Nov 04 01:28:25.402401 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:25.402378    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-z5r64"]
Nov 04 01:28:25.967767 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:25.967694    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc"
Nov 04 01:28:26.081775 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:26.081733    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c2697761-a360-4c3c-a5d5-c96f06998bb3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2697761-a360-4c3c-a5d5-c96f06998bb3-istio-envoy\") pod \"c2697761-a360-4c3c-a5d5-c96f06998bb3\" (UID: \"c2697761-a360-4c3c-a5d5-c96f06998bb3\") "
Nov 04 01:28:26.081978 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:26.081860    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c2697761-a360-4c3c-a5d5-c96f06998bb3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2697761-a360-4c3c-a5d5-c96f06998bb3-istio-data\") pod \"c2697761-a360-4c3c-a5d5-c96f06998bb3\" (UID: \"c2697761-a360-4c3c-a5d5-c96f06998bb3\") "
Nov 04 01:28:26.081978 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:26.081888    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c2697761-a360-4c3c-a5d5-c96f06998bb3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2697761-a360-4c3c-a5d5-c96f06998bb3-istio-podinfo\") pod \"c2697761-a360-4c3c-a5d5-c96f06998bb3\" (UID: \"c2697761-a360-4c3c-a5d5-c96f06998bb3\") "
Nov 04 01:28:26.081978 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:26.081963    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c2697761-a360-4c3c-a5d5-c96f06998bb3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2697761-a360-4c3c-a5d5-c96f06998bb3-workload-socket\") pod \"c2697761-a360-4c3c-a5d5-c96f06998bb3\" (UID: \"c2697761-a360-4c3c-a5d5-c96f06998bb3\") "
Nov 04 01:28:26.082155 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:26.081993    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c2697761-a360-4c3c-a5d5-c96f06998bb3-istio-token\" (UniqueName: \"kubernetes.io/projected/c2697761-a360-4c3c-a5d5-c96f06998bb3-istio-token\") pod \"c2697761-a360-4c3c-a5d5-c96f06998bb3\" (UID: \"c2697761-a360-4c3c-a5d5-c96f06998bb3\") "
Nov 04 01:28:26.082155 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:26.082023    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c2697761-a360-4c3c-a5d5-c96f06998bb3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2697761-a360-4c3c-a5d5-c96f06998bb3-credential-socket\") pod \"c2697761-a360-4c3c-a5d5-c96f06998bb3\" (UID: \"c2697761-a360-4c3c-a5d5-c96f06998bb3\") "
Nov 04 01:28:26.082155 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:26.082061    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c2697761-a360-4c3c-a5d5-c96f06998bb3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2697761-a360-4c3c-a5d5-c96f06998bb3-istiod-ca-cert\") pod \"c2697761-a360-4c3c-a5d5-c96f06998bb3\" (UID: \"c2697761-a360-4c3c-a5d5-c96f06998bb3\") "
Nov 04 01:28:26.082316 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:26.082169    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c2697761-a360-4c3c-a5d5-c96f06998bb3-kube-api-access-gdbjp\" (UniqueName: \"kubernetes.io/projected/c2697761-a360-4c3c-a5d5-c96f06998bb3-kube-api-access-gdbjp\") pod \"c2697761-a360-4c3c-a5d5-c96f06998bb3\" (UID: \"c2697761-a360-4c3c-a5d5-c96f06998bb3\") "
Nov 04 01:28:26.082316 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:26.082169    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2697761-a360-4c3c-a5d5-c96f06998bb3-istio-data" pod "c2697761-a360-4c3c-a5d5-c96f06998bb3" (UID: "c2697761-a360-4c3c-a5d5-c96f06998bb3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:26.082316 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:26.082207    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c2697761-a360-4c3c-a5d5-c96f06998bb3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2697761-a360-4c3c-a5d5-c96f06998bb3-workload-certs\") pod \"c2697761-a360-4c3c-a5d5-c96f06998bb3\" (UID: \"c2697761-a360-4c3c-a5d5-c96f06998bb3\") "
Nov 04 01:28:26.082472 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:26.082409    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2697761-a360-4c3c-a5d5-c96f06998bb3-workload-socket" pod "c2697761-a360-4c3c-a5d5-c96f06998bb3" (UID: "c2697761-a360-4c3c-a5d5-c96f06998bb3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:26.082897 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:26.082716    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2697761-a360-4c3c-a5d5-c96f06998bb3-workload-certs" pod "c2697761-a360-4c3c-a5d5-c96f06998bb3" (UID: "c2697761-a360-4c3c-a5d5-c96f06998bb3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:26.082897 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:26.082867    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2697761-a360-4c3c-a5d5-c96f06998bb3-credential-socket" pod "c2697761-a360-4c3c-a5d5-c96f06998bb3" (UID: "c2697761-a360-4c3c-a5d5-c96f06998bb3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:26.083082 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:26.082942    5068 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2697761-a360-4c3c-a5d5-c96f06998bb3-istio-data\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:26.083082 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:26.082960    5068 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2697761-a360-4c3c-a5d5-c96f06998bb3-workload-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:26.083082 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:26.082974    5068 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2697761-a360-4c3c-a5d5-c96f06998bb3-workload-certs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:26.083082 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:26.082988    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2697761-a360-4c3c-a5d5-c96f06998bb3-istiod-ca-cert" pod "c2697761-a360-4c3c-a5d5-c96f06998bb3" (UID: "c2697761-a360-4c3c-a5d5-c96f06998bb3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:28:26.084330 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:26.084300    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2697761-a360-4c3c-a5d5-c96f06998bb3-istio-envoy" pod "c2697761-a360-4c3c-a5d5-c96f06998bb3" (UID: "c2697761-a360-4c3c-a5d5-c96f06998bb3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:26.084451 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:26.084430    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2697761-a360-4c3c-a5d5-c96f06998bb3-istio-token" pod "c2697761-a360-4c3c-a5d5-c96f06998bb3" (UID: "c2697761-a360-4c3c-a5d5-c96f06998bb3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:28:26.085088 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:26.085057    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c2697761-a360-4c3c-a5d5-c96f06998bb3-istio-podinfo" pod "c2697761-a360-4c3c-a5d5-c96f06998bb3" (UID: "c2697761-a360-4c3c-a5d5-c96f06998bb3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:28:26.085204 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:26.085144    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2697761-a360-4c3c-a5d5-c96f06998bb3-kube-api-access-gdbjp" pod "c2697761-a360-4c3c-a5d5-c96f06998bb3" (UID: "c2697761-a360-4c3c-a5d5-c96f06998bb3"). InnerVolumeSpecName "kube-api-access-gdbjp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:28:26.113475 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:26.113446    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aae6d603-4415-436c-917f-448a09092e29" path="/var/lib/kubelet/pods/aae6d603-4415-436c-917f-448a09092e29/volumes"
Nov 04 01:28:26.113973 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:26.113958    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d8725463-9c1f-4cee-b5ec-28fb3ca9f939" path="/var/lib/kubelet/pods/d8725463-9c1f-4cee-b5ec-28fb3ca9f939/volumes"
Nov 04 01:28:26.185003 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:26.184784    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gdbjp\" (UniqueName: \"kubernetes.io/projected/c2697761-a360-4c3c-a5d5-c96f06998bb3-kube-api-access-gdbjp\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:26.185003 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:26.184819    5068 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2697761-a360-4c3c-a5d5-c96f06998bb3-istio-envoy\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:26.185003 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:26.184833    5068 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2697761-a360-4c3c-a5d5-c96f06998bb3-istio-podinfo\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:26.185003 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:26.184847    5068 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2697761-a360-4c3c-a5d5-c96f06998bb3-istio-token\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:26.185003 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:26.184862    5068 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2697761-a360-4c3c-a5d5-c96f06998bb3-credential-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:26.185003 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:26.184876    5068 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2697761-a360-4c3c-a5d5-c96f06998bb3-istiod-ca-cert\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:26.370443 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:26.370405    5068 generic.go:356] "Generic (PLEG): container finished" podID="c2697761-a360-4c3c-a5d5-c96f06998bb3" containerID="33372768c1e40b5058e82519f32767016312d6ddf1759497cd14862b7611ec20" exitCode=0
Nov 04 01:28:26.370703 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:26.370466    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc"
Nov 04 01:28:26.370703 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:26.370547    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc" event={"ID":"c2697761-a360-4c3c-a5d5-c96f06998bb3","Type":"ContainerDied","Data":"33372768c1e40b5058e82519f32767016312d6ddf1759497cd14862b7611ec20"}
Nov 04 01:28:26.370703 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:26.370598    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc" event={"ID":"c2697761-a360-4c3c-a5d5-c96f06998bb3","Type":"ContainerDied","Data":"dc60c08b7e72fa4cca15658416a983a302b6deabf782c3570df3d08b5b364815"}
Nov 04 01:28:26.370703 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:26.370670    5068 scope.go:122] "RemoveContainer" containerID="33372768c1e40b5058e82519f32767016312d6ddf1759497cd14862b7611ec20"
Nov 04 01:28:26.455658 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:26.439200    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc"]
Nov 04 01:28:26.483539 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:26.483187    5068 scope.go:122] "RemoveContainer" containerID="33372768c1e40b5058e82519f32767016312d6ddf1759497cd14862b7611ec20"
Nov 04 01:28:26.483980 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:28:26.483872    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33372768c1e40b5058e82519f32767016312d6ddf1759497cd14862b7611ec20\": container with ID starting with 33372768c1e40b5058e82519f32767016312d6ddf1759497cd14862b7611ec20 not found: ID does not exist" containerID="33372768c1e40b5058e82519f32767016312d6ddf1759497cd14862b7611ec20"
Nov 04 01:28:26.483980 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:26.483912    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33372768c1e40b5058e82519f32767016312d6ddf1759497cd14862b7611ec20"} err="failed to get container status \"33372768c1e40b5058e82519f32767016312d6ddf1759497cd14862b7611ec20\": rpc error: code = NotFound desc = could not find container \"33372768c1e40b5058e82519f32767016312d6ddf1759497cd14862b7611ec20\": container with ID starting with 33372768c1e40b5058e82519f32767016312d6ddf1759497cd14862b7611ec20 not found: ID does not exist"
Nov 04 01:28:26.525097 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:26.522163    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-875528-openshift-default-57bd759bd9-n2pwc"]
Nov 04 01:28:28.114457 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:28.114421    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c2697761-a360-4c3c-a5d5-c96f06998bb3" path="/var/lib/kubelet/pods/c2697761-a360-4c3c-a5d5-c96f06998bb3/volumes"
Nov 04 01:28:28.180335 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:28.180301    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:28:30.633333 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:30.633291    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82" podUID="6cdd01de-ff61-494a-bdb3-c67025a49440" containerName="main" containerID="cri-o://58804afceec88781291367b70788b6abd8d2a0ab7cfe27e8f62decaf5086d7df" gracePeriod=45
Nov 04 01:28:30.640992 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:30.640960    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm" podUID="f9e7abd2-8737-4357-a778-d6492fd12f3f" containerName="main" containerID="cri-o://e9df8cb91833ffe22046558b5ecd859e003ce091e8e0d4f84c4970cf2091e993" gracePeriod=45
Nov 04 01:28:30.680148 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:30.680113    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz" podUID="4ccafa61-1c9b-4595-a511-3404d7479526" containerName="main" containerID="cri-o://a2ff4fb0c4fcceff17ddbfb82eeb035a68edab36cb506afd2a4836dffef76c75" gracePeriod=45
Nov 04 01:28:30.694407 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:30.694360    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9" podUID="530fc578-6622-4c42-a384-02e73bf17b1f" containerName="main" containerID="cri-o://6260d6e9440205c0e82ed5b4f6239f47fb7f94a97253f8f2f22703596d1129cd" gracePeriod=45
Nov 04 01:28:30.789636 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:30.787017    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9" podUID="530fc578-6622-4c42-a384-02e73bf17b1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": dial tcp 10.132.0.60:8000: connect: connection refused"
Nov 04 01:28:30.886354 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:30.886018    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz" podUID="4ccafa61-1c9b-4595-a511-3404d7479526" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": dial tcp 10.132.0.63:8000: connect: connection refused"
Nov 04 01:28:30.896550 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:30.896512    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-ba844c-7bc4dfcfd-nwk5b"]
Nov 04 01:28:31.072706 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.072668    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82"
Nov 04 01:28:31.103470 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.103437    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm"
Nov 04 01:28:31.114547 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.114448    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9"
Nov 04 01:28:31.122511 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.122483    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz"
Nov 04 01:28:31.203619 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.203506    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f9e7abd2-8737-4357-a778-d6492fd12f3f-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9e7abd2-8737-4357-a778-d6492fd12f3f-model-cache\") pod \"f9e7abd2-8737-4357-a778-d6492fd12f3f\" (UID: \"f9e7abd2-8737-4357-a778-d6492fd12f3f\") "
Nov 04 01:28:31.203797 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.203646    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f9e7abd2-8737-4357-a778-d6492fd12f3f-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9e7abd2-8737-4357-a778-d6492fd12f3f-tmp-dir\") pod \"f9e7abd2-8737-4357-a778-d6492fd12f3f\" (UID: \"f9e7abd2-8737-4357-a778-d6492fd12f3f\") "
Nov 04 01:28:31.203797 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.203680    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6cdd01de-ff61-494a-bdb3-c67025a49440-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6cdd01de-ff61-494a-bdb3-c67025a49440-tmp-dir\") pod \"6cdd01de-ff61-494a-bdb3-c67025a49440\" (UID: \"6cdd01de-ff61-494a-bdb3-c67025a49440\") "
Nov 04 01:28:31.203797 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.203731    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/530fc578-6622-4c42-a384-02e73bf17b1f-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/530fc578-6622-4c42-a384-02e73bf17b1f-tmp-dir\") pod \"530fc578-6622-4c42-a384-02e73bf17b1f\" (UID: \"530fc578-6622-4c42-a384-02e73bf17b1f\") "
Nov 04 01:28:31.203797 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.203762    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f9e7abd2-8737-4357-a778-d6492fd12f3f-kube-api-access-qd6p9\" (UniqueName: \"kubernetes.io/projected/f9e7abd2-8737-4357-a778-d6492fd12f3f-kube-api-access-qd6p9\") pod \"f9e7abd2-8737-4357-a778-d6492fd12f3f\" (UID: \"f9e7abd2-8737-4357-a778-d6492fd12f3f\") "
Nov 04 01:28:31.204043 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.203797    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/530fc578-6622-4c42-a384-02e73bf17b1f-tls-certs\" (UniqueName: \"kubernetes.io/secret/530fc578-6622-4c42-a384-02e73bf17b1f-tls-certs\") pod \"530fc578-6622-4c42-a384-02e73bf17b1f\" (UID: \"530fc578-6622-4c42-a384-02e73bf17b1f\") "
Nov 04 01:28:31.204043 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.203994    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6cdd01de-ff61-494a-bdb3-c67025a49440-home\" (UniqueName: \"kubernetes.io/empty-dir/6cdd01de-ff61-494a-bdb3-c67025a49440-home\") pod \"6cdd01de-ff61-494a-bdb3-c67025a49440\" (UID: \"6cdd01de-ff61-494a-bdb3-c67025a49440\") "
Nov 04 01:28:31.204043 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.204028    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/530fc578-6622-4c42-a384-02e73bf17b1f-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/530fc578-6622-4c42-a384-02e73bf17b1f-model-cache\") pod \"530fc578-6622-4c42-a384-02e73bf17b1f\" (UID: \"530fc578-6622-4c42-a384-02e73bf17b1f\") "
Nov 04 01:28:31.204200 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.204061    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4ccafa61-1c9b-4595-a511-3404d7479526-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ccafa61-1c9b-4595-a511-3404d7479526-model-cache\") pod \"4ccafa61-1c9b-4595-a511-3404d7479526\" (UID: \"4ccafa61-1c9b-4595-a511-3404d7479526\") "
Nov 04 01:28:31.204200 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.204092    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6cdd01de-ff61-494a-bdb3-c67025a49440-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6cdd01de-ff61-494a-bdb3-c67025a49440-model-cache\") pod \"6cdd01de-ff61-494a-bdb3-c67025a49440\" (UID: \"6cdd01de-ff61-494a-bdb3-c67025a49440\") "
Nov 04 01:28:31.204200 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.204189    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4ccafa61-1c9b-4595-a511-3404d7479526-home\" (UniqueName: \"kubernetes.io/empty-dir/4ccafa61-1c9b-4595-a511-3404d7479526-home\") pod \"4ccafa61-1c9b-4595-a511-3404d7479526\" (UID: \"4ccafa61-1c9b-4595-a511-3404d7479526\") "
Nov 04 01:28:31.205035 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.204238    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4ccafa61-1c9b-4595-a511-3404d7479526-kube-api-access-62ghr\" (UniqueName: \"kubernetes.io/projected/4ccafa61-1c9b-4595-a511-3404d7479526-kube-api-access-62ghr\") pod \"4ccafa61-1c9b-4595-a511-3404d7479526\" (UID: \"4ccafa61-1c9b-4595-a511-3404d7479526\") "
Nov 04 01:28:31.205035 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.204319    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6cdd01de-ff61-494a-bdb3-c67025a49440-kube-api-access-lxrl7\" (UniqueName: \"kubernetes.io/projected/6cdd01de-ff61-494a-bdb3-c67025a49440-kube-api-access-lxrl7\") pod \"6cdd01de-ff61-494a-bdb3-c67025a49440\" (UID: \"6cdd01de-ff61-494a-bdb3-c67025a49440\") "
Nov 04 01:28:31.205035 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.204358    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4ccafa61-1c9b-4595-a511-3404d7479526-dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ccafa61-1c9b-4595-a511-3404d7479526-dshm\") pod \"4ccafa61-1c9b-4595-a511-3404d7479526\" (UID: \"4ccafa61-1c9b-4595-a511-3404d7479526\") "
Nov 04 01:28:31.205035 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.204403    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/530fc578-6622-4c42-a384-02e73bf17b1f-home\" (UniqueName: \"kubernetes.io/empty-dir/530fc578-6622-4c42-a384-02e73bf17b1f-home\") pod \"530fc578-6622-4c42-a384-02e73bf17b1f\" (UID: \"530fc578-6622-4c42-a384-02e73bf17b1f\") "
Nov 04 01:28:31.205035 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.204417    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9e7abd2-8737-4357-a778-d6492fd12f3f-model-cache" pod "f9e7abd2-8737-4357-a778-d6492fd12f3f" (UID: "f9e7abd2-8737-4357-a778-d6492fd12f3f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:31.205035 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.204479    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6cdd01de-ff61-494a-bdb3-c67025a49440-tls-certs\" (UniqueName: \"kubernetes.io/secret/6cdd01de-ff61-494a-bdb3-c67025a49440-tls-certs\") pod \"6cdd01de-ff61-494a-bdb3-c67025a49440\" (UID: \"6cdd01de-ff61-494a-bdb3-c67025a49440\") "
Nov 04 01:28:31.205035 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.204510    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4ccafa61-1c9b-4595-a511-3404d7479526-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ccafa61-1c9b-4595-a511-3404d7479526-tmp-dir\") pod \"4ccafa61-1c9b-4595-a511-3404d7479526\" (UID: \"4ccafa61-1c9b-4595-a511-3404d7479526\") "
Nov 04 01:28:31.205035 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.204539    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f9e7abd2-8737-4357-a778-d6492fd12f3f-dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9e7abd2-8737-4357-a778-d6492fd12f3f-dshm\") pod \"f9e7abd2-8737-4357-a778-d6492fd12f3f\" (UID: \"f9e7abd2-8737-4357-a778-d6492fd12f3f\") "
Nov 04 01:28:31.205035 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.204573    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f9e7abd2-8737-4357-a778-d6492fd12f3f-tls-certs\" (UniqueName: \"kubernetes.io/secret/f9e7abd2-8737-4357-a778-d6492fd12f3f-tls-certs\") pod \"f9e7abd2-8737-4357-a778-d6492fd12f3f\" (UID: \"f9e7abd2-8737-4357-a778-d6492fd12f3f\") "
Nov 04 01:28:31.205035 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.204626    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6cdd01de-ff61-494a-bdb3-c67025a49440-dshm\" (UniqueName: \"kubernetes.io/empty-dir/6cdd01de-ff61-494a-bdb3-c67025a49440-dshm\") pod \"6cdd01de-ff61-494a-bdb3-c67025a49440\" (UID: \"6cdd01de-ff61-494a-bdb3-c67025a49440\") "
Nov 04 01:28:31.205035 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.204680    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/530fc578-6622-4c42-a384-02e73bf17b1f-model-cache" pod "530fc578-6622-4c42-a384-02e73bf17b1f" (UID: "530fc578-6622-4c42-a384-02e73bf17b1f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:31.205035 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.204690    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f9e7abd2-8737-4357-a778-d6492fd12f3f-home\" (UniqueName: \"kubernetes.io/empty-dir/f9e7abd2-8737-4357-a778-d6492fd12f3f-home\") pod \"f9e7abd2-8737-4357-a778-d6492fd12f3f\" (UID: \"f9e7abd2-8737-4357-a778-d6492fd12f3f\") "
Nov 04 01:28:31.205035 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.204738    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/530fc578-6622-4c42-a384-02e73bf17b1f-dshm\" (UniqueName: \"kubernetes.io/empty-dir/530fc578-6622-4c42-a384-02e73bf17b1f-dshm\") pod \"530fc578-6622-4c42-a384-02e73bf17b1f\" (UID: \"530fc578-6622-4c42-a384-02e73bf17b1f\") "
Nov 04 01:28:31.205035 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.204845    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4ccafa61-1c9b-4595-a511-3404d7479526-tls-certs\" (UniqueName: \"kubernetes.io/secret/4ccafa61-1c9b-4595-a511-3404d7479526-tls-certs\") pod \"4ccafa61-1c9b-4595-a511-3404d7479526\" (UID: \"4ccafa61-1c9b-4595-a511-3404d7479526\") "
Nov 04 01:28:31.205035 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.204889    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/530fc578-6622-4c42-a384-02e73bf17b1f-kube-api-access-xj55g\" (UniqueName: \"kubernetes.io/projected/530fc578-6622-4c42-a384-02e73bf17b1f-kube-api-access-xj55g\") pod \"530fc578-6622-4c42-a384-02e73bf17b1f\" (UID: \"530fc578-6622-4c42-a384-02e73bf17b1f\") "
Nov 04 01:28:31.205836 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.205753    5068 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9e7abd2-8737-4357-a778-d6492fd12f3f-model-cache\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:31.205836 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.205775    5068 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/530fc578-6622-4c42-a384-02e73bf17b1f-model-cache\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:31.206487 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.204845    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9e7abd2-8737-4357-a778-d6492fd12f3f-tmp-dir" pod "f9e7abd2-8737-4357-a778-d6492fd12f3f" (UID: "f9e7abd2-8737-4357-a778-d6492fd12f3f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:31.206487 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.205005    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cdd01de-ff61-494a-bdb3-c67025a49440-tmp-dir" pod "6cdd01de-ff61-494a-bdb3-c67025a49440" (UID: "6cdd01de-ff61-494a-bdb3-c67025a49440"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:31.206487 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.205166    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/530fc578-6622-4c42-a384-02e73bf17b1f-tmp-dir" pod "530fc578-6622-4c42-a384-02e73bf17b1f" (UID: "530fc578-6622-4c42-a384-02e73bf17b1f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:31.206796 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.206732    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cdd01de-ff61-494a-bdb3-c67025a49440-home" pod "6cdd01de-ff61-494a-bdb3-c67025a49440" (UID: "6cdd01de-ff61-494a-bdb3-c67025a49440"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:31.207358 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.207337    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/530fc578-6622-4c42-a384-02e73bf17b1f-home" pod "530fc578-6622-4c42-a384-02e73bf17b1f" (UID: "530fc578-6622-4c42-a384-02e73bf17b1f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:31.208110 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.208060    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ccafa61-1c9b-4595-a511-3404d7479526-home" pod "4ccafa61-1c9b-4595-a511-3404d7479526" (UID: "4ccafa61-1c9b-4595-a511-3404d7479526"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:31.208286 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.208257    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cdd01de-ff61-494a-bdb3-c67025a49440-model-cache" pod "6cdd01de-ff61-494a-bdb3-c67025a49440" (UID: "6cdd01de-ff61-494a-bdb3-c67025a49440"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:31.208695 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.208417    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ccafa61-1c9b-4595-a511-3404d7479526-model-cache" pod "4ccafa61-1c9b-4595-a511-3404d7479526" (UID: "4ccafa61-1c9b-4595-a511-3404d7479526"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:31.212101 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.211997    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ccafa61-1c9b-4595-a511-3404d7479526-kube-api-access-62ghr" pod "4ccafa61-1c9b-4595-a511-3404d7479526" (UID: "4ccafa61-1c9b-4595-a511-3404d7479526"). InnerVolumeSpecName "kube-api-access-62ghr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:28:31.213280 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.213037    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6cdd01de-ff61-494a-bdb3-c67025a49440-tls-certs" pod "6cdd01de-ff61-494a-bdb3-c67025a49440" (UID: "6cdd01de-ff61-494a-bdb3-c67025a49440"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:28:31.213713 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.213421    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9e7abd2-8737-4357-a778-d6492fd12f3f-home" pod "f9e7abd2-8737-4357-a778-d6492fd12f3f" (UID: "f9e7abd2-8737-4357-a778-d6492fd12f3f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:31.213713 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.213542    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9e7abd2-8737-4357-a778-d6492fd12f3f-tls-certs" pod "f9e7abd2-8737-4357-a778-d6492fd12f3f" (UID: "f9e7abd2-8737-4357-a778-d6492fd12f3f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:28:31.213713 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.213602    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/530fc578-6622-4c42-a384-02e73bf17b1f-dshm" pod "530fc578-6622-4c42-a384-02e73bf17b1f" (UID: "530fc578-6622-4c42-a384-02e73bf17b1f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:31.214017 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.213820    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ccafa61-1c9b-4595-a511-3404d7479526-tmp-dir" pod "4ccafa61-1c9b-4595-a511-3404d7479526" (UID: "4ccafa61-1c9b-4595-a511-3404d7479526"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:31.217471 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.217288    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9e7abd2-8737-4357-a778-d6492fd12f3f-kube-api-access-qd6p9" pod "f9e7abd2-8737-4357-a778-d6492fd12f3f" (UID: "f9e7abd2-8737-4357-a778-d6492fd12f3f"). InnerVolumeSpecName "kube-api-access-qd6p9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:28:31.218107 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.218075    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ccafa61-1c9b-4595-a511-3404d7479526-tls-certs" pod "4ccafa61-1c9b-4595-a511-3404d7479526" (UID: "4ccafa61-1c9b-4595-a511-3404d7479526"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:28:31.218457 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.218333    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/530fc578-6622-4c42-a384-02e73bf17b1f-kube-api-access-xj55g" pod "530fc578-6622-4c42-a384-02e73bf17b1f" (UID: "530fc578-6622-4c42-a384-02e73bf17b1f"). InnerVolumeSpecName "kube-api-access-xj55g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:28:31.218457 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.218431    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9e7abd2-8737-4357-a778-d6492fd12f3f-dshm" pod "f9e7abd2-8737-4357-a778-d6492fd12f3f" (UID: "f9e7abd2-8737-4357-a778-d6492fd12f3f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:31.218735 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.218710    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6cdd01de-ff61-494a-bdb3-c67025a49440-kube-api-access-lxrl7" pod "6cdd01de-ff61-494a-bdb3-c67025a49440" (UID: "6cdd01de-ff61-494a-bdb3-c67025a49440"). InnerVolumeSpecName "kube-api-access-lxrl7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:28:31.218952 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.218897    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/530fc578-6622-4c42-a384-02e73bf17b1f-tls-certs" pod "530fc578-6622-4c42-a384-02e73bf17b1f" (UID: "530fc578-6622-4c42-a384-02e73bf17b1f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:28:31.220253 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.220222    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ccafa61-1c9b-4595-a511-3404d7479526-dshm" pod "4ccafa61-1c9b-4595-a511-3404d7479526" (UID: "4ccafa61-1c9b-4595-a511-3404d7479526"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:31.220458 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.220431    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cdd01de-ff61-494a-bdb3-c67025a49440-dshm" pod "6cdd01de-ff61-494a-bdb3-c67025a49440" (UID: "6cdd01de-ff61-494a-bdb3-c67025a49440"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:31.341588 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.337164    5068 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/530fc578-6622-4c42-a384-02e73bf17b1f-tls-certs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:31.341588 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.337199    5068 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6cdd01de-ff61-494a-bdb3-c67025a49440-home\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:31.341588 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.337214    5068 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ccafa61-1c9b-4595-a511-3404d7479526-model-cache\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:31.341588 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.337228    5068 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6cdd01de-ff61-494a-bdb3-c67025a49440-model-cache\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:31.341588 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.337242    5068 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ccafa61-1c9b-4595-a511-3404d7479526-home\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:31.341588 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.337259    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-62ghr\" (UniqueName: \"kubernetes.io/projected/4ccafa61-1c9b-4595-a511-3404d7479526-kube-api-access-62ghr\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:31.341588 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.337273    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lxrl7\" (UniqueName: \"kubernetes.io/projected/6cdd01de-ff61-494a-bdb3-c67025a49440-kube-api-access-lxrl7\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:31.341588 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.337286    5068 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ccafa61-1c9b-4595-a511-3404d7479526-dshm\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:31.341588 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.337299    5068 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/530fc578-6622-4c42-a384-02e73bf17b1f-home\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:31.341588 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.337311    5068 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6cdd01de-ff61-494a-bdb3-c67025a49440-tls-certs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:31.341588 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.337323    5068 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ccafa61-1c9b-4595-a511-3404d7479526-tmp-dir\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:31.341588 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.337335    5068 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9e7abd2-8737-4357-a778-d6492fd12f3f-dshm\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:31.341588 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.337348    5068 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9e7abd2-8737-4357-a778-d6492fd12f3f-tls-certs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:31.341588 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.337360    5068 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6cdd01de-ff61-494a-bdb3-c67025a49440-dshm\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:31.341588 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.337374    5068 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9e7abd2-8737-4357-a778-d6492fd12f3f-home\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:31.341588 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.337386    5068 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/530fc578-6622-4c42-a384-02e73bf17b1f-dshm\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:31.341588 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.337398    5068 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ccafa61-1c9b-4595-a511-3404d7479526-tls-certs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:31.341588 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.337410    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xj55g\" (UniqueName: \"kubernetes.io/projected/530fc578-6622-4c42-a384-02e73bf17b1f-kube-api-access-xj55g\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:31.341588 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.337424    5068 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9e7abd2-8737-4357-a778-d6492fd12f3f-tmp-dir\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:31.341588 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.337437    5068 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6cdd01de-ff61-494a-bdb3-c67025a49440-tmp-dir\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:31.341588 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.337451    5068 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/530fc578-6622-4c42-a384-02e73bf17b1f-tmp-dir\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:31.341588 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.337465    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qd6p9\" (UniqueName: \"kubernetes.io/projected/f9e7abd2-8737-4357-a778-d6492fd12f3f-kube-api-access-qd6p9\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:31.450999 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.445179    5068 generic.go:356] "Generic (PLEG): container finished" podID="4ccafa61-1c9b-4595-a511-3404d7479526" containerID="a2ff4fb0c4fcceff17ddbfb82eeb035a68edab36cb506afd2a4836dffef76c75" exitCode=0
Nov 04 01:28:31.450999 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.445302    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz" event={"ID":"4ccafa61-1c9b-4595-a511-3404d7479526","Type":"ContainerDied","Data":"a2ff4fb0c4fcceff17ddbfb82eeb035a68edab36cb506afd2a4836dffef76c75"}
Nov 04 01:28:31.450999 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.445339    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz" event={"ID":"4ccafa61-1c9b-4595-a511-3404d7479526","Type":"ContainerDied","Data":"040690e7e691376e95f02cd647194b58ae610f2b4bb5a308a6939e3375feed0d"}
Nov 04 01:28:31.450999 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.445359    5068 scope.go:122] "RemoveContainer" containerID="a2ff4fb0c4fcceff17ddbfb82eeb035a68edab36cb506afd2a4836dffef76c75"
Nov 04 01:28:31.450999 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.445494    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz"
Nov 04 01:28:31.465877 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.462103    5068 generic.go:356] "Generic (PLEG): container finished" podID="6cdd01de-ff61-494a-bdb3-c67025a49440" containerID="58804afceec88781291367b70788b6abd8d2a0ab7cfe27e8f62decaf5086d7df" exitCode=0
Nov 04 01:28:31.465877 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.462356    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82"
Nov 04 01:28:31.466729 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.466378    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82" event={"ID":"6cdd01de-ff61-494a-bdb3-c67025a49440","Type":"ContainerDied","Data":"58804afceec88781291367b70788b6abd8d2a0ab7cfe27e8f62decaf5086d7df"}
Nov 04 01:28:31.466729 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.466427    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82" event={"ID":"6cdd01de-ff61-494a-bdb3-c67025a49440","Type":"ContainerDied","Data":"f2e472238d1374a293eceb5065faea219edb94c5f6e30e8777da87018ecdd7ec"}
Nov 04 01:28:31.477053 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.476560    5068 generic.go:356] "Generic (PLEG): container finished" podID="530fc578-6622-4c42-a384-02e73bf17b1f" containerID="6260d6e9440205c0e82ed5b4f6239f47fb7f94a97253f8f2f22703596d1129cd" exitCode=0
Nov 04 01:28:31.477053 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.476729    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9"
Nov 04 01:28:31.477053 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.476867    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9" event={"ID":"530fc578-6622-4c42-a384-02e73bf17b1f","Type":"ContainerDied","Data":"6260d6e9440205c0e82ed5b4f6239f47fb7f94a97253f8f2f22703596d1129cd"}
Nov 04 01:28:31.477053 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.476915    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9" event={"ID":"530fc578-6622-4c42-a384-02e73bf17b1f","Type":"ContainerDied","Data":"eb559794dfad2b60135913f8a294b65938ec4c631df795e1cbbf7f67da4a244c"}
Nov 04 01:28:31.481663 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.480470    5068 generic.go:356] "Generic (PLEG): container finished" podID="f9e7abd2-8737-4357-a778-d6492fd12f3f" containerID="e9df8cb91833ffe22046558b5ecd859e003ce091e8e0d4f84c4970cf2091e993" exitCode=0
Nov 04 01:28:31.481663 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.480540    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm"
Nov 04 01:28:31.481663 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.480544    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm" event={"ID":"f9e7abd2-8737-4357-a778-d6492fd12f3f","Type":"ContainerDied","Data":"e9df8cb91833ffe22046558b5ecd859e003ce091e8e0d4f84c4970cf2091e993"}
Nov 04 01:28:31.481663 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.480571    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm" event={"ID":"f9e7abd2-8737-4357-a778-d6492fd12f3f","Type":"ContainerDied","Data":"d0c3bca9bd0037a535d862cf3791e5d369a2327ed552664ec679423a20063a74"}
Nov 04 01:28:31.490475 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.490366    5068 scope.go:122] "RemoveContainer" containerID="a2ff4fb0c4fcceff17ddbfb82eeb035a68edab36cb506afd2a4836dffef76c75"
Nov 04 01:28:31.493735 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:28:31.492783    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2ff4fb0c4fcceff17ddbfb82eeb035a68edab36cb506afd2a4836dffef76c75\": container with ID starting with a2ff4fb0c4fcceff17ddbfb82eeb035a68edab36cb506afd2a4836dffef76c75 not found: ID does not exist" containerID="a2ff4fb0c4fcceff17ddbfb82eeb035a68edab36cb506afd2a4836dffef76c75"
Nov 04 01:28:31.493735 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.492821    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2ff4fb0c4fcceff17ddbfb82eeb035a68edab36cb506afd2a4836dffef76c75"} err="failed to get container status \"a2ff4fb0c4fcceff17ddbfb82eeb035a68edab36cb506afd2a4836dffef76c75\": rpc error: code = NotFound desc = could not find container \"a2ff4fb0c4fcceff17ddbfb82eeb035a68edab36cb506afd2a4836dffef76c75\": container with ID starting with a2ff4fb0c4fcceff17ddbfb82eeb035a68edab36cb506afd2a4836dffef76c75 not found: ID does not exist"
Nov 04 01:28:31.493735 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.492992    5068 scope.go:122] "RemoveContainer" containerID="58804afceec88781291367b70788b6abd8d2a0ab7cfe27e8f62decaf5086d7df"
Nov 04 01:28:31.497971 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.497947    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz"]
Nov 04 01:28:31.504743 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.504721    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-ba844c/premium-w4-ba844c-kserve-79b7f75f5d-r6hfz"]
Nov 04 01:28:31.517933 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.517905    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9"]
Nov 04 01:28:31.518707 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.518690    5068 scope.go:122] "RemoveContainer" containerID="58804afceec88781291367b70788b6abd8d2a0ab7cfe27e8f62decaf5086d7df"
Nov 04 01:28:31.519038 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:28:31.519013    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58804afceec88781291367b70788b6abd8d2a0ab7cfe27e8f62decaf5086d7df\": container with ID starting with 58804afceec88781291367b70788b6abd8d2a0ab7cfe27e8f62decaf5086d7df not found: ID does not exist" containerID="58804afceec88781291367b70788b6abd8d2a0ab7cfe27e8f62decaf5086d7df"
Nov 04 01:28:31.519138 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.519047    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58804afceec88781291367b70788b6abd8d2a0ab7cfe27e8f62decaf5086d7df"} err="failed to get container status \"58804afceec88781291367b70788b6abd8d2a0ab7cfe27e8f62decaf5086d7df\": rpc error: code = NotFound desc = could not find container \"58804afceec88781291367b70788b6abd8d2a0ab7cfe27e8f62decaf5086d7df\": container with ID starting with 58804afceec88781291367b70788b6abd8d2a0ab7cfe27e8f62decaf5086d7df not found: ID does not exist"
Nov 04 01:28:31.519138 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.519070    5068 scope.go:122] "RemoveContainer" containerID="6260d6e9440205c0e82ed5b4f6239f47fb7f94a97253f8f2f22703596d1129cd"
Nov 04 01:28:31.520851 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.520830    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-ba844c/simulator-w4-ba844c-kserve-5f54cdc8c9-lkhp9"]
Nov 04 01:28:31.528314 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.528298    5068 scope.go:122] "RemoveContainer" containerID="6260d6e9440205c0e82ed5b4f6239f47fb7f94a97253f8f2f22703596d1129cd"
Nov 04 01:28:31.528581 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:28:31.528562    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6260d6e9440205c0e82ed5b4f6239f47fb7f94a97253f8f2f22703596d1129cd\": container with ID starting with 6260d6e9440205c0e82ed5b4f6239f47fb7f94a97253f8f2f22703596d1129cd not found: ID does not exist" containerID="6260d6e9440205c0e82ed5b4f6239f47fb7f94a97253f8f2f22703596d1129cd"
Nov 04 01:28:31.528665 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.528585    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6260d6e9440205c0e82ed5b4f6239f47fb7f94a97253f8f2f22703596d1129cd"} err="failed to get container status \"6260d6e9440205c0e82ed5b4f6239f47fb7f94a97253f8f2f22703596d1129cd\": rpc error: code = NotFound desc = could not find container \"6260d6e9440205c0e82ed5b4f6239f47fb7f94a97253f8f2f22703596d1129cd\": container with ID starting with 6260d6e9440205c0e82ed5b4f6239f47fb7f94a97253f8f2f22703596d1129cd not found: ID does not exist"
Nov 04 01:28:31.528665 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.528602    5068 scope.go:122] "RemoveContainer" containerID="e9df8cb91833ffe22046558b5ecd859e003ce091e8e0d4f84c4970cf2091e993"
Nov 04 01:28:31.534687 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.534664    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82"]
Nov 04 01:28:31.538701 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.538680    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-ba844c/distinct-2-w4-ba844c-kserve-748d585646-m9p82"]
Nov 04 01:28:31.539680 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.539654    5068 scope.go:122] "RemoveContainer" containerID="e9df8cb91833ffe22046558b5ecd859e003ce091e8e0d4f84c4970cf2091e993"
Nov 04 01:28:31.540008 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:28:31.539984    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9df8cb91833ffe22046558b5ecd859e003ce091e8e0d4f84c4970cf2091e993\": container with ID starting with e9df8cb91833ffe22046558b5ecd859e003ce091e8e0d4f84c4970cf2091e993 not found: ID does not exist" containerID="e9df8cb91833ffe22046558b5ecd859e003ce091e8e0d4f84c4970cf2091e993"
Nov 04 01:28:31.540089 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.540016    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9df8cb91833ffe22046558b5ecd859e003ce091e8e0d4f84c4970cf2091e993"} err="failed to get container status \"e9df8cb91833ffe22046558b5ecd859e003ce091e8e0d4f84c4970cf2091e993\": rpc error: code = NotFound desc = could not find container \"e9df8cb91833ffe22046558b5ecd859e003ce091e8e0d4f84c4970cf2091e993\": container with ID starting with e9df8cb91833ffe22046558b5ecd859e003ce091e8e0d4f84c4970cf2091e993 not found: ID does not exist"
Nov 04 01:28:31.552497 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.552463    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm"]
Nov 04 01:28:31.557047 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:31.557019    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-ba844c/distinct-w4-ba844c-kserve-97db78785-cg9xm"]
Nov 04 01:28:32.113682 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:32.113602    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4ccafa61-1c9b-4595-a511-3404d7479526" path="/var/lib/kubelet/pods/4ccafa61-1c9b-4595-a511-3404d7479526/volumes"
Nov 04 01:28:32.114195 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:32.114179    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="530fc578-6622-4c42-a384-02e73bf17b1f" path="/var/lib/kubelet/pods/530fc578-6622-4c42-a384-02e73bf17b1f/volumes"
Nov 04 01:28:32.114584 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:32.114571    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6cdd01de-ff61-494a-bdb3-c67025a49440" path="/var/lib/kubelet/pods/6cdd01de-ff61-494a-bdb3-c67025a49440/volumes"
Nov 04 01:28:32.115314 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:32.115299    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f9e7abd2-8737-4357-a778-d6492fd12f3f" path="/var/lib/kubelet/pods/f9e7abd2-8737-4357-a778-d6492fd12f3f/volumes"
Nov 04 01:28:35.897590 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:35.897547    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w4-ba844c-7bc4dfcfd-nwk5b" podUID="e0946b4c-7a9f-4044-8f44-53484315882d" containerName="payload-processing" containerID="cri-o://b0d96574a62da39c743f1fec23623671952015364b05212fbe6fe334b0aa634c" gracePeriod=25
Nov 04 01:28:42.419096 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:42.417649    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w4-ba844c-7bc4dfcfd-nwk5b" podUID="e0946b4c-7a9f-4044-8f44-53484315882d" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:28:43.452336 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:28:43.452301    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:28:52.415631 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:52.415520    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w4-ba844c-7bc4dfcfd-nwk5b" podUID="e0946b4c-7a9f-4044-8f44-53484315882d" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:28:56.212092 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:56.212065    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-ba844c-7bc4dfcfd-nwk5b"
Nov 04 01:28:56.350963 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:56.350919    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e0946b4c-7a9f-4044-8f44-53484315882d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0946b4c-7a9f-4044-8f44-53484315882d-plugins-config-volume\") pod \"e0946b4c-7a9f-4044-8f44-53484315882d\" (UID: \"e0946b4c-7a9f-4044-8f44-53484315882d\") "
Nov 04 01:28:56.351172 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:56.351147    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e0946b4c-7a9f-4044-8f44-53484315882d-kube-api-access-cxj5h\" (UniqueName: \"kubernetes.io/projected/e0946b4c-7a9f-4044-8f44-53484315882d-kube-api-access-cxj5h\") pod \"e0946b4c-7a9f-4044-8f44-53484315882d\" (UID: \"e0946b4c-7a9f-4044-8f44-53484315882d\") "
Nov 04 01:28:56.351590 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:56.351549    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0946b4c-7a9f-4044-8f44-53484315882d-plugins-config-volume" pod "e0946b4c-7a9f-4044-8f44-53484315882d" (UID: "e0946b4c-7a9f-4044-8f44-53484315882d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:28:56.351764 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:56.351744    5068 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0946b4c-7a9f-4044-8f44-53484315882d-plugins-config-volume\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:56.353465 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:56.353441    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0946b4c-7a9f-4044-8f44-53484315882d-kube-api-access-cxj5h" pod "e0946b4c-7a9f-4044-8f44-53484315882d" (UID: "e0946b4c-7a9f-4044-8f44-53484315882d"). InnerVolumeSpecName "kube-api-access-cxj5h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:28:56.453344 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:56.453250    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cxj5h\" (UniqueName: \"kubernetes.io/projected/e0946b4c-7a9f-4044-8f44-53484315882d-kube-api-access-cxj5h\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:56.724735 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:56.724644    5068 generic.go:356] "Generic (PLEG): container finished" podID="e0946b4c-7a9f-4044-8f44-53484315882d" containerID="b0d96574a62da39c743f1fec23623671952015364b05212fbe6fe334b0aa634c" exitCode=0
Nov 04 01:28:56.724735 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:56.724700    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-ba844c-7bc4dfcfd-nwk5b"
Nov 04 01:28:56.724958 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:56.724734    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-ba844c-7bc4dfcfd-nwk5b" event={"ID":"e0946b4c-7a9f-4044-8f44-53484315882d","Type":"ContainerDied","Data":"b0d96574a62da39c743f1fec23623671952015364b05212fbe6fe334b0aa634c"}
Nov 04 01:28:56.724958 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:56.724795    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-ba844c-7bc4dfcfd-nwk5b" event={"ID":"e0946b4c-7a9f-4044-8f44-53484315882d","Type":"ContainerDied","Data":"da4afd994971d474b1fa4e462ca9fdc3102938f6d2d64ec8e683069a7b917947"}
Nov 04 01:28:56.724958 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:56.724819    5068 scope.go:122] "RemoveContainer" containerID="b0d96574a62da39c743f1fec23623671952015364b05212fbe6fe334b0aa634c"
Nov 04 01:28:56.735595 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:56.735566    5068 scope.go:122] "RemoveContainer" containerID="b0d96574a62da39c743f1fec23623671952015364b05212fbe6fe334b0aa634c"
Nov 04 01:28:56.735929 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:28:56.735906    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0d96574a62da39c743f1fec23623671952015364b05212fbe6fe334b0aa634c\": container with ID starting with b0d96574a62da39c743f1fec23623671952015364b05212fbe6fe334b0aa634c not found: ID does not exist" containerID="b0d96574a62da39c743f1fec23623671952015364b05212fbe6fe334b0aa634c"
Nov 04 01:28:56.736028 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:56.735945    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0d96574a62da39c743f1fec23623671952015364b05212fbe6fe334b0aa634c"} err="failed to get container status \"b0d96574a62da39c743f1fec23623671952015364b05212fbe6fe334b0aa634c\": rpc error: code = NotFound desc = could not find container \"b0d96574a62da39c743f1fec23623671952015364b05212fbe6fe334b0aa634c\": container with ID starting with b0d96574a62da39c743f1fec23623671952015364b05212fbe6fe334b0aa634c not found: ID does not exist"
Nov 04 01:28:56.751958 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:56.751929    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-ba844c-7bc4dfcfd-nwk5b"]
Nov 04 01:28:56.755971 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:56.755940    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-ba844c-7bc4dfcfd-nwk5b"]
Nov 04 01:28:58.114193 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:58.114160    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e0946b4c-7a9f-4044-8f44-53484315882d" path="/var/lib/kubelet/pods/e0946b4c-7a9f-4044-8f44-53484315882d/volumes"
Nov 04 01:28:59.993881 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:59.993303    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h"]
Nov 04 01:28:59.993881 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:28:59.993552    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h" podUID="fc92d968-9cb0-46bf-bf06-7f75d63a12b2" containerName="istio-proxy" containerID="cri-o://420c92dec053736258e6fe8120170bcf809dfbcb65a2881e6c1ab5abc537759a" gracePeriod=30
Nov 04 01:29:00.009500 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:00.005138    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg"]
Nov 04 01:29:00.009500 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:00.005393    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg" podUID="11913ebb-58e6-4b68-83ba-f4618a61e457" containerName="istio-proxy" containerID="cri-o://d1da1b14b02280e26da09a101c39b1e54e0fab795030b56bca3d381cbb0e2ab5" gracePeriod=30
Nov 04 01:29:00.010722 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:00.010680    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947"]
Nov 04 01:29:00.011146 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:00.011056    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947" podUID="a5f098fc-01c1-4e24-8611-e2b6aed954dc" containerName="istio-proxy" containerID="cri-o://52538cce5f64b8e64773a33fb18c455f844d3e14e49cbd8b598453fc39e9bb46" gracePeriod=30
Nov 04 01:29:05.326950 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.326910    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h"
Nov 04 01:29:05.371844 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.370786    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg"
Nov 04 01:29:05.395384 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.395352    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947"
Nov 04 01:29:05.443952 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.443904    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5f098fc-01c1-4e24-8611-e2b6aed954dc-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5f098fc-01c1-4e24-8611-e2b6aed954dc-workload-certs\") pod \"a5f098fc-01c1-4e24-8611-e2b6aed954dc\" (UID: \"a5f098fc-01c1-4e24-8611-e2b6aed954dc\") "
Nov 04 01:29:05.443952 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.443954    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a5f098fc-01c1-4e24-8611-e2b6aed954dc-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5f098fc-01c1-4e24-8611-e2b6aed954dc-istio-podinfo\") pod \"a5f098fc-01c1-4e24-8611-e2b6aed954dc\" (UID: \"a5f098fc-01c1-4e24-8611-e2b6aed954dc\") "
Nov 04 01:29:05.444218 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.444046    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5f098fc-01c1-4e24-8611-e2b6aed954dc-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5f098fc-01c1-4e24-8611-e2b6aed954dc-credential-socket\") pod \"a5f098fc-01c1-4e24-8611-e2b6aed954dc\" (UID: \"a5f098fc-01c1-4e24-8611-e2b6aed954dc\") "
Nov 04 01:29:05.444218 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.444077    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/11913ebb-58e6-4b68-83ba-f4618a61e457-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11913ebb-58e6-4b68-83ba-f4618a61e457-credential-socket\") pod \"11913ebb-58e6-4b68-83ba-f4618a61e457\" (UID: \"11913ebb-58e6-4b68-83ba-f4618a61e457\") "
Nov 04 01:29:05.444218 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.444110    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-workload-certs\") pod \"fc92d968-9cb0-46bf-bf06-7f75d63a12b2\" (UID: \"fc92d968-9cb0-46bf-bf06-7f75d63a12b2\") "
Nov 04 01:29:05.444218 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.444150    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-credential-socket\") pod \"fc92d968-9cb0-46bf-bf06-7f75d63a12b2\" (UID: \"fc92d968-9cb0-46bf-bf06-7f75d63a12b2\") "
Nov 04 01:29:05.444218 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.444184    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-istio-envoy\") pod \"fc92d968-9cb0-46bf-bf06-7f75d63a12b2\" (UID: \"fc92d968-9cb0-46bf-bf06-7f75d63a12b2\") "
Nov 04 01:29:05.444492 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.444219    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-workload-socket\") pod \"fc92d968-9cb0-46bf-bf06-7f75d63a12b2\" (UID: \"fc92d968-9cb0-46bf-bf06-7f75d63a12b2\") "
Nov 04 01:29:05.444492 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.444250    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/11913ebb-58e6-4b68-83ba-f4618a61e457-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11913ebb-58e6-4b68-83ba-f4618a61e457-istiod-ca-cert\") pod \"11913ebb-58e6-4b68-83ba-f4618a61e457\" (UID: \"11913ebb-58e6-4b68-83ba-f4618a61e457\") "
Nov 04 01:29:05.444492 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.444283    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5f098fc-01c1-4e24-8611-e2b6aed954dc-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5f098fc-01c1-4e24-8611-e2b6aed954dc-istio-envoy\") pod \"a5f098fc-01c1-4e24-8611-e2b6aed954dc\" (UID: \"a5f098fc-01c1-4e24-8611-e2b6aed954dc\") "
Nov 04 01:29:05.444492 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.444314    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-kube-api-access-8r2gw\" (UniqueName: \"kubernetes.io/projected/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-kube-api-access-8r2gw\") pod \"fc92d968-9cb0-46bf-bf06-7f75d63a12b2\" (UID: \"fc92d968-9cb0-46bf-bf06-7f75d63a12b2\") "
Nov 04 01:29:05.444492 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.444348    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-istio-podinfo\") pod \"fc92d968-9cb0-46bf-bf06-7f75d63a12b2\" (UID: \"fc92d968-9cb0-46bf-bf06-7f75d63a12b2\") "
Nov 04 01:29:05.444492 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.444403    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5f098fc-01c1-4e24-8611-e2b6aed954dc-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5f098fc-01c1-4e24-8611-e2b6aed954dc-workload-socket\") pod \"a5f098fc-01c1-4e24-8611-e2b6aed954dc\" (UID: \"a5f098fc-01c1-4e24-8611-e2b6aed954dc\") "
Nov 04 01:29:05.444492 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.444437    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/11913ebb-58e6-4b68-83ba-f4618a61e457-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11913ebb-58e6-4b68-83ba-f4618a61e457-istio-podinfo\") pod \"11913ebb-58e6-4b68-83ba-f4618a61e457\" (UID: \"11913ebb-58e6-4b68-83ba-f4618a61e457\") "
Nov 04 01:29:05.444890 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.444497    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a5f098fc-01c1-4e24-8611-e2b6aed954dc-istio-token\" (UniqueName: \"kubernetes.io/projected/a5f098fc-01c1-4e24-8611-e2b6aed954dc-istio-token\") pod \"a5f098fc-01c1-4e24-8611-e2b6aed954dc\" (UID: \"a5f098fc-01c1-4e24-8611-e2b6aed954dc\") "
Nov 04 01:29:05.444890 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.444526    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/11913ebb-58e6-4b68-83ba-f4618a61e457-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11913ebb-58e6-4b68-83ba-f4618a61e457-istio-data\") pod \"11913ebb-58e6-4b68-83ba-f4618a61e457\" (UID: \"11913ebb-58e6-4b68-83ba-f4618a61e457\") "
Nov 04 01:29:05.444890 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.444553    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a5f098fc-01c1-4e24-8611-e2b6aed954dc-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5f098fc-01c1-4e24-8611-e2b6aed954dc-istiod-ca-cert\") pod \"a5f098fc-01c1-4e24-8611-e2b6aed954dc\" (UID: \"a5f098fc-01c1-4e24-8611-e2b6aed954dc\") "
Nov 04 01:29:05.444890 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.444624    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5f098fc-01c1-4e24-8611-e2b6aed954dc-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5f098fc-01c1-4e24-8611-e2b6aed954dc-istio-data\") pod \"a5f098fc-01c1-4e24-8611-e2b6aed954dc\" (UID: \"a5f098fc-01c1-4e24-8611-e2b6aed954dc\") "
Nov 04 01:29:05.445094 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.445030    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/11913ebb-58e6-4b68-83ba-f4618a61e457-istio-token\" (UniqueName: \"kubernetes.io/projected/11913ebb-58e6-4b68-83ba-f4618a61e457-istio-token\") pod \"11913ebb-58e6-4b68-83ba-f4618a61e457\" (UID: \"11913ebb-58e6-4b68-83ba-f4618a61e457\") "
Nov 04 01:29:05.445094 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.445076    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a5f098fc-01c1-4e24-8611-e2b6aed954dc-kube-api-access-gpx7g\" (UniqueName: \"kubernetes.io/projected/a5f098fc-01c1-4e24-8611-e2b6aed954dc-kube-api-access-gpx7g\") pod \"a5f098fc-01c1-4e24-8611-e2b6aed954dc\" (UID: \"a5f098fc-01c1-4e24-8611-e2b6aed954dc\") "
Nov 04 01:29:05.445221 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.445196    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/11913ebb-58e6-4b68-83ba-f4618a61e457-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11913ebb-58e6-4b68-83ba-f4618a61e457-workload-socket\") pod \"11913ebb-58e6-4b68-83ba-f4618a61e457\" (UID: \"11913ebb-58e6-4b68-83ba-f4618a61e457\") "
Nov 04 01:29:05.445307 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.445278    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/11913ebb-58e6-4b68-83ba-f4618a61e457-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11913ebb-58e6-4b68-83ba-f4618a61e457-workload-certs\") pod \"11913ebb-58e6-4b68-83ba-f4618a61e457\" (UID: \"11913ebb-58e6-4b68-83ba-f4618a61e457\") "
Nov 04 01:29:05.445366 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.445314    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-istiod-ca-cert\") pod \"fc92d968-9cb0-46bf-bf06-7f75d63a12b2\" (UID: \"fc92d968-9cb0-46bf-bf06-7f75d63a12b2\") "
Nov 04 01:29:05.445366 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.445343    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/11913ebb-58e6-4b68-83ba-f4618a61e457-kube-api-access-j8nnk\" (UniqueName: \"kubernetes.io/projected/11913ebb-58e6-4b68-83ba-f4618a61e457-kube-api-access-j8nnk\") pod \"11913ebb-58e6-4b68-83ba-f4618a61e457\" (UID: \"11913ebb-58e6-4b68-83ba-f4618a61e457\") "
Nov 04 01:29:05.445473 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.445375    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/11913ebb-58e6-4b68-83ba-f4618a61e457-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11913ebb-58e6-4b68-83ba-f4618a61e457-istio-envoy\") pod \"11913ebb-58e6-4b68-83ba-f4618a61e457\" (UID: \"11913ebb-58e6-4b68-83ba-f4618a61e457\") "
Nov 04 01:29:05.445473 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.445437    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-istio-token\" (UniqueName: \"kubernetes.io/projected/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-istio-token\") pod \"fc92d968-9cb0-46bf-bf06-7f75d63a12b2\" (UID: \"fc92d968-9cb0-46bf-bf06-7f75d63a12b2\") "
Nov 04 01:29:05.445473 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.445464    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-istio-data\") pod \"fc92d968-9cb0-46bf-bf06-7f75d63a12b2\" (UID: \"fc92d968-9cb0-46bf-bf06-7f75d63a12b2\") "
Nov 04 01:29:05.448628 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.446150    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11913ebb-58e6-4b68-83ba-f4618a61e457-workload-socket" pod "11913ebb-58e6-4b68-83ba-f4618a61e457" (UID: "11913ebb-58e6-4b68-83ba-f4618a61e457"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:29:05.448628 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.446320    5068 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11913ebb-58e6-4b68-83ba-f4618a61e457-workload-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:05.448628 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.446400    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11913ebb-58e6-4b68-83ba-f4618a61e457-istio-data" pod "11913ebb-58e6-4b68-83ba-f4618a61e457" (UID: "11913ebb-58e6-4b68-83ba-f4618a61e457"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:29:05.448628 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.446406    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-istio-data" pod "fc92d968-9cb0-46bf-bf06-7f75d63a12b2" (UID: "fc92d968-9cb0-46bf-bf06-7f75d63a12b2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:29:05.448628 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.446736    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11913ebb-58e6-4b68-83ba-f4618a61e457-credential-socket" pod "11913ebb-58e6-4b68-83ba-f4618a61e457" (UID: "11913ebb-58e6-4b68-83ba-f4618a61e457"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:29:05.448628 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.447084    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-istiod-ca-cert" pod "fc92d968-9cb0-46bf-bf06-7f75d63a12b2" (UID: "fc92d968-9cb0-46bf-bf06-7f75d63a12b2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:29:05.448628 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.447296    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5f098fc-01c1-4e24-8611-e2b6aed954dc-workload-certs" pod "a5f098fc-01c1-4e24-8611-e2b6aed954dc" (UID: "a5f098fc-01c1-4e24-8611-e2b6aed954dc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:29:05.448628 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.447808    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11913ebb-58e6-4b68-83ba-f4618a61e457-workload-certs" pod "11913ebb-58e6-4b68-83ba-f4618a61e457" (UID: "11913ebb-58e6-4b68-83ba-f4618a61e457"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:29:05.448628 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.448029    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5f098fc-01c1-4e24-8611-e2b6aed954dc-credential-socket" pod "a5f098fc-01c1-4e24-8611-e2b6aed954dc" (UID: "a5f098fc-01c1-4e24-8611-e2b6aed954dc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:29:05.448628 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.448213    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-credential-socket" pod "fc92d968-9cb0-46bf-bf06-7f75d63a12b2" (UID: "fc92d968-9cb0-46bf-bf06-7f75d63a12b2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:29:05.448628 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.448368    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-workload-certs" pod "fc92d968-9cb0-46bf-bf06-7f75d63a12b2" (UID: "fc92d968-9cb0-46bf-bf06-7f75d63a12b2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:29:05.449701 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.449401    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11913ebb-58e6-4b68-83ba-f4618a61e457-istiod-ca-cert" pod "11913ebb-58e6-4b68-83ba-f4618a61e457" (UID: "11913ebb-58e6-4b68-83ba-f4618a61e457"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:29:05.450023 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.449855    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-workload-socket" pod "fc92d968-9cb0-46bf-bf06-7f75d63a12b2" (UID: "fc92d968-9cb0-46bf-bf06-7f75d63a12b2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:29:05.450466 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.450226    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5f098fc-01c1-4e24-8611-e2b6aed954dc-istiod-ca-cert" pod "a5f098fc-01c1-4e24-8611-e2b6aed954dc" (UID: "a5f098fc-01c1-4e24-8611-e2b6aed954dc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:29:05.454664 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.454640    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5f098fc-01c1-4e24-8611-e2b6aed954dc-kube-api-access-gpx7g" pod "a5f098fc-01c1-4e24-8611-e2b6aed954dc" (UID: "a5f098fc-01c1-4e24-8611-e2b6aed954dc"). InnerVolumeSpecName "kube-api-access-gpx7g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:29:05.456176 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.455831    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5f098fc-01c1-4e24-8611-e2b6aed954dc-istio-data" pod "a5f098fc-01c1-4e24-8611-e2b6aed954dc" (UID: "a5f098fc-01c1-4e24-8611-e2b6aed954dc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:29:05.456176 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.456146    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5f098fc-01c1-4e24-8611-e2b6aed954dc-workload-socket" pod "a5f098fc-01c1-4e24-8611-e2b6aed954dc" (UID: "a5f098fc-01c1-4e24-8611-e2b6aed954dc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:29:05.457094 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.457057    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-kube-api-access-8r2gw" pod "fc92d968-9cb0-46bf-bf06-7f75d63a12b2" (UID: "fc92d968-9cb0-46bf-bf06-7f75d63a12b2"). InnerVolumeSpecName "kube-api-access-8r2gw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:29:05.459995 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.459958    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-istio-token" pod "fc92d968-9cb0-46bf-bf06-7f75d63a12b2" (UID: "fc92d968-9cb0-46bf-bf06-7f75d63a12b2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:29:05.461945 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.461920    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-istio-podinfo" pod "fc92d968-9cb0-46bf-bf06-7f75d63a12b2" (UID: "fc92d968-9cb0-46bf-bf06-7f75d63a12b2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:29:05.462054 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.462037    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-istio-envoy" pod "fc92d968-9cb0-46bf-bf06-7f75d63a12b2" (UID: "fc92d968-9cb0-46bf-bf06-7f75d63a12b2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:29:05.462128 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.462110    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a5f098fc-01c1-4e24-8611-e2b6aed954dc-istio-podinfo" pod "a5f098fc-01c1-4e24-8611-e2b6aed954dc" (UID: "a5f098fc-01c1-4e24-8611-e2b6aed954dc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:29:05.462233 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.462217    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5f098fc-01c1-4e24-8611-e2b6aed954dc-istio-token" pod "a5f098fc-01c1-4e24-8611-e2b6aed954dc" (UID: "a5f098fc-01c1-4e24-8611-e2b6aed954dc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:29:05.462308 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.462293    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11913ebb-58e6-4b68-83ba-f4618a61e457-kube-api-access-j8nnk" pod "11913ebb-58e6-4b68-83ba-f4618a61e457" (UID: "11913ebb-58e6-4b68-83ba-f4618a61e457"). InnerVolumeSpecName "kube-api-access-j8nnk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:29:05.462385 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.462370    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/11913ebb-58e6-4b68-83ba-f4618a61e457-istio-podinfo" pod "11913ebb-58e6-4b68-83ba-f4618a61e457" (UID: "11913ebb-58e6-4b68-83ba-f4618a61e457"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:29:05.462460 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.462445    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11913ebb-58e6-4b68-83ba-f4618a61e457-istio-token" pod "11913ebb-58e6-4b68-83ba-f4618a61e457" (UID: "11913ebb-58e6-4b68-83ba-f4618a61e457"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:29:05.462856 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.462828    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11913ebb-58e6-4b68-83ba-f4618a61e457-istio-envoy" pod "11913ebb-58e6-4b68-83ba-f4618a61e457" (UID: "11913ebb-58e6-4b68-83ba-f4618a61e457"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:29:05.464338 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.464312    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5f098fc-01c1-4e24-8611-e2b6aed954dc-istio-envoy" pod "a5f098fc-01c1-4e24-8611-e2b6aed954dc" (UID: "a5f098fc-01c1-4e24-8611-e2b6aed954dc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:29:05.562682 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.551488    5068 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5f098fc-01c1-4e24-8611-e2b6aed954dc-credential-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:05.562682 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.551533    5068 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11913ebb-58e6-4b68-83ba-f4618a61e457-credential-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:05.562682 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.551551    5068 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-workload-certs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:05.562682 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.551564    5068 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-credential-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:05.562682 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.551587    5068 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-istio-envoy\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:05.562682 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.551600    5068 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-workload-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:05.562682 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.551630    5068 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11913ebb-58e6-4b68-83ba-f4618a61e457-istiod-ca-cert\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:05.562682 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.551645    5068 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5f098fc-01c1-4e24-8611-e2b6aed954dc-istio-envoy\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:05.562682 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.551658    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8r2gw\" (UniqueName: \"kubernetes.io/projected/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-kube-api-access-8r2gw\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:05.562682 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.551674    5068 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-istio-podinfo\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:05.562682 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.551689    5068 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5f098fc-01c1-4e24-8611-e2b6aed954dc-workload-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:05.562682 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.551702    5068 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11913ebb-58e6-4b68-83ba-f4618a61e457-istio-podinfo\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:05.562682 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.551715    5068 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5f098fc-01c1-4e24-8611-e2b6aed954dc-istio-token\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:05.562682 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.551729    5068 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11913ebb-58e6-4b68-83ba-f4618a61e457-istio-data\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:05.562682 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.551742    5068 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5f098fc-01c1-4e24-8611-e2b6aed954dc-istiod-ca-cert\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:05.562682 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.551755    5068 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5f098fc-01c1-4e24-8611-e2b6aed954dc-istio-data\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:05.562682 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.551769    5068 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11913ebb-58e6-4b68-83ba-f4618a61e457-istio-token\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:05.562682 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.551783    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gpx7g\" (UniqueName: \"kubernetes.io/projected/a5f098fc-01c1-4e24-8611-e2b6aed954dc-kube-api-access-gpx7g\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:05.562682 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.551798    5068 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11913ebb-58e6-4b68-83ba-f4618a61e457-workload-certs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:05.562682 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.551811    5068 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-istiod-ca-cert\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:05.562682 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.551824    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j8nnk\" (UniqueName: \"kubernetes.io/projected/11913ebb-58e6-4b68-83ba-f4618a61e457-kube-api-access-j8nnk\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:05.562682 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.551838    5068 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11913ebb-58e6-4b68-83ba-f4618a61e457-istio-envoy\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:05.562682 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.551851    5068 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-istio-token\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:05.562682 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.551864    5068 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc92d968-9cb0-46bf-bf06-7f75d63a12b2-istio-data\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:05.562682 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.551877    5068 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5f098fc-01c1-4e24-8611-e2b6aed954dc-workload-certs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:05.562682 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.551890    5068 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5f098fc-01c1-4e24-8611-e2b6aed954dc-istio-podinfo\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:05.791923 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.791888    5068 generic.go:356] "Generic (PLEG): container finished" podID="fc92d968-9cb0-46bf-bf06-7f75d63a12b2" containerID="420c92dec053736258e6fe8120170bcf809dfbcb65a2881e6c1ab5abc537759a" exitCode=0
Nov 04 01:29:05.792072 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.791942    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h"
Nov 04 01:29:05.792072 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.791966    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h" event={"ID":"fc92d968-9cb0-46bf-bf06-7f75d63a12b2","Type":"ContainerDied","Data":"420c92dec053736258e6fe8120170bcf809dfbcb65a2881e6c1ab5abc537759a"}
Nov 04 01:29:05.792072 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.792023    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h" event={"ID":"fc92d968-9cb0-46bf-bf06-7f75d63a12b2","Type":"ContainerDied","Data":"a4972e35aa46350d470d8b5d7f56d7d517ea8bbce0cf5bedabe8cf70d411389a"}
Nov 04 01:29:05.792072 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.792049    5068 scope.go:122] "RemoveContainer" containerID="420c92dec053736258e6fe8120170bcf809dfbcb65a2881e6c1ab5abc537759a"
Nov 04 01:29:05.793602 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.793574    5068 generic.go:356] "Generic (PLEG): container finished" podID="a5f098fc-01c1-4e24-8611-e2b6aed954dc" containerID="52538cce5f64b8e64773a33fb18c455f844d3e14e49cbd8b598453fc39e9bb46" exitCode=0
Nov 04 01:29:05.793764 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.793629    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947" event={"ID":"a5f098fc-01c1-4e24-8611-e2b6aed954dc","Type":"ContainerDied","Data":"52538cce5f64b8e64773a33fb18c455f844d3e14e49cbd8b598453fc39e9bb46"}
Nov 04 01:29:05.793764 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.793664    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947"
Nov 04 01:29:05.793764 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.793682    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947" event={"ID":"a5f098fc-01c1-4e24-8611-e2b6aed954dc","Type":"ContainerDied","Data":"039d1dbbfdfb13e1ab8bfe8ce380964f9bfa6796f84c655bedb0bc11b72f3458"}
Nov 04 01:29:05.795360 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.795323    5068 generic.go:356] "Generic (PLEG): container finished" podID="11913ebb-58e6-4b68-83ba-f4618a61e457" containerID="d1da1b14b02280e26da09a101c39b1e54e0fab795030b56bca3d381cbb0e2ab5" exitCode=0
Nov 04 01:29:05.795492 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.795378    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg" event={"ID":"11913ebb-58e6-4b68-83ba-f4618a61e457","Type":"ContainerDied","Data":"d1da1b14b02280e26da09a101c39b1e54e0fab795030b56bca3d381cbb0e2ab5"}
Nov 04 01:29:05.795492 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.795389    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg"
Nov 04 01:29:05.795492 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.795417    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg" event={"ID":"11913ebb-58e6-4b68-83ba-f4618a61e457","Type":"ContainerDied","Data":"189548a7706956037081b13c805ee5bc137331e6edd36b58fd5d1160e1e1371e"}
Nov 04 01:29:05.807410 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.807287    5068 scope.go:122] "RemoveContainer" containerID="420c92dec053736258e6fe8120170bcf809dfbcb65a2881e6c1ab5abc537759a"
Nov 04 01:29:05.807580 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:29:05.807526    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"420c92dec053736258e6fe8120170bcf809dfbcb65a2881e6c1ab5abc537759a\": container with ID starting with 420c92dec053736258e6fe8120170bcf809dfbcb65a2881e6c1ab5abc537759a not found: ID does not exist" containerID="420c92dec053736258e6fe8120170bcf809dfbcb65a2881e6c1ab5abc537759a"
Nov 04 01:29:05.807580 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.807557    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"420c92dec053736258e6fe8120170bcf809dfbcb65a2881e6c1ab5abc537759a"} err="failed to get container status \"420c92dec053736258e6fe8120170bcf809dfbcb65a2881e6c1ab5abc537759a\": rpc error: code = NotFound desc = could not find container \"420c92dec053736258e6fe8120170bcf809dfbcb65a2881e6c1ab5abc537759a\": container with ID starting with 420c92dec053736258e6fe8120170bcf809dfbcb65a2881e6c1ab5abc537759a not found: ID does not exist"
Nov 04 01:29:05.807740 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.807587    5068 scope.go:122] "RemoveContainer" containerID="52538cce5f64b8e64773a33fb18c455f844d3e14e49cbd8b598453fc39e9bb46"
Nov 04 01:29:05.817140 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.817112    5068 scope.go:122] "RemoveContainer" containerID="52538cce5f64b8e64773a33fb18c455f844d3e14e49cbd8b598453fc39e9bb46"
Nov 04 01:29:05.817394 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:29:05.817376    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52538cce5f64b8e64773a33fb18c455f844d3e14e49cbd8b598453fc39e9bb46\": container with ID starting with 52538cce5f64b8e64773a33fb18c455f844d3e14e49cbd8b598453fc39e9bb46 not found: ID does not exist" containerID="52538cce5f64b8e64773a33fb18c455f844d3e14e49cbd8b598453fc39e9bb46"
Nov 04 01:29:05.817473 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.817403    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52538cce5f64b8e64773a33fb18c455f844d3e14e49cbd8b598453fc39e9bb46"} err="failed to get container status \"52538cce5f64b8e64773a33fb18c455f844d3e14e49cbd8b598453fc39e9bb46\": rpc error: code = NotFound desc = could not find container \"52538cce5f64b8e64773a33fb18c455f844d3e14e49cbd8b598453fc39e9bb46\": container with ID starting with 52538cce5f64b8e64773a33fb18c455f844d3e14e49cbd8b598453fc39e9bb46 not found: ID does not exist"
Nov 04 01:29:05.817473 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.817424    5068 scope.go:122] "RemoveContainer" containerID="d1da1b14b02280e26da09a101c39b1e54e0fab795030b56bca3d381cbb0e2ab5"
Nov 04 01:29:05.826132 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.826116    5068 scope.go:122] "RemoveContainer" containerID="d1da1b14b02280e26da09a101c39b1e54e0fab795030b56bca3d381cbb0e2ab5"
Nov 04 01:29:05.826433 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:29:05.826413    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1da1b14b02280e26da09a101c39b1e54e0fab795030b56bca3d381cbb0e2ab5\": container with ID starting with d1da1b14b02280e26da09a101c39b1e54e0fab795030b56bca3d381cbb0e2ab5 not found: ID does not exist" containerID="d1da1b14b02280e26da09a101c39b1e54e0fab795030b56bca3d381cbb0e2ab5"
Nov 04 01:29:05.826512 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.826444    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1da1b14b02280e26da09a101c39b1e54e0fab795030b56bca3d381cbb0e2ab5"} err="failed to get container status \"d1da1b14b02280e26da09a101c39b1e54e0fab795030b56bca3d381cbb0e2ab5\": rpc error: code = NotFound desc = could not find container \"d1da1b14b02280e26da09a101c39b1e54e0fab795030b56bca3d381cbb0e2ab5\": container with ID starting with d1da1b14b02280e26da09a101c39b1e54e0fab795030b56bca3d381cbb0e2ab5 not found: ID does not exist"
Nov 04 01:29:05.830665 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.830642    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h"]
Nov 04 01:29:05.844747 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.844722    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-5lg2h"]
Nov 04 01:29:05.852725 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.852704    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg"]
Nov 04 01:29:05.857133 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.857112    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-7vzsg"]
Nov 04 01:29:05.872935 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.872902    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947"]
Nov 04 01:29:05.877909 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:05.877882    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-ba844c-openshift-default-549db56fd5-vf947"]
Nov 04 01:29:06.113399 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:06.113322    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="11913ebb-58e6-4b68-83ba-f4618a61e457" path="/var/lib/kubelet/pods/11913ebb-58e6-4b68-83ba-f4618a61e457/volumes"
Nov 04 01:29:06.113867 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:06.113852    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a5f098fc-01c1-4e24-8611-e2b6aed954dc" path="/var/lib/kubelet/pods/a5f098fc-01c1-4e24-8611-e2b6aed954dc/volumes"
Nov 04 01:29:06.114269 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:06.114256    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fc92d968-9cb0-46bf-bf06-7f75d63a12b2" path="/var/lib/kubelet/pods/fc92d968-9cb0-46bf-bf06-7f75d63a12b2/volumes"
Nov 04 01:29:08.345447 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:08.345411    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:29:13.451941 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:29:13.451904    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:29:14.216759 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:14.216717    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6ecd1f-857447647c-54bjb"]
Nov 04 01:29:14.217014 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:14.216972    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ecd1f-857447647c-54bjb" podUID="c9dfccbc-dc8d-4569-bac9-6847e6658b8c" containerName="maas-api" containerID="cri-o://71a4fea5e3f5d50da81a6657560c471be4842ed3101e12454279f5f3d958ad72" gracePeriod=30
Nov 04 01:29:14.624047 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:14.624020    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ecd1f-857447647c-54bjb"
Nov 04 01:29:14.721647 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:14.721598    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c9dfccbc-dc8d-4569-bac9-6847e6658b8c-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9dfccbc-dc8d-4569-bac9-6847e6658b8c-openshift-service-ca\") pod \"c9dfccbc-dc8d-4569-bac9-6847e6658b8c\" (UID: \"c9dfccbc-dc8d-4569-bac9-6847e6658b8c\") "
Nov 04 01:29:14.721822 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:14.721685    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c9dfccbc-dc8d-4569-bac9-6847e6658b8c-metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9dfccbc-dc8d-4569-bac9-6847e6658b8c-metrics-certs\") pod \"c9dfccbc-dc8d-4569-bac9-6847e6658b8c\" (UID: \"c9dfccbc-dc8d-4569-bac9-6847e6658b8c\") "
Nov 04 01:29:14.721822 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:14.721817    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c9dfccbc-dc8d-4569-bac9-6847e6658b8c-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9dfccbc-dc8d-4569-bac9-6847e6658b8c-maas-api-tls\") pod \"c9dfccbc-dc8d-4569-bac9-6847e6658b8c\" (UID: \"c9dfccbc-dc8d-4569-bac9-6847e6658b8c\") "
Nov 04 01:29:14.721967 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:14.721876    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c9dfccbc-dc8d-4569-bac9-6847e6658b8c-kube-api-access-kpwd8\" (UniqueName: \"kubernetes.io/projected/c9dfccbc-dc8d-4569-bac9-6847e6658b8c-kube-api-access-kpwd8\") pod \"c9dfccbc-dc8d-4569-bac9-6847e6658b8c\" (UID: \"c9dfccbc-dc8d-4569-bac9-6847e6658b8c\") "
Nov 04 01:29:14.722027 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:14.721998    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9dfccbc-dc8d-4569-bac9-6847e6658b8c-openshift-service-ca" pod "c9dfccbc-dc8d-4569-bac9-6847e6658b8c" (UID: "c9dfccbc-dc8d-4569-bac9-6847e6658b8c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:29:14.722368 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:14.722346    5068 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9dfccbc-dc8d-4569-bac9-6847e6658b8c-openshift-service-ca\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:14.724659 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:14.724581    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9dfccbc-dc8d-4569-bac9-6847e6658b8c-maas-api-tls" pod "c9dfccbc-dc8d-4569-bac9-6847e6658b8c" (UID: "c9dfccbc-dc8d-4569-bac9-6847e6658b8c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:29:14.724796 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:14.724630    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9dfccbc-dc8d-4569-bac9-6847e6658b8c-metrics-certs" pod "c9dfccbc-dc8d-4569-bac9-6847e6658b8c" (UID: "c9dfccbc-dc8d-4569-bac9-6847e6658b8c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:29:14.724796 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:14.724779    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9dfccbc-dc8d-4569-bac9-6847e6658b8c-kube-api-access-kpwd8" pod "c9dfccbc-dc8d-4569-bac9-6847e6658b8c" (UID: "c9dfccbc-dc8d-4569-bac9-6847e6658b8c"). InnerVolumeSpecName "kube-api-access-kpwd8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:29:14.823308 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:14.823276    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kpwd8\" (UniqueName: \"kubernetes.io/projected/c9dfccbc-dc8d-4569-bac9-6847e6658b8c-kube-api-access-kpwd8\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:14.823308 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:14.823306    5068 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9dfccbc-dc8d-4569-bac9-6847e6658b8c-metrics-certs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:14.823513 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:14.823320    5068 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9dfccbc-dc8d-4569-bac9-6847e6658b8c-maas-api-tls\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:14.898111 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:14.898079    5068 generic.go:356] "Generic (PLEG): container finished" podID="c9dfccbc-dc8d-4569-bac9-6847e6658b8c" containerID="71a4fea5e3f5d50da81a6657560c471be4842ed3101e12454279f5f3d958ad72" exitCode=0
Nov 04 01:29:14.898307 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:14.898126    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ecd1f-857447647c-54bjb"
Nov 04 01:29:14.898307 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:14.898119    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ecd1f-857447647c-54bjb" event={"ID":"c9dfccbc-dc8d-4569-bac9-6847e6658b8c","Type":"ContainerDied","Data":"71a4fea5e3f5d50da81a6657560c471be4842ed3101e12454279f5f3d958ad72"}
Nov 04 01:29:14.898307 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:14.898261    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ecd1f-857447647c-54bjb" event={"ID":"c9dfccbc-dc8d-4569-bac9-6847e6658b8c","Type":"ContainerDied","Data":"37947f6b07314ae714194af4aaf8a66024ea0f907edb42a4294eba7051a65c95"}
Nov 04 01:29:14.898307 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:14.898290    5068 scope.go:122] "RemoveContainer" containerID="71a4fea5e3f5d50da81a6657560c471be4842ed3101e12454279f5f3d958ad72"
Nov 04 01:29:14.913212 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:14.913193    5068 scope.go:122] "RemoveContainer" containerID="71a4fea5e3f5d50da81a6657560c471be4842ed3101e12454279f5f3d958ad72"
Nov 04 01:29:14.913681 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:29:14.913603    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71a4fea5e3f5d50da81a6657560c471be4842ed3101e12454279f5f3d958ad72\": container with ID starting with 71a4fea5e3f5d50da81a6657560c471be4842ed3101e12454279f5f3d958ad72 not found: ID does not exist" containerID="71a4fea5e3f5d50da81a6657560c471be4842ed3101e12454279f5f3d958ad72"
Nov 04 01:29:14.913844 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:14.913690    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71a4fea5e3f5d50da81a6657560c471be4842ed3101e12454279f5f3d958ad72"} err="failed to get container status \"71a4fea5e3f5d50da81a6657560c471be4842ed3101e12454279f5f3d958ad72\": rpc error: code = NotFound desc = could not find container \"71a4fea5e3f5d50da81a6657560c471be4842ed3101e12454279f5f3d958ad72\": container with ID starting with 71a4fea5e3f5d50da81a6657560c471be4842ed3101e12454279f5f3d958ad72 not found: ID does not exist"
Nov 04 01:29:14.927778 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:14.927741    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6ecd1f-857447647c-54bjb"]
Nov 04 01:29:14.931259 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:14.931237    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6ecd1f-857447647c-54bjb"]
Nov 04 01:29:16.112953 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:16.112920    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c9dfccbc-dc8d-4569-bac9-6847e6658b8c" path="/var/lib/kubelet/pods/c9dfccbc-dc8d-4569-bac9-6847e6658b8c/volumes"
Nov 04 01:29:27.470350 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:27.470315    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5"]
Nov 04 01:29:30.794338 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:30.794298    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:29:42.410483 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.410440    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds"]
Nov 04 01:29:42.411934 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.411901    5068 cpu_manager.go:414] "containerMap: removing container" podUID="fc92d968-9cb0-46bf-bf06-7f75d63a12b2" containerName="istio-proxy"
Nov 04 01:29:42.411934 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.411933    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fc92d968-9cb0-46bf-bf06-7f75d63a12b2" containerName="istio-proxy"
Nov 04 01:29:42.412142 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.411954    5068 cpu_manager.go:414] "containerMap: removing container" podUID="aae6d603-4415-436c-917f-448a09092e29" containerName="istio-proxy"
Nov 04 01:29:42.412142 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.411962    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aae6d603-4415-436c-917f-448a09092e29" containerName="istio-proxy"
Nov 04 01:29:42.412142 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.411983    5068 cpu_manager.go:414] "containerMap: removing container" podUID="530fc578-6622-4c42-a384-02e73bf17b1f" containerName="main"
Nov 04 01:29:42.412142 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.411990    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="530fc578-6622-4c42-a384-02e73bf17b1f" containerName="main"
Nov 04 01:29:42.412142 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.411999    5068 cpu_manager.go:414] "containerMap: removing container" podUID="11913ebb-58e6-4b68-83ba-f4618a61e457" containerName="istio-proxy"
Nov 04 01:29:42.412142 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.412006    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="11913ebb-58e6-4b68-83ba-f4618a61e457" containerName="istio-proxy"
Nov 04 01:29:42.412142 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.412031    5068 cpu_manager.go:414] "containerMap: removing container" podUID="4ccafa61-1c9b-4595-a511-3404d7479526" containerName="main"
Nov 04 01:29:42.412142 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.412038    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4ccafa61-1c9b-4595-a511-3404d7479526" containerName="main"
Nov 04 01:29:42.412142 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.412047    5068 cpu_manager.go:414] "containerMap: removing container" podUID="c2697761-a360-4c3c-a5d5-c96f06998bb3" containerName="istio-proxy"
Nov 04 01:29:42.412142 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.412056    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c2697761-a360-4c3c-a5d5-c96f06998bb3" containerName="istio-proxy"
Nov 04 01:29:42.412142 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.412074    5068 cpu_manager.go:414] "containerMap: removing container" podUID="c9dfccbc-dc8d-4569-bac9-6847e6658b8c" containerName="maas-api"
Nov 04 01:29:42.412142 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.412081    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c9dfccbc-dc8d-4569-bac9-6847e6658b8c" containerName="maas-api"
Nov 04 01:29:42.412142 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.412131    5068 cpu_manager.go:414] "containerMap: removing container" podUID="a5f098fc-01c1-4e24-8611-e2b6aed954dc" containerName="istio-proxy"
Nov 04 01:29:42.412142 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.412138    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a5f098fc-01c1-4e24-8611-e2b6aed954dc" containerName="istio-proxy"
Nov 04 01:29:42.412910 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.412153    5068 cpu_manager.go:414] "containerMap: removing container" podUID="e0946b4c-7a9f-4044-8f44-53484315882d" containerName="payload-processing"
Nov 04 01:29:42.412910 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.412160    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e0946b4c-7a9f-4044-8f44-53484315882d" containerName="payload-processing"
Nov 04 01:29:42.412910 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.412177    5068 cpu_manager.go:414] "containerMap: removing container" podUID="d8725463-9c1f-4cee-b5ec-28fb3ca9f939" containerName="istio-proxy"
Nov 04 01:29:42.412910 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.412186    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d8725463-9c1f-4cee-b5ec-28fb3ca9f939" containerName="istio-proxy"
Nov 04 01:29:42.412910 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.412208    5068 cpu_manager.go:414] "containerMap: removing container" podUID="f9e7abd2-8737-4357-a778-d6492fd12f3f" containerName="main"
Nov 04 01:29:42.412910 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.412215    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f9e7abd2-8737-4357-a778-d6492fd12f3f" containerName="main"
Nov 04 01:29:42.412910 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.412230    5068 cpu_manager.go:414] "containerMap: removing container" podUID="6cdd01de-ff61-494a-bdb3-c67025a49440" containerName="main"
Nov 04 01:29:42.412910 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.412237    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6cdd01de-ff61-494a-bdb3-c67025a49440" containerName="main"
Nov 04 01:29:42.412910 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.412376    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="e0946b4c-7a9f-4044-8f44-53484315882d" containerName="payload-processing"
Nov 04 01:29:42.412910 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.412389    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="d8725463-9c1f-4cee-b5ec-28fb3ca9f939" containerName="istio-proxy"
Nov 04 01:29:42.412910 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.412403    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="f9e7abd2-8737-4357-a778-d6492fd12f3f" containerName="main"
Nov 04 01:29:42.412910 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.412414    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="6cdd01de-ff61-494a-bdb3-c67025a49440" containerName="main"
Nov 04 01:29:42.412910 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.412425    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="fc92d968-9cb0-46bf-bf06-7f75d63a12b2" containerName="istio-proxy"
Nov 04 01:29:42.412910 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.412437    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="aae6d603-4415-436c-917f-448a09092e29" containerName="istio-proxy"
Nov 04 01:29:42.412910 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.412447    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="530fc578-6622-4c42-a384-02e73bf17b1f" containerName="main"
Nov 04 01:29:42.412910 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.412457    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="11913ebb-58e6-4b68-83ba-f4618a61e457" containerName="istio-proxy"
Nov 04 01:29:42.412910 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.412469    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="4ccafa61-1c9b-4595-a511-3404d7479526" containerName="main"
Nov 04 01:29:42.412910 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.412479    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="c2697761-a360-4c3c-a5d5-c96f06998bb3" containerName="istio-proxy"
Nov 04 01:29:42.412910 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.412489    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="c9dfccbc-dc8d-4569-bac9-6847e6658b8c" containerName="maas-api"
Nov 04 01:29:42.412910 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.412505    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="a5f098fc-01c1-4e24-8611-e2b6aed954dc" containerName="istio-proxy"
Nov 04 01:29:42.416956 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.416929    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds"
Nov 04 01:29:42.431860 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.430956    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-2aefc5-openshift-default-dockercfg-tlkxk\""
Nov 04 01:29:42.431860 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.431552    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds"]
Nov 04 01:29:42.494276 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.494215    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5" podUID="60f5c87a-48f8-4ad9-847b-cff571cc9340" containerName="main" containerID="cri-o://d0e8cee1f4946b2b00040a0af013b5fc0187f86d45cb571c1d2e02b00da992f6" gracePeriod=45
Nov 04 01:29:42.541685 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.540858    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d091cbea-bbd1-4242-a1c9-f72628c8e019-workload-socket\") pod \"e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds\" (UID: \"d091cbea-bbd1-4242-a1c9-f72628c8e019\") " pod="openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds"
Nov 04 01:29:42.541685 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.540916    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hcdlr\" (UniqueName: \"kubernetes.io/projected/d091cbea-bbd1-4242-a1c9-f72628c8e019-kube-api-access-hcdlr\") pod \"e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds\" (UID: \"d091cbea-bbd1-4242-a1c9-f72628c8e019\") " pod="openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds"
Nov 04 01:29:42.541685 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.540950    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d091cbea-bbd1-4242-a1c9-f72628c8e019-credential-socket\") pod \"e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds\" (UID: \"d091cbea-bbd1-4242-a1c9-f72628c8e019\") " pod="openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds"
Nov 04 01:29:42.541685 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.540983    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d091cbea-bbd1-4242-a1c9-f72628c8e019-istiod-ca-cert\") pod \"e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds\" (UID: \"d091cbea-bbd1-4242-a1c9-f72628c8e019\") " pod="openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds"
Nov 04 01:29:42.541685 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.541022    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d091cbea-bbd1-4242-a1c9-f72628c8e019-istio-data\") pod \"e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds\" (UID: \"d091cbea-bbd1-4242-a1c9-f72628c8e019\") " pod="openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds"
Nov 04 01:29:42.541685 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.541108    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d091cbea-bbd1-4242-a1c9-f72628c8e019-istio-podinfo\") pod \"e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds\" (UID: \"d091cbea-bbd1-4242-a1c9-f72628c8e019\") " pod="openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds"
Nov 04 01:29:42.541685 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.541200    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d091cbea-bbd1-4242-a1c9-f72628c8e019-istio-envoy\") pod \"e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds\" (UID: \"d091cbea-bbd1-4242-a1c9-f72628c8e019\") " pod="openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds"
Nov 04 01:29:42.541685 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.541265    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d091cbea-bbd1-4242-a1c9-f72628c8e019-workload-certs\") pod \"e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds\" (UID: \"d091cbea-bbd1-4242-a1c9-f72628c8e019\") " pod="openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds"
Nov 04 01:29:42.541685 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.541318    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d091cbea-bbd1-4242-a1c9-f72628c8e019-istio-token\") pod \"e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds\" (UID: \"d091cbea-bbd1-4242-a1c9-f72628c8e019\") " pod="openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds"
Nov 04 01:29:42.668704 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.643957    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d091cbea-bbd1-4242-a1c9-f72628c8e019-workload-socket\") pod \"e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds\" (UID: \"d091cbea-bbd1-4242-a1c9-f72628c8e019\") " pod="openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds"
Nov 04 01:29:42.668704 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.644011    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hcdlr\" (UniqueName: \"kubernetes.io/projected/d091cbea-bbd1-4242-a1c9-f72628c8e019-kube-api-access-hcdlr\") pod \"e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds\" (UID: \"d091cbea-bbd1-4242-a1c9-f72628c8e019\") " pod="openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds"
Nov 04 01:29:42.668704 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.644046    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d091cbea-bbd1-4242-a1c9-f72628c8e019-credential-socket\") pod \"e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds\" (UID: \"d091cbea-bbd1-4242-a1c9-f72628c8e019\") " pod="openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds"
Nov 04 01:29:42.668704 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.644083    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d091cbea-bbd1-4242-a1c9-f72628c8e019-istiod-ca-cert\") pod \"e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds\" (UID: \"d091cbea-bbd1-4242-a1c9-f72628c8e019\") " pod="openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds"
Nov 04 01:29:42.668704 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.644123    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d091cbea-bbd1-4242-a1c9-f72628c8e019-istio-data\") pod \"e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds\" (UID: \"d091cbea-bbd1-4242-a1c9-f72628c8e019\") " pod="openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds"
Nov 04 01:29:42.668704 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.644243    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d091cbea-bbd1-4242-a1c9-f72628c8e019-istio-podinfo\") pod \"e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds\" (UID: \"d091cbea-bbd1-4242-a1c9-f72628c8e019\") " pod="openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds"
Nov 04 01:29:42.668704 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.644333    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d091cbea-bbd1-4242-a1c9-f72628c8e019-istio-envoy\") pod \"e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds\" (UID: \"d091cbea-bbd1-4242-a1c9-f72628c8e019\") " pod="openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds"
Nov 04 01:29:42.668704 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.644397    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d091cbea-bbd1-4242-a1c9-f72628c8e019-workload-certs\") pod \"e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds\" (UID: \"d091cbea-bbd1-4242-a1c9-f72628c8e019\") " pod="openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds"
Nov 04 01:29:42.668704 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.644448    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d091cbea-bbd1-4242-a1c9-f72628c8e019-istio-token\") pod \"e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds\" (UID: \"d091cbea-bbd1-4242-a1c9-f72628c8e019\") " pod="openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds"
Nov 04 01:29:42.668704 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.645242    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d091cbea-bbd1-4242-a1c9-f72628c8e019-workload-socket\") pod \"e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds\" (UID: \"d091cbea-bbd1-4242-a1c9-f72628c8e019\") " pod="openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds"
Nov 04 01:29:42.669793 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.669692    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d091cbea-bbd1-4242-a1c9-f72628c8e019-credential-socket\") pod \"e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds\" (UID: \"d091cbea-bbd1-4242-a1c9-f72628c8e019\") " pod="openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds"
Nov 04 01:29:42.673080 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.672730    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d091cbea-bbd1-4242-a1c9-f72628c8e019-istiod-ca-cert\") pod \"e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds\" (UID: \"d091cbea-bbd1-4242-a1c9-f72628c8e019\") " pod="openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds"
Nov 04 01:29:42.673080 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.672894    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d091cbea-bbd1-4242-a1c9-f72628c8e019-istio-data\") pod \"e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds\" (UID: \"d091cbea-bbd1-4242-a1c9-f72628c8e019\") " pod="openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds"
Nov 04 01:29:42.673080 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.673031    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d091cbea-bbd1-4242-a1c9-f72628c8e019-workload-certs\") pod \"e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds\" (UID: \"d091cbea-bbd1-4242-a1c9-f72628c8e019\") " pod="openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds"
Nov 04 01:29:42.737648 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.726053    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d091cbea-bbd1-4242-a1c9-f72628c8e019-istio-podinfo\") pod \"e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds\" (UID: \"d091cbea-bbd1-4242-a1c9-f72628c8e019\") " pod="openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds"
Nov 04 01:29:42.737648 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.729074    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hcdlr\" (UniqueName: \"kubernetes.io/projected/d091cbea-bbd1-4242-a1c9-f72628c8e019-kube-api-access-hcdlr\") pod \"e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds\" (UID: \"d091cbea-bbd1-4242-a1c9-f72628c8e019\") " pod="openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds"
Nov 04 01:29:42.737648 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.734108    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d091cbea-bbd1-4242-a1c9-f72628c8e019-istio-token\") pod \"e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds\" (UID: \"d091cbea-bbd1-4242-a1c9-f72628c8e019\") " pod="openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds"
Nov 04 01:29:42.737648 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.734566    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d091cbea-bbd1-4242-a1c9-f72628c8e019-istio-envoy\") pod \"e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds\" (UID: \"d091cbea-bbd1-4242-a1c9-f72628c8e019\") " pod="openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds"
Nov 04 01:29:42.743639 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:42.741923    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds"
Nov 04 01:29:43.099312 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:43.099279    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5"
Nov 04 01:29:43.153314 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:43.152421    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/60f5c87a-48f8-4ad9-847b-cff571cc9340-home\" (UniqueName: \"kubernetes.io/empty-dir/60f5c87a-48f8-4ad9-847b-cff571cc9340-home\") pod \"60f5c87a-48f8-4ad9-847b-cff571cc9340\" (UID: \"60f5c87a-48f8-4ad9-847b-cff571cc9340\") "
Nov 04 01:29:43.153314 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:43.152475    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/60f5c87a-48f8-4ad9-847b-cff571cc9340-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60f5c87a-48f8-4ad9-847b-cff571cc9340-tmp-dir\") pod \"60f5c87a-48f8-4ad9-847b-cff571cc9340\" (UID: \"60f5c87a-48f8-4ad9-847b-cff571cc9340\") "
Nov 04 01:29:43.153314 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:43.152539    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/60f5c87a-48f8-4ad9-847b-cff571cc9340-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60f5c87a-48f8-4ad9-847b-cff571cc9340-model-cache\") pod \"60f5c87a-48f8-4ad9-847b-cff571cc9340\" (UID: \"60f5c87a-48f8-4ad9-847b-cff571cc9340\") "
Nov 04 01:29:43.153314 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:43.152586    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/60f5c87a-48f8-4ad9-847b-cff571cc9340-kube-api-access-jpnkh\" (UniqueName: \"kubernetes.io/projected/60f5c87a-48f8-4ad9-847b-cff571cc9340-kube-api-access-jpnkh\") pod \"60f5c87a-48f8-4ad9-847b-cff571cc9340\" (UID: \"60f5c87a-48f8-4ad9-847b-cff571cc9340\") "
Nov 04 01:29:43.153314 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:43.152665    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/60f5c87a-48f8-4ad9-847b-cff571cc9340-dshm\" (UniqueName: \"kubernetes.io/empty-dir/60f5c87a-48f8-4ad9-847b-cff571cc9340-dshm\") pod \"60f5c87a-48f8-4ad9-847b-cff571cc9340\" (UID: \"60f5c87a-48f8-4ad9-847b-cff571cc9340\") "
Nov 04 01:29:43.153314 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:43.152788    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/60f5c87a-48f8-4ad9-847b-cff571cc9340-tls-certs\" (UniqueName: \"kubernetes.io/secret/60f5c87a-48f8-4ad9-847b-cff571cc9340-tls-certs\") pod \"60f5c87a-48f8-4ad9-847b-cff571cc9340\" (UID: \"60f5c87a-48f8-4ad9-847b-cff571cc9340\") "
Nov 04 01:29:43.153314 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:43.152785    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60f5c87a-48f8-4ad9-847b-cff571cc9340-home" pod "60f5c87a-48f8-4ad9-847b-cff571cc9340" (UID: "60f5c87a-48f8-4ad9-847b-cff571cc9340"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:29:43.153314 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:43.152971    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60f5c87a-48f8-4ad9-847b-cff571cc9340-model-cache" pod "60f5c87a-48f8-4ad9-847b-cff571cc9340" (UID: "60f5c87a-48f8-4ad9-847b-cff571cc9340"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:29:43.154147 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:43.153353    5068 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60f5c87a-48f8-4ad9-847b-cff571cc9340-model-cache\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:43.154147 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:43.153388    5068 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60f5c87a-48f8-4ad9-847b-cff571cc9340-home\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:43.154147 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:43.153912    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60f5c87a-48f8-4ad9-847b-cff571cc9340-tmp-dir" pod "60f5c87a-48f8-4ad9-847b-cff571cc9340" (UID: "60f5c87a-48f8-4ad9-847b-cff571cc9340"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:29:43.157037 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:29:43.156938    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd091cbea_bbd1_4242_a1c9_f72628c8e019.slice/crio-19e7ce512d603be2825ac5eccff4f78dedb5541ed8e64de0c8dba4f00a275ea0 WatchSource:0}: Error finding container 19e7ce512d603be2825ac5eccff4f78dedb5541ed8e64de0c8dba4f00a275ea0: Status 404 returned error can't find the container with id 19e7ce512d603be2825ac5eccff4f78dedb5541ed8e64de0c8dba4f00a275ea0
Nov 04 01:29:43.168527 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:43.157935    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60f5c87a-48f8-4ad9-847b-cff571cc9340-dshm" pod "60f5c87a-48f8-4ad9-847b-cff571cc9340" (UID: "60f5c87a-48f8-4ad9-847b-cff571cc9340"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:29:43.168527 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:43.167122    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:29:43.168527 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:43.167393    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:29:43.168527 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:43.167437    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:29:43.172960 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:43.172188    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds"]
Nov 04 01:29:43.191286 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:43.189205    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60f5c87a-48f8-4ad9-847b-cff571cc9340-kube-api-access-jpnkh" pod "60f5c87a-48f8-4ad9-847b-cff571cc9340" (UID: "60f5c87a-48f8-4ad9-847b-cff571cc9340"). InnerVolumeSpecName "kube-api-access-jpnkh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:29:43.191691 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:43.191334    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60f5c87a-48f8-4ad9-847b-cff571cc9340-tls-certs" pod "60f5c87a-48f8-4ad9-847b-cff571cc9340" (UID: "60f5c87a-48f8-4ad9-847b-cff571cc9340"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:29:43.193565 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:43.193426    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds" event={"ID":"d091cbea-bbd1-4242-a1c9-f72628c8e019","Type":"ContainerStarted","Data":"19e7ce512d603be2825ac5eccff4f78dedb5541ed8e64de0c8dba4f00a275ea0"}
Nov 04 01:29:43.200800 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:43.200762    5068 generic.go:356] "Generic (PLEG): container finished" podID="60f5c87a-48f8-4ad9-847b-cff571cc9340" containerID="d0e8cee1f4946b2b00040a0af013b5fc0187f86d45cb571c1d2e02b00da992f6" exitCode=0
Nov 04 01:29:43.200943 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:43.200905    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5" event={"ID":"60f5c87a-48f8-4ad9-847b-cff571cc9340","Type":"ContainerDied","Data":"d0e8cee1f4946b2b00040a0af013b5fc0187f86d45cb571c1d2e02b00da992f6"}
Nov 04 01:29:43.200994 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:43.200958    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5" event={"ID":"60f5c87a-48f8-4ad9-847b-cff571cc9340","Type":"ContainerDied","Data":"b84fa3d3cedca73535ffa0f430e2408672672778410408a592b1f8d180c85df5"}
Nov 04 01:29:43.200994 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:43.200980    5068 scope.go:122] "RemoveContainer" containerID="d0e8cee1f4946b2b00040a0af013b5fc0187f86d45cb571c1d2e02b00da992f6"
Nov 04 01:29:43.201182 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:43.201168    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5"
Nov 04 01:29:43.223295 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:43.222854    5068 scope.go:122] "RemoveContainer" containerID="d0e8cee1f4946b2b00040a0af013b5fc0187f86d45cb571c1d2e02b00da992f6"
Nov 04 01:29:43.223512 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:29:43.223489    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0e8cee1f4946b2b00040a0af013b5fc0187f86d45cb571c1d2e02b00da992f6\": container with ID starting with d0e8cee1f4946b2b00040a0af013b5fc0187f86d45cb571c1d2e02b00da992f6 not found: ID does not exist" containerID="d0e8cee1f4946b2b00040a0af013b5fc0187f86d45cb571c1d2e02b00da992f6"
Nov 04 01:29:43.223662 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:43.223526    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0e8cee1f4946b2b00040a0af013b5fc0187f86d45cb571c1d2e02b00da992f6"} err="failed to get container status \"d0e8cee1f4946b2b00040a0af013b5fc0187f86d45cb571c1d2e02b00da992f6\": rpc error: code = NotFound desc = could not find container \"d0e8cee1f4946b2b00040a0af013b5fc0187f86d45cb571c1d2e02b00da992f6\": container with ID starting with d0e8cee1f4946b2b00040a0af013b5fc0187f86d45cb571c1d2e02b00da992f6 not found: ID does not exist"
Nov 04 01:29:43.255688 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:43.254732    5068 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60f5c87a-48f8-4ad9-847b-cff571cc9340-tmp-dir\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:43.255688 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:43.254765    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jpnkh\" (UniqueName: \"kubernetes.io/projected/60f5c87a-48f8-4ad9-847b-cff571cc9340-kube-api-access-jpnkh\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:43.255688 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:43.254779    5068 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60f5c87a-48f8-4ad9-847b-cff571cc9340-dshm\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:43.255688 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:43.254793    5068 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60f5c87a-48f8-4ad9-847b-cff571cc9340-tls-certs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:43.273793 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:43.273731    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5"]
Nov 04 01:29:43.276621 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:43.276587    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-a5115a/simulator-w3-a5115a-kserve-569c5c47bd-wrkv5"]
Nov 04 01:29:43.475673 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:29:43.472953    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:29:44.113577 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:44.113542    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="60f5c87a-48f8-4ad9-847b-cff571cc9340" path="/var/lib/kubelet/pods/60f5c87a-48f8-4ad9-847b-cff571cc9340/volumes"
Nov 04 01:29:44.208933 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:44.208895    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds" event={"ID":"d091cbea-bbd1-4242-a1c9-f72628c8e019","Type":"ContainerStarted","Data":"83ccac7852dbaeb0e4eb4548000acaa575b366d9683b7fddd5d982ff86117b82"}
Nov 04 01:29:44.235642 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:44.235555    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds" podStartSLOduration=2.235536783 podStartE2EDuration="2.235536783s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:29:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:29:44.232205461 +0000 UTC m=+1624.278766260" watchObservedRunningTime="2026-11-04 01:29:44.235536783 +0000 UTC m=+1624.282097573"
Nov 04 01:29:44.742811 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:44.742770    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds"
Nov 04 01:29:44.748779 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:44.748750    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds"
Nov 04 01:29:45.215942 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:45.215910    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds"
Nov 04 01:29:45.217373 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:45.217347    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds"
Nov 04 01:29:48.050701 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:48.050661    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p"]
Nov 04 01:29:48.051249 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:48.051232    5068 cpu_manager.go:414] "containerMap: removing container" podUID="60f5c87a-48f8-4ad9-847b-cff571cc9340" containerName="main"
Nov 04 01:29:48.051297 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:48.051250    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="60f5c87a-48f8-4ad9-847b-cff571cc9340" containerName="main"
Nov 04 01:29:48.051350 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:48.051340    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="60f5c87a-48f8-4ad9-847b-cff571cc9340" containerName="main"
Nov 04 01:29:48.056144 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:48.056119    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p"
Nov 04 01:29:48.067663 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:48.067635    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p"]
Nov 04 01:29:48.132335 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:48.132302    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-8974fa-openshift-default-dockercfg-h4tg2\""
Nov 04 01:29:48.154108 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:48.154069    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-credential-socket\") pod \"e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p\" (UID: \"332ac8c5-1996-489e-8a9c-3efcdc3e9ce5\") " pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p"
Nov 04 01:29:48.164854 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:48.164803    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-istio-podinfo\") pod \"e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p\" (UID: \"332ac8c5-1996-489e-8a9c-3efcdc3e9ce5\") " pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p"
Nov 04 01:29:48.189925 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:48.189893    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-workload-socket\") pod \"e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p\" (UID: \"332ac8c5-1996-489e-8a9c-3efcdc3e9ce5\") " pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p"
Nov 04 01:29:48.193756 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:48.190211    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-istio-data\") pod \"e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p\" (UID: \"332ac8c5-1996-489e-8a9c-3efcdc3e9ce5\") " pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p"
Nov 04 01:29:48.210097 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:48.210064    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-istio-envoy\") pod \"e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p\" (UID: \"332ac8c5-1996-489e-8a9c-3efcdc3e9ce5\") " pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p"
Nov 04 01:29:48.227066 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:48.227019    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-istiod-ca-cert\") pod \"e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p\" (UID: \"332ac8c5-1996-489e-8a9c-3efcdc3e9ce5\") " pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p"
Nov 04 01:29:48.230729 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:48.228030    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-istio-token\") pod \"e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p\" (UID: \"332ac8c5-1996-489e-8a9c-3efcdc3e9ce5\") " pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p"
Nov 04 01:29:48.231372 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:48.231330    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqgxd\" (UniqueName: \"kubernetes.io/projected/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-kube-api-access-dqgxd\") pod \"e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p\" (UID: \"332ac8c5-1996-489e-8a9c-3efcdc3e9ce5\") " pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p"
Nov 04 01:29:48.231686 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:48.231663    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-workload-certs\") pod \"e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p\" (UID: \"332ac8c5-1996-489e-8a9c-3efcdc3e9ce5\") " pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p"
Nov 04 01:29:48.333057 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:48.332906    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-istio-podinfo\") pod \"e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p\" (UID: \"332ac8c5-1996-489e-8a9c-3efcdc3e9ce5\") " pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p"
Nov 04 01:29:48.333335 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:48.333308    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-workload-socket\") pod \"e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p\" (UID: \"332ac8c5-1996-489e-8a9c-3efcdc3e9ce5\") " pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p"
Nov 04 01:29:48.333503 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:48.333370    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-istio-data\") pod \"e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p\" (UID: \"332ac8c5-1996-489e-8a9c-3efcdc3e9ce5\") " pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p"
Nov 04 01:29:48.333503 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:48.333423    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-istio-envoy\") pod \"e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p\" (UID: \"332ac8c5-1996-489e-8a9c-3efcdc3e9ce5\") " pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p"
Nov 04 01:29:48.333503 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:48.333467    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-istiod-ca-cert\") pod \"e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p\" (UID: \"332ac8c5-1996-489e-8a9c-3efcdc3e9ce5\") " pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p"
Nov 04 01:29:48.333742 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:48.333549    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-istio-token\") pod \"e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p\" (UID: \"332ac8c5-1996-489e-8a9c-3efcdc3e9ce5\") " pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p"
Nov 04 01:29:48.333742 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:48.333603    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqgxd\" (UniqueName: \"kubernetes.io/projected/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-kube-api-access-dqgxd\") pod \"e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p\" (UID: \"332ac8c5-1996-489e-8a9c-3efcdc3e9ce5\") " pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p"
Nov 04 01:29:48.333742 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:48.333671    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-workload-certs\") pod \"e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p\" (UID: \"332ac8c5-1996-489e-8a9c-3efcdc3e9ce5\") " pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p"
Nov 04 01:29:48.333742 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:48.333726    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-credential-socket\") pod \"e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p\" (UID: \"332ac8c5-1996-489e-8a9c-3efcdc3e9ce5\") " pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p"
Nov 04 01:29:48.333890 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:48.333777    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-istio-data\") pod \"e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p\" (UID: \"332ac8c5-1996-489e-8a9c-3efcdc3e9ce5\") " pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p"
Nov 04 01:29:48.333890 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:48.333814    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-workload-socket\") pod \"e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p\" (UID: \"332ac8c5-1996-489e-8a9c-3efcdc3e9ce5\") " pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p"
Nov 04 01:29:48.334074 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:48.334048    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-credential-socket\") pod \"e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p\" (UID: \"332ac8c5-1996-489e-8a9c-3efcdc3e9ce5\") " pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p"
Nov 04 01:29:48.334285 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:48.334256    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-workload-certs\") pod \"e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p\" (UID: \"332ac8c5-1996-489e-8a9c-3efcdc3e9ce5\") " pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p"
Nov 04 01:29:48.334582 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:48.334562    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-istiod-ca-cert\") pod \"e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p\" (UID: \"332ac8c5-1996-489e-8a9c-3efcdc3e9ce5\") " pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p"
Nov 04 01:29:48.336098 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:48.336079    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-istio-envoy\") pod \"e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p\" (UID: \"332ac8c5-1996-489e-8a9c-3efcdc3e9ce5\") " pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p"
Nov 04 01:29:48.336343 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:48.336318    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-istio-podinfo\") pod \"e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p\" (UID: \"332ac8c5-1996-489e-8a9c-3efcdc3e9ce5\") " pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p"
Nov 04 01:29:48.347351 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:48.347323    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqgxd\" (UniqueName: \"kubernetes.io/projected/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-kube-api-access-dqgxd\") pod \"e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p\" (UID: \"332ac8c5-1996-489e-8a9c-3efcdc3e9ce5\") " pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p"
Nov 04 01:29:48.347842 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:48.347819    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-istio-token\") pod \"e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p\" (UID: \"332ac8c5-1996-489e-8a9c-3efcdc3e9ce5\") " pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p"
Nov 04 01:29:48.374163 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:48.374122    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p"
Nov 04 01:29:48.663863 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:48.663831    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p"]
Nov 04 01:29:48.664799 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:29:48.664749    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod332ac8c5_1996_489e_8a9c_3efcdc3e9ce5.slice/crio-c57cb378f3e12feda9da3149c056ae9e3bb981f60feffa3a77e374e809376781 WatchSource:0}: Error finding container c57cb378f3e12feda9da3149c056ae9e3bb981f60feffa3a77e374e809376781: Status 404 returned error can't find the container with id c57cb378f3e12feda9da3149c056ae9e3bb981f60feffa3a77e374e809376781
Nov 04 01:29:48.667527 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:48.667493    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:29:48.667662 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:48.667568    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:29:48.667662 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:48.667647    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:29:49.280314 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:49.280278    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p" event={"ID":"332ac8c5-1996-489e-8a9c-3efcdc3e9ce5","Type":"ContainerStarted","Data":"e537136d60705f76291e4528a66c0e723533a1043618eecaba0687ef47fb1030"}
Nov 04 01:29:49.280314 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:49.280330    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p" event={"ID":"332ac8c5-1996-489e-8a9c-3efcdc3e9ce5","Type":"ContainerStarted","Data":"c57cb378f3e12feda9da3149c056ae9e3bb981f60feffa3a77e374e809376781"}
Nov 04 01:29:49.307353 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:49.307302    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p" podStartSLOduration=1.307288785 podStartE2EDuration="1.307288785s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:29:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:29:49.30368733 +0000 UTC m=+1629.350248122" watchObservedRunningTime="2026-11-04 01:29:49.307288785 +0000 UTC m=+1629.353849575"
Nov 04 01:29:49.375634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:49.375293    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p"
Nov 04 01:29:49.377014 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:49.376983    5068 patch_prober.go:29] interesting pod/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.125:15021/healthz/ready\": dial tcp 10.132.0.125:15021: connect: connection refused" start-of-body=
Nov 04 01:29:49.377157 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:49.377037    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p" podUID="332ac8c5-1996-489e-8a9c-3efcdc3e9ce5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.125:15021/healthz/ready\": dial tcp 10.132.0.125:15021: connect: connection refused"
Nov 04 01:29:50.374718 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:50.374674    5068 patch_prober.go:29] interesting pod/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.125:15021/healthz/ready\": dial tcp 10.132.0.125:15021: connect: connection refused" start-of-body=
Nov 04 01:29:50.375255 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:50.374736    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p" podUID="332ac8c5-1996-489e-8a9c-3efcdc3e9ce5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.125:15021/healthz/ready\": dial tcp 10.132.0.125:15021: connect: connection refused"
Nov 04 01:29:51.375899 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:51.375865    5068 patch_prober.go:29] interesting pod/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.125:15021/healthz/ready\": dial tcp 10.132.0.125:15021: connect: connection refused" start-of-body=
Nov 04 01:29:51.376353 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:51.375920    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p" podUID="332ac8c5-1996-489e-8a9c-3efcdc3e9ce5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.125:15021/healthz/ready\": dial tcp 10.132.0.125:15021: connect: connection refused"
Nov 04 01:29:52.122861 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:52.122735    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-a5115a-8754c4f46-g677p"]
Nov 04 01:29:52.123369 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:52.123308    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-a5115a-8754c4f46-g677p" podUID="2ccb4aaa-fcb8-4797-8295-9e8521de235b" containerName="maas-api" containerID="cri-o://bfde4d33a227dce129a9bb2a790e1597fc69017131f4ec9dc5e342a600ad2a96" gracePeriod=30
Nov 04 01:29:52.358016 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:52.357975    5068 generic.go:356] "Generic (PLEG): container finished" podID="2ccb4aaa-fcb8-4797-8295-9e8521de235b" containerID="bfde4d33a227dce129a9bb2a790e1597fc69017131f4ec9dc5e342a600ad2a96" exitCode=0
Nov 04 01:29:52.358375 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:52.358350    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-a5115a-8754c4f46-g677p" event={"ID":"2ccb4aaa-fcb8-4797-8295-9e8521de235b","Type":"ContainerDied","Data":"bfde4d33a227dce129a9bb2a790e1597fc69017131f4ec9dc5e342a600ad2a96"}
Nov 04 01:29:52.375191 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:52.375094    5068 patch_prober.go:29] interesting pod/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.125:15021/healthz/ready\": dial tcp 10.132.0.125:15021: connect: connection refused" start-of-body=
Nov 04 01:29:52.375503 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:52.375475    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p" podUID="332ac8c5-1996-489e-8a9c-3efcdc3e9ce5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.125:15021/healthz/ready\": dial tcp 10.132.0.125:15021: connect: connection refused"
Nov 04 01:29:52.687152 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:52.686774    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-a5115a-8754c4f46-g677p"
Nov 04 01:29:52.884427 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:52.884385    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2ccb4aaa-fcb8-4797-8295-9e8521de235b-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ccb4aaa-fcb8-4797-8295-9e8521de235b-openshift-service-ca\") pod \"2ccb4aaa-fcb8-4797-8295-9e8521de235b\" (UID: \"2ccb4aaa-fcb8-4797-8295-9e8521de235b\") "
Nov 04 01:29:52.884427 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:52.884432    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2ccb4aaa-fcb8-4797-8295-9e8521de235b-kube-api-access-wbgjn\" (UniqueName: \"kubernetes.io/projected/2ccb4aaa-fcb8-4797-8295-9e8521de235b-kube-api-access-wbgjn\") pod \"2ccb4aaa-fcb8-4797-8295-9e8521de235b\" (UID: \"2ccb4aaa-fcb8-4797-8295-9e8521de235b\") "
Nov 04 01:29:52.884740 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:52.884465    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2ccb4aaa-fcb8-4797-8295-9e8521de235b-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ccb4aaa-fcb8-4797-8295-9e8521de235b-maas-api-tls\") pod \"2ccb4aaa-fcb8-4797-8295-9e8521de235b\" (UID: \"2ccb4aaa-fcb8-4797-8295-9e8521de235b\") "
Nov 04 01:29:52.884740 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:52.884583    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2ccb4aaa-fcb8-4797-8295-9e8521de235b-metrics-certs\" (UniqueName: \"kubernetes.io/secret/2ccb4aaa-fcb8-4797-8295-9e8521de235b-metrics-certs\") pod \"2ccb4aaa-fcb8-4797-8295-9e8521de235b\" (UID: \"2ccb4aaa-fcb8-4797-8295-9e8521de235b\") "
Nov 04 01:29:52.884867 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:52.884838    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ccb4aaa-fcb8-4797-8295-9e8521de235b-openshift-service-ca" pod "2ccb4aaa-fcb8-4797-8295-9e8521de235b" (UID: "2ccb4aaa-fcb8-4797-8295-9e8521de235b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:29:52.885133 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:52.885108    5068 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ccb4aaa-fcb8-4797-8295-9e8521de235b-openshift-service-ca\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:52.888035 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:52.886783    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ccb4aaa-fcb8-4797-8295-9e8521de235b-kube-api-access-wbgjn" pod "2ccb4aaa-fcb8-4797-8295-9e8521de235b" (UID: "2ccb4aaa-fcb8-4797-8295-9e8521de235b"). InnerVolumeSpecName "kube-api-access-wbgjn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:29:52.888035 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:52.886902    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ccb4aaa-fcb8-4797-8295-9e8521de235b-metrics-certs" pod "2ccb4aaa-fcb8-4797-8295-9e8521de235b" (UID: "2ccb4aaa-fcb8-4797-8295-9e8521de235b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:29:52.888815 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:52.888319    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ccb4aaa-fcb8-4797-8295-9e8521de235b-maas-api-tls" pod "2ccb4aaa-fcb8-4797-8295-9e8521de235b" (UID: "2ccb4aaa-fcb8-4797-8295-9e8521de235b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:29:52.985913 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:52.985834    5068 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2ccb4aaa-fcb8-4797-8295-9e8521de235b-metrics-certs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:52.985913 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:52.985866    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wbgjn\" (UniqueName: \"kubernetes.io/projected/2ccb4aaa-fcb8-4797-8295-9e8521de235b-kube-api-access-wbgjn\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:52.985913 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:52.985876    5068 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ccb4aaa-fcb8-4797-8295-9e8521de235b-maas-api-tls\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:53.374711 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:53.374678    5068 patch_prober.go:29] interesting pod/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.125:15021/healthz/ready\": dial tcp 10.132.0.125:15021: connect: connection refused" start-of-body=
Nov 04 01:29:53.374929 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:53.374735    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p" podUID="332ac8c5-1996-489e-8a9c-3efcdc3e9ce5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.125:15021/healthz/ready\": dial tcp 10.132.0.125:15021: connect: connection refused"
Nov 04 01:29:53.377664 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:53.377641    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-a5115a-8754c4f46-g677p"
Nov 04 01:29:53.377805 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:53.377666    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-a5115a-8754c4f46-g677p" event={"ID":"2ccb4aaa-fcb8-4797-8295-9e8521de235b","Type":"ContainerDied","Data":"533a36ee81bde5d3a6726c54b3c69ce6c93b7a6fec9b8e6a98d478e8d3f91e5b"}
Nov 04 01:29:53.377805 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:53.377736    5068 scope.go:122] "RemoveContainer" containerID="bfde4d33a227dce129a9bb2a790e1597fc69017131f4ec9dc5e342a600ad2a96"
Nov 04 01:29:53.413707 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:53.413669    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-a5115a-8754c4f46-g677p"]
Nov 04 01:29:53.430542 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:53.430505    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-a5115a-8754c4f46-g677p"]
Nov 04 01:29:54.112657 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:54.112624    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2ccb4aaa-fcb8-4797-8295-9e8521de235b" path="/var/lib/kubelet/pods/2ccb4aaa-fcb8-4797-8295-9e8521de235b/volumes"
Nov 04 01:29:54.376221 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:54.376140    5068 patch_prober.go:29] interesting pod/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.125:15021/healthz/ready\": dial tcp 10.132.0.125:15021: connect: connection refused" start-of-body=
Nov 04 01:29:54.376457 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:54.376430    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p" podUID="332ac8c5-1996-489e-8a9c-3efcdc3e9ce5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.125:15021/healthz/ready\": dial tcp 10.132.0.125:15021: connect: connection refused"
Nov 04 01:29:54.584528 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:54.581746    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-a362b6/test-model-a362b6-kserve-68474d95dc-xhrtw"]
Nov 04 01:29:54.584528 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:54.582595    5068 cpu_manager.go:414] "containerMap: removing container" podUID="2ccb4aaa-fcb8-4797-8295-9e8521de235b" containerName="maas-api"
Nov 04 01:29:54.584528 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:54.582654    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2ccb4aaa-fcb8-4797-8295-9e8521de235b" containerName="maas-api"
Nov 04 01:29:54.584528 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:54.582804    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="2ccb4aaa-fcb8-4797-8295-9e8521de235b" containerName="maas-api"
Nov 04 01:29:54.588215 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:54.588177    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-a362b6/test-model-a362b6-kserve-68474d95dc-xhrtw"]
Nov 04 01:29:54.588396 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:54.588333    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a362b6/test-model-a362b6-kserve-68474d95dc-xhrtw"
Nov 04 01:29:54.601233 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:54.601209    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-a362b6\"/\"default-dockercfg-wdn75\""
Nov 04 01:29:54.601472 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:54.601457    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-a362b6\"/\"test-model-a362b6-kserve-self-signed-certs\""
Nov 04 01:29:54.610949 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:54.610921    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c58619a-e60c-420d-a1b8-e1e54d7dce0e-tls-certs\") pod \"test-model-a362b6-kserve-68474d95dc-xhrtw\" (UID: \"7c58619a-e60c-420d-a1b8-e1e54d7dce0e\") " pod="ai-tenant-e2e-inf-a-a362b6/test-model-a362b6-kserve-68474d95dc-xhrtw"
Nov 04 01:29:54.611177 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:54.611154    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c58619a-e60c-420d-a1b8-e1e54d7dce0e-tmp-dir\") pod \"test-model-a362b6-kserve-68474d95dc-xhrtw\" (UID: \"7c58619a-e60c-420d-a1b8-e1e54d7dce0e\") " pod="ai-tenant-e2e-inf-a-a362b6/test-model-a362b6-kserve-68474d95dc-xhrtw"
Nov 04 01:29:54.611326 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:54.611309    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c58619a-e60c-420d-a1b8-e1e54d7dce0e-dshm\") pod \"test-model-a362b6-kserve-68474d95dc-xhrtw\" (UID: \"7c58619a-e60c-420d-a1b8-e1e54d7dce0e\") " pod="ai-tenant-e2e-inf-a-a362b6/test-model-a362b6-kserve-68474d95dc-xhrtw"
Nov 04 01:29:54.611465 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:54.611451    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c58619a-e60c-420d-a1b8-e1e54d7dce0e-home\") pod \"test-model-a362b6-kserve-68474d95dc-xhrtw\" (UID: \"7c58619a-e60c-420d-a1b8-e1e54d7dce0e\") " pod="ai-tenant-e2e-inf-a-a362b6/test-model-a362b6-kserve-68474d95dc-xhrtw"
Nov 04 01:29:54.611561 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:54.611548    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c58619a-e60c-420d-a1b8-e1e54d7dce0e-model-cache\") pod \"test-model-a362b6-kserve-68474d95dc-xhrtw\" (UID: \"7c58619a-e60c-420d-a1b8-e1e54d7dce0e\") " pod="ai-tenant-e2e-inf-a-a362b6/test-model-a362b6-kserve-68474d95dc-xhrtw"
Nov 04 01:29:54.611807 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:54.611774    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5d6jz\" (UniqueName: \"kubernetes.io/projected/7c58619a-e60c-420d-a1b8-e1e54d7dce0e-kube-api-access-5d6jz\") pod \"test-model-a362b6-kserve-68474d95dc-xhrtw\" (UID: \"7c58619a-e60c-420d-a1b8-e1e54d7dce0e\") " pod="ai-tenant-e2e-inf-a-a362b6/test-model-a362b6-kserve-68474d95dc-xhrtw"
Nov 04 01:29:54.655174 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:54.655080    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-a362b6\"/\"openshift-service-ca.crt\""
Nov 04 01:29:54.655627 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:54.655578    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-a362b6\"/\"kube-root-ca.crt\""
Nov 04 01:29:54.712999 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:54.712962    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c58619a-e60c-420d-a1b8-e1e54d7dce0e-tls-certs\") pod \"test-model-a362b6-kserve-68474d95dc-xhrtw\" (UID: \"7c58619a-e60c-420d-a1b8-e1e54d7dce0e\") " pod="ai-tenant-e2e-inf-a-a362b6/test-model-a362b6-kserve-68474d95dc-xhrtw"
Nov 04 01:29:54.713203 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:54.713028    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c58619a-e60c-420d-a1b8-e1e54d7dce0e-tmp-dir\") pod \"test-model-a362b6-kserve-68474d95dc-xhrtw\" (UID: \"7c58619a-e60c-420d-a1b8-e1e54d7dce0e\") " pod="ai-tenant-e2e-inf-a-a362b6/test-model-a362b6-kserve-68474d95dc-xhrtw"
Nov 04 01:29:54.713203 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:54.713077    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c58619a-e60c-420d-a1b8-e1e54d7dce0e-dshm\") pod \"test-model-a362b6-kserve-68474d95dc-xhrtw\" (UID: \"7c58619a-e60c-420d-a1b8-e1e54d7dce0e\") " pod="ai-tenant-e2e-inf-a-a362b6/test-model-a362b6-kserve-68474d95dc-xhrtw"
Nov 04 01:29:54.713203 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:54.713127    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c58619a-e60c-420d-a1b8-e1e54d7dce0e-home\") pod \"test-model-a362b6-kserve-68474d95dc-xhrtw\" (UID: \"7c58619a-e60c-420d-a1b8-e1e54d7dce0e\") " pod="ai-tenant-e2e-inf-a-a362b6/test-model-a362b6-kserve-68474d95dc-xhrtw"
Nov 04 01:29:54.713203 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:54.713157    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c58619a-e60c-420d-a1b8-e1e54d7dce0e-model-cache\") pod \"test-model-a362b6-kserve-68474d95dc-xhrtw\" (UID: \"7c58619a-e60c-420d-a1b8-e1e54d7dce0e\") " pod="ai-tenant-e2e-inf-a-a362b6/test-model-a362b6-kserve-68474d95dc-xhrtw"
Nov 04 01:29:54.713203 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:54.713197    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5d6jz\" (UniqueName: \"kubernetes.io/projected/7c58619a-e60c-420d-a1b8-e1e54d7dce0e-kube-api-access-5d6jz\") pod \"test-model-a362b6-kserve-68474d95dc-xhrtw\" (UID: \"7c58619a-e60c-420d-a1b8-e1e54d7dce0e\") " pod="ai-tenant-e2e-inf-a-a362b6/test-model-a362b6-kserve-68474d95dc-xhrtw"
Nov 04 01:29:54.713565 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:54.713537    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c58619a-e60c-420d-a1b8-e1e54d7dce0e-tmp-dir\") pod \"test-model-a362b6-kserve-68474d95dc-xhrtw\" (UID: \"7c58619a-e60c-420d-a1b8-e1e54d7dce0e\") " pod="ai-tenant-e2e-inf-a-a362b6/test-model-a362b6-kserve-68474d95dc-xhrtw"
Nov 04 01:29:54.713828 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:54.713805    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c58619a-e60c-420d-a1b8-e1e54d7dce0e-home\") pod \"test-model-a362b6-kserve-68474d95dc-xhrtw\" (UID: \"7c58619a-e60c-420d-a1b8-e1e54d7dce0e\") " pod="ai-tenant-e2e-inf-a-a362b6/test-model-a362b6-kserve-68474d95dc-xhrtw"
Nov 04 01:29:54.714405 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:54.714379    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c58619a-e60c-420d-a1b8-e1e54d7dce0e-model-cache\") pod \"test-model-a362b6-kserve-68474d95dc-xhrtw\" (UID: \"7c58619a-e60c-420d-a1b8-e1e54d7dce0e\") " pod="ai-tenant-e2e-inf-a-a362b6/test-model-a362b6-kserve-68474d95dc-xhrtw"
Nov 04 01:29:54.716880 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:54.716856    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c58619a-e60c-420d-a1b8-e1e54d7dce0e-tls-certs\") pod \"test-model-a362b6-kserve-68474d95dc-xhrtw\" (UID: \"7c58619a-e60c-420d-a1b8-e1e54d7dce0e\") " pod="ai-tenant-e2e-inf-a-a362b6/test-model-a362b6-kserve-68474d95dc-xhrtw"
Nov 04 01:29:54.716987 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:54.716865    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c58619a-e60c-420d-a1b8-e1e54d7dce0e-dshm\") pod \"test-model-a362b6-kserve-68474d95dc-xhrtw\" (UID: \"7c58619a-e60c-420d-a1b8-e1e54d7dce0e\") " pod="ai-tenant-e2e-inf-a-a362b6/test-model-a362b6-kserve-68474d95dc-xhrtw"
Nov 04 01:29:54.726116 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:54.726088    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5d6jz\" (UniqueName: \"kubernetes.io/projected/7c58619a-e60c-420d-a1b8-e1e54d7dce0e-kube-api-access-5d6jz\") pod \"test-model-a362b6-kserve-68474d95dc-xhrtw\" (UID: \"7c58619a-e60c-420d-a1b8-e1e54d7dce0e\") " pod="ai-tenant-e2e-inf-a-a362b6/test-model-a362b6-kserve-68474d95dc-xhrtw"
Nov 04 01:29:54.908897 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:54.908801    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a362b6/test-model-a362b6-kserve-68474d95dc-xhrtw"
Nov 04 01:29:55.232885 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:55.231789    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-a362b6/test-model-a362b6-kserve-68474d95dc-xhrtw"]
Nov 04 01:29:55.380074 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:55.379910    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p"
Nov 04 01:29:55.381338 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:55.381316    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p"
Nov 04 01:29:55.382484 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:55.382314    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p"
Nov 04 01:29:55.420778 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:55.420746    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a362b6/test-model-a362b6-kserve-68474d95dc-xhrtw" event={"ID":"7c58619a-e60c-420d-a1b8-e1e54d7dce0e","Type":"ContainerStarted","Data":"d363be720088ee8941a834ea37b924a7b66ae0225556fbc6c28e4fae38c26071"}
Nov 04 01:29:55.420930 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:55.420788    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a362b6/test-model-a362b6-kserve-68474d95dc-xhrtw" event={"ID":"7c58619a-e60c-420d-a1b8-e1e54d7dce0e","Type":"ContainerStarted","Data":"b290996edcd5e0d1981beddab75e9e2f3db1fff020a994d7fe212b551e6da84e"}
Nov 04 01:29:55.470631 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:55.468056    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-a362b6/test-model-a362b6-kserve-68474d95dc-xhrtw" podStartSLOduration=1.468036338 podStartE2EDuration="1.468036338s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:29:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:29:55.46326452 +0000 UTC m=+1635.509825325" watchObservedRunningTime="2026-11-04 01:29:55.468036338 +0000 UTC m=+1635.514597129"
Nov 04 01:29:55.909412 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:55.909379    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-a362b6/test-model-a362b6-kserve-68474d95dc-xhrtw"
Nov 04 01:29:56.344656 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:29:56.344623    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:30:00.147110 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:00.147078    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-5ce9e8-29895930-f4rf2"]
Nov 04 01:30:00.151133 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:00.151109    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-5ce9e8-29895930-f4rf2"
Nov 04 01:30:00.165124 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:00.165089    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-5ce9e8-29895930-f4rf2"]
Nov 04 01:30:00.178839 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:00.178810    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-362631-29895930-snfmr"]
Nov 04 01:30:00.191259 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:00.191234    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-a362b6-29895930-2fqh4"]
Nov 04 01:30:00.191892 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:00.191867    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-362631-29895930-snfmr"
Nov 04 01:30:00.214289 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:00.214256    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96h6w\" (UniqueName: \"kubernetes.io/projected/c6ddf85e-cd63-46b0-9f07-f6c93d158181-kube-api-access-96h6w\") pod \"maas-api-key-cleanup-e2e-shared-b-w1-5ce9e8-29895930-f4rf2\" (UID: \"c6ddf85e-cd63-46b0-9f07-f6c93d158181\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-5ce9e8-29895930-f4rf2"
Nov 04 01:30:00.214460 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:00.214341    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6ddf85e-cd63-46b0-9f07-f6c93d158181-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w1-5ce9e8-29895930-f4rf2\" (UID: \"c6ddf85e-cd63-46b0-9f07-f6c93d158181\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-5ce9e8-29895930-f4rf2"
Nov 04 01:30:00.222881 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:00.222856    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-362631-29895930-snfmr"]
Nov 04 01:30:00.223076 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:00.223051    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-a362b6-29895930-2fqh4"]
Nov 04 01:30:00.223165 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:00.222946    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-a362b6-29895930-2fqh4"
Nov 04 01:30:00.265372 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:00.265342    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qdgjq\""
Nov 04 01:30:00.315330 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:00.315298    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-824pn\" (UniqueName: \"kubernetes.io/projected/78c3da50-fc89-4b70-ae04-79c623854724-kube-api-access-824pn\") pod \"maas-api-key-cleanup-e2e-inf-b-362631-29895930-snfmr\" (UID: \"78c3da50-fc89-4b70-ae04-79c623854724\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-362631-29895930-snfmr"
Nov 04 01:30:00.315497 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:00.315335    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78c3da50-fc89-4b70-ae04-79c623854724-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-b-362631-29895930-snfmr\" (UID: \"78c3da50-fc89-4b70-ae04-79c623854724\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-362631-29895930-snfmr"
Nov 04 01:30:00.315497 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:00.315367    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/343b7e45-62f4-4533-a07c-0732c85a5b32-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-a-a362b6-29895930-2fqh4\" (UID: \"343b7e45-62f4-4533-a07c-0732c85a5b32\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-a362b6-29895930-2fqh4"
Nov 04 01:30:00.315497 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:00.315483    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96h6w\" (UniqueName: \"kubernetes.io/projected/c6ddf85e-cd63-46b0-9f07-f6c93d158181-kube-api-access-96h6w\") pod \"maas-api-key-cleanup-e2e-shared-b-w1-5ce9e8-29895930-f4rf2\" (UID: \"c6ddf85e-cd63-46b0-9f07-f6c93d158181\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-5ce9e8-29895930-f4rf2"
Nov 04 01:30:00.315665 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:00.315634    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lnqf\" (UniqueName: \"kubernetes.io/projected/343b7e45-62f4-4533-a07c-0732c85a5b32-kube-api-access-6lnqf\") pod \"maas-api-key-cleanup-e2e-inf-a-a362b6-29895930-2fqh4\" (UID: \"343b7e45-62f4-4533-a07c-0732c85a5b32\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-a362b6-29895930-2fqh4"
Nov 04 01:30:00.315754 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:00.315725    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6ddf85e-cd63-46b0-9f07-f6c93d158181-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w1-5ce9e8-29895930-f4rf2\" (UID: \"c6ddf85e-cd63-46b0-9f07-f6c93d158181\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-5ce9e8-29895930-f4rf2"
Nov 04 01:30:00.316312 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:00.316294    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6ddf85e-cd63-46b0-9f07-f6c93d158181-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w1-5ce9e8-29895930-f4rf2\" (UID: \"c6ddf85e-cd63-46b0-9f07-f6c93d158181\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-5ce9e8-29895930-f4rf2"
Nov 04 01:30:00.327845 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:00.327821    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-96h6w\" (UniqueName: \"kubernetes.io/projected/c6ddf85e-cd63-46b0-9f07-f6c93d158181-kube-api-access-96h6w\") pod \"maas-api-key-cleanup-e2e-shared-b-w1-5ce9e8-29895930-f4rf2\" (UID: \"c6ddf85e-cd63-46b0-9f07-f6c93d158181\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-5ce9e8-29895930-f4rf2"
Nov 04 01:30:00.417558 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:00.417460    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-824pn\" (UniqueName: \"kubernetes.io/projected/78c3da50-fc89-4b70-ae04-79c623854724-kube-api-access-824pn\") pod \"maas-api-key-cleanup-e2e-inf-b-362631-29895930-snfmr\" (UID: \"78c3da50-fc89-4b70-ae04-79c623854724\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-362631-29895930-snfmr"
Nov 04 01:30:00.417558 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:00.417522    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78c3da50-fc89-4b70-ae04-79c623854724-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-b-362631-29895930-snfmr\" (UID: \"78c3da50-fc89-4b70-ae04-79c623854724\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-362631-29895930-snfmr"
Nov 04 01:30:00.417558 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:00.417554    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/343b7e45-62f4-4533-a07c-0732c85a5b32-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-a-a362b6-29895930-2fqh4\" (UID: \"343b7e45-62f4-4533-a07c-0732c85a5b32\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-a362b6-29895930-2fqh4"
Nov 04 01:30:00.417845 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:00.417657    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lnqf\" (UniqueName: \"kubernetes.io/projected/343b7e45-62f4-4533-a07c-0732c85a5b32-kube-api-access-6lnqf\") pod \"maas-api-key-cleanup-e2e-inf-a-a362b6-29895930-2fqh4\" (UID: \"343b7e45-62f4-4533-a07c-0732c85a5b32\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-a362b6-29895930-2fqh4"
Nov 04 01:30:00.418539 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:00.418289    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/343b7e45-62f4-4533-a07c-0732c85a5b32-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-a-a362b6-29895930-2fqh4\" (UID: \"343b7e45-62f4-4533-a07c-0732c85a5b32\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-a362b6-29895930-2fqh4"
Nov 04 01:30:00.418539 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:00.418363    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78c3da50-fc89-4b70-ae04-79c623854724-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-b-362631-29895930-snfmr\" (UID: \"78c3da50-fc89-4b70-ae04-79c623854724\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-362631-29895930-snfmr"
Nov 04 01:30:00.430078 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:00.430050    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lnqf\" (UniqueName: \"kubernetes.io/projected/343b7e45-62f4-4533-a07c-0732c85a5b32-kube-api-access-6lnqf\") pod \"maas-api-key-cleanup-e2e-inf-a-a362b6-29895930-2fqh4\" (UID: \"343b7e45-62f4-4533-a07c-0732c85a5b32\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-a362b6-29895930-2fqh4"
Nov 04 01:30:00.432251 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:00.432227    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-824pn\" (UniqueName: \"kubernetes.io/projected/78c3da50-fc89-4b70-ae04-79c623854724-kube-api-access-824pn\") pod \"maas-api-key-cleanup-e2e-inf-b-362631-29895930-snfmr\" (UID: \"78c3da50-fc89-4b70-ae04-79c623854724\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-362631-29895930-snfmr"
Nov 04 01:30:00.482007 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:00.481972    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-5ce9e8-29895930-f4rf2"
Nov 04 01:30:00.544865 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:00.544826    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-362631-29895930-snfmr"
Nov 04 01:30:00.564246 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:00.564179    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-a362b6-29895930-2fqh4"
Nov 04 01:30:00.778493 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:00.778458    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-5ce9e8-29895930-f4rf2"]
Nov 04 01:30:00.779041 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:30:00.779012    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6ddf85e_cd63_46b0_9f07_f6c93d158181.slice/crio-009cf0ee606065fc8063b15cfc5339a2f5e99741037bed03852a49aaa284ea19 WatchSource:0}: Error finding container 009cf0ee606065fc8063b15cfc5339a2f5e99741037bed03852a49aaa284ea19: Status 404 returned error can't find the container with id 009cf0ee606065fc8063b15cfc5339a2f5e99741037bed03852a49aaa284ea19
Nov 04 01:30:00.870735 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:30:00.870157    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78c3da50_fc89_4b70_ae04_79c623854724.slice/crio-a5c203623039a44f9c9e76e9cc91c42de26aeec8470309f438983ccb0e43bf22 WatchSource:0}: Error finding container a5c203623039a44f9c9e76e9cc91c42de26aeec8470309f438983ccb0e43bf22: Status 404 returned error can't find the container with id a5c203623039a44f9c9e76e9cc91c42de26aeec8470309f438983ccb0e43bf22
Nov 04 01:30:00.870735 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:00.870563    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-362631-29895930-snfmr"]
Nov 04 01:30:00.896221 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:00.896190    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-a362b6-29895930-2fqh4"]
Nov 04 01:30:00.897973 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:30:00.897945    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod343b7e45_62f4_4533_a07c_0732c85a5b32.slice/crio-bb9f0ebf64fbe12f736e23fcc746be097f09beac4f42d0ba91f6c8bbe61df15e WatchSource:0}: Error finding container bb9f0ebf64fbe12f736e23fcc746be097f09beac4f42d0ba91f6c8bbe61df15e: Status 404 returned error can't find the container with id bb9f0ebf64fbe12f736e23fcc746be097f09beac4f42d0ba91f6c8bbe61df15e
Nov 04 01:30:01.473833 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:01.473794    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-362631-29895930-snfmr" event={"ID":"78c3da50-fc89-4b70-ae04-79c623854724","Type":"ContainerStarted","Data":"069c42caffc7fa365352e38809530eaa30ab66fdbb3fb53492741163840d2fcc"}
Nov 04 01:30:01.474309 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:01.473858    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-362631-29895930-snfmr" event={"ID":"78c3da50-fc89-4b70-ae04-79c623854724","Type":"ContainerStarted","Data":"a5c203623039a44f9c9e76e9cc91c42de26aeec8470309f438983ccb0e43bf22"}
Nov 04 01:30:01.475548 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:01.475517    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-5ce9e8-29895930-f4rf2" event={"ID":"c6ddf85e-cd63-46b0-9f07-f6c93d158181","Type":"ContainerStarted","Data":"f5e8dcf8c8a7e34681b0e5009ecb604651c683ba039753a03653222239e1adb4"}
Nov 04 01:30:01.475742 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:01.475578    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-5ce9e8-29895930-f4rf2" event={"ID":"c6ddf85e-cd63-46b0-9f07-f6c93d158181","Type":"ContainerStarted","Data":"009cf0ee606065fc8063b15cfc5339a2f5e99741037bed03852a49aaa284ea19"}
Nov 04 01:30:01.477418 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:01.477391    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-a362b6-29895930-2fqh4" event={"ID":"343b7e45-62f4-4533-a07c-0732c85a5b32","Type":"ContainerStarted","Data":"377b882f04986dfe61c8e2f0298f6f5cbfb2e4b2c31a2fe16d33bdc3723081d1"}
Nov 04 01:30:01.477520 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:01.477440    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-a362b6-29895930-2fqh4" event={"ID":"343b7e45-62f4-4533-a07c-0732c85a5b32","Type":"ContainerStarted","Data":"bb9f0ebf64fbe12f736e23fcc746be097f09beac4f42d0ba91f6c8bbe61df15e"}
Nov 04 01:30:01.493383 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:01.493331    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-362631-29895930-snfmr" podStartSLOduration=1.493317801 podStartE2EDuration="1.493317801s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:30:01.489479172 +0000 UTC m=+1641.536039962" watchObservedRunningTime="2026-11-04 01:30:01.493317801 +0000 UTC m=+1641.539878590"
Nov 04 01:30:01.508591 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:01.508437    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-5ce9e8-29895930-f4rf2" podStartSLOduration=1.5084239560000001 podStartE2EDuration="1.508423956s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:30:01.504953108 +0000 UTC m=+1641.551513923" watchObservedRunningTime="2026-11-04 01:30:01.508423956 +0000 UTC m=+1641.554984745"
Nov 04 01:30:01.522591 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:01.522531    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-a362b6-29895930-2fqh4" podStartSLOduration=1.522514312 podStartE2EDuration="1.522514312s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:30:01.521429376 +0000 UTC m=+1641.567990166" watchObservedRunningTime="2026-11-04 01:30:01.522514312 +0000 UTC m=+1641.569075102"
Nov 04 01:30:02.235059 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:02.232037    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-362631/test-model-362631-kserve-59474b9c57-fqrxf"]
Nov 04 01:30:02.236219 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:02.236193    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-362631/test-model-362631-kserve-59474b9c57-fqrxf"
Nov 04 01:30:02.244758 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:02.244715    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-362631\"/\"openshift-service-ca.crt\""
Nov 04 01:30:02.244900 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:02.244806    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-362631\"/\"kube-root-ca.crt\""
Nov 04 01:30:02.246764 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:02.246334    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-362631\"/\"default-dockercfg-9zhct\""
Nov 04 01:30:02.246881 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:02.246862    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-362631\"/\"test-model-362631-kserve-self-signed-certs\""
Nov 04 01:30:02.252933 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:02.252911    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-362631/test-model-362631-kserve-59474b9c57-fqrxf"]
Nov 04 01:30:02.345523 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:02.345421    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a9762e4-1ac3-45c8-8324-c007bddda95f-tmp-dir\") pod \"test-model-362631-kserve-59474b9c57-fqrxf\" (UID: \"8a9762e4-1ac3-45c8-8324-c007bddda95f\") " pod="ai-tenant-e2e-inf-b-362631/test-model-362631-kserve-59474b9c57-fqrxf"
Nov 04 01:30:02.345523 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:02.345512    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a9762e4-1ac3-45c8-8324-c007bddda95f-tls-certs\") pod \"test-model-362631-kserve-59474b9c57-fqrxf\" (UID: \"8a9762e4-1ac3-45c8-8324-c007bddda95f\") " pod="ai-tenant-e2e-inf-b-362631/test-model-362631-kserve-59474b9c57-fqrxf"
Nov 04 01:30:02.345763 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:02.345565    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mx276\" (UniqueName: \"kubernetes.io/projected/8a9762e4-1ac3-45c8-8324-c007bddda95f-kube-api-access-mx276\") pod \"test-model-362631-kserve-59474b9c57-fqrxf\" (UID: \"8a9762e4-1ac3-45c8-8324-c007bddda95f\") " pod="ai-tenant-e2e-inf-b-362631/test-model-362631-kserve-59474b9c57-fqrxf"
Nov 04 01:30:02.345763 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:02.345629    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a9762e4-1ac3-45c8-8324-c007bddda95f-home\") pod \"test-model-362631-kserve-59474b9c57-fqrxf\" (UID: \"8a9762e4-1ac3-45c8-8324-c007bddda95f\") " pod="ai-tenant-e2e-inf-b-362631/test-model-362631-kserve-59474b9c57-fqrxf"
Nov 04 01:30:02.345763 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:02.345669    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a9762e4-1ac3-45c8-8324-c007bddda95f-model-cache\") pod \"test-model-362631-kserve-59474b9c57-fqrxf\" (UID: \"8a9762e4-1ac3-45c8-8324-c007bddda95f\") " pod="ai-tenant-e2e-inf-b-362631/test-model-362631-kserve-59474b9c57-fqrxf"
Nov 04 01:30:02.345876 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:02.345787    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a9762e4-1ac3-45c8-8324-c007bddda95f-dshm\") pod \"test-model-362631-kserve-59474b9c57-fqrxf\" (UID: \"8a9762e4-1ac3-45c8-8324-c007bddda95f\") " pod="ai-tenant-e2e-inf-b-362631/test-model-362631-kserve-59474b9c57-fqrxf"
Nov 04 01:30:02.457628 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:02.453428    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a9762e4-1ac3-45c8-8324-c007bddda95f-dshm\") pod \"test-model-362631-kserve-59474b9c57-fqrxf\" (UID: \"8a9762e4-1ac3-45c8-8324-c007bddda95f\") " pod="ai-tenant-e2e-inf-b-362631/test-model-362631-kserve-59474b9c57-fqrxf"
Nov 04 01:30:02.457628 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:02.454032    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a9762e4-1ac3-45c8-8324-c007bddda95f-tmp-dir\") pod \"test-model-362631-kserve-59474b9c57-fqrxf\" (UID: \"8a9762e4-1ac3-45c8-8324-c007bddda95f\") " pod="ai-tenant-e2e-inf-b-362631/test-model-362631-kserve-59474b9c57-fqrxf"
Nov 04 01:30:02.457628 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:02.454108    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a9762e4-1ac3-45c8-8324-c007bddda95f-tls-certs\") pod \"test-model-362631-kserve-59474b9c57-fqrxf\" (UID: \"8a9762e4-1ac3-45c8-8324-c007bddda95f\") " pod="ai-tenant-e2e-inf-b-362631/test-model-362631-kserve-59474b9c57-fqrxf"
Nov 04 01:30:02.457628 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:02.454168    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mx276\" (UniqueName: \"kubernetes.io/projected/8a9762e4-1ac3-45c8-8324-c007bddda95f-kube-api-access-mx276\") pod \"test-model-362631-kserve-59474b9c57-fqrxf\" (UID: \"8a9762e4-1ac3-45c8-8324-c007bddda95f\") " pod="ai-tenant-e2e-inf-b-362631/test-model-362631-kserve-59474b9c57-fqrxf"
Nov 04 01:30:02.457628 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:02.454218    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a9762e4-1ac3-45c8-8324-c007bddda95f-home\") pod \"test-model-362631-kserve-59474b9c57-fqrxf\" (UID: \"8a9762e4-1ac3-45c8-8324-c007bddda95f\") " pod="ai-tenant-e2e-inf-b-362631/test-model-362631-kserve-59474b9c57-fqrxf"
Nov 04 01:30:02.457628 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:02.454259    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a9762e4-1ac3-45c8-8324-c007bddda95f-model-cache\") pod \"test-model-362631-kserve-59474b9c57-fqrxf\" (UID: \"8a9762e4-1ac3-45c8-8324-c007bddda95f\") " pod="ai-tenant-e2e-inf-b-362631/test-model-362631-kserve-59474b9c57-fqrxf"
Nov 04 01:30:02.457628 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:02.454630    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a9762e4-1ac3-45c8-8324-c007bddda95f-model-cache\") pod \"test-model-362631-kserve-59474b9c57-fqrxf\" (UID: \"8a9762e4-1ac3-45c8-8324-c007bddda95f\") " pod="ai-tenant-e2e-inf-b-362631/test-model-362631-kserve-59474b9c57-fqrxf"
Nov 04 01:30:02.457628 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:02.454868    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a9762e4-1ac3-45c8-8324-c007bddda95f-tmp-dir\") pod \"test-model-362631-kserve-59474b9c57-fqrxf\" (UID: \"8a9762e4-1ac3-45c8-8324-c007bddda95f\") " pod="ai-tenant-e2e-inf-b-362631/test-model-362631-kserve-59474b9c57-fqrxf"
Nov 04 01:30:02.457628 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:02.455708    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a9762e4-1ac3-45c8-8324-c007bddda95f-home\") pod \"test-model-362631-kserve-59474b9c57-fqrxf\" (UID: \"8a9762e4-1ac3-45c8-8324-c007bddda95f\") " pod="ai-tenant-e2e-inf-b-362631/test-model-362631-kserve-59474b9c57-fqrxf"
Nov 04 01:30:02.476232 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:02.475463    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mx276\" (UniqueName: \"kubernetes.io/projected/8a9762e4-1ac3-45c8-8324-c007bddda95f-kube-api-access-mx276\") pod \"test-model-362631-kserve-59474b9c57-fqrxf\" (UID: \"8a9762e4-1ac3-45c8-8324-c007bddda95f\") " pod="ai-tenant-e2e-inf-b-362631/test-model-362631-kserve-59474b9c57-fqrxf"
Nov 04 01:30:02.512750 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:02.492883    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a9762e4-1ac3-45c8-8324-c007bddda95f-dshm\") pod \"test-model-362631-kserve-59474b9c57-fqrxf\" (UID: \"8a9762e4-1ac3-45c8-8324-c007bddda95f\") " pod="ai-tenant-e2e-inf-b-362631/test-model-362631-kserve-59474b9c57-fqrxf"
Nov 04 01:30:02.518872 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:02.518797    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a9762e4-1ac3-45c8-8324-c007bddda95f-tls-certs\") pod \"test-model-362631-kserve-59474b9c57-fqrxf\" (UID: \"8a9762e4-1ac3-45c8-8324-c007bddda95f\") " pod="ai-tenant-e2e-inf-b-362631/test-model-362631-kserve-59474b9c57-fqrxf"
Nov 04 01:30:02.521751 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:02.521711    5068 generic.go:356] "Generic (PLEG): container finished" podID="343b7e45-62f4-4533-a07c-0732c85a5b32" containerID="377b882f04986dfe61c8e2f0298f6f5cbfb2e4b2c31a2fe16d33bdc3723081d1" exitCode=60
Nov 04 01:30:02.521894 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:02.521775    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-a362b6-29895930-2fqh4" event={"ID":"343b7e45-62f4-4533-a07c-0732c85a5b32","Type":"ContainerDied","Data":"377b882f04986dfe61c8e2f0298f6f5cbfb2e4b2c31a2fe16d33bdc3723081d1"}
Nov 04 01:30:02.522694 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:02.522674    5068 scope.go:122] "RemoveContainer" containerID="377b882f04986dfe61c8e2f0298f6f5cbfb2e4b2c31a2fe16d33bdc3723081d1"
Nov 04 01:30:02.526094 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:02.526073    5068 generic.go:356] "Generic (PLEG): container finished" podID="78c3da50-fc89-4b70-ae04-79c623854724" containerID="069c42caffc7fa365352e38809530eaa30ab66fdbb3fb53492741163840d2fcc" exitCode=60
Nov 04 01:30:02.526181 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:02.526134    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-362631-29895930-snfmr" event={"ID":"78c3da50-fc89-4b70-ae04-79c623854724","Type":"ContainerDied","Data":"069c42caffc7fa365352e38809530eaa30ab66fdbb3fb53492741163840d2fcc"}
Nov 04 01:30:02.526448 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:02.526431    5068 scope.go:122] "RemoveContainer" containerID="069c42caffc7fa365352e38809530eaa30ab66fdbb3fb53492741163840d2fcc"
Nov 04 01:30:02.528236 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:02.528213    5068 generic.go:356] "Generic (PLEG): container finished" podID="c6ddf85e-cd63-46b0-9f07-f6c93d158181" containerID="f5e8dcf8c8a7e34681b0e5009ecb604651c683ba039753a03653222239e1adb4" exitCode=60
Nov 04 01:30:02.528334 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:02.528262    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-5ce9e8-29895930-f4rf2" event={"ID":"c6ddf85e-cd63-46b0-9f07-f6c93d158181","Type":"ContainerDied","Data":"f5e8dcf8c8a7e34681b0e5009ecb604651c683ba039753a03653222239e1adb4"}
Nov 04 01:30:02.528506 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:02.528487    5068 scope.go:122] "RemoveContainer" containerID="f5e8dcf8c8a7e34681b0e5009ecb604651c683ba039753a03653222239e1adb4"
Nov 04 01:30:02.573942 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:02.573740    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-362631/test-model-362631-kserve-59474b9c57-fqrxf"
Nov 04 01:30:02.896296 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:02.896250    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-362631/test-model-362631-kserve-59474b9c57-fqrxf"]
Nov 04 01:30:02.897298 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:30:02.897271    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8a9762e4_1ac3_45c8_8324_c007bddda95f.slice/crio-4a6bcc8978ab27050484a67549babb039546c93d8e97e4d767dd8d85ebdfe8e6 WatchSource:0}: Error finding container 4a6bcc8978ab27050484a67549babb039546c93d8e97e4d767dd8d85ebdfe8e6: Status 404 returned error can't find the container with id 4a6bcc8978ab27050484a67549babb039546c93d8e97e4d767dd8d85ebdfe8e6
Nov 04 01:30:03.535974 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:03.535935    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-a362b6-29895930-2fqh4" event={"ID":"343b7e45-62f4-4533-a07c-0732c85a5b32","Type":"ContainerStarted","Data":"4e378bedc2016bc8cafc7b12f2ae0b6ed56981b62756755add5ce2b3849e4c77"}
Nov 04 01:30:03.537911 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:03.537887    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-362631-29895930-snfmr" event={"ID":"78c3da50-fc89-4b70-ae04-79c623854724","Type":"ContainerStarted","Data":"1fc8ec5959077f6c281079fb3886eded211a674a26d7ccf57de161f5220ce0c2"}
Nov 04 01:30:03.544646 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:03.539946    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-5ce9e8-29895930-f4rf2" event={"ID":"c6ddf85e-cd63-46b0-9f07-f6c93d158181","Type":"ContainerStarted","Data":"31d0778b113db4365c71bbafad8c4e018efbe6963f2d9518f878ecff86f4fca2"}
Nov 04 01:30:03.544646 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:03.541907    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-362631/test-model-362631-kserve-59474b9c57-fqrxf" event={"ID":"8a9762e4-1ac3-45c8-8324-c007bddda95f","Type":"ContainerStarted","Data":"50b4795c849b34582c2f8c0fe086ad9ad859df80fb6600250017d509b588d270"}
Nov 04 01:30:03.544646 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:03.541937    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-362631/test-model-362631-kserve-59474b9c57-fqrxf" event={"ID":"8a9762e4-1ac3-45c8-8324-c007bddda95f","Type":"ContainerStarted","Data":"4a6bcc8978ab27050484a67549babb039546c93d8e97e4d767dd8d85ebdfe8e6"}
Nov 04 01:30:03.592634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:03.588718    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-362631/test-model-362631-kserve-59474b9c57-fqrxf"
Nov 04 01:30:03.625654 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:03.622135    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-362631/test-model-362631-kserve-59474b9c57-fqrxf" podStartSLOduration=1.622115642 podStartE2EDuration="1.622115642s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:30:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:30:03.615885114 +0000 UTC m=+1643.662445961" watchObservedRunningTime="2026-11-04 01:30:03.622115642 +0000 UTC m=+1643.668676433"
Nov 04 01:30:04.551672 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:04.551598    5068 generic.go:356] "Generic (PLEG): container finished" podID="343b7e45-62f4-4533-a07c-0732c85a5b32" containerID="4e378bedc2016bc8cafc7b12f2ae0b6ed56981b62756755add5ce2b3849e4c77" exitCode=60
Nov 04 01:30:04.551672 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:04.551661    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-a362b6-29895930-2fqh4" event={"ID":"343b7e45-62f4-4533-a07c-0732c85a5b32","Type":"ContainerDied","Data":"4e378bedc2016bc8cafc7b12f2ae0b6ed56981b62756755add5ce2b3849e4c77"}
Nov 04 01:30:04.552205 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:04.551725    5068 scope.go:122] "RemoveContainer" containerID="377b882f04986dfe61c8e2f0298f6f5cbfb2e4b2c31a2fe16d33bdc3723081d1"
Nov 04 01:30:04.552671 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:04.552643    5068 scope.go:122] "RemoveContainer" containerID="4e378bedc2016bc8cafc7b12f2ae0b6ed56981b62756755add5ce2b3849e4c77"
Nov 04 01:30:04.553247 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:30:04.552955    5068 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-a-a362b6-29895930-2fqh4_odh-ai-gateway-infra(343b7e45-62f4-4533-a07c-0732c85a5b32)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-a362b6-29895930-2fqh4" podUID="343b7e45-62f4-4533-a07c-0732c85a5b32"
Nov 04 01:30:04.554780 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:04.554748    5068 generic.go:356] "Generic (PLEG): container finished" podID="78c3da50-fc89-4b70-ae04-79c623854724" containerID="1fc8ec5959077f6c281079fb3886eded211a674a26d7ccf57de161f5220ce0c2" exitCode=60
Nov 04 01:30:04.554917 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:04.554839    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-362631-29895930-snfmr" event={"ID":"78c3da50-fc89-4b70-ae04-79c623854724","Type":"ContainerDied","Data":"1fc8ec5959077f6c281079fb3886eded211a674a26d7ccf57de161f5220ce0c2"}
Nov 04 01:30:04.555255 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:04.555114    5068 scope.go:122] "RemoveContainer" containerID="1fc8ec5959077f6c281079fb3886eded211a674a26d7ccf57de161f5220ce0c2"
Nov 04 01:30:04.555559 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:30:04.555449    5068 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-b-362631-29895930-snfmr_odh-ai-gateway-infra(78c3da50-fc89-4b70-ae04-79c623854724)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-362631-29895930-snfmr" podUID="78c3da50-fc89-4b70-ae04-79c623854724"
Nov 04 01:30:04.557513 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:04.557486    5068 generic.go:356] "Generic (PLEG): container finished" podID="c6ddf85e-cd63-46b0-9f07-f6c93d158181" containerID="31d0778b113db4365c71bbafad8c4e018efbe6963f2d9518f878ecff86f4fca2" exitCode=60
Nov 04 01:30:04.557713 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:04.557679    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-5ce9e8-29895930-f4rf2" event={"ID":"c6ddf85e-cd63-46b0-9f07-f6c93d158181","Type":"ContainerDied","Data":"31d0778b113db4365c71bbafad8c4e018efbe6963f2d9518f878ecff86f4fca2"}
Nov 04 01:30:04.557863 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:04.557796    5068 scope.go:122] "RemoveContainer" containerID="31d0778b113db4365c71bbafad8c4e018efbe6963f2d9518f878ecff86f4fca2"
Nov 04 01:30:04.558265 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:30:04.558233    5068 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w1-5ce9e8-29895930-f4rf2_odh-ai-gateway-infra(c6ddf85e-cd63-46b0-9f07-f6c93d158181)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-5ce9e8-29895930-f4rf2" podUID="c6ddf85e-cd63-46b0-9f07-f6c93d158181"
Nov 04 01:30:04.567809 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:04.567760    5068 scope.go:122] "RemoveContainer" containerID="069c42caffc7fa365352e38809530eaa30ab66fdbb3fb53492741163840d2fcc"
Nov 04 01:30:04.583592 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:04.583564    5068 scope.go:122] "RemoveContainer" containerID="f5e8dcf8c8a7e34681b0e5009ecb604651c683ba039753a03653222239e1adb4"
Nov 04 01:30:04.909427 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:04.909344    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-a362b6/test-model-a362b6-kserve-68474d95dc-xhrtw"
Nov 04 01:30:04.922537 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:04.922504    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-a362b6/test-model-a362b6-kserve-68474d95dc-xhrtw"
Nov 04 01:30:05.568280 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:05.568246    5068 scope.go:122] "RemoveContainer" containerID="4e378bedc2016bc8cafc7b12f2ae0b6ed56981b62756755add5ce2b3849e4c77"
Nov 04 01:30:05.568788 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:30:05.568571    5068 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-a-a362b6-29895930-2fqh4_odh-ai-gateway-infra(343b7e45-62f4-4533-a07c-0732c85a5b32)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-a362b6-29895930-2fqh4" podUID="343b7e45-62f4-4533-a07c-0732c85a5b32"
Nov 04 01:30:05.570198 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:05.570177    5068 scope.go:122] "RemoveContainer" containerID="1fc8ec5959077f6c281079fb3886eded211a674a26d7ccf57de161f5220ce0c2"
Nov 04 01:30:05.570412 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:30:05.570390    5068 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-b-362631-29895930-snfmr_odh-ai-gateway-infra(78c3da50-fc89-4b70-ae04-79c623854724)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-362631-29895930-snfmr" podUID="78c3da50-fc89-4b70-ae04-79c623854724"
Nov 04 01:30:05.572334 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:05.572294    5068 scope.go:122] "RemoveContainer" containerID="31d0778b113db4365c71bbafad8c4e018efbe6963f2d9518f878ecff86f4fca2"
Nov 04 01:30:05.572754 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:30:05.572729    5068 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w1-5ce9e8-29895930-f4rf2_odh-ai-gateway-infra(c6ddf85e-cd63-46b0-9f07-f6c93d158181)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-5ce9e8-29895930-f4rf2" podUID="c6ddf85e-cd63-46b0-9f07-f6c93d158181"
Nov 04 01:30:05.623650 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:05.619901    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-a362b6/test-model-a362b6-kserve-68474d95dc-xhrtw"
Nov 04 01:30:07.818291 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:07.818256    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds"]
Nov 04 01:30:07.819224 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:07.819193    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds" podUID="84f5c469-8fea-4b88-969b-d1eed184c268" containerName="istio-proxy" containerID="cri-o://9feeca1a762b85e8f9c6a4a63effaf66094f94dbab3b637db2ce7bec889f8796" gracePeriod=30
Nov 04 01:30:09.982039 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:09.981989    5068 patch_prober.go:29] interesting pod/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 01:30:09.982496 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:09.982078    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds" podUID="84f5c469-8fea-4b88-969b-d1eed184c268" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 01:30:12.574832 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:12.574790    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-362631/test-model-362631-kserve-59474b9c57-fqrxf"
Nov 04 01:30:12.589682 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:12.589644    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-362631/test-model-362631-kserve-59474b9c57-fqrxf"
Nov 04 01:30:12.731929 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:12.731892    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-362631/test-model-362631-kserve-59474b9c57-fqrxf"
Nov 04 01:30:13.152659 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:13.152541    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds"
Nov 04 01:30:13.272736 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:13.271973    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/84f5c469-8fea-4b88-969b-d1eed184c268-kube-api-access-z8dc6\" (UniqueName: \"kubernetes.io/projected/84f5c469-8fea-4b88-969b-d1eed184c268-kube-api-access-z8dc6\") pod \"84f5c469-8fea-4b88-969b-d1eed184c268\" (UID: \"84f5c469-8fea-4b88-969b-d1eed184c268\") "
Nov 04 01:30:13.272736 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:13.272057    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/84f5c469-8fea-4b88-969b-d1eed184c268-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84f5c469-8fea-4b88-969b-d1eed184c268-istio-podinfo\") pod \"84f5c469-8fea-4b88-969b-d1eed184c268\" (UID: \"84f5c469-8fea-4b88-969b-d1eed184c268\") "
Nov 04 01:30:13.272736 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:13.272128    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/84f5c469-8fea-4b88-969b-d1eed184c268-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84f5c469-8fea-4b88-969b-d1eed184c268-credential-socket\") pod \"84f5c469-8fea-4b88-969b-d1eed184c268\" (UID: \"84f5c469-8fea-4b88-969b-d1eed184c268\") "
Nov 04 01:30:13.272736 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:13.272197    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/84f5c469-8fea-4b88-969b-d1eed184c268-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84f5c469-8fea-4b88-969b-d1eed184c268-istio-data\") pod \"84f5c469-8fea-4b88-969b-d1eed184c268\" (UID: \"84f5c469-8fea-4b88-969b-d1eed184c268\") "
Nov 04 01:30:13.272736 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:13.272237    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/84f5c469-8fea-4b88-969b-d1eed184c268-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84f5c469-8fea-4b88-969b-d1eed184c268-workload-certs\") pod \"84f5c469-8fea-4b88-969b-d1eed184c268\" (UID: \"84f5c469-8fea-4b88-969b-d1eed184c268\") "
Nov 04 01:30:13.272736 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:13.272273    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/84f5c469-8fea-4b88-969b-d1eed184c268-istio-token\" (UniqueName: \"kubernetes.io/projected/84f5c469-8fea-4b88-969b-d1eed184c268-istio-token\") pod \"84f5c469-8fea-4b88-969b-d1eed184c268\" (UID: \"84f5c469-8fea-4b88-969b-d1eed184c268\") "
Nov 04 01:30:13.272736 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:13.272316    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/84f5c469-8fea-4b88-969b-d1eed184c268-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84f5c469-8fea-4b88-969b-d1eed184c268-istio-envoy\") pod \"84f5c469-8fea-4b88-969b-d1eed184c268\" (UID: \"84f5c469-8fea-4b88-969b-d1eed184c268\") "
Nov 04 01:30:13.272736 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:13.272356    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/84f5c469-8fea-4b88-969b-d1eed184c268-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84f5c469-8fea-4b88-969b-d1eed184c268-istiod-ca-cert\") pod \"84f5c469-8fea-4b88-969b-d1eed184c268\" (UID: \"84f5c469-8fea-4b88-969b-d1eed184c268\") "
Nov 04 01:30:13.272736 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:13.272391    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/84f5c469-8fea-4b88-969b-d1eed184c268-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84f5c469-8fea-4b88-969b-d1eed184c268-workload-socket\") pod \"84f5c469-8fea-4b88-969b-d1eed184c268\" (UID: \"84f5c469-8fea-4b88-969b-d1eed184c268\") "
Nov 04 01:30:13.272736 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:13.272578    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84f5c469-8fea-4b88-969b-d1eed184c268-credential-socket" pod "84f5c469-8fea-4b88-969b-d1eed184c268" (UID: "84f5c469-8fea-4b88-969b-d1eed184c268"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:30:13.272736 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:13.272589    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84f5c469-8fea-4b88-969b-d1eed184c268-istio-data" pod "84f5c469-8fea-4b88-969b-d1eed184c268" (UID: "84f5c469-8fea-4b88-969b-d1eed184c268"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:30:13.273536 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:13.272858    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84f5c469-8fea-4b88-969b-d1eed184c268-workload-certs" pod "84f5c469-8fea-4b88-969b-d1eed184c268" (UID: "84f5c469-8fea-4b88-969b-d1eed184c268"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:30:13.273536 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:13.272989    5068 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84f5c469-8fea-4b88-969b-d1eed184c268-credential-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:30:13.273536 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:13.273010    5068 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84f5c469-8fea-4b88-969b-d1eed184c268-istio-data\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:30:13.273536 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:13.273025    5068 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84f5c469-8fea-4b88-969b-d1eed184c268-workload-certs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:30:13.273536 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:13.273049    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84f5c469-8fea-4b88-969b-d1eed184c268-istiod-ca-cert" pod "84f5c469-8fea-4b88-969b-d1eed184c268" (UID: "84f5c469-8fea-4b88-969b-d1eed184c268"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:30:13.273536 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:13.273417    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84f5c469-8fea-4b88-969b-d1eed184c268-workload-socket" pod "84f5c469-8fea-4b88-969b-d1eed184c268" (UID: "84f5c469-8fea-4b88-969b-d1eed184c268"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:30:13.275715 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:13.275676    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/84f5c469-8fea-4b88-969b-d1eed184c268-istio-podinfo" pod "84f5c469-8fea-4b88-969b-d1eed184c268" (UID: "84f5c469-8fea-4b88-969b-d1eed184c268"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:30:13.276042 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:13.276010    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84f5c469-8fea-4b88-969b-d1eed184c268-istio-token" pod "84f5c469-8fea-4b88-969b-d1eed184c268" (UID: "84f5c469-8fea-4b88-969b-d1eed184c268"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:30:13.276354 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:13.276329    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84f5c469-8fea-4b88-969b-d1eed184c268-istio-envoy" pod "84f5c469-8fea-4b88-969b-d1eed184c268" (UID: "84f5c469-8fea-4b88-969b-d1eed184c268"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:30:13.276430 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:13.276374    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84f5c469-8fea-4b88-969b-d1eed184c268-kube-api-access-z8dc6" pod "84f5c469-8fea-4b88-969b-d1eed184c268" (UID: "84f5c469-8fea-4b88-969b-d1eed184c268"). InnerVolumeSpecName "kube-api-access-z8dc6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:30:13.374255 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:13.374158    5068 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84f5c469-8fea-4b88-969b-d1eed184c268-istio-token\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:30:13.374255 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:13.374190    5068 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84f5c469-8fea-4b88-969b-d1eed184c268-istio-envoy\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:30:13.374255 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:13.374200    5068 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84f5c469-8fea-4b88-969b-d1eed184c268-istiod-ca-cert\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:30:13.374255 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:13.374210    5068 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84f5c469-8fea-4b88-969b-d1eed184c268-workload-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:30:13.374255 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:13.374224    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z8dc6\" (UniqueName: \"kubernetes.io/projected/84f5c469-8fea-4b88-969b-d1eed184c268-kube-api-access-z8dc6\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:30:13.374255 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:13.374238    5068 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84f5c469-8fea-4b88-969b-d1eed184c268-istio-podinfo\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:30:13.483591 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:30:13.483543    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:30:13.737080 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:13.736985    5068 generic.go:356] "Generic (PLEG): container finished" podID="84f5c469-8fea-4b88-969b-d1eed184c268" containerID="9feeca1a762b85e8f9c6a4a63effaf66094f94dbab3b637db2ce7bec889f8796" exitCode=0
Nov 04 01:30:13.738258 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:13.738229    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds"
Nov 04 01:30:13.738394 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:13.738296    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds" event={"ID":"84f5c469-8fea-4b88-969b-d1eed184c268","Type":"ContainerDied","Data":"9feeca1a762b85e8f9c6a4a63effaf66094f94dbab3b637db2ce7bec889f8796"}
Nov 04 01:30:13.738394 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:13.738389    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds" event={"ID":"84f5c469-8fea-4b88-969b-d1eed184c268","Type":"ContainerDied","Data":"bfc6c1b78e3829b43e9eeb64f02f881242a0b8cfd6c3abe3a70a47acc5211bb0"}
Nov 04 01:30:13.738495 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:13.738427    5068 scope.go:122] "RemoveContainer" containerID="9feeca1a762b85e8f9c6a4a63effaf66094f94dbab3b637db2ce7bec889f8796"
Nov 04 01:30:13.757153 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:13.757133    5068 scope.go:122] "RemoveContainer" containerID="9feeca1a762b85e8f9c6a4a63effaf66094f94dbab3b637db2ce7bec889f8796"
Nov 04 01:30:13.757443 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:30:13.757418    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9feeca1a762b85e8f9c6a4a63effaf66094f94dbab3b637db2ce7bec889f8796\": container with ID starting with 9feeca1a762b85e8f9c6a4a63effaf66094f94dbab3b637db2ce7bec889f8796 not found: ID does not exist" containerID="9feeca1a762b85e8f9c6a4a63effaf66094f94dbab3b637db2ce7bec889f8796"
Nov 04 01:30:13.757522 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:13.757456    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9feeca1a762b85e8f9c6a4a63effaf66094f94dbab3b637db2ce7bec889f8796"} err="failed to get container status \"9feeca1a762b85e8f9c6a4a63effaf66094f94dbab3b637db2ce7bec889f8796\": rpc error: code = NotFound desc = could not find container \"9feeca1a762b85e8f9c6a4a63effaf66094f94dbab3b637db2ce7bec889f8796\": container with ID starting with 9feeca1a762b85e8f9c6a4a63effaf66094f94dbab3b637db2ce7bec889f8796 not found: ID does not exist"
Nov 04 01:30:13.767305 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:13.767279    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds"]
Nov 04 01:30:13.772057 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:13.772034    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-a5115a-openshift-default-59f76958f9-glnds"]
Nov 04 01:30:14.127404 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:14.127356    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="84f5c469-8fea-4b88-969b-d1eed184c268" path="/var/lib/kubelet/pods/84f5c469-8fea-4b88-969b-d1eed184c268/volumes"
Nov 04 01:30:15.142275 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:15.142214    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:30:16.106094 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:16.106065    5068 scope.go:122] "RemoveContainer" containerID="1fc8ec5959077f6c281079fb3886eded211a674a26d7ccf57de161f5220ce0c2"
Nov 04 01:30:16.257811 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:16.257769    5068 patch_prober.go:29] interesting pod/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.124:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:30:16.258298 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:16.257836    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds" podUID="d091cbea-bbd1-4242-a1c9-f72628c8e019" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.124:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:30:16.779150 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:16.779112    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-362631-29895930-snfmr" event={"ID":"78c3da50-fc89-4b70-ae04-79c623854724","Type":"ContainerStarted","Data":"f7d3de550a100a78934ca8f596fc6bfa4c2d6066510fafe3bb8f9e48cdc2c55c"}
Nov 04 01:30:17.019231 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:17.019193    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-2aefc5/test-model-2aefc5-kserve-5f499d668c-7wq6h"]
Nov 04 01:30:17.019959 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:17.019933    5068 cpu_manager.go:414] "containerMap: removing container" podUID="84f5c469-8fea-4b88-969b-d1eed184c268" containerName="istio-proxy"
Nov 04 01:30:17.020135 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:17.020099    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="84f5c469-8fea-4b88-969b-d1eed184c268" containerName="istio-proxy"
Nov 04 01:30:17.020300 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:17.020286    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="84f5c469-8fea-4b88-969b-d1eed184c268" containerName="istio-proxy"
Nov 04 01:30:17.026499 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:17.026475    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-2aefc5/test-model-2aefc5-kserve-5f499d668c-7wq6h"
Nov 04 01:30:17.035284 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:17.035251    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-2aefc5/test-model-2aefc5-kserve-5f499d668c-7wq6h"]
Nov 04 01:30:17.037069 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:17.036985    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-2aefc5\"/\"kube-root-ca.crt\""
Nov 04 01:30:17.037069 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:17.036988    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-2aefc5\"/\"openshift-service-ca.crt\""
Nov 04 01:30:17.037069 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:17.037062    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-2aefc5\"/\"default-dockercfg-sjfz4\""
Nov 04 01:30:17.038139 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:17.038119    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-2aefc5\"/\"test-model-2aefc5-kserve-self-signed-certs\""
Nov 04 01:30:17.105940 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:17.105909    5068 scope.go:122] "RemoveContainer" containerID="31d0778b113db4365c71bbafad8c4e018efbe6963f2d9518f878ecff86f4fca2"
Nov 04 01:30:17.106337 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:17.106317    5068 scope.go:122] "RemoveContainer" containerID="4e378bedc2016bc8cafc7b12f2ae0b6ed56981b62756755add5ce2b3849e4c77"
Nov 04 01:30:17.165364 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:17.165327    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d2564ce-bab1-4bf5-b7dd-d2077596cf43-tls-certs\") pod \"test-model-2aefc5-kserve-5f499d668c-7wq6h\" (UID: \"4d2564ce-bab1-4bf5-b7dd-d2077596cf43\") " pod="ai-tenant-e2e-inf-b-2aefc5/test-model-2aefc5-kserve-5f499d668c-7wq6h"
Nov 04 01:30:17.165496 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:17.165398    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-428cq\" (UniqueName: \"kubernetes.io/projected/4d2564ce-bab1-4bf5-b7dd-d2077596cf43-kube-api-access-428cq\") pod \"test-model-2aefc5-kserve-5f499d668c-7wq6h\" (UID: \"4d2564ce-bab1-4bf5-b7dd-d2077596cf43\") " pod="ai-tenant-e2e-inf-b-2aefc5/test-model-2aefc5-kserve-5f499d668c-7wq6h"
Nov 04 01:30:17.165561 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:17.165500    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4d2564ce-bab1-4bf5-b7dd-d2077596cf43-dshm\") pod \"test-model-2aefc5-kserve-5f499d668c-7wq6h\" (UID: \"4d2564ce-bab1-4bf5-b7dd-d2077596cf43\") " pod="ai-tenant-e2e-inf-b-2aefc5/test-model-2aefc5-kserve-5f499d668c-7wq6h"
Nov 04 01:30:17.165703 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:17.165680    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4d2564ce-bab1-4bf5-b7dd-d2077596cf43-model-cache\") pod \"test-model-2aefc5-kserve-5f499d668c-7wq6h\" (UID: \"4d2564ce-bab1-4bf5-b7dd-d2077596cf43\") " pod="ai-tenant-e2e-inf-b-2aefc5/test-model-2aefc5-kserve-5f499d668c-7wq6h"
Nov 04 01:30:17.165766 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:17.165737    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4d2564ce-bab1-4bf5-b7dd-d2077596cf43-home\") pod \"test-model-2aefc5-kserve-5f499d668c-7wq6h\" (UID: \"4d2564ce-bab1-4bf5-b7dd-d2077596cf43\") " pod="ai-tenant-e2e-inf-b-2aefc5/test-model-2aefc5-kserve-5f499d668c-7wq6h"
Nov 04 01:30:17.165824 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:17.165786    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d2564ce-bab1-4bf5-b7dd-d2077596cf43-tmp-dir\") pod \"test-model-2aefc5-kserve-5f499d668c-7wq6h\" (UID: \"4d2564ce-bab1-4bf5-b7dd-d2077596cf43\") " pod="ai-tenant-e2e-inf-b-2aefc5/test-model-2aefc5-kserve-5f499d668c-7wq6h"
Nov 04 01:30:17.269846 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:17.268031    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4d2564ce-bab1-4bf5-b7dd-d2077596cf43-model-cache\") pod \"test-model-2aefc5-kserve-5f499d668c-7wq6h\" (UID: \"4d2564ce-bab1-4bf5-b7dd-d2077596cf43\") " pod="ai-tenant-e2e-inf-b-2aefc5/test-model-2aefc5-kserve-5f499d668c-7wq6h"
Nov 04 01:30:17.269846 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:17.268097    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4d2564ce-bab1-4bf5-b7dd-d2077596cf43-home\") pod \"test-model-2aefc5-kserve-5f499d668c-7wq6h\" (UID: \"4d2564ce-bab1-4bf5-b7dd-d2077596cf43\") " pod="ai-tenant-e2e-inf-b-2aefc5/test-model-2aefc5-kserve-5f499d668c-7wq6h"
Nov 04 01:30:17.269846 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:17.268126    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d2564ce-bab1-4bf5-b7dd-d2077596cf43-tmp-dir\") pod \"test-model-2aefc5-kserve-5f499d668c-7wq6h\" (UID: \"4d2564ce-bab1-4bf5-b7dd-d2077596cf43\") " pod="ai-tenant-e2e-inf-b-2aefc5/test-model-2aefc5-kserve-5f499d668c-7wq6h"
Nov 04 01:30:17.269846 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:17.268369    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d2564ce-bab1-4bf5-b7dd-d2077596cf43-tls-certs\") pod \"test-model-2aefc5-kserve-5f499d668c-7wq6h\" (UID: \"4d2564ce-bab1-4bf5-b7dd-d2077596cf43\") " pod="ai-tenant-e2e-inf-b-2aefc5/test-model-2aefc5-kserve-5f499d668c-7wq6h"
Nov 04 01:30:17.269846 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:17.268432    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-428cq\" (UniqueName: \"kubernetes.io/projected/4d2564ce-bab1-4bf5-b7dd-d2077596cf43-kube-api-access-428cq\") pod \"test-model-2aefc5-kserve-5f499d668c-7wq6h\" (UID: \"4d2564ce-bab1-4bf5-b7dd-d2077596cf43\") " pod="ai-tenant-e2e-inf-b-2aefc5/test-model-2aefc5-kserve-5f499d668c-7wq6h"
Nov 04 01:30:17.269846 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:17.268473    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4d2564ce-bab1-4bf5-b7dd-d2077596cf43-dshm\") pod \"test-model-2aefc5-kserve-5f499d668c-7wq6h\" (UID: \"4d2564ce-bab1-4bf5-b7dd-d2077596cf43\") " pod="ai-tenant-e2e-inf-b-2aefc5/test-model-2aefc5-kserve-5f499d668c-7wq6h"
Nov 04 01:30:17.269846 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:17.269492    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4d2564ce-bab1-4bf5-b7dd-d2077596cf43-model-cache\") pod \"test-model-2aefc5-kserve-5f499d668c-7wq6h\" (UID: \"4d2564ce-bab1-4bf5-b7dd-d2077596cf43\") " pod="ai-tenant-e2e-inf-b-2aefc5/test-model-2aefc5-kserve-5f499d668c-7wq6h"
Nov 04 01:30:17.278343 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:17.278307    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d2564ce-bab1-4bf5-b7dd-d2077596cf43-tmp-dir\") pod \"test-model-2aefc5-kserve-5f499d668c-7wq6h\" (UID: \"4d2564ce-bab1-4bf5-b7dd-d2077596cf43\") " pod="ai-tenant-e2e-inf-b-2aefc5/test-model-2aefc5-kserve-5f499d668c-7wq6h"
Nov 04 01:30:17.283109 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:17.283022    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4d2564ce-bab1-4bf5-b7dd-d2077596cf43-home\") pod \"test-model-2aefc5-kserve-5f499d668c-7wq6h\" (UID: \"4d2564ce-bab1-4bf5-b7dd-d2077596cf43\") " pod="ai-tenant-e2e-inf-b-2aefc5/test-model-2aefc5-kserve-5f499d668c-7wq6h"
Nov 04 01:30:17.340375 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:17.340341    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4d2564ce-bab1-4bf5-b7dd-d2077596cf43-dshm\") pod \"test-model-2aefc5-kserve-5f499d668c-7wq6h\" (UID: \"4d2564ce-bab1-4bf5-b7dd-d2077596cf43\") " pod="ai-tenant-e2e-inf-b-2aefc5/test-model-2aefc5-kserve-5f499d668c-7wq6h"
Nov 04 01:30:17.341494 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:17.341467    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-428cq\" (UniqueName: \"kubernetes.io/projected/4d2564ce-bab1-4bf5-b7dd-d2077596cf43-kube-api-access-428cq\") pod \"test-model-2aefc5-kserve-5f499d668c-7wq6h\" (UID: \"4d2564ce-bab1-4bf5-b7dd-d2077596cf43\") " pod="ai-tenant-e2e-inf-b-2aefc5/test-model-2aefc5-kserve-5f499d668c-7wq6h"
Nov 04 01:30:17.341818 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:17.341797    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d2564ce-bab1-4bf5-b7dd-d2077596cf43-tls-certs\") pod \"test-model-2aefc5-kserve-5f499d668c-7wq6h\" (UID: \"4d2564ce-bab1-4bf5-b7dd-d2077596cf43\") " pod="ai-tenant-e2e-inf-b-2aefc5/test-model-2aefc5-kserve-5f499d668c-7wq6h"
Nov 04 01:30:17.392095 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:17.392065    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-2aefc5/test-model-2aefc5-kserve-5f499d668c-7wq6h"
Nov 04 01:30:17.721997 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:17.721805    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-2aefc5/test-model-2aefc5-kserve-5f499d668c-7wq6h"]
Nov 04 01:30:17.792146 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:17.792103    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-2aefc5/test-model-2aefc5-kserve-5f499d668c-7wq6h" event={"ID":"4d2564ce-bab1-4bf5-b7dd-d2077596cf43","Type":"ContainerStarted","Data":"20c952bc5824420745569cf99baaaea22016ae7b5319ca8da5accd369277ded0"}
Nov 04 01:30:17.793954 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:17.793928    5068 generic.go:356] "Generic (PLEG): container finished" podID="d091cbea-bbd1-4242-a1c9-f72628c8e019" containerID="83ccac7852dbaeb0e4eb4548000acaa575b366d9683b7fddd5d982ff86117b82" exitCode=0
Nov 04 01:30:17.794082 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:17.794031    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds" event={"ID":"d091cbea-bbd1-4242-a1c9-f72628c8e019","Type":"ContainerDied","Data":"83ccac7852dbaeb0e4eb4548000acaa575b366d9683b7fddd5d982ff86117b82"}
Nov 04 01:30:17.794495 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:17.794470    5068 scope.go:122] "RemoveContainer" containerID="83ccac7852dbaeb0e4eb4548000acaa575b366d9683b7fddd5d982ff86117b82"
Nov 04 01:30:17.796554 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:17.796520    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:30:17.796679 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:17.796624    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:30:17.796679 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:17.796671    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:30:17.796800 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:17.796763    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-a362b6-29895930-2fqh4" event={"ID":"343b7e45-62f4-4533-a07c-0732c85a5b32","Type":"ContainerStarted","Data":"ae81027403394656522494b771d38f24178b0384a36b1ef55c238c8305bab4b2"}
Nov 04 01:30:17.799142 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:17.799092    5068 generic.go:356] "Generic (PLEG): container finished" podID="78c3da50-fc89-4b70-ae04-79c623854724" containerID="f7d3de550a100a78934ca8f596fc6bfa4c2d6066510fafe3bb8f9e48cdc2c55c" exitCode=60
Nov 04 01:30:17.799247 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:17.799164    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-362631-29895930-snfmr" event={"ID":"78c3da50-fc89-4b70-ae04-79c623854724","Type":"ContainerDied","Data":"f7d3de550a100a78934ca8f596fc6bfa4c2d6066510fafe3bb8f9e48cdc2c55c"}
Nov 04 01:30:17.799247 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:17.799223    5068 scope.go:122] "RemoveContainer" containerID="1fc8ec5959077f6c281079fb3886eded211a674a26d7ccf57de161f5220ce0c2"
Nov 04 01:30:17.799465 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:17.799450    5068 scope.go:122] "RemoveContainer" containerID="f7d3de550a100a78934ca8f596fc6bfa4c2d6066510fafe3bb8f9e48cdc2c55c"
Nov 04 01:30:17.799695 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:30:17.799675    5068 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-b-362631-29895930-snfmr_odh-ai-gateway-infra(78c3da50-fc89-4b70-ae04-79c623854724)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-362631-29895930-snfmr" podUID="78c3da50-fc89-4b70-ae04-79c623854724"
Nov 04 01:30:17.803205 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:17.803126    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-5ce9e8-29895930-f4rf2" event={"ID":"c6ddf85e-cd63-46b0-9f07-f6c93d158181","Type":"ContainerStarted","Data":"d04232c25a77e81ef73981cb4b7f81c179a41421164d2ce64f478ee6e1786698"}
Nov 04 01:30:17.810566 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:17.810458    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-362631-29895930-snfmr"]
Nov 04 01:30:18.116405 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:18.116326    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w"]
Nov 04 01:30:18.120358 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:18.120331    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w"
Nov 04 01:30:18.153077 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:18.153010    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w"]
Nov 04 01:30:18.287623 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:18.287568    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58b0b993-d096-479a-9cc8-907633a20652-workload-socket\") pod \"e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w\" (UID: \"58b0b993-d096-479a-9cc8-907633a20652\") " pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w"
Nov 04 01:30:18.288149 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:18.287721    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58b0b993-d096-479a-9cc8-907633a20652-workload-certs\") pod \"e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w\" (UID: \"58b0b993-d096-479a-9cc8-907633a20652\") " pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w"
Nov 04 01:30:18.288149 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:18.287773    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5pmd\" (UniqueName: \"kubernetes.io/projected/58b0b993-d096-479a-9cc8-907633a20652-kube-api-access-x5pmd\") pod \"e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w\" (UID: \"58b0b993-d096-479a-9cc8-907633a20652\") " pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w"
Nov 04 01:30:18.288149 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:18.287820    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58b0b993-d096-479a-9cc8-907633a20652-istio-token\") pod \"e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w\" (UID: \"58b0b993-d096-479a-9cc8-907633a20652\") " pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w"
Nov 04 01:30:18.288149 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:18.287845    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58b0b993-d096-479a-9cc8-907633a20652-istio-data\") pod \"e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w\" (UID: \"58b0b993-d096-479a-9cc8-907633a20652\") " pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w"
Nov 04 01:30:18.288149 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:18.287870    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58b0b993-d096-479a-9cc8-907633a20652-istio-envoy\") pod \"e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w\" (UID: \"58b0b993-d096-479a-9cc8-907633a20652\") " pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w"
Nov 04 01:30:18.288149 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:18.287906    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58b0b993-d096-479a-9cc8-907633a20652-istio-podinfo\") pod \"e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w\" (UID: \"58b0b993-d096-479a-9cc8-907633a20652\") " pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w"
Nov 04 01:30:18.288149 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:18.287981    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58b0b993-d096-479a-9cc8-907633a20652-istiod-ca-cert\") pod \"e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w\" (UID: \"58b0b993-d096-479a-9cc8-907633a20652\") " pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w"
Nov 04 01:30:18.288149 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:18.288004    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58b0b993-d096-479a-9cc8-907633a20652-credential-socket\") pod \"e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w\" (UID: \"58b0b993-d096-479a-9cc8-907633a20652\") " pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w"
Nov 04 01:30:18.390798 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:18.389334    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58b0b993-d096-479a-9cc8-907633a20652-istio-token\") pod \"e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w\" (UID: \"58b0b993-d096-479a-9cc8-907633a20652\") " pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w"
Nov 04 01:30:18.390798 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:18.389396    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58b0b993-d096-479a-9cc8-907633a20652-istio-data\") pod \"e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w\" (UID: \"58b0b993-d096-479a-9cc8-907633a20652\") " pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w"
Nov 04 01:30:18.390798 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:18.389427    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58b0b993-d096-479a-9cc8-907633a20652-istio-envoy\") pod \"e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w\" (UID: \"58b0b993-d096-479a-9cc8-907633a20652\") " pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w"
Nov 04 01:30:18.390798 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:18.389462    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58b0b993-d096-479a-9cc8-907633a20652-istio-podinfo\") pod \"e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w\" (UID: \"58b0b993-d096-479a-9cc8-907633a20652\") " pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w"
Nov 04 01:30:18.390798 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:18.389538    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58b0b993-d096-479a-9cc8-907633a20652-istiod-ca-cert\") pod \"e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w\" (UID: \"58b0b993-d096-479a-9cc8-907633a20652\") " pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w"
Nov 04 01:30:18.390798 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:18.389567    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58b0b993-d096-479a-9cc8-907633a20652-credential-socket\") pod \"e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w\" (UID: \"58b0b993-d096-479a-9cc8-907633a20652\") " pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w"
Nov 04 01:30:18.390798 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:18.389689    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58b0b993-d096-479a-9cc8-907633a20652-workload-socket\") pod \"e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w\" (UID: \"58b0b993-d096-479a-9cc8-907633a20652\") " pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w"
Nov 04 01:30:18.390798 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:18.389747    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58b0b993-d096-479a-9cc8-907633a20652-workload-certs\") pod \"e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w\" (UID: \"58b0b993-d096-479a-9cc8-907633a20652\") " pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w"
Nov 04 01:30:18.390798 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:18.389789    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5pmd\" (UniqueName: \"kubernetes.io/projected/58b0b993-d096-479a-9cc8-907633a20652-kube-api-access-x5pmd\") pod \"e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w\" (UID: \"58b0b993-d096-479a-9cc8-907633a20652\") " pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w"
Nov 04 01:30:18.390798 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:18.390409    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58b0b993-d096-479a-9cc8-907633a20652-credential-socket\") pod \"e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w\" (UID: \"58b0b993-d096-479a-9cc8-907633a20652\") " pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w"
Nov 04 01:30:18.390798 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:18.390453    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58b0b993-d096-479a-9cc8-907633a20652-workload-socket\") pod \"e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w\" (UID: \"58b0b993-d096-479a-9cc8-907633a20652\") " pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w"
Nov 04 01:30:18.390798 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:18.390701    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58b0b993-d096-479a-9cc8-907633a20652-workload-certs\") pod \"e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w\" (UID: \"58b0b993-d096-479a-9cc8-907633a20652\") " pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w"
Nov 04 01:30:18.391518 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:18.391149    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58b0b993-d096-479a-9cc8-907633a20652-istiod-ca-cert\") pod \"e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w\" (UID: \"58b0b993-d096-479a-9cc8-907633a20652\") " pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w"
Nov 04 01:30:18.391518 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:18.391489    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58b0b993-d096-479a-9cc8-907633a20652-istio-data\") pod \"e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w\" (UID: \"58b0b993-d096-479a-9cc8-907633a20652\") " pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w"
Nov 04 01:30:18.396643 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:18.395844    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58b0b993-d096-479a-9cc8-907633a20652-istio-envoy\") pod \"e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w\" (UID: \"58b0b993-d096-479a-9cc8-907633a20652\") " pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w"
Nov 04 01:30:18.396828 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:18.396658    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58b0b993-d096-479a-9cc8-907633a20652-istio-podinfo\") pod \"e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w\" (UID: \"58b0b993-d096-479a-9cc8-907633a20652\") " pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w"
Nov 04 01:30:18.401749 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:18.401722    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5pmd\" (UniqueName: \"kubernetes.io/projected/58b0b993-d096-479a-9cc8-907633a20652-kube-api-access-x5pmd\") pod \"e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w\" (UID: \"58b0b993-d096-479a-9cc8-907633a20652\") " pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w"
Nov 04 01:30:18.403187 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:18.403162    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58b0b993-d096-479a-9cc8-907633a20652-istio-token\") pod \"e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w\" (UID: \"58b0b993-d096-479a-9cc8-907633a20652\") " pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w"
Nov 04 01:30:18.475054 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:18.475002    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w"
Nov 04 01:30:18.775162 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:18.775123    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w"]
Nov 04 01:30:18.782758 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:18.782705    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:30:18.782867 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:18.782838    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:30:18.782941 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:18.782917    5068 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 01:30:18.811407 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:18.811380    5068 generic.go:356] "Generic (PLEG): container finished" podID="343b7e45-62f4-4533-a07c-0732c85a5b32" containerID="ae81027403394656522494b771d38f24178b0384a36b1ef55c238c8305bab4b2" exitCode=60
Nov 04 01:30:18.811523 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:18.811447    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-a362b6-29895930-2fqh4" event={"ID":"343b7e45-62f4-4533-a07c-0732c85a5b32","Type":"ContainerDied","Data":"ae81027403394656522494b771d38f24178b0384a36b1ef55c238c8305bab4b2"}
Nov 04 01:30:18.811523 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:18.811503    5068 scope.go:122] "RemoveContainer" containerID="4e378bedc2016bc8cafc7b12f2ae0b6ed56981b62756755add5ce2b3849e4c77"
Nov 04 01:30:18.812005 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:18.811969    5068 scope.go:122] "RemoveContainer" containerID="ae81027403394656522494b771d38f24178b0384a36b1ef55c238c8305bab4b2"
Nov 04 01:30:18.812849 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:30:18.812822    5068 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-a-a362b6-29895930-2fqh4_odh-ai-gateway-infra(343b7e45-62f4-4533-a07c-0732c85a5b32)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-a362b6-29895930-2fqh4" podUID="343b7e45-62f4-4533-a07c-0732c85a5b32"
Nov 04 01:30:18.816477 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:18.816450    5068 generic.go:356] "Generic (PLEG): container finished" podID="c6ddf85e-cd63-46b0-9f07-f6c93d158181" containerID="d04232c25a77e81ef73981cb4b7f81c179a41421164d2ce64f478ee6e1786698" exitCode=60
Nov 04 01:30:18.816571 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:18.816553    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-5ce9e8-29895930-f4rf2" event={"ID":"c6ddf85e-cd63-46b0-9f07-f6c93d158181","Type":"ContainerDied","Data":"d04232c25a77e81ef73981cb4b7f81c179a41421164d2ce64f478ee6e1786698"}
Nov 04 01:30:18.817030 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:18.817012    5068 scope.go:122] "RemoveContainer" containerID="d04232c25a77e81ef73981cb4b7f81c179a41421164d2ce64f478ee6e1786698"
Nov 04 01:30:18.817305 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:30:18.817284    5068 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w1-5ce9e8-29895930-f4rf2_odh-ai-gateway-infra(c6ddf85e-cd63-46b0-9f07-f6c93d158181)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-5ce9e8-29895930-f4rf2" podUID="c6ddf85e-cd63-46b0-9f07-f6c93d158181"
Nov 04 01:30:18.818815 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:18.818765    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w" event={"ID":"58b0b993-d096-479a-9cc8-907633a20652","Type":"ContainerStarted","Data":"f2e4c8c5d0519c002ab58b549232c893b7276cb384903b666af17226ca837df2"}
Nov 04 01:30:18.823646 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:18.822903    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-2aefc5/test-model-2aefc5-kserve-5f499d668c-7wq6h" event={"ID":"4d2564ce-bab1-4bf5-b7dd-d2077596cf43","Type":"ContainerStarted","Data":"cf2115852c54e2b28a71d3c7609932079d65653f6e6bbe0c36cc9bb11e52d9b4"}
Nov 04 01:30:18.829512 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:18.829482    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds" event={"ID":"d091cbea-bbd1-4242-a1c9-f72628c8e019","Type":"ContainerStarted","Data":"eaa4c5eea767c3ccf8446ae61f36401d100c1a10573e89fb9c86a1b43f53ba69"}
Nov 04 01:30:18.829673 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:18.829650    5068 scope.go:122] "RemoveContainer" containerID="31d0778b113db4365c71bbafad8c4e018efbe6963f2d9518f878ecff86f4fca2"
Nov 04 01:30:18.885504 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:18.885445    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-2aefc5/test-model-2aefc5-kserve-5f499d668c-7wq6h" podStartSLOduration=2.885422698 podStartE2EDuration="2.885422698s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:30:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:30:18.882595987 +0000 UTC m=+1658.929156778" watchObservedRunningTime="2026-11-04 01:30:18.885422698 +0000 UTC m=+1658.931983488"
Nov 04 01:30:18.901110 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:18.901090    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-a362b6-29895930-2fqh4"]
Nov 04 01:30:18.911047 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:18.911020    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-5ce9e8-29895930-f4rf2"]
Nov 04 01:30:19.232730 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:19.230838    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-362631-29895930-snfmr"
Nov 04 01:30:19.394145 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:19.394113    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-2aefc5/test-model-2aefc5-kserve-5f499d668c-7wq6h"
Nov 04 01:30:19.412621 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:19.412568    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/78c3da50-fc89-4b70-ae04-79c623854724-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78c3da50-fc89-4b70-ae04-79c623854724-maas-api-service-ca\") pod \"78c3da50-fc89-4b70-ae04-79c623854724\" (UID: \"78c3da50-fc89-4b70-ae04-79c623854724\") "
Nov 04 01:30:19.413427 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:19.413307    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78c3da50-fc89-4b70-ae04-79c623854724-maas-api-service-ca" pod "78c3da50-fc89-4b70-ae04-79c623854724" (UID: "78c3da50-fc89-4b70-ae04-79c623854724"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:30:19.413988 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:19.413966    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/78c3da50-fc89-4b70-ae04-79c623854724-kube-api-access-824pn\" (UniqueName: \"kubernetes.io/projected/78c3da50-fc89-4b70-ae04-79c623854724-kube-api-access-824pn\") pod \"78c3da50-fc89-4b70-ae04-79c623854724\" (UID: \"78c3da50-fc89-4b70-ae04-79c623854724\") "
Nov 04 01:30:19.414789 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:19.414774    5068 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78c3da50-fc89-4b70-ae04-79c623854724-maas-api-service-ca\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:30:19.415933 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:19.415911    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78c3da50-fc89-4b70-ae04-79c623854724-kube-api-access-824pn" pod "78c3da50-fc89-4b70-ae04-79c623854724" (UID: "78c3da50-fc89-4b70-ae04-79c623854724"). InnerVolumeSpecName "kube-api-access-824pn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:30:19.527640 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:19.524790    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-824pn\" (UniqueName: \"kubernetes.io/projected/78c3da50-fc89-4b70-ae04-79c623854724-kube-api-access-824pn\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:30:19.748550 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:19.748461    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds"
Nov 04 01:30:19.748923 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:19.748897    5068 patch_prober.go:29] interesting pod/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.124:15021/healthz/ready\": dial tcp 10.132.0.124:15021: connect: connection refused" start-of-body=
Nov 04 01:30:19.749111 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:19.749087    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds" podUID="d091cbea-bbd1-4242-a1c9-f72628c8e019" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.124:15021/healthz/ready\": dial tcp 10.132.0.124:15021: connect: connection refused"
Nov 04 01:30:19.839209 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:19.839100    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-362631-29895930-snfmr"
Nov 04 01:30:19.840078 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:19.839593    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-362631-29895930-snfmr" event={"ID":"78c3da50-fc89-4b70-ae04-79c623854724","Type":"ContainerDied","Data":"a5c203623039a44f9c9e76e9cc91c42de26aeec8470309f438983ccb0e43bf22"}
Nov 04 01:30:19.840378 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:19.840273    5068 scope.go:122] "RemoveContainer" containerID="f7d3de550a100a78934ca8f596fc6bfa4c2d6066510fafe3bb8f9e48cdc2c55c"
Nov 04 01:30:19.852629 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:19.851698    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w" event={"ID":"58b0b993-d096-479a-9cc8-907633a20652","Type":"ContainerStarted","Data":"a7eede906a16647bc27b5dcfd88316037dc9200979a03f9e11dbbbc70a422b8f"}
Nov 04 01:30:19.943636 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:19.939636    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w" podStartSLOduration=1.939599743 podStartE2EDuration="1.939599743s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:30:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:30:19.93834261 +0000 UTC m=+1659.984903402" watchObservedRunningTime="2026-11-04 01:30:19.939599743 +0000 UTC m=+1659.986160533"
Nov 04 01:30:20.009744 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:20.009702    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-362631-29895930-snfmr"]
Nov 04 01:30:20.011192 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:20.011165    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-362631-29895930-snfmr"]
Nov 04 01:30:20.135209 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:20.135174    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="78c3da50-fc89-4b70-ae04-79c623854724" path="/var/lib/kubelet/pods/78c3da50-fc89-4b70-ae04-79c623854724/volumes"
Nov 04 01:30:20.381633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:20.378095    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-a362b6-29895930-2fqh4"
Nov 04 01:30:20.434561 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:20.433836    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-5ce9e8-29895930-f4rf2"
Nov 04 01:30:20.516535 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:20.516505    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w"
Nov 04 01:30:20.516904 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:20.516881    5068 patch_prober.go:29] interesting pod/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.132:15021/healthz/ready\": dial tcp 10.132.0.132:15021: connect: connection refused" start-of-body=
Nov 04 01:30:20.517073 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:20.517050    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w" podUID="58b0b993-d096-479a-9cc8-907633a20652" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.132:15021/healthz/ready\": dial tcp 10.132.0.132:15021: connect: connection refused"
Nov 04 01:30:20.518539 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:20.518506    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c6ddf85e-cd63-46b0-9f07-f6c93d158181-kube-api-access-96h6w\" (UniqueName: \"kubernetes.io/projected/c6ddf85e-cd63-46b0-9f07-f6c93d158181-kube-api-access-96h6w\") pod \"c6ddf85e-cd63-46b0-9f07-f6c93d158181\" (UID: \"c6ddf85e-cd63-46b0-9f07-f6c93d158181\") "
Nov 04 01:30:20.518714 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:20.518552    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/343b7e45-62f4-4533-a07c-0732c85a5b32-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/343b7e45-62f4-4533-a07c-0732c85a5b32-maas-api-service-ca\") pod \"343b7e45-62f4-4533-a07c-0732c85a5b32\" (UID: \"343b7e45-62f4-4533-a07c-0732c85a5b32\") "
Nov 04 01:30:20.518714 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:20.518590    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c6ddf85e-cd63-46b0-9f07-f6c93d158181-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6ddf85e-cd63-46b0-9f07-f6c93d158181-maas-api-service-ca\") pod \"c6ddf85e-cd63-46b0-9f07-f6c93d158181\" (UID: \"c6ddf85e-cd63-46b0-9f07-f6c93d158181\") "
Nov 04 01:30:20.518714 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:20.518691    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/343b7e45-62f4-4533-a07c-0732c85a5b32-kube-api-access-6lnqf\" (UniqueName: \"kubernetes.io/projected/343b7e45-62f4-4533-a07c-0732c85a5b32-kube-api-access-6lnqf\") pod \"343b7e45-62f4-4533-a07c-0732c85a5b32\" (UID: \"343b7e45-62f4-4533-a07c-0732c85a5b32\") "
Nov 04 01:30:20.519791 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:20.519749    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/343b7e45-62f4-4533-a07c-0732c85a5b32-maas-api-service-ca" pod "343b7e45-62f4-4533-a07c-0732c85a5b32" (UID: "343b7e45-62f4-4533-a07c-0732c85a5b32"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:30:20.520338 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:20.520312    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6ddf85e-cd63-46b0-9f07-f6c93d158181-maas-api-service-ca" pod "c6ddf85e-cd63-46b0-9f07-f6c93d158181" (UID: "c6ddf85e-cd63-46b0-9f07-f6c93d158181"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:30:20.521771 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:20.521738    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6ddf85e-cd63-46b0-9f07-f6c93d158181-kube-api-access-96h6w" pod "c6ddf85e-cd63-46b0-9f07-f6c93d158181" (UID: "c6ddf85e-cd63-46b0-9f07-f6c93d158181"). InnerVolumeSpecName "kube-api-access-96h6w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:30:20.521972 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:20.521950    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/343b7e45-62f4-4533-a07c-0732c85a5b32-kube-api-access-6lnqf" pod "343b7e45-62f4-4533-a07c-0732c85a5b32" (UID: "343b7e45-62f4-4533-a07c-0732c85a5b32"). InnerVolumeSpecName "kube-api-access-6lnqf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:30:20.686717 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:20.685663    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-96h6w\" (UniqueName: \"kubernetes.io/projected/c6ddf85e-cd63-46b0-9f07-f6c93d158181-kube-api-access-96h6w\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:30:20.686717 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:20.685710    5068 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/343b7e45-62f4-4533-a07c-0732c85a5b32-maas-api-service-ca\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:30:20.686717 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:20.685727    5068 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6ddf85e-cd63-46b0-9f07-f6c93d158181-maas-api-service-ca\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:30:20.686717 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:20.685742    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6lnqf\" (UniqueName: \"kubernetes.io/projected/343b7e45-62f4-4533-a07c-0732c85a5b32-kube-api-access-6lnqf\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:30:20.762649 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:20.756226    5068 patch_prober.go:29] interesting pod/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.124:15021/healthz/ready\": dial tcp 10.132.0.124:15021: connect: connection refused" start-of-body=
Nov 04 01:30:20.762649 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:20.756286    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds" podUID="d091cbea-bbd1-4242-a1c9-f72628c8e019" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.124:15021/healthz/ready\": dial tcp 10.132.0.124:15021: connect: connection refused"
Nov 04 01:30:20.910671 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:20.910464    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-5ce9e8-29895930-f4rf2" event={"ID":"c6ddf85e-cd63-46b0-9f07-f6c93d158181","Type":"ContainerDied","Data":"009cf0ee606065fc8063b15cfc5339a2f5e99741037bed03852a49aaa284ea19"}
Nov 04 01:30:20.910671 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:20.910532    5068 scope.go:122] "RemoveContainer" containerID="d04232c25a77e81ef73981cb4b7f81c179a41421164d2ce64f478ee6e1786698"
Nov 04 01:30:20.911639 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:20.911594    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-5ce9e8-29895930-f4rf2"
Nov 04 01:30:20.913412 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:20.913388    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-a362b6-29895930-2fqh4"
Nov 04 01:30:20.917797 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:20.917772    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-a362b6-29895930-2fqh4" event={"ID":"343b7e45-62f4-4533-a07c-0732c85a5b32","Type":"ContainerDied","Data":"bb9f0ebf64fbe12f736e23fcc746be097f09beac4f42d0ba91f6c8bbe61df15e"}
Nov 04 01:30:20.924648 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:20.924192    5068 scope.go:122] "RemoveContainer" containerID="ae81027403394656522494b771d38f24178b0384a36b1ef55c238c8305bab4b2"
Nov 04 01:30:20.966764 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:20.966669    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-5ce9e8-29895930-f4rf2"]
Nov 04 01:30:20.990731 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:20.990658    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-5ce9e8-29895930-f4rf2"]
Nov 04 01:30:21.004979 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:21.004924    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-a362b6-29895930-2fqh4"]
Nov 04 01:30:21.016828 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:21.016794    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-a362b6-29895930-2fqh4"]
Nov 04 01:30:21.485641 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:21.481018    5068 patch_prober.go:29] interesting pod/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.132:15021/healthz/ready\": dial tcp 10.132.0.132:15021: connect: connection refused" start-of-body=
Nov 04 01:30:21.485641 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:21.481086    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w" podUID="58b0b993-d096-479a-9cc8-907633a20652" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.132:15021/healthz/ready\": dial tcp 10.132.0.132:15021: connect: connection refused"
Nov 04 01:30:21.743397 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:21.742970    5068 patch_prober.go:29] interesting pod/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.124:15021/healthz/ready\": dial tcp 10.132.0.124:15021: connect: connection refused" start-of-body=
Nov 04 01:30:21.743397 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:21.743026    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds" podUID="d091cbea-bbd1-4242-a1c9-f72628c8e019" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.124:15021/healthz/ready\": dial tcp 10.132.0.124:15021: connect: connection refused"
Nov 04 01:30:22.120030 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:22.119993    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="343b7e45-62f4-4533-a07c-0732c85a5b32" path="/var/lib/kubelet/pods/343b7e45-62f4-4533-a07c-0732c85a5b32/volumes"
Nov 04 01:30:22.120450 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:22.120434    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c6ddf85e-cd63-46b0-9f07-f6c93d158181" path="/var/lib/kubelet/pods/c6ddf85e-cd63-46b0-9f07-f6c93d158181/volumes"
Nov 04 01:30:22.479692 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:22.479012    5068 patch_prober.go:29] interesting pod/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.132:15021/healthz/ready\": dial tcp 10.132.0.132:15021: connect: connection refused" start-of-body=
Nov 04 01:30:22.479692 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:22.479076    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w" podUID="58b0b993-d096-479a-9cc8-907633a20652" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.132:15021/healthz/ready\": dial tcp 10.132.0.132:15021: connect: connection refused"
Nov 04 01:30:22.760694 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:22.747975    5068 patch_prober.go:29] interesting pod/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.124:15021/healthz/ready\": dial tcp 10.132.0.124:15021: connect: connection refused" start-of-body=
Nov 04 01:30:22.760694 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:22.748042    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds" podUID="d091cbea-bbd1-4242-a1c9-f72628c8e019" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.124:15021/healthz/ready\": dial tcp 10.132.0.124:15021: connect: connection refused"
Nov 04 01:30:23.488250 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:23.488215    5068 patch_prober.go:29] interesting pod/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.132:15021/healthz/ready\": dial tcp 10.132.0.132:15021: connect: connection refused" start-of-body=
Nov 04 01:30:23.488543 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:23.488516    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w" podUID="58b0b993-d096-479a-9cc8-907633a20652" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.132:15021/healthz/ready\": dial tcp 10.132.0.132:15021: connect: connection refused"
Nov 04 01:30:23.743098 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:23.743004    5068 patch_prober.go:29] interesting pod/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.124:15021/healthz/ready\": dial tcp 10.132.0.124:15021: connect: connection refused" start-of-body=
Nov 04 01:30:23.743364 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:23.743334    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds" podUID="d091cbea-bbd1-4242-a1c9-f72628c8e019" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.124:15021/healthz/ready\": dial tcp 10.132.0.124:15021: connect: connection refused"
Nov 04 01:30:24.476248 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:24.476205    5068 patch_prober.go:29] interesting pod/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.132:15021/healthz/ready\": dial tcp 10.132.0.132:15021: connect: connection refused" start-of-body=
Nov 04 01:30:24.476710 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:24.476275    5068 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w" podUID="58b0b993-d096-479a-9cc8-907633a20652" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.132:15021/healthz/ready\": dial tcp 10.132.0.132:15021: connect: connection refused"
Nov 04 01:30:24.753998 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:24.753908    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds"
Nov 04 01:30:24.755680 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:24.755657    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds"
Nov 04 01:30:24.760013 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:24.759990    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds"
Nov 04 01:30:25.482807 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:25.482771    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w"
Nov 04 01:30:25.483645 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:25.483601    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w"
Nov 04 01:30:25.484431 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:25.484413    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w"
Nov 04 01:30:27.392874 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:27.392830    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-2aefc5/test-model-2aefc5-kserve-5f499d668c-7wq6h"
Nov 04 01:30:27.408731 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:27.408703    5068 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-2aefc5/test-model-2aefc5-kserve-5f499d668c-7wq6h"
Nov 04 01:30:28.014430 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:28.014389    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-2aefc5/test-model-2aefc5-kserve-5f499d668c-7wq6h"
Nov 04 01:30:33.244748 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:33.244714    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:30:43.450439 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:30:43.450403    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:30:50.768042 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:30:50.768002    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:31:12.144939 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:31:12.144905    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:31:13.456939 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:31:13.456892    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:31:26.748206 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:31:26.748170    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:31:43.451424 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:31:43.451376    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:31:45.312436 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:31:45.312397    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:32:01.664563 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:01.664526    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-a362b6/test-model-a362b6-kserve-68474d95dc-xhrtw"]
Nov 04 01:32:06.987483 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:06.987444    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:32:12.843959 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:12.843916    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-362631/test-model-362631-kserve-59474b9c57-fqrxf"]
Nov 04 01:32:13.451251 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:32:13.451209    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:32:16.696677 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:16.696598    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-a362b6/test-model-a362b6-kserve-68474d95dc-xhrtw" podUID="7c58619a-e60c-420d-a1b8-e1e54d7dce0e" containerName="main" containerID="cri-o://d363be720088ee8941a834ea37b924a7b66ae0225556fbc6c28e4fae38c26071" gracePeriod=45
Nov 04 01:32:16.996212 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:16.996181    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a362b6/test-model-a362b6-kserve-68474d95dc-xhrtw"
Nov 04 01:32:17.007525 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:17.007490    5068 generic.go:356] "Generic (PLEG): container finished" podID="7c58619a-e60c-420d-a1b8-e1e54d7dce0e" containerID="d363be720088ee8941a834ea37b924a7b66ae0225556fbc6c28e4fae38c26071" exitCode=0
Nov 04 01:32:17.007700 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:17.007533    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a362b6/test-model-a362b6-kserve-68474d95dc-xhrtw"
Nov 04 01:32:17.007700 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:17.007554    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a362b6/test-model-a362b6-kserve-68474d95dc-xhrtw" event={"ID":"7c58619a-e60c-420d-a1b8-e1e54d7dce0e","Type":"ContainerDied","Data":"d363be720088ee8941a834ea37b924a7b66ae0225556fbc6c28e4fae38c26071"}
Nov 04 01:32:17.008700 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:17.007601    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a362b6/test-model-a362b6-kserve-68474d95dc-xhrtw" event={"ID":"7c58619a-e60c-420d-a1b8-e1e54d7dce0e","Type":"ContainerDied","Data":"b290996edcd5e0d1981beddab75e9e2f3db1fff020a994d7fe212b551e6da84e"}
Nov 04 01:32:17.008777 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:17.007632    5068 scope.go:122] "RemoveContainer" containerID="d363be720088ee8941a834ea37b924a7b66ae0225556fbc6c28e4fae38c26071"
Nov 04 01:32:17.019807 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:17.019779    5068 scope.go:122] "RemoveContainer" containerID="d363be720088ee8941a834ea37b924a7b66ae0225556fbc6c28e4fae38c26071"
Nov 04 01:32:17.020100 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:32:17.020069    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d363be720088ee8941a834ea37b924a7b66ae0225556fbc6c28e4fae38c26071\": container with ID starting with d363be720088ee8941a834ea37b924a7b66ae0225556fbc6c28e4fae38c26071 not found: ID does not exist" containerID="d363be720088ee8941a834ea37b924a7b66ae0225556fbc6c28e4fae38c26071"
Nov 04 01:32:17.020213 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:17.020108    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d363be720088ee8941a834ea37b924a7b66ae0225556fbc6c28e4fae38c26071"} err="failed to get container status \"d363be720088ee8941a834ea37b924a7b66ae0225556fbc6c28e4fae38c26071\": rpc error: code = NotFound desc = could not find container \"d363be720088ee8941a834ea37b924a7b66ae0225556fbc6c28e4fae38c26071\": container with ID starting with d363be720088ee8941a834ea37b924a7b66ae0225556fbc6c28e4fae38c26071 not found: ID does not exist"
Nov 04 01:32:17.085628 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:17.085579    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7c58619a-e60c-420d-a1b8-e1e54d7dce0e-home\" (UniqueName: \"kubernetes.io/empty-dir/7c58619a-e60c-420d-a1b8-e1e54d7dce0e-home\") pod \"7c58619a-e60c-420d-a1b8-e1e54d7dce0e\" (UID: \"7c58619a-e60c-420d-a1b8-e1e54d7dce0e\") "
Nov 04 01:32:17.085820 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:17.085678    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7c58619a-e60c-420d-a1b8-e1e54d7dce0e-tls-certs\" (UniqueName: \"kubernetes.io/secret/7c58619a-e60c-420d-a1b8-e1e54d7dce0e-tls-certs\") pod \"7c58619a-e60c-420d-a1b8-e1e54d7dce0e\" (UID: \"7c58619a-e60c-420d-a1b8-e1e54d7dce0e\") "
Nov 04 01:32:17.085820 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:17.085734    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7c58619a-e60c-420d-a1b8-e1e54d7dce0e-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c58619a-e60c-420d-a1b8-e1e54d7dce0e-model-cache\") pod \"7c58619a-e60c-420d-a1b8-e1e54d7dce0e\" (UID: \"7c58619a-e60c-420d-a1b8-e1e54d7dce0e\") "
Nov 04 01:32:17.085951 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:17.085837    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7c58619a-e60c-420d-a1b8-e1e54d7dce0e-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c58619a-e60c-420d-a1b8-e1e54d7dce0e-tmp-dir\") pod \"7c58619a-e60c-420d-a1b8-e1e54d7dce0e\" (UID: \"7c58619a-e60c-420d-a1b8-e1e54d7dce0e\") "
Nov 04 01:32:17.085951 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:17.085875    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7c58619a-e60c-420d-a1b8-e1e54d7dce0e-dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c58619a-e60c-420d-a1b8-e1e54d7dce0e-dshm\") pod \"7c58619a-e60c-420d-a1b8-e1e54d7dce0e\" (UID: \"7c58619a-e60c-420d-a1b8-e1e54d7dce0e\") "
Nov 04 01:32:17.085951 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:17.085900    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c58619a-e60c-420d-a1b8-e1e54d7dce0e-home" pod "7c58619a-e60c-420d-a1b8-e1e54d7dce0e" (UID: "7c58619a-e60c-420d-a1b8-e1e54d7dce0e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:32:17.086095 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:17.085913    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7c58619a-e60c-420d-a1b8-e1e54d7dce0e-kube-api-access-5d6jz\" (UniqueName: \"kubernetes.io/projected/7c58619a-e60c-420d-a1b8-e1e54d7dce0e-kube-api-access-5d6jz\") pod \"7c58619a-e60c-420d-a1b8-e1e54d7dce0e\" (UID: \"7c58619a-e60c-420d-a1b8-e1e54d7dce0e\") "
Nov 04 01:32:17.086095 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:17.086009    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c58619a-e60c-420d-a1b8-e1e54d7dce0e-model-cache" pod "7c58619a-e60c-420d-a1b8-e1e54d7dce0e" (UID: "7c58619a-e60c-420d-a1b8-e1e54d7dce0e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:32:17.086191 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:17.086121    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c58619a-e60c-420d-a1b8-e1e54d7dce0e-tmp-dir" pod "7c58619a-e60c-420d-a1b8-e1e54d7dce0e" (UID: "7c58619a-e60c-420d-a1b8-e1e54d7dce0e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:32:17.086547 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:17.086524    5068 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c58619a-e60c-420d-a1b8-e1e54d7dce0e-home\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:17.086547 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:17.086548    5068 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c58619a-e60c-420d-a1b8-e1e54d7dce0e-model-cache\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:17.086724 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:17.086560    5068 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c58619a-e60c-420d-a1b8-e1e54d7dce0e-tmp-dir\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:17.088162 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:17.088132    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c58619a-e60c-420d-a1b8-e1e54d7dce0e-dshm" pod "7c58619a-e60c-420d-a1b8-e1e54d7dce0e" (UID: "7c58619a-e60c-420d-a1b8-e1e54d7dce0e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:32:17.088266 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:17.088192    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c58619a-e60c-420d-a1b8-e1e54d7dce0e-tls-certs" pod "7c58619a-e60c-420d-a1b8-e1e54d7dce0e" (UID: "7c58619a-e60c-420d-a1b8-e1e54d7dce0e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:32:17.088320 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:17.088302    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c58619a-e60c-420d-a1b8-e1e54d7dce0e-kube-api-access-5d6jz" pod "7c58619a-e60c-420d-a1b8-e1e54d7dce0e" (UID: "7c58619a-e60c-420d-a1b8-e1e54d7dce0e"). InnerVolumeSpecName "kube-api-access-5d6jz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:32:17.188017 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:17.187971    5068 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c58619a-e60c-420d-a1b8-e1e54d7dce0e-tls-certs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:17.188017 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:17.188010    5068 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c58619a-e60c-420d-a1b8-e1e54d7dce0e-dshm\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:17.188017 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:17.188023    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5d6jz\" (UniqueName: \"kubernetes.io/projected/7c58619a-e60c-420d-a1b8-e1e54d7dce0e-kube-api-access-5d6jz\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:17.355911 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:17.355830    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-a362b6/test-model-a362b6-kserve-68474d95dc-xhrtw"]
Nov 04 01:32:17.379478 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:17.379399    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-a362b6/test-model-a362b6-kserve-68474d95dc-xhrtw"]
Nov 04 01:32:18.113925 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:18.113885    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7c58619a-e60c-420d-a1b8-e1e54d7dce0e" path="/var/lib/kubelet/pods/7c58619a-e60c-420d-a1b8-e1e54d7dce0e/volumes"
Nov 04 01:32:26.960799 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:26.960764    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:32:27.868735 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:27.868698    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-362631/test-model-362631-kserve-59474b9c57-fqrxf" podUID="8a9762e4-1ac3-45c8-8324-c007bddda95f" containerName="main" containerID="cri-o://50b4795c849b34582c2f8c0fe086ad9ad859df80fb6600250017d509b588d270" gracePeriod=45
Nov 04 01:32:28.113936 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:28.113897    5068 generic.go:356] "Generic (PLEG): container finished" podID="8a9762e4-1ac3-45c8-8324-c007bddda95f" containerID="50b4795c849b34582c2f8c0fe086ad9ad859df80fb6600250017d509b588d270" exitCode=0
Nov 04 01:32:28.117175 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:28.117148    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-362631/test-model-362631-kserve-59474b9c57-fqrxf" event={"ID":"8a9762e4-1ac3-45c8-8324-c007bddda95f","Type":"ContainerDied","Data":"50b4795c849b34582c2f8c0fe086ad9ad859df80fb6600250017d509b588d270"}
Nov 04 01:32:28.178005 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:28.177978    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-362631/test-model-362631-kserve-59474b9c57-fqrxf"
Nov 04 01:32:28.246534 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:28.246498    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8a9762e4-1ac3-45c8-8324-c007bddda95f-kube-api-access-mx276\" (UniqueName: \"kubernetes.io/projected/8a9762e4-1ac3-45c8-8324-c007bddda95f-kube-api-access-mx276\") pod \"8a9762e4-1ac3-45c8-8324-c007bddda95f\" (UID: \"8a9762e4-1ac3-45c8-8324-c007bddda95f\") "
Nov 04 01:32:28.246749 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:28.246668    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8a9762e4-1ac3-45c8-8324-c007bddda95f-tls-certs\" (UniqueName: \"kubernetes.io/secret/8a9762e4-1ac3-45c8-8324-c007bddda95f-tls-certs\") pod \"8a9762e4-1ac3-45c8-8324-c007bddda95f\" (UID: \"8a9762e4-1ac3-45c8-8324-c007bddda95f\") "
Nov 04 01:32:28.246749 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:28.246730    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8a9762e4-1ac3-45c8-8324-c007bddda95f-dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a9762e4-1ac3-45c8-8324-c007bddda95f-dshm\") pod \"8a9762e4-1ac3-45c8-8324-c007bddda95f\" (UID: \"8a9762e4-1ac3-45c8-8324-c007bddda95f\") "
Nov 04 01:32:28.246875 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:28.246803    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8a9762e4-1ac3-45c8-8324-c007bddda95f-home\" (UniqueName: \"kubernetes.io/empty-dir/8a9762e4-1ac3-45c8-8324-c007bddda95f-home\") pod \"8a9762e4-1ac3-45c8-8324-c007bddda95f\" (UID: \"8a9762e4-1ac3-45c8-8324-c007bddda95f\") "
Nov 04 01:32:28.246929 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:28.246908    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8a9762e4-1ac3-45c8-8324-c007bddda95f-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a9762e4-1ac3-45c8-8324-c007bddda95f-tmp-dir\") pod \"8a9762e4-1ac3-45c8-8324-c007bddda95f\" (UID: \"8a9762e4-1ac3-45c8-8324-c007bddda95f\") "
Nov 04 01:32:28.246979 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:28.246944    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8a9762e4-1ac3-45c8-8324-c007bddda95f-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a9762e4-1ac3-45c8-8324-c007bddda95f-model-cache\") pod \"8a9762e4-1ac3-45c8-8324-c007bddda95f\" (UID: \"8a9762e4-1ac3-45c8-8324-c007bddda95f\") "
Nov 04 01:32:28.248105 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:28.248074    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a9762e4-1ac3-45c8-8324-c007bddda95f-tmp-dir" pod "8a9762e4-1ac3-45c8-8324-c007bddda95f" (UID: "8a9762e4-1ac3-45c8-8324-c007bddda95f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:32:28.248248 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:28.248216    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a9762e4-1ac3-45c8-8324-c007bddda95f-model-cache" pod "8a9762e4-1ac3-45c8-8324-c007bddda95f" (UID: "8a9762e4-1ac3-45c8-8324-c007bddda95f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:32:28.248485 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:28.248460    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a9762e4-1ac3-45c8-8324-c007bddda95f-home" pod "8a9762e4-1ac3-45c8-8324-c007bddda95f" (UID: "8a9762e4-1ac3-45c8-8324-c007bddda95f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:32:28.250157 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:28.250134    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a9762e4-1ac3-45c8-8324-c007bddda95f-dshm" pod "8a9762e4-1ac3-45c8-8324-c007bddda95f" (UID: "8a9762e4-1ac3-45c8-8324-c007bddda95f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:32:28.250245 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:28.250182    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a9762e4-1ac3-45c8-8324-c007bddda95f-kube-api-access-mx276" pod "8a9762e4-1ac3-45c8-8324-c007bddda95f" (UID: "8a9762e4-1ac3-45c8-8324-c007bddda95f"). InnerVolumeSpecName "kube-api-access-mx276". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:32:28.250746 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:28.250728    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a9762e4-1ac3-45c8-8324-c007bddda95f-tls-certs" pod "8a9762e4-1ac3-45c8-8324-c007bddda95f" (UID: "8a9762e4-1ac3-45c8-8324-c007bddda95f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:32:28.349287 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:28.349245    5068 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a9762e4-1ac3-45c8-8324-c007bddda95f-tmp-dir\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:28.349287 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:28.349286    5068 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a9762e4-1ac3-45c8-8324-c007bddda95f-model-cache\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:28.349524 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:28.349302    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mx276\" (UniqueName: \"kubernetes.io/projected/8a9762e4-1ac3-45c8-8324-c007bddda95f-kube-api-access-mx276\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:28.349524 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:28.349317    5068 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a9762e4-1ac3-45c8-8324-c007bddda95f-tls-certs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:28.349524 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:28.349331    5068 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a9762e4-1ac3-45c8-8324-c007bddda95f-dshm\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:28.349524 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:28.349343    5068 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a9762e4-1ac3-45c8-8324-c007bddda95f-home\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:29.122838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:29.122803    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-362631/test-model-362631-kserve-59474b9c57-fqrxf" event={"ID":"8a9762e4-1ac3-45c8-8324-c007bddda95f","Type":"ContainerDied","Data":"4a6bcc8978ab27050484a67549babb039546c93d8e97e4d767dd8d85ebdfe8e6"}
Nov 04 01:32:29.122838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:29.122834    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-362631/test-model-362631-kserve-59474b9c57-fqrxf"
Nov 04 01:32:29.123380 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:29.122852    5068 scope.go:122] "RemoveContainer" containerID="50b4795c849b34582c2f8c0fe086ad9ad859df80fb6600250017d509b588d270"
Nov 04 01:32:29.149647 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:29.149603    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-362631/test-model-362631-kserve-59474b9c57-fqrxf"]
Nov 04 01:32:29.154541 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:29.154516    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-362631/test-model-362631-kserve-59474b9c57-fqrxf"]
Nov 04 01:32:30.114679 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:30.114648    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8a9762e4-1ac3-45c8-8324-c007bddda95f" path="/var/lib/kubelet/pods/8a9762e4-1ac3-45c8-8324-c007bddda95f/volumes"
Nov 04 01:32:32.223571 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:32.223532    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-2aefc5/test-model-2aefc5-kserve-5f499d668c-7wq6h"]
Nov 04 01:32:41.153806 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:41.153767    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:32:43.451512 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:32:43.451478    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:32:44.406034 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:44.406002    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tr8b2_21d1d23b-7b6e-4ab5-9593-cf7784fffea2/console-operator/2.log"
Nov 04 01:32:44.503135 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:44.503104    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tr8b2_21d1d23b-7b6e-4ab5-9593-cf7784fffea2/console-operator/2.log"
Nov 04 01:32:47.249192 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:47.249121    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-2aefc5/test-model-2aefc5-kserve-5f499d668c-7wq6h" podUID="4d2564ce-bab1-4bf5-b7dd-d2077596cf43" containerName="main" containerID="cri-o://cf2115852c54e2b28a71d3c7609932079d65653f6e6bbe0c36cc9bb11e52d9b4" gracePeriod=45
Nov 04 01:32:47.565412 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:47.565378    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-2aefc5/test-model-2aefc5-kserve-5f499d668c-7wq6h"
Nov 04 01:32:47.633461 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:47.633421    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d2564ce-bab1-4bf5-b7dd-d2077596cf43-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d2564ce-bab1-4bf5-b7dd-d2077596cf43-tmp-dir\") pod \"4d2564ce-bab1-4bf5-b7dd-d2077596cf43\" (UID: \"4d2564ce-bab1-4bf5-b7dd-d2077596cf43\") "
Nov 04 01:32:47.633646 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:47.633525    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d2564ce-bab1-4bf5-b7dd-d2077596cf43-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4d2564ce-bab1-4bf5-b7dd-d2077596cf43-model-cache\") pod \"4d2564ce-bab1-4bf5-b7dd-d2077596cf43\" (UID: \"4d2564ce-bab1-4bf5-b7dd-d2077596cf43\") "
Nov 04 01:32:47.633697 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:47.633659    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d2564ce-bab1-4bf5-b7dd-d2077596cf43-dshm\" (UniqueName: \"kubernetes.io/empty-dir/4d2564ce-bab1-4bf5-b7dd-d2077596cf43-dshm\") pod \"4d2564ce-bab1-4bf5-b7dd-d2077596cf43\" (UID: \"4d2564ce-bab1-4bf5-b7dd-d2077596cf43\") "
Nov 04 01:32:47.633747 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:47.633720    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4d2564ce-bab1-4bf5-b7dd-d2077596cf43-tls-certs\" (UniqueName: \"kubernetes.io/secret/4d2564ce-bab1-4bf5-b7dd-d2077596cf43-tls-certs\") pod \"4d2564ce-bab1-4bf5-b7dd-d2077596cf43\" (UID: \"4d2564ce-bab1-4bf5-b7dd-d2077596cf43\") "
Nov 04 01:32:47.633747 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:47.633731    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d2564ce-bab1-4bf5-b7dd-d2077596cf43-tmp-dir" pod "4d2564ce-bab1-4bf5-b7dd-d2077596cf43" (UID: "4d2564ce-bab1-4bf5-b7dd-d2077596cf43"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:32:47.633833 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:47.633782    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4d2564ce-bab1-4bf5-b7dd-d2077596cf43-kube-api-access-428cq\" (UniqueName: \"kubernetes.io/projected/4d2564ce-bab1-4bf5-b7dd-d2077596cf43-kube-api-access-428cq\") pod \"4d2564ce-bab1-4bf5-b7dd-d2077596cf43\" (UID: \"4d2564ce-bab1-4bf5-b7dd-d2077596cf43\") "
Nov 04 01:32:47.633833 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:47.633813    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d2564ce-bab1-4bf5-b7dd-d2077596cf43-home\" (UniqueName: \"kubernetes.io/empty-dir/4d2564ce-bab1-4bf5-b7dd-d2077596cf43-home\") pod \"4d2564ce-bab1-4bf5-b7dd-d2077596cf43\" (UID: \"4d2564ce-bab1-4bf5-b7dd-d2077596cf43\") "
Nov 04 01:32:47.633833 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:47.633818    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d2564ce-bab1-4bf5-b7dd-d2077596cf43-model-cache" pod "4d2564ce-bab1-4bf5-b7dd-d2077596cf43" (UID: "4d2564ce-bab1-4bf5-b7dd-d2077596cf43"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:32:47.634136 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:47.634101    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d2564ce-bab1-4bf5-b7dd-d2077596cf43-home" pod "4d2564ce-bab1-4bf5-b7dd-d2077596cf43" (UID: "4d2564ce-bab1-4bf5-b7dd-d2077596cf43"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:32:47.634408 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:47.634389    5068 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4d2564ce-bab1-4bf5-b7dd-d2077596cf43-home\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:47.634481 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:47.634413    5068 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d2564ce-bab1-4bf5-b7dd-d2077596cf43-tmp-dir\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:47.634481 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:47.634428    5068 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4d2564ce-bab1-4bf5-b7dd-d2077596cf43-model-cache\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:47.636887 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:47.636861    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d2564ce-bab1-4bf5-b7dd-d2077596cf43-tls-certs" pod "4d2564ce-bab1-4bf5-b7dd-d2077596cf43" (UID: "4d2564ce-bab1-4bf5-b7dd-d2077596cf43"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:32:47.637064 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:47.637042    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d2564ce-bab1-4bf5-b7dd-d2077596cf43-kube-api-access-428cq" pod "4d2564ce-bab1-4bf5-b7dd-d2077596cf43" (UID: "4d2564ce-bab1-4bf5-b7dd-d2077596cf43"). InnerVolumeSpecName "kube-api-access-428cq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:32:47.645020 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:47.644992    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d2564ce-bab1-4bf5-b7dd-d2077596cf43-dshm" pod "4d2564ce-bab1-4bf5-b7dd-d2077596cf43" (UID: "4d2564ce-bab1-4bf5-b7dd-d2077596cf43"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:32:47.735214 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:47.735184    5068 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4d2564ce-bab1-4bf5-b7dd-d2077596cf43-dshm\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:47.735214 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:47.735210    5068 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d2564ce-bab1-4bf5-b7dd-d2077596cf43-tls-certs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:47.735214 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:47.735219    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-428cq\" (UniqueName: \"kubernetes.io/projected/4d2564ce-bab1-4bf5-b7dd-d2077596cf43-kube-api-access-428cq\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:48.330285 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:48.330244    5068 generic.go:356] "Generic (PLEG): container finished" podID="4d2564ce-bab1-4bf5-b7dd-d2077596cf43" containerID="cf2115852c54e2b28a71d3c7609932079d65653f6e6bbe0c36cc9bb11e52d9b4" exitCode=0
Nov 04 01:32:48.330855 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:48.330306    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-2aefc5/test-model-2aefc5-kserve-5f499d668c-7wq6h"
Nov 04 01:32:48.330855 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:48.330330    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-2aefc5/test-model-2aefc5-kserve-5f499d668c-7wq6h" event={"ID":"4d2564ce-bab1-4bf5-b7dd-d2077596cf43","Type":"ContainerDied","Data":"cf2115852c54e2b28a71d3c7609932079d65653f6e6bbe0c36cc9bb11e52d9b4"}
Nov 04 01:32:48.330855 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:48.330418    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-2aefc5/test-model-2aefc5-kserve-5f499d668c-7wq6h" event={"ID":"4d2564ce-bab1-4bf5-b7dd-d2077596cf43","Type":"ContainerDied","Data":"20c952bc5824420745569cf99baaaea22016ae7b5319ca8da5accd369277ded0"}
Nov 04 01:32:48.330855 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:48.330483    5068 scope.go:122] "RemoveContainer" containerID="cf2115852c54e2b28a71d3c7609932079d65653f6e6bbe0c36cc9bb11e52d9b4"
Nov 04 01:32:48.346893 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:48.346869    5068 scope.go:122] "RemoveContainer" containerID="cf2115852c54e2b28a71d3c7609932079d65653f6e6bbe0c36cc9bb11e52d9b4"
Nov 04 01:32:48.347229 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:32:48.347198    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf2115852c54e2b28a71d3c7609932079d65653f6e6bbe0c36cc9bb11e52d9b4\": container with ID starting with cf2115852c54e2b28a71d3c7609932079d65653f6e6bbe0c36cc9bb11e52d9b4 not found: ID does not exist" containerID="cf2115852c54e2b28a71d3c7609932079d65653f6e6bbe0c36cc9bb11e52d9b4"
Nov 04 01:32:48.347318 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:48.347232    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf2115852c54e2b28a71d3c7609932079d65653f6e6bbe0c36cc9bb11e52d9b4"} err="failed to get container status \"cf2115852c54e2b28a71d3c7609932079d65653f6e6bbe0c36cc9bb11e52d9b4\": rpc error: code = NotFound desc = could not find container \"cf2115852c54e2b28a71d3c7609932079d65653f6e6bbe0c36cc9bb11e52d9b4\": container with ID starting with cf2115852c54e2b28a71d3c7609932079d65653f6e6bbe0c36cc9bb11e52d9b4 not found: ID does not exist"
Nov 04 01:32:48.354737 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:48.354711    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-2aefc5/test-model-2aefc5-kserve-5f499d668c-7wq6h"]
Nov 04 01:32:48.360099 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:48.360076    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-2aefc5/test-model-2aefc5-kserve-5f499d668c-7wq6h"]
Nov 04 01:32:50.115033 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:50.114998    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4d2564ce-bab1-4bf5-b7dd-d2077596cf43" path="/var/lib/kubelet/pods/4d2564ce-bab1-4bf5-b7dd-d2077596cf43/volumes"
Nov 04 01:32:50.956495 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:50.956455    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:32:53.277515 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:53.277464    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w"]
Nov 04 01:32:53.278153 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:53.277847    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w" podUID="58b0b993-d096-479a-9cc8-907633a20652" containerName="istio-proxy" containerID="cri-o://a7eede906a16647bc27b5dcfd88316037dc9200979a03f9e11dbbbc70a422b8f" gracePeriod=30
Nov 04 01:32:53.284840 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:53.284805    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p"]
Nov 04 01:32:53.285114 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:53.285070    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p" podUID="332ac8c5-1996-489e-8a9c-3efcdc3e9ce5" containerName="istio-proxy" containerID="cri-o://e537136d60705f76291e4528a66c0e723533a1043618eecaba0687ef47fb1030" gracePeriod=30
Nov 04 01:32:55.383698 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:55.383656    5068 patch_prober.go:29] interesting pod/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 01:32:55.384100 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:55.383712    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p" podUID="332ac8c5-1996-489e-8a9c-3efcdc3e9ce5" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 01:32:55.485913 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:55.485872    5068 patch_prober.go:29] interesting pod/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 01:32:55.486084 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:55.485926    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w" podUID="58b0b993-d096-479a-9cc8-907633a20652" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 01:32:58.417084 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.417047    5068 generic.go:356] "Generic (PLEG): container finished" podID="332ac8c5-1996-489e-8a9c-3efcdc3e9ce5" containerID="e537136d60705f76291e4528a66c0e723533a1043618eecaba0687ef47fb1030" exitCode=0
Nov 04 01:32:58.417497 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.417116    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p" event={"ID":"332ac8c5-1996-489e-8a9c-3efcdc3e9ce5","Type":"ContainerDied","Data":"e537136d60705f76291e4528a66c0e723533a1043618eecaba0687ef47fb1030"}
Nov 04 01:32:58.419700 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.419259    5068 generic.go:356] "Generic (PLEG): container finished" podID="58b0b993-d096-479a-9cc8-907633a20652" containerID="a7eede906a16647bc27b5dcfd88316037dc9200979a03f9e11dbbbc70a422b8f" exitCode=0
Nov 04 01:32:58.419700 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.419325    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w" event={"ID":"58b0b993-d096-479a-9cc8-907633a20652","Type":"ContainerDied","Data":"a7eede906a16647bc27b5dcfd88316037dc9200979a03f9e11dbbbc70a422b8f"}
Nov 04 01:32:58.565314 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.565285    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w"
Nov 04 01:32:58.606871 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.606847    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p"
Nov 04 01:32:58.608826 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.608803    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/58b0b993-d096-479a-9cc8-907633a20652-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58b0b993-d096-479a-9cc8-907633a20652-istio-data\") pod \"58b0b993-d096-479a-9cc8-907633a20652\" (UID: \"58b0b993-d096-479a-9cc8-907633a20652\") "
Nov 04 01:32:58.608933 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.608906    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/58b0b993-d096-479a-9cc8-907633a20652-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58b0b993-d096-479a-9cc8-907633a20652-workload-socket\") pod \"58b0b993-d096-479a-9cc8-907633a20652\" (UID: \"58b0b993-d096-479a-9cc8-907633a20652\") "
Nov 04 01:32:58.608986 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.608942    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/58b0b993-d096-479a-9cc8-907633a20652-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58b0b993-d096-479a-9cc8-907633a20652-workload-certs\") pod \"58b0b993-d096-479a-9cc8-907633a20652\" (UID: \"58b0b993-d096-479a-9cc8-907633a20652\") "
Nov 04 01:32:58.609036 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.608989    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/58b0b993-d096-479a-9cc8-907633a20652-istio-token\" (UniqueName: \"kubernetes.io/projected/58b0b993-d096-479a-9cc8-907633a20652-istio-token\") pod \"58b0b993-d096-479a-9cc8-907633a20652\" (UID: \"58b0b993-d096-479a-9cc8-907633a20652\") "
Nov 04 01:32:58.609036 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.609021    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/58b0b993-d096-479a-9cc8-907633a20652-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58b0b993-d096-479a-9cc8-907633a20652-credential-socket\") pod \"58b0b993-d096-479a-9cc8-907633a20652\" (UID: \"58b0b993-d096-479a-9cc8-907633a20652\") "
Nov 04 01:32:58.609143 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.609087    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/58b0b993-d096-479a-9cc8-907633a20652-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58b0b993-d096-479a-9cc8-907633a20652-istiod-ca-cert\") pod \"58b0b993-d096-479a-9cc8-907633a20652\" (UID: \"58b0b993-d096-479a-9cc8-907633a20652\") "
Nov 04 01:32:58.609143 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.609086    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58b0b993-d096-479a-9cc8-907633a20652-istio-data" pod "58b0b993-d096-479a-9cc8-907633a20652" (UID: "58b0b993-d096-479a-9cc8-907633a20652"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:32:58.609143 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.609117    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/58b0b993-d096-479a-9cc8-907633a20652-kube-api-access-x5pmd\" (UniqueName: \"kubernetes.io/projected/58b0b993-d096-479a-9cc8-907633a20652-kube-api-access-x5pmd\") pod \"58b0b993-d096-479a-9cc8-907633a20652\" (UID: \"58b0b993-d096-479a-9cc8-907633a20652\") "
Nov 04 01:32:58.609288 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.609200    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58b0b993-d096-479a-9cc8-907633a20652-workload-socket" pod "58b0b993-d096-479a-9cc8-907633a20652" (UID: "58b0b993-d096-479a-9cc8-907633a20652"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:32:58.609288 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.609208    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/58b0b993-d096-479a-9cc8-907633a20652-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58b0b993-d096-479a-9cc8-907633a20652-istio-envoy\") pod \"58b0b993-d096-479a-9cc8-907633a20652\" (UID: \"58b0b993-d096-479a-9cc8-907633a20652\") "
Nov 04 01:32:58.609288 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.609256    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/58b0b993-d096-479a-9cc8-907633a20652-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58b0b993-d096-479a-9cc8-907633a20652-istio-podinfo\") pod \"58b0b993-d096-479a-9cc8-907633a20652\" (UID: \"58b0b993-d096-479a-9cc8-907633a20652\") "
Nov 04 01:32:58.609451 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.609356    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58b0b993-d096-479a-9cc8-907633a20652-workload-certs" pod "58b0b993-d096-479a-9cc8-907633a20652" (UID: "58b0b993-d096-479a-9cc8-907633a20652"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:32:58.609506 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.609474    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58b0b993-d096-479a-9cc8-907633a20652-istiod-ca-cert" pod "58b0b993-d096-479a-9cc8-907633a20652" (UID: "58b0b993-d096-479a-9cc8-907633a20652"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:32:58.609845 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.609813    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58b0b993-d096-479a-9cc8-907633a20652-credential-socket" pod "58b0b993-d096-479a-9cc8-907633a20652" (UID: "58b0b993-d096-479a-9cc8-907633a20652"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:32:58.609952 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.609910    5068 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58b0b993-d096-479a-9cc8-907633a20652-istiod-ca-cert\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:58.609952 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.609930    5068 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58b0b993-d096-479a-9cc8-907633a20652-istio-data\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:58.609952 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.609943    5068 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58b0b993-d096-479a-9cc8-907633a20652-workload-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:58.609952 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.609953    5068 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58b0b993-d096-479a-9cc8-907633a20652-workload-certs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:58.611835 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.611810    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58b0b993-d096-479a-9cc8-907633a20652-istio-envoy" pod "58b0b993-d096-479a-9cc8-907633a20652" (UID: "58b0b993-d096-479a-9cc8-907633a20652"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:32:58.611985 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.611967    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58b0b993-d096-479a-9cc8-907633a20652-kube-api-access-x5pmd" pod "58b0b993-d096-479a-9cc8-907633a20652" (UID: "58b0b993-d096-479a-9cc8-907633a20652"). InnerVolumeSpecName "kube-api-access-x5pmd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:32:58.612061 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.612016    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58b0b993-d096-479a-9cc8-907633a20652-istio-token" pod "58b0b993-d096-479a-9cc8-907633a20652" (UID: "58b0b993-d096-479a-9cc8-907633a20652"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:32:58.612190 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.612169    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/58b0b993-d096-479a-9cc8-907633a20652-istio-podinfo" pod "58b0b993-d096-479a-9cc8-907633a20652" (UID: "58b0b993-d096-479a-9cc8-907633a20652"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:32:58.711312 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.711280    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-istio-envoy\") pod \"332ac8c5-1996-489e-8a9c-3efcdc3e9ce5\" (UID: \"332ac8c5-1996-489e-8a9c-3efcdc3e9ce5\") "
Nov 04 01:32:58.711503 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.711319    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-workload-socket\") pod \"332ac8c5-1996-489e-8a9c-3efcdc3e9ce5\" (UID: \"332ac8c5-1996-489e-8a9c-3efcdc3e9ce5\") "
Nov 04 01:32:58.711503 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.711341    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-istiod-ca-cert\") pod \"332ac8c5-1996-489e-8a9c-3efcdc3e9ce5\" (UID: \"332ac8c5-1996-489e-8a9c-3efcdc3e9ce5\") "
Nov 04 01:32:58.711503 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.711385    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-kube-api-access-dqgxd\" (UniqueName: \"kubernetes.io/projected/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-kube-api-access-dqgxd\") pod \"332ac8c5-1996-489e-8a9c-3efcdc3e9ce5\" (UID: \"332ac8c5-1996-489e-8a9c-3efcdc3e9ce5\") "
Nov 04 01:32:58.711503 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.711486    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-istio-podinfo\") pod \"332ac8c5-1996-489e-8a9c-3efcdc3e9ce5\" (UID: \"332ac8c5-1996-489e-8a9c-3efcdc3e9ce5\") "
Nov 04 01:32:58.711780 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.711620    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-credential-socket\") pod \"332ac8c5-1996-489e-8a9c-3efcdc3e9ce5\" (UID: \"332ac8c5-1996-489e-8a9c-3efcdc3e9ce5\") "
Nov 04 01:32:58.711780 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.711677    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-workload-socket" pod "332ac8c5-1996-489e-8a9c-3efcdc3e9ce5" (UID: "332ac8c5-1996-489e-8a9c-3efcdc3e9ce5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:32:58.711780 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.711712    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-workload-certs\") pod \"332ac8c5-1996-489e-8a9c-3efcdc3e9ce5\" (UID: \"332ac8c5-1996-489e-8a9c-3efcdc3e9ce5\") "
Nov 04 01:32:58.711954 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.711786    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-istio-data\") pod \"332ac8c5-1996-489e-8a9c-3efcdc3e9ce5\" (UID: \"332ac8c5-1996-489e-8a9c-3efcdc3e9ce5\") "
Nov 04 01:32:58.711954 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.711819    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-istio-token\" (UniqueName: \"kubernetes.io/projected/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-istio-token\") pod \"332ac8c5-1996-489e-8a9c-3efcdc3e9ce5\" (UID: \"332ac8c5-1996-489e-8a9c-3efcdc3e9ce5\") "
Nov 04 01:32:58.711954 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.711902    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-workload-certs" pod "332ac8c5-1996-489e-8a9c-3efcdc3e9ce5" (UID: "332ac8c5-1996-489e-8a9c-3efcdc3e9ce5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:32:58.711954 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.711927    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-istiod-ca-cert" pod "332ac8c5-1996-489e-8a9c-3efcdc3e9ce5" (UID: "332ac8c5-1996-489e-8a9c-3efcdc3e9ce5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:32:58.712166 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.712127    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-istio-data" pod "332ac8c5-1996-489e-8a9c-3efcdc3e9ce5" (UID: "332ac8c5-1996-489e-8a9c-3efcdc3e9ce5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:32:58.712166 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.712139    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-credential-socket" pod "332ac8c5-1996-489e-8a9c-3efcdc3e9ce5" (UID: "332ac8c5-1996-489e-8a9c-3efcdc3e9ce5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:32:58.712441 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.712424    5068 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58b0b993-d096-479a-9cc8-907633a20652-istio-token\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:58.712518 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.712448    5068 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58b0b993-d096-479a-9cc8-907633a20652-credential-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:58.712518 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.712465    5068 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-credential-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:58.712518 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.712479    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x5pmd\" (UniqueName: \"kubernetes.io/projected/58b0b993-d096-479a-9cc8-907633a20652-kube-api-access-x5pmd\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:58.712518 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.712494    5068 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-workload-certs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:58.712518 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.712508    5068 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-istio-data\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:58.712768 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.712522    5068 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58b0b993-d096-479a-9cc8-907633a20652-istio-envoy\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:58.712768 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.712535    5068 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58b0b993-d096-479a-9cc8-907633a20652-istio-podinfo\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:58.712768 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.712549    5068 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-workload-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:58.712768 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.712563    5068 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-istiod-ca-cert\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:58.713908 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.713888    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-istio-envoy" pod "332ac8c5-1996-489e-8a9c-3efcdc3e9ce5" (UID: "332ac8c5-1996-489e-8a9c-3efcdc3e9ce5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:32:58.714175 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.714150    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-istio-podinfo" pod "332ac8c5-1996-489e-8a9c-3efcdc3e9ce5" (UID: "332ac8c5-1996-489e-8a9c-3efcdc3e9ce5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:32:58.714252 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.714185    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-kube-api-access-dqgxd" pod "332ac8c5-1996-489e-8a9c-3efcdc3e9ce5" (UID: "332ac8c5-1996-489e-8a9c-3efcdc3e9ce5"). InnerVolumeSpecName "kube-api-access-dqgxd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:32:58.714516 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.714496    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-istio-token" pod "332ac8c5-1996-489e-8a9c-3efcdc3e9ce5" (UID: "332ac8c5-1996-489e-8a9c-3efcdc3e9ce5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:32:58.813739 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.813658    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dqgxd\" (UniqueName: \"kubernetes.io/projected/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-kube-api-access-dqgxd\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:58.813739 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.813686    5068 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-istio-podinfo\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:58.813739 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.813700    5068 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-istio-token\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:58.813739 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:58.813712    5068 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5-istio-envoy\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:59.427644 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:59.427580    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w"
Nov 04 01:32:59.427644 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:59.427593    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w" event={"ID":"58b0b993-d096-479a-9cc8-907633a20652","Type":"ContainerDied","Data":"f2e4c8c5d0519c002ab58b549232c893b7276cb384903b666af17226ca837df2"}
Nov 04 01:32:59.428182 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:59.427701    5068 scope.go:122] "RemoveContainer" containerID="a7eede906a16647bc27b5dcfd88316037dc9200979a03f9e11dbbbc70a422b8f"
Nov 04 01:32:59.429262 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:59.429218    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p"
Nov 04 01:32:59.429362 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:59.429249    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p" event={"ID":"332ac8c5-1996-489e-8a9c-3efcdc3e9ce5","Type":"ContainerDied","Data":"c57cb378f3e12feda9da3149c056ae9e3bb981f60feffa3a77e374e809376781"}
Nov 04 01:32:59.437966 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:59.437946    5068 scope.go:122] "RemoveContainer" containerID="e537136d60705f76291e4528a66c0e723533a1043618eecaba0687ef47fb1030"
Nov 04 01:32:59.463063 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:59.463039    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p"]
Nov 04 01:32:59.467684 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:59.467660    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-lgp8p"]
Nov 04 01:32:59.479724 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:59.479698    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w"]
Nov 04 01:32:59.484256 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:32:59.484234    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-8974fa-openshift-default-78d69f7b89-pz76w"]
Nov 04 01:33:00.113891 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:00.113861    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="332ac8c5-1996-489e-8a9c-3efcdc3e9ce5" path="/var/lib/kubelet/pods/332ac8c5-1996-489e-8a9c-3efcdc3e9ce5/volumes"
Nov 04 01:33:00.114353 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:00.114339    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="58b0b993-d096-479a-9cc8-907633a20652" path="/var/lib/kubelet/pods/58b0b993-d096-479a-9cc8-907633a20652/volumes"
Nov 04 01:33:03.372254 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:03.372221    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-35980d-86c7c666d4-bphw8"]
Nov 04 01:33:03.388841 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:03.386692    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w0-35980d-97689d97f-79zvl"]
Nov 04 01:33:05.996718 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:05.996659    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-35980d-64c6d6dc7d-j2s9z"]
Nov 04 01:33:05.997239 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:05.996917    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-35980d-64c6d6dc7d-j2s9z" podUID="1ac72a77-4112-4df4-ada5-ea100fe5dfb2" containerName="maas-api" containerID="cri-o://80a657cbe2cb99374f1212521d12dcf6951caa5b6fb3dfd71a2dadb62d1c3190" gracePeriod=30
Nov 04 01:33:06.358689 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:06.358656    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-35980d-64c6d6dc7d-j2s9z"
Nov 04 01:33:06.427519 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:06.427481    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1ac72a77-4112-4df4-ada5-ea100fe5dfb2-metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ac72a77-4112-4df4-ada5-ea100fe5dfb2-metrics-certs\") pod \"1ac72a77-4112-4df4-ada5-ea100fe5dfb2\" (UID: \"1ac72a77-4112-4df4-ada5-ea100fe5dfb2\") "
Nov 04 01:33:06.427759 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:06.427534    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1ac72a77-4112-4df4-ada5-ea100fe5dfb2-kube-api-access-cszvz\" (UniqueName: \"kubernetes.io/projected/1ac72a77-4112-4df4-ada5-ea100fe5dfb2-kube-api-access-cszvz\") pod \"1ac72a77-4112-4df4-ada5-ea100fe5dfb2\" (UID: \"1ac72a77-4112-4df4-ada5-ea100fe5dfb2\") "
Nov 04 01:33:06.427759 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:06.427707    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1ac72a77-4112-4df4-ada5-ea100fe5dfb2-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ac72a77-4112-4df4-ada5-ea100fe5dfb2-openshift-service-ca\") pod \"1ac72a77-4112-4df4-ada5-ea100fe5dfb2\" (UID: \"1ac72a77-4112-4df4-ada5-ea100fe5dfb2\") "
Nov 04 01:33:06.427889 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:06.427819    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1ac72a77-4112-4df4-ada5-ea100fe5dfb2-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ac72a77-4112-4df4-ada5-ea100fe5dfb2-maas-api-tls\") pod \"1ac72a77-4112-4df4-ada5-ea100fe5dfb2\" (UID: \"1ac72a77-4112-4df4-ada5-ea100fe5dfb2\") "
Nov 04 01:33:06.428151 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:06.428124    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ac72a77-4112-4df4-ada5-ea100fe5dfb2-openshift-service-ca" pod "1ac72a77-4112-4df4-ada5-ea100fe5dfb2" (UID: "1ac72a77-4112-4df4-ada5-ea100fe5dfb2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:33:06.428319 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:06.428281    5068 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ac72a77-4112-4df4-ada5-ea100fe5dfb2-openshift-service-ca\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:06.430044 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:06.430009    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ac72a77-4112-4df4-ada5-ea100fe5dfb2-kube-api-access-cszvz" pod "1ac72a77-4112-4df4-ada5-ea100fe5dfb2" (UID: "1ac72a77-4112-4df4-ada5-ea100fe5dfb2"). InnerVolumeSpecName "kube-api-access-cszvz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:33:06.430179 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:06.430148    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ac72a77-4112-4df4-ada5-ea100fe5dfb2-maas-api-tls" pod "1ac72a77-4112-4df4-ada5-ea100fe5dfb2" (UID: "1ac72a77-4112-4df4-ada5-ea100fe5dfb2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:33:06.430308 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:06.430192    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ac72a77-4112-4df4-ada5-ea100fe5dfb2-metrics-certs" pod "1ac72a77-4112-4df4-ada5-ea100fe5dfb2" (UID: "1ac72a77-4112-4df4-ada5-ea100fe5dfb2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:33:06.493451 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:06.493418    5068 generic.go:356] "Generic (PLEG): container finished" podID="1ac72a77-4112-4df4-ada5-ea100fe5dfb2" containerID="80a657cbe2cb99374f1212521d12dcf6951caa5b6fb3dfd71a2dadb62d1c3190" exitCode=0
Nov 04 01:33:06.493643 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:06.493464    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-35980d-64c6d6dc7d-j2s9z"
Nov 04 01:33:06.493643 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:06.493469    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-35980d-64c6d6dc7d-j2s9z" event={"ID":"1ac72a77-4112-4df4-ada5-ea100fe5dfb2","Type":"ContainerDied","Data":"80a657cbe2cb99374f1212521d12dcf6951caa5b6fb3dfd71a2dadb62d1c3190"}
Nov 04 01:33:06.493643 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:06.493568    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-35980d-64c6d6dc7d-j2s9z" event={"ID":"1ac72a77-4112-4df4-ada5-ea100fe5dfb2","Type":"ContainerDied","Data":"b320bf44a930e7e6b1cdbd5a3a59652cf1a3a5edc72e55b423eea3ccdd087617"}
Nov 04 01:33:06.493643 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:06.493590    5068 scope.go:122] "RemoveContainer" containerID="80a657cbe2cb99374f1212521d12dcf6951caa5b6fb3dfd71a2dadb62d1c3190"
Nov 04 01:33:06.509787 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:06.509767    5068 scope.go:122] "RemoveContainer" containerID="80a657cbe2cb99374f1212521d12dcf6951caa5b6fb3dfd71a2dadb62d1c3190"
Nov 04 01:33:06.510081 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:33:06.510060    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80a657cbe2cb99374f1212521d12dcf6951caa5b6fb3dfd71a2dadb62d1c3190\": container with ID starting with 80a657cbe2cb99374f1212521d12dcf6951caa5b6fb3dfd71a2dadb62d1c3190 not found: ID does not exist" containerID="80a657cbe2cb99374f1212521d12dcf6951caa5b6fb3dfd71a2dadb62d1c3190"
Nov 04 01:33:06.510136 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:06.510089    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80a657cbe2cb99374f1212521d12dcf6951caa5b6fb3dfd71a2dadb62d1c3190"} err="failed to get container status \"80a657cbe2cb99374f1212521d12dcf6951caa5b6fb3dfd71a2dadb62d1c3190\": rpc error: code = NotFound desc = could not find container \"80a657cbe2cb99374f1212521d12dcf6951caa5b6fb3dfd71a2dadb62d1c3190\": container with ID starting with 80a657cbe2cb99374f1212521d12dcf6951caa5b6fb3dfd71a2dadb62d1c3190 not found: ID does not exist"
Nov 04 01:33:06.520861 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:06.520829    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-35980d-64c6d6dc7d-j2s9z"]
Nov 04 01:33:06.524261 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:06.524238    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-35980d-64c6d6dc7d-j2s9z"]
Nov 04 01:33:06.530008 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:06.529987    5068 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ac72a77-4112-4df4-ada5-ea100fe5dfb2-maas-api-tls\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:06.530008 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:06.530009    5068 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ac72a77-4112-4df4-ada5-ea100fe5dfb2-metrics-certs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:06.530121 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:06.530021    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cszvz\" (UniqueName: \"kubernetes.io/projected/1ac72a77-4112-4df4-ada5-ea100fe5dfb2-kube-api-access-cszvz\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:07.169368 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:07.169320    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:33:08.113679 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:08.113644    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1ac72a77-4112-4df4-ada5-ea100fe5dfb2" path="/var/lib/kubelet/pods/1ac72a77-4112-4df4-ada5-ea100fe5dfb2/volumes"
Nov 04 01:33:08.373249 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:08.373145    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-35980d-86c7c666d4-bphw8" podUID="788ddd8c-788c-4575-8208-fd949c0bcc38" containerName="payload-processing" containerID="cri-o://cb04e6904827a62277a15e0cde41bf75fffe9c369d86b99af009e928a14c8070" gracePeriod=25
Nov 04 01:33:08.387794 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:08.387759    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-35980d-97689d97f-79zvl" podUID="34c18742-f9bd-4f63-8e99-6a63af9c5999" containerName="payload-pre-processing" containerID="cri-o://14a5bc89d5ef6b9a779cf9e51da432801213a60c157f9edbaf58b3941efdb64f" gracePeriod=25
Nov 04 01:33:08.514073 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:08.514044    5068 generic.go:356] "Generic (PLEG): container finished" podID="34c18742-f9bd-4f63-8e99-6a63af9c5999" containerID="14a5bc89d5ef6b9a779cf9e51da432801213a60c157f9edbaf58b3941efdb64f" exitCode=0
Nov 04 01:33:08.514290 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:08.514222    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-35980d-97689d97f-79zvl" event={"ID":"34c18742-f9bd-4f63-8e99-6a63af9c5999","Type":"ContainerDied","Data":"14a5bc89d5ef6b9a779cf9e51da432801213a60c157f9edbaf58b3941efdb64f"}
Nov 04 01:33:08.519025 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:08.518346    5068 generic.go:356] "Generic (PLEG): container finished" podID="788ddd8c-788c-4575-8208-fd949c0bcc38" containerID="cb04e6904827a62277a15e0cde41bf75fffe9c369d86b99af009e928a14c8070" exitCode=0
Nov 04 01:33:08.519025 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:08.518484    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-35980d-86c7c666d4-bphw8" event={"ID":"788ddd8c-788c-4575-8208-fd949c0bcc38","Type":"ContainerDied","Data":"cb04e6904827a62277a15e0cde41bf75fffe9c369d86b99af009e928a14c8070"}
Nov 04 01:33:08.553716 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:08.552572    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds"]
Nov 04 01:33:08.553716 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:08.552890    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds" podUID="d091cbea-bbd1-4242-a1c9-f72628c8e019" containerName="istio-proxy" containerID="cri-o://eaa4c5eea767c3ccf8446ae61f36401d100c1a10573e89fb9c86a1b43f53ba69" gracePeriod=30
Nov 04 01:33:08.819120 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:08.819091    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-35980d-86c7c666d4-bphw8"
Nov 04 01:33:08.822692 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:08.822672    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-35980d-97689d97f-79zvl"
Nov 04 01:33:08.907735 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:08.907704    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/788ddd8c-788c-4575-8208-fd949c0bcc38-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/788ddd8c-788c-4575-8208-fd949c0bcc38-plugins-config-volume\") pod \"788ddd8c-788c-4575-8208-fd949c0bcc38\" (UID: \"788ddd8c-788c-4575-8208-fd949c0bcc38\") "
Nov 04 01:33:08.907927 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:08.907823    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/788ddd8c-788c-4575-8208-fd949c0bcc38-kube-api-access-jrzv9\" (UniqueName: \"kubernetes.io/projected/788ddd8c-788c-4575-8208-fd949c0bcc38-kube-api-access-jrzv9\") pod \"788ddd8c-788c-4575-8208-fd949c0bcc38\" (UID: \"788ddd8c-788c-4575-8208-fd949c0bcc38\") "
Nov 04 01:33:08.908250 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:08.908224    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/788ddd8c-788c-4575-8208-fd949c0bcc38-plugins-config-volume" pod "788ddd8c-788c-4575-8208-fd949c0bcc38" (UID: "788ddd8c-788c-4575-8208-fd949c0bcc38"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:33:08.910054 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:08.910030    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/788ddd8c-788c-4575-8208-fd949c0bcc38-kube-api-access-jrzv9" pod "788ddd8c-788c-4575-8208-fd949c0bcc38" (UID: "788ddd8c-788c-4575-8208-fd949c0bcc38"). InnerVolumeSpecName "kube-api-access-jrzv9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:33:09.009174 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:09.009132    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/34c18742-f9bd-4f63-8e99-6a63af9c5999-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34c18742-f9bd-4f63-8e99-6a63af9c5999-plugins-config-volume\") pod \"34c18742-f9bd-4f63-8e99-6a63af9c5999\" (UID: \"34c18742-f9bd-4f63-8e99-6a63af9c5999\") "
Nov 04 01:33:09.009378 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:09.009330    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/34c18742-f9bd-4f63-8e99-6a63af9c5999-kube-api-access-q6wj8\" (UniqueName: \"kubernetes.io/projected/34c18742-f9bd-4f63-8e99-6a63af9c5999-kube-api-access-q6wj8\") pod \"34c18742-f9bd-4f63-8e99-6a63af9c5999\" (UID: \"34c18742-f9bd-4f63-8e99-6a63af9c5999\") "
Nov 04 01:33:09.009663 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:09.009597    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34c18742-f9bd-4f63-8e99-6a63af9c5999-plugins-config-volume" pod "34c18742-f9bd-4f63-8e99-6a63af9c5999" (UID: "34c18742-f9bd-4f63-8e99-6a63af9c5999"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:33:09.009799 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:09.009782    5068 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/788ddd8c-788c-4575-8208-fd949c0bcc38-plugins-config-volume\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:09.009853 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:09.009802    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jrzv9\" (UniqueName: \"kubernetes.io/projected/788ddd8c-788c-4575-8208-fd949c0bcc38-kube-api-access-jrzv9\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:09.009853 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:09.009812    5068 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34c18742-f9bd-4f63-8e99-6a63af9c5999-plugins-config-volume\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:09.011453 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:09.011434    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34c18742-f9bd-4f63-8e99-6a63af9c5999-kube-api-access-q6wj8" pod "34c18742-f9bd-4f63-8e99-6a63af9c5999" (UID: "34c18742-f9bd-4f63-8e99-6a63af9c5999"). InnerVolumeSpecName "kube-api-access-q6wj8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:33:09.111186 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:09.111101    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q6wj8\" (UniqueName: \"kubernetes.io/projected/34c18742-f9bd-4f63-8e99-6a63af9c5999-kube-api-access-q6wj8\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:09.547644 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:09.535257    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-35980d-97689d97f-79zvl" event={"ID":"34c18742-f9bd-4f63-8e99-6a63af9c5999","Type":"ContainerDied","Data":"85592b8fb5cf2d1a9d87721364945868692a402b755a0411d5795d95ceca791a"}
Nov 04 01:33:09.547644 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:09.535344    5068 scope.go:122] "RemoveContainer" containerID="14a5bc89d5ef6b9a779cf9e51da432801213a60c157f9edbaf58b3941efdb64f"
Nov 04 01:33:09.547644 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:09.535530    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-35980d-97689d97f-79zvl"
Nov 04 01:33:09.561550 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:09.560564    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-35980d-86c7c666d4-bphw8" event={"ID":"788ddd8c-788c-4575-8208-fd949c0bcc38","Type":"ContainerDied","Data":"737a101b8059408e2cd997c96145df8c0fb86fd0edb12a656d72100492c7cc36"}
Nov 04 01:33:09.561550 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:09.560748    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-35980d-86c7c666d4-bphw8"
Nov 04 01:33:09.575668 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:09.575129    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w0-35980d-97689d97f-79zvl"]
Nov 04 01:33:09.578682 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:09.578128    5068 scope.go:122] "RemoveContainer" containerID="cb04e6904827a62277a15e0cde41bf75fffe9c369d86b99af009e928a14c8070"
Nov 04 01:33:09.579680 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:09.579658    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w0-35980d-97689d97f-79zvl"]
Nov 04 01:33:09.593546 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:09.593496    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-35980d-86c7c666d4-bphw8"]
Nov 04 01:33:09.597535 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:09.597510    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-35980d-86c7c666d4-bphw8"]
Nov 04 01:33:09.758593 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:09.758507    5068 patch_prober.go:29] interesting pod/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 01:33:09.758752 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:09.758585    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds" podUID="d091cbea-bbd1-4242-a1c9-f72628c8e019" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 01:33:10.113918 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:10.113883    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="34c18742-f9bd-4f63-8e99-6a63af9c5999" path="/var/lib/kubelet/pods/34c18742-f9bd-4f63-8e99-6a63af9c5999/volumes"
Nov 04 01:33:10.114344 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:10.114330    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="788ddd8c-788c-4575-8208-fd949c0bcc38" path="/var/lib/kubelet/pods/788ddd8c-788c-4575-8208-fd949c0bcc38/volumes"
Nov 04 01:33:13.450701 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:33:13.450664    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:33:13.830587 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:13.830559    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds"
Nov 04 01:33:13.874249 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:13.874209    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d091cbea-bbd1-4242-a1c9-f72628c8e019-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d091cbea-bbd1-4242-a1c9-f72628c8e019-istio-envoy\") pod \"d091cbea-bbd1-4242-a1c9-f72628c8e019\" (UID: \"d091cbea-bbd1-4242-a1c9-f72628c8e019\") "
Nov 04 01:33:13.874431 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:13.874268    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d091cbea-bbd1-4242-a1c9-f72628c8e019-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d091cbea-bbd1-4242-a1c9-f72628c8e019-workload-socket\") pod \"d091cbea-bbd1-4242-a1c9-f72628c8e019\" (UID: \"d091cbea-bbd1-4242-a1c9-f72628c8e019\") "
Nov 04 01:33:13.874431 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:13.874361    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d091cbea-bbd1-4242-a1c9-f72628c8e019-istio-token\" (UniqueName: \"kubernetes.io/projected/d091cbea-bbd1-4242-a1c9-f72628c8e019-istio-token\") pod \"d091cbea-bbd1-4242-a1c9-f72628c8e019\" (UID: \"d091cbea-bbd1-4242-a1c9-f72628c8e019\") "
Nov 04 01:33:13.874431 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:13.874397    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d091cbea-bbd1-4242-a1c9-f72628c8e019-kube-api-access-hcdlr\" (UniqueName: \"kubernetes.io/projected/d091cbea-bbd1-4242-a1c9-f72628c8e019-kube-api-access-hcdlr\") pod \"d091cbea-bbd1-4242-a1c9-f72628c8e019\" (UID: \"d091cbea-bbd1-4242-a1c9-f72628c8e019\") "
Nov 04 01:33:13.874657 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:13.874444    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d091cbea-bbd1-4242-a1c9-f72628c8e019-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d091cbea-bbd1-4242-a1c9-f72628c8e019-istio-podinfo\") pod \"d091cbea-bbd1-4242-a1c9-f72628c8e019\" (UID: \"d091cbea-bbd1-4242-a1c9-f72628c8e019\") "
Nov 04 01:33:13.874657 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:13.874474    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d091cbea-bbd1-4242-a1c9-f72628c8e019-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d091cbea-bbd1-4242-a1c9-f72628c8e019-istio-data\") pod \"d091cbea-bbd1-4242-a1c9-f72628c8e019\" (UID: \"d091cbea-bbd1-4242-a1c9-f72628c8e019\") "
Nov 04 01:33:13.874657 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:13.874568    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d091cbea-bbd1-4242-a1c9-f72628c8e019-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d091cbea-bbd1-4242-a1c9-f72628c8e019-credential-socket\") pod \"d091cbea-bbd1-4242-a1c9-f72628c8e019\" (UID: \"d091cbea-bbd1-4242-a1c9-f72628c8e019\") "
Nov 04 01:33:13.874657 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:13.874625    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d091cbea-bbd1-4242-a1c9-f72628c8e019-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d091cbea-bbd1-4242-a1c9-f72628c8e019-workload-certs\") pod \"d091cbea-bbd1-4242-a1c9-f72628c8e019\" (UID: \"d091cbea-bbd1-4242-a1c9-f72628c8e019\") "
Nov 04 01:33:13.874877 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:13.874661    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d091cbea-bbd1-4242-a1c9-f72628c8e019-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d091cbea-bbd1-4242-a1c9-f72628c8e019-istiod-ca-cert\") pod \"d091cbea-bbd1-4242-a1c9-f72628c8e019\" (UID: \"d091cbea-bbd1-4242-a1c9-f72628c8e019\") "
Nov 04 01:33:13.875447 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:13.875423    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d091cbea-bbd1-4242-a1c9-f72628c8e019-istiod-ca-cert" pod "d091cbea-bbd1-4242-a1c9-f72628c8e019" (UID: "d091cbea-bbd1-4242-a1c9-f72628c8e019"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:33:13.876267 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:13.876075    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d091cbea-bbd1-4242-a1c9-f72628c8e019-credential-socket" pod "d091cbea-bbd1-4242-a1c9-f72628c8e019" (UID: "d091cbea-bbd1-4242-a1c9-f72628c8e019"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:33:13.876462 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:13.876426    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d091cbea-bbd1-4242-a1c9-f72628c8e019-istio-data" pod "d091cbea-bbd1-4242-a1c9-f72628c8e019" (UID: "d091cbea-bbd1-4242-a1c9-f72628c8e019"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:33:13.876577 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:13.876546    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d091cbea-bbd1-4242-a1c9-f72628c8e019-workload-socket" pod "d091cbea-bbd1-4242-a1c9-f72628c8e019" (UID: "d091cbea-bbd1-4242-a1c9-f72628c8e019"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:33:13.876670 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:13.876598    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d091cbea-bbd1-4242-a1c9-f72628c8e019-workload-certs" pod "d091cbea-bbd1-4242-a1c9-f72628c8e019" (UID: "d091cbea-bbd1-4242-a1c9-f72628c8e019"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:33:13.878375 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:13.878005    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d091cbea-bbd1-4242-a1c9-f72628c8e019-istio-envoy" pod "d091cbea-bbd1-4242-a1c9-f72628c8e019" (UID: "d091cbea-bbd1-4242-a1c9-f72628c8e019"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:33:13.878519 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:13.878498    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d091cbea-bbd1-4242-a1c9-f72628c8e019-istio-token" pod "d091cbea-bbd1-4242-a1c9-f72628c8e019" (UID: "d091cbea-bbd1-4242-a1c9-f72628c8e019"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:33:13.878776 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:13.878758    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d091cbea-bbd1-4242-a1c9-f72628c8e019-istio-podinfo" pod "d091cbea-bbd1-4242-a1c9-f72628c8e019" (UID: "d091cbea-bbd1-4242-a1c9-f72628c8e019"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:33:13.879986 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:13.879957    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d091cbea-bbd1-4242-a1c9-f72628c8e019-kube-api-access-hcdlr" pod "d091cbea-bbd1-4242-a1c9-f72628c8e019" (UID: "d091cbea-bbd1-4242-a1c9-f72628c8e019"). InnerVolumeSpecName "kube-api-access-hcdlr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:33:13.976059 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:13.976020    5068 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d091cbea-bbd1-4242-a1c9-f72628c8e019-istio-podinfo\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:13.976059 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:13.976058    5068 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d091cbea-bbd1-4242-a1c9-f72628c8e019-istio-data\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:13.976241 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:13.976073    5068 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d091cbea-bbd1-4242-a1c9-f72628c8e019-credential-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:13.976241 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:13.976087    5068 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d091cbea-bbd1-4242-a1c9-f72628c8e019-workload-certs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:13.976241 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:13.976101    5068 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d091cbea-bbd1-4242-a1c9-f72628c8e019-istiod-ca-cert\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:13.976241 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:13.976114    5068 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d091cbea-bbd1-4242-a1c9-f72628c8e019-istio-envoy\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:13.976241 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:13.976127    5068 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d091cbea-bbd1-4242-a1c9-f72628c8e019-workload-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:13.976241 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:13.976140    5068 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d091cbea-bbd1-4242-a1c9-f72628c8e019-istio-token\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:13.976241 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:13.976154    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hcdlr\" (UniqueName: \"kubernetes.io/projected/d091cbea-bbd1-4242-a1c9-f72628c8e019-kube-api-access-hcdlr\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:14.601317 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:14.601283    5068 generic.go:356] "Generic (PLEG): container finished" podID="d091cbea-bbd1-4242-a1c9-f72628c8e019" containerID="eaa4c5eea767c3ccf8446ae61f36401d100c1a10573e89fb9c86a1b43f53ba69" exitCode=0
Nov 04 01:33:14.601908 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:14.601357    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds"
Nov 04 01:33:14.601908 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:14.601362    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds" event={"ID":"d091cbea-bbd1-4242-a1c9-f72628c8e019","Type":"ContainerDied","Data":"eaa4c5eea767c3ccf8446ae61f36401d100c1a10573e89fb9c86a1b43f53ba69"}
Nov 04 01:33:14.601908 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:14.601423    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds" event={"ID":"d091cbea-bbd1-4242-a1c9-f72628c8e019","Type":"ContainerDied","Data":"19e7ce512d603be2825ac5eccff4f78dedb5541ed8e64de0c8dba4f00a275ea0"}
Nov 04 01:33:14.601908 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:14.601447    5068 scope.go:122] "RemoveContainer" containerID="eaa4c5eea767c3ccf8446ae61f36401d100c1a10573e89fb9c86a1b43f53ba69"
Nov 04 01:33:14.612992 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:14.612965    5068 scope.go:122] "RemoveContainer" containerID="83ccac7852dbaeb0e4eb4548000acaa575b366d9683b7fddd5d982ff86117b82"
Nov 04 01:33:14.622196 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:14.622169    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds"]
Nov 04 01:33:14.622809 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:14.622786    5068 scope.go:122] "RemoveContainer" containerID="eaa4c5eea767c3ccf8446ae61f36401d100c1a10573e89fb9c86a1b43f53ba69"
Nov 04 01:33:14.623076 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:33:14.623058    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eaa4c5eea767c3ccf8446ae61f36401d100c1a10573e89fb9c86a1b43f53ba69\": container with ID starting with eaa4c5eea767c3ccf8446ae61f36401d100c1a10573e89fb9c86a1b43f53ba69 not found: ID does not exist" containerID="eaa4c5eea767c3ccf8446ae61f36401d100c1a10573e89fb9c86a1b43f53ba69"
Nov 04 01:33:14.623150 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:14.623083    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eaa4c5eea767c3ccf8446ae61f36401d100c1a10573e89fb9c86a1b43f53ba69"} err="failed to get container status \"eaa4c5eea767c3ccf8446ae61f36401d100c1a10573e89fb9c86a1b43f53ba69\": rpc error: code = NotFound desc = could not find container \"eaa4c5eea767c3ccf8446ae61f36401d100c1a10573e89fb9c86a1b43f53ba69\": container with ID starting with eaa4c5eea767c3ccf8446ae61f36401d100c1a10573e89fb9c86a1b43f53ba69 not found: ID does not exist"
Nov 04 01:33:14.623150 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:14.623101    5068 scope.go:122] "RemoveContainer" containerID="83ccac7852dbaeb0e4eb4548000acaa575b366d9683b7fddd5d982ff86117b82"
Nov 04 01:33:14.623349 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:33:14.623334    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83ccac7852dbaeb0e4eb4548000acaa575b366d9683b7fddd5d982ff86117b82\": container with ID starting with 83ccac7852dbaeb0e4eb4548000acaa575b366d9683b7fddd5d982ff86117b82 not found: ID does not exist" containerID="83ccac7852dbaeb0e4eb4548000acaa575b366d9683b7fddd5d982ff86117b82"
Nov 04 01:33:14.623392 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:14.623352    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83ccac7852dbaeb0e4eb4548000acaa575b366d9683b7fddd5d982ff86117b82"} err="failed to get container status \"83ccac7852dbaeb0e4eb4548000acaa575b366d9683b7fddd5d982ff86117b82\": rpc error: code = NotFound desc = could not find container \"83ccac7852dbaeb0e4eb4548000acaa575b366d9683b7fddd5d982ff86117b82\": container with ID starting with 83ccac7852dbaeb0e4eb4548000acaa575b366d9683b7fddd5d982ff86117b82 not found: ID does not exist"
Nov 04 01:33:14.628910 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:14.628886    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-2aefc5-openshift-default-859f95fb89-prvds"]
Nov 04 01:33:16.112337 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:16.112304    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d091cbea-bbd1-4242-a1c9-f72628c8e019" path="/var/lib/kubelet/pods/d091cbea-bbd1-4242-a1c9-f72628c8e019/volumes"
Nov 04 01:33:16.708581 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:16.708544    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:33:17.924076 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:17.924036    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0741ef-f87fdfbbb-cqfcq"]
Nov 04 01:33:17.925710 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:17.924336    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0741ef-f87fdfbbb-cqfcq" podUID="4f6317cf-be73-46fc-bf2d-b694dc34e258" containerName="maas-api" containerID="cri-o://89e72ef21de86aa4fdebedc38ce6f3020505fc12f1b6614d0478f86fa31b374d" gracePeriod=30
Nov 04 01:33:18.402216 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:18.402188    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0741ef-f87fdfbbb-cqfcq"
Nov 04 01:33:18.440550 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:18.440509    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4f6317cf-be73-46fc-bf2d-b694dc34e258-metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f6317cf-be73-46fc-bf2d-b694dc34e258-metrics-certs\") pod \"4f6317cf-be73-46fc-bf2d-b694dc34e258\" (UID: \"4f6317cf-be73-46fc-bf2d-b694dc34e258\") "
Nov 04 01:33:18.440756 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:18.440660    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4f6317cf-be73-46fc-bf2d-b694dc34e258-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f6317cf-be73-46fc-bf2d-b694dc34e258-maas-api-tls\") pod \"4f6317cf-be73-46fc-bf2d-b694dc34e258\" (UID: \"4f6317cf-be73-46fc-bf2d-b694dc34e258\") "
Nov 04 01:33:18.440828 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:18.440783    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4f6317cf-be73-46fc-bf2d-b694dc34e258-kube-api-access-vqmxr\" (UniqueName: \"kubernetes.io/projected/4f6317cf-be73-46fc-bf2d-b694dc34e258-kube-api-access-vqmxr\") pod \"4f6317cf-be73-46fc-bf2d-b694dc34e258\" (UID: \"4f6317cf-be73-46fc-bf2d-b694dc34e258\") "
Nov 04 01:33:18.440885 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:18.440839    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4f6317cf-be73-46fc-bf2d-b694dc34e258-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f6317cf-be73-46fc-bf2d-b694dc34e258-openshift-service-ca\") pod \"4f6317cf-be73-46fc-bf2d-b694dc34e258\" (UID: \"4f6317cf-be73-46fc-bf2d-b694dc34e258\") "
Nov 04 01:33:18.441373 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:18.441345    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f6317cf-be73-46fc-bf2d-b694dc34e258-openshift-service-ca" pod "4f6317cf-be73-46fc-bf2d-b694dc34e258" (UID: "4f6317cf-be73-46fc-bf2d-b694dc34e258"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:33:18.443225 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:18.443160    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f6317cf-be73-46fc-bf2d-b694dc34e258-kube-api-access-vqmxr" pod "4f6317cf-be73-46fc-bf2d-b694dc34e258" (UID: "4f6317cf-be73-46fc-bf2d-b694dc34e258"). InnerVolumeSpecName "kube-api-access-vqmxr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:33:18.443529 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:18.443420    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f6317cf-be73-46fc-bf2d-b694dc34e258-maas-api-tls" pod "4f6317cf-be73-46fc-bf2d-b694dc34e258" (UID: "4f6317cf-be73-46fc-bf2d-b694dc34e258"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:33:18.446014 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:18.445976    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f6317cf-be73-46fc-bf2d-b694dc34e258-metrics-certs" pod "4f6317cf-be73-46fc-bf2d-b694dc34e258" (UID: "4f6317cf-be73-46fc-bf2d-b694dc34e258"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:33:18.542601 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:18.542556    5068 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f6317cf-be73-46fc-bf2d-b694dc34e258-metrics-certs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:18.542601 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:18.542598    5068 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f6317cf-be73-46fc-bf2d-b694dc34e258-maas-api-tls\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:18.542601 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:18.542623    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vqmxr\" (UniqueName: \"kubernetes.io/projected/4f6317cf-be73-46fc-bf2d-b694dc34e258-kube-api-access-vqmxr\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:18.542948 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:18.542636    5068 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f6317cf-be73-46fc-bf2d-b694dc34e258-openshift-service-ca\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:18.627417 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:18.627334    5068 generic.go:356] "Generic (PLEG): container finished" podID="4f6317cf-be73-46fc-bf2d-b694dc34e258" containerID="89e72ef21de86aa4fdebedc38ce6f3020505fc12f1b6614d0478f86fa31b374d" exitCode=0
Nov 04 01:33:18.627417 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:18.627378    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0741ef-f87fdfbbb-cqfcq"
Nov 04 01:33:18.627585 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:18.627424    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0741ef-f87fdfbbb-cqfcq" event={"ID":"4f6317cf-be73-46fc-bf2d-b694dc34e258","Type":"ContainerDied","Data":"89e72ef21de86aa4fdebedc38ce6f3020505fc12f1b6614d0478f86fa31b374d"}
Nov 04 01:33:18.627585 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:18.627476    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0741ef-f87fdfbbb-cqfcq" event={"ID":"4f6317cf-be73-46fc-bf2d-b694dc34e258","Type":"ContainerDied","Data":"5f46307d415725da580d186e45a25830bba5f3cecfc61f5386502f7158cdc56e"}
Nov 04 01:33:18.627585 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:18.627500    5068 scope.go:122] "RemoveContainer" containerID="89e72ef21de86aa4fdebedc38ce6f3020505fc12f1b6614d0478f86fa31b374d"
Nov 04 01:33:18.637916 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:18.637894    5068 scope.go:122] "RemoveContainer" containerID="89e72ef21de86aa4fdebedc38ce6f3020505fc12f1b6614d0478f86fa31b374d"
Nov 04 01:33:18.638206 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:33:18.638186    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89e72ef21de86aa4fdebedc38ce6f3020505fc12f1b6614d0478f86fa31b374d\": container with ID starting with 89e72ef21de86aa4fdebedc38ce6f3020505fc12f1b6614d0478f86fa31b374d not found: ID does not exist" containerID="89e72ef21de86aa4fdebedc38ce6f3020505fc12f1b6614d0478f86fa31b374d"
Nov 04 01:33:18.638293 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:18.638219    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89e72ef21de86aa4fdebedc38ce6f3020505fc12f1b6614d0478f86fa31b374d"} err="failed to get container status \"89e72ef21de86aa4fdebedc38ce6f3020505fc12f1b6614d0478f86fa31b374d\": rpc error: code = NotFound desc = could not find container \"89e72ef21de86aa4fdebedc38ce6f3020505fc12f1b6614d0478f86fa31b374d\": container with ID starting with 89e72ef21de86aa4fdebedc38ce6f3020505fc12f1b6614d0478f86fa31b374d not found: ID does not exist"
Nov 04 01:33:18.654706 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:18.654674    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0741ef-f87fdfbbb-cqfcq"]
Nov 04 01:33:18.657729 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:18.657693    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0741ef-f87fdfbbb-cqfcq"]
Nov 04 01:33:20.114537 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:20.114463    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4f6317cf-be73-46fc-bf2d-b694dc34e258" path="/var/lib/kubelet/pods/4f6317cf-be73-46fc-bf2d-b694dc34e258/volumes"
Nov 04 01:33:24.015589 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:24.015545    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-6dac99-6d46fbff94-fcfwv"]
Nov 04 01:33:29.016173 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:29.016099    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-6dac99-6d46fbff94-fcfwv" podUID="98e65cde-ca44-4ad9-9a57-ce81c0369a0f" containerName="payload-processing" containerID="cri-o://307a28b2c7e0cbeb01d080fae154cff7f63bb3598b2f756d178747115c5b319e" gracePeriod=25
Nov 04 01:33:29.030700 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:29.030665    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:33:29.899050 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:29.899017    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-5ce9e8-5cd7d58c4b-k77tv"]
Nov 04 01:33:29.915389 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:29.915358    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-5ce9e8-7bf85dc8f5-l96mq"]
Nov 04 01:33:33.420601 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:33.420563    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w0-6dac99-6d46fbff94-fcfwv" podUID="98e65cde-ca44-4ad9-9a57-ce81c0369a0f" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:33:34.899344 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:34.899286    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-5ce9e8-5cd7d58c4b-k77tv" podUID="88233bfe-a2e5-4dc2-b4ab-d6f9772f7219" containerName="payload-processing" containerID="cri-o://21788304f95e42ce848e84d5514d3a7a0ea02d135c1672e694d726baec748e43" gracePeriod=25
Nov 04 01:33:34.915938 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:34.915903    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-5ce9e8-7bf85dc8f5-l96mq" podUID="bda9909c-2480-4f83-9cef-3d14f19568ac" containerName="payload-pre-processing" containerID="cri-o://6264710adb7ce1fa23bdd9c75f6eb6d415d26a5be4862cf4504eaa9bbd4056e0" gracePeriod=25
Nov 04 01:33:35.258569 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:35.258541    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-5ce9e8-5cd7d58c4b-k77tv"
Nov 04 01:33:35.306232 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:35.306205    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-5ce9e8-7bf85dc8f5-l96mq"
Nov 04 01:33:35.395543 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:35.395511    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/88233bfe-a2e5-4dc2-b4ab-d6f9772f7219-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88233bfe-a2e5-4dc2-b4ab-d6f9772f7219-plugins-config-volume\") pod \"88233bfe-a2e5-4dc2-b4ab-d6f9772f7219\" (UID: \"88233bfe-a2e5-4dc2-b4ab-d6f9772f7219\") "
Nov 04 01:33:35.395766 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:35.395563    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bda9909c-2480-4f83-9cef-3d14f19568ac-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bda9909c-2480-4f83-9cef-3d14f19568ac-plugins-config-volume\") pod \"bda9909c-2480-4f83-9cef-3d14f19568ac\" (UID: \"bda9909c-2480-4f83-9cef-3d14f19568ac\") "
Nov 04 01:33:35.395866 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:35.395721    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/88233bfe-a2e5-4dc2-b4ab-d6f9772f7219-kube-api-access-khlqs\" (UniqueName: \"kubernetes.io/projected/88233bfe-a2e5-4dc2-b4ab-d6f9772f7219-kube-api-access-khlqs\") pod \"88233bfe-a2e5-4dc2-b4ab-d6f9772f7219\" (UID: \"88233bfe-a2e5-4dc2-b4ab-d6f9772f7219\") "
Nov 04 01:33:35.395977 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:35.395952    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88233bfe-a2e5-4dc2-b4ab-d6f9772f7219-plugins-config-volume" pod "88233bfe-a2e5-4dc2-b4ab-d6f9772f7219" (UID: "88233bfe-a2e5-4dc2-b4ab-d6f9772f7219"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:33:35.395977 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:35.395970    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bda9909c-2480-4f83-9cef-3d14f19568ac-kube-api-access-v7v7t\" (UniqueName: \"kubernetes.io/projected/bda9909c-2480-4f83-9cef-3d14f19568ac-kube-api-access-v7v7t\") pod \"bda9909c-2480-4f83-9cef-3d14f19568ac\" (UID: \"bda9909c-2480-4f83-9cef-3d14f19568ac\") "
Nov 04 01:33:35.396092 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:35.395957    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bda9909c-2480-4f83-9cef-3d14f19568ac-plugins-config-volume" pod "bda9909c-2480-4f83-9cef-3d14f19568ac" (UID: "bda9909c-2480-4f83-9cef-3d14f19568ac"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:33:35.396420 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:35.396397    5068 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88233bfe-a2e5-4dc2-b4ab-d6f9772f7219-plugins-config-volume\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:35.396508 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:35.396422    5068 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bda9909c-2480-4f83-9cef-3d14f19568ac-plugins-config-volume\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:35.397956 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:35.397934    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bda9909c-2480-4f83-9cef-3d14f19568ac-kube-api-access-v7v7t" pod "bda9909c-2480-4f83-9cef-3d14f19568ac" (UID: "bda9909c-2480-4f83-9cef-3d14f19568ac"). InnerVolumeSpecName "kube-api-access-v7v7t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:33:35.398038 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:35.398028    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88233bfe-a2e5-4dc2-b4ab-d6f9772f7219-kube-api-access-khlqs" pod "88233bfe-a2e5-4dc2-b4ab-d6f9772f7219" (UID: "88233bfe-a2e5-4dc2-b4ab-d6f9772f7219"). InnerVolumeSpecName "kube-api-access-khlqs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:33:35.497906 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:35.497822    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v7v7t\" (UniqueName: \"kubernetes.io/projected/bda9909c-2480-4f83-9cef-3d14f19568ac-kube-api-access-v7v7t\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:35.497906 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:35.497860    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-khlqs\" (UniqueName: \"kubernetes.io/projected/88233bfe-a2e5-4dc2-b4ab-d6f9772f7219-kube-api-access-khlqs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:35.808807 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:35.808770    5068 generic.go:356] "Generic (PLEG): container finished" podID="bda9909c-2480-4f83-9cef-3d14f19568ac" containerID="6264710adb7ce1fa23bdd9c75f6eb6d415d26a5be4862cf4504eaa9bbd4056e0" exitCode=0
Nov 04 01:33:35.809019 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:35.808821    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-5ce9e8-7bf85dc8f5-l96mq"
Nov 04 01:33:35.809019 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:35.808856    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-5ce9e8-7bf85dc8f5-l96mq" event={"ID":"bda9909c-2480-4f83-9cef-3d14f19568ac","Type":"ContainerDied","Data":"6264710adb7ce1fa23bdd9c75f6eb6d415d26a5be4862cf4504eaa9bbd4056e0"}
Nov 04 01:33:35.809019 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:35.808915    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-5ce9e8-7bf85dc8f5-l96mq" event={"ID":"bda9909c-2480-4f83-9cef-3d14f19568ac","Type":"ContainerDied","Data":"013eadc4e241c9027fbdce878623b813e4703745ba06f845a3ce0d3179152203"}
Nov 04 01:33:35.809019 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:35.808942    5068 scope.go:122] "RemoveContainer" containerID="6264710adb7ce1fa23bdd9c75f6eb6d415d26a5be4862cf4504eaa9bbd4056e0"
Nov 04 01:33:35.810623 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:35.810582    5068 generic.go:356] "Generic (PLEG): container finished" podID="88233bfe-a2e5-4dc2-b4ab-d6f9772f7219" containerID="21788304f95e42ce848e84d5514d3a7a0ea02d135c1672e694d726baec748e43" exitCode=0
Nov 04 01:33:35.810734 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:35.810644    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-5ce9e8-5cd7d58c4b-k77tv"
Nov 04 01:33:35.810734 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:35.810660    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-5ce9e8-5cd7d58c4b-k77tv" event={"ID":"88233bfe-a2e5-4dc2-b4ab-d6f9772f7219","Type":"ContainerDied","Data":"21788304f95e42ce848e84d5514d3a7a0ea02d135c1672e694d726baec748e43"}
Nov 04 01:33:35.810734 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:35.810720    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-5ce9e8-5cd7d58c4b-k77tv" event={"ID":"88233bfe-a2e5-4dc2-b4ab-d6f9772f7219","Type":"ContainerDied","Data":"edf860fa0900e2a898dd51bfb2c03d3a5dbf5b8e2be60b2495b702e68007235d"}
Nov 04 01:33:35.832008 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:35.831985    5068 scope.go:122] "RemoveContainer" containerID="6264710adb7ce1fa23bdd9c75f6eb6d415d26a5be4862cf4504eaa9bbd4056e0"
Nov 04 01:33:35.832290 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:33:35.832269    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6264710adb7ce1fa23bdd9c75f6eb6d415d26a5be4862cf4504eaa9bbd4056e0\": container with ID starting with 6264710adb7ce1fa23bdd9c75f6eb6d415d26a5be4862cf4504eaa9bbd4056e0 not found: ID does not exist" containerID="6264710adb7ce1fa23bdd9c75f6eb6d415d26a5be4862cf4504eaa9bbd4056e0"
Nov 04 01:33:35.832348 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:35.832297    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6264710adb7ce1fa23bdd9c75f6eb6d415d26a5be4862cf4504eaa9bbd4056e0"} err="failed to get container status \"6264710adb7ce1fa23bdd9c75f6eb6d415d26a5be4862cf4504eaa9bbd4056e0\": rpc error: code = NotFound desc = could not find container \"6264710adb7ce1fa23bdd9c75f6eb6d415d26a5be4862cf4504eaa9bbd4056e0\": container with ID starting with 6264710adb7ce1fa23bdd9c75f6eb6d415d26a5be4862cf4504eaa9bbd4056e0 not found: ID does not exist"
Nov 04 01:33:35.832348 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:35.832312    5068 scope.go:122] "RemoveContainer" containerID="21788304f95e42ce848e84d5514d3a7a0ea02d135c1672e694d726baec748e43"
Nov 04 01:33:35.845892 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:35.843340    5068 scope.go:122] "RemoveContainer" containerID="21788304f95e42ce848e84d5514d3a7a0ea02d135c1672e694d726baec748e43"
Nov 04 01:33:35.845892 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:35.843470    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-5ce9e8-7bf85dc8f5-l96mq"]
Nov 04 01:33:35.845892 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:33:35.843667    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21788304f95e42ce848e84d5514d3a7a0ea02d135c1672e694d726baec748e43\": container with ID starting with 21788304f95e42ce848e84d5514d3a7a0ea02d135c1672e694d726baec748e43 not found: ID does not exist" containerID="21788304f95e42ce848e84d5514d3a7a0ea02d135c1672e694d726baec748e43"
Nov 04 01:33:35.845892 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:35.843699    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21788304f95e42ce848e84d5514d3a7a0ea02d135c1672e694d726baec748e43"} err="failed to get container status \"21788304f95e42ce848e84d5514d3a7a0ea02d135c1672e694d726baec748e43\": rpc error: code = NotFound desc = could not find container \"21788304f95e42ce848e84d5514d3a7a0ea02d135c1672e694d726baec748e43\": container with ID starting with 21788304f95e42ce848e84d5514d3a7a0ea02d135c1672e694d726baec748e43 not found: ID does not exist"
Nov 04 01:33:35.846282 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:35.846259    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-5ce9e8-7bf85dc8f5-l96mq"]
Nov 04 01:33:35.858465 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:35.858441    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-5ce9e8-5cd7d58c4b-k77tv"]
Nov 04 01:33:35.861626 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:35.861589    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-5ce9e8-5cd7d58c4b-k77tv"]
Nov 04 01:33:36.114443 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:36.114361    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="88233bfe-a2e5-4dc2-b4ab-d6f9772f7219" path="/var/lib/kubelet/pods/88233bfe-a2e5-4dc2-b4ab-d6f9772f7219/volumes"
Nov 04 01:33:36.114919 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:36.114903    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bda9909c-2480-4f83-9cef-3d14f19568ac" path="/var/lib/kubelet/pods/bda9909c-2480-4f83-9cef-3d14f19568ac/volumes"
Nov 04 01:33:38.921701 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:38.921652    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:33:43.419962 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:43.419922    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w0-6dac99-6d46fbff94-fcfwv" podUID="98e65cde-ca44-4ad9-9a57-ce81c0369a0f" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:33:43.478711 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:33:43.478675    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:33:44.072291 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:44.072252    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf"]
Nov 04 01:33:47.830322 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:47.830283    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:33:49.824197 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:49.824168    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-6dac99-6d46fbff94-fcfwv"
Nov 04 01:33:49.918306 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:49.918271    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/98e65cde-ca44-4ad9-9a57-ce81c0369a0f-kube-api-access-7b7zh\" (UniqueName: \"kubernetes.io/projected/98e65cde-ca44-4ad9-9a57-ce81c0369a0f-kube-api-access-7b7zh\") pod \"98e65cde-ca44-4ad9-9a57-ce81c0369a0f\" (UID: \"98e65cde-ca44-4ad9-9a57-ce81c0369a0f\") "
Nov 04 01:33:49.918646 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:49.918342    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/98e65cde-ca44-4ad9-9a57-ce81c0369a0f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98e65cde-ca44-4ad9-9a57-ce81c0369a0f-plugins-config-volume\") pod \"98e65cde-ca44-4ad9-9a57-ce81c0369a0f\" (UID: \"98e65cde-ca44-4ad9-9a57-ce81c0369a0f\") "
Nov 04 01:33:49.918865 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:49.918838    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98e65cde-ca44-4ad9-9a57-ce81c0369a0f-plugins-config-volume" pod "98e65cde-ca44-4ad9-9a57-ce81c0369a0f" (UID: "98e65cde-ca44-4ad9-9a57-ce81c0369a0f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:33:49.921461 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:49.921434    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98e65cde-ca44-4ad9-9a57-ce81c0369a0f-kube-api-access-7b7zh" pod "98e65cde-ca44-4ad9-9a57-ce81c0369a0f" (UID: "98e65cde-ca44-4ad9-9a57-ce81c0369a0f"). InnerVolumeSpecName "kube-api-access-7b7zh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:33:49.938621 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:49.938573    5068 generic.go:356] "Generic (PLEG): container finished" podID="98e65cde-ca44-4ad9-9a57-ce81c0369a0f" containerID="307a28b2c7e0cbeb01d080fae154cff7f63bb3598b2f756d178747115c5b319e" exitCode=0
Nov 04 01:33:49.938917 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:49.938757    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-6dac99-6d46fbff94-fcfwv"
Nov 04 01:33:49.940061 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:49.938780    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-6dac99-6d46fbff94-fcfwv" event={"ID":"98e65cde-ca44-4ad9-9a57-ce81c0369a0f","Type":"ContainerDied","Data":"307a28b2c7e0cbeb01d080fae154cff7f63bb3598b2f756d178747115c5b319e"}
Nov 04 01:33:49.940061 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:49.939787    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-6dac99-6d46fbff94-fcfwv" event={"ID":"98e65cde-ca44-4ad9-9a57-ce81c0369a0f","Type":"ContainerDied","Data":"37a4b9244fa6db19d9f73cdd829f052f3ec264918e26c3a72fc41b4bee4db92c"}
Nov 04 01:33:49.940061 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:49.939827    5068 scope.go:122] "RemoveContainer" containerID="307a28b2c7e0cbeb01d080fae154cff7f63bb3598b2f756d178747115c5b319e"
Nov 04 01:33:49.966899 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:49.966850    5068 scope.go:122] "RemoveContainer" containerID="307a28b2c7e0cbeb01d080fae154cff7f63bb3598b2f756d178747115c5b319e"
Nov 04 01:33:49.968193 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:33:49.968090    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"307a28b2c7e0cbeb01d080fae154cff7f63bb3598b2f756d178747115c5b319e\": container with ID starting with 307a28b2c7e0cbeb01d080fae154cff7f63bb3598b2f756d178747115c5b319e not found: ID does not exist" containerID="307a28b2c7e0cbeb01d080fae154cff7f63bb3598b2f756d178747115c5b319e"
Nov 04 01:33:49.968193 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:49.968129    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"307a28b2c7e0cbeb01d080fae154cff7f63bb3598b2f756d178747115c5b319e"} err="failed to get container status \"307a28b2c7e0cbeb01d080fae154cff7f63bb3598b2f756d178747115c5b319e\": rpc error: code = NotFound desc = could not find container \"307a28b2c7e0cbeb01d080fae154cff7f63bb3598b2f756d178747115c5b319e\": container with ID starting with 307a28b2c7e0cbeb01d080fae154cff7f63bb3598b2f756d178747115c5b319e not found: ID does not exist"
Nov 04 01:33:49.983179 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:49.983147    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-6dac99-6d46fbff94-fcfwv"]
Nov 04 01:33:49.987109 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:49.987079    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-6dac99-6d46fbff94-fcfwv"]
Nov 04 01:33:50.019428 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:50.019395    5068 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98e65cde-ca44-4ad9-9a57-ce81c0369a0f-plugins-config-volume\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:50.019428 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:50.019425    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7b7zh\" (UniqueName: \"kubernetes.io/projected/98e65cde-ca44-4ad9-9a57-ce81c0369a0f-kube-api-access-7b7zh\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:50.113106 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:50.113070    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="98e65cde-ca44-4ad9-9a57-ce81c0369a0f" path="/var/lib/kubelet/pods/98e65cde-ca44-4ad9-9a57-ce81c0369a0f/volumes"
Nov 04 01:33:56.153570 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:56.153534    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v"]
Nov 04 01:33:56.154312 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:56.154282    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v" podUID="9b28aa81-39ec-4761-a076-0e5a2db936cd" containerName="istio-proxy" containerID="cri-o://192b0defb9cd1abc1b2ab2408f5e12dc2ee1612bc6af3470f72a482a2d34d0d2" gracePeriod=30
Nov 04 01:33:56.161313 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:56.161283    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz"]
Nov 04 01:33:56.161533 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:56.161509    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz" podUID="52491e2f-36e5-4e6b-ae82-5795752be764" containerName="istio-proxy" containerID="cri-o://4fc8b329222454fe0d27fff9ece227fe9b9cf9bf10f190d65b6e90478bc9a6c3" gracePeriod=30
Nov 04 01:33:56.169215 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:56.169186    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj"]
Nov 04 01:33:56.169554 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:56.169527    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj" podUID="239d3dec-a8fd-45e4-82ec-1ed4e1d01c72" containerName="istio-proxy" containerID="cri-o://9f3de262a8c7582a7320d9a0685f7ed27c0e5cdc196de4ffea24d76cbfdada64" gracePeriod=30
Nov 04 01:33:59.138308 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:59.138235    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf" podUID="32f60f59-a98f-46ba-9fb4-8420f77998da" containerName="main" containerID="cri-o://06300cf6bf05237f990c2cf9a7f6a09de989309df2b86f1ecfd8f967ddd31612" gracePeriod=45
Nov 04 01:33:59.411463 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:59.411438    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf"
Nov 04 01:33:59.503971 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:59.503933    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/32f60f59-a98f-46ba-9fb4-8420f77998da-kube-api-access-rsxlk\" (UniqueName: \"kubernetes.io/projected/32f60f59-a98f-46ba-9fb4-8420f77998da-kube-api-access-rsxlk\") pod \"32f60f59-a98f-46ba-9fb4-8420f77998da\" (UID: \"32f60f59-a98f-46ba-9fb4-8420f77998da\") "
Nov 04 01:33:59.504148 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:59.504000    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/32f60f59-a98f-46ba-9fb4-8420f77998da-home\" (UniqueName: \"kubernetes.io/empty-dir/32f60f59-a98f-46ba-9fb4-8420f77998da-home\") pod \"32f60f59-a98f-46ba-9fb4-8420f77998da\" (UID: \"32f60f59-a98f-46ba-9fb4-8420f77998da\") "
Nov 04 01:33:59.504148 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:59.504030    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/32f60f59-a98f-46ba-9fb4-8420f77998da-tls-certs\" (UniqueName: \"kubernetes.io/secret/32f60f59-a98f-46ba-9fb4-8420f77998da-tls-certs\") pod \"32f60f59-a98f-46ba-9fb4-8420f77998da\" (UID: \"32f60f59-a98f-46ba-9fb4-8420f77998da\") "
Nov 04 01:33:59.504148 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:59.504115    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/32f60f59-a98f-46ba-9fb4-8420f77998da-dshm\" (UniqueName: \"kubernetes.io/empty-dir/32f60f59-a98f-46ba-9fb4-8420f77998da-dshm\") pod \"32f60f59-a98f-46ba-9fb4-8420f77998da\" (UID: \"32f60f59-a98f-46ba-9fb4-8420f77998da\") "
Nov 04 01:33:59.504303 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:59.504155    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/32f60f59-a98f-46ba-9fb4-8420f77998da-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/32f60f59-a98f-46ba-9fb4-8420f77998da-model-cache\") pod \"32f60f59-a98f-46ba-9fb4-8420f77998da\" (UID: \"32f60f59-a98f-46ba-9fb4-8420f77998da\") "
Nov 04 01:33:59.504303 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:59.504208    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/32f60f59-a98f-46ba-9fb4-8420f77998da-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32f60f59-a98f-46ba-9fb4-8420f77998da-tmp-dir\") pod \"32f60f59-a98f-46ba-9fb4-8420f77998da\" (UID: \"32f60f59-a98f-46ba-9fb4-8420f77998da\") "
Nov 04 01:33:59.504419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:59.504346    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32f60f59-a98f-46ba-9fb4-8420f77998da-home" pod "32f60f59-a98f-46ba-9fb4-8420f77998da" (UID: "32f60f59-a98f-46ba-9fb4-8420f77998da"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:33:59.504510 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:59.504478    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32f60f59-a98f-46ba-9fb4-8420f77998da-model-cache" pod "32f60f59-a98f-46ba-9fb4-8420f77998da" (UID: "32f60f59-a98f-46ba-9fb4-8420f77998da"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:33:59.504715 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:59.504684    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32f60f59-a98f-46ba-9fb4-8420f77998da-tmp-dir" pod "32f60f59-a98f-46ba-9fb4-8420f77998da" (UID: "32f60f59-a98f-46ba-9fb4-8420f77998da"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:33:59.504877 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:59.504860    5068 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32f60f59-a98f-46ba-9fb4-8420f77998da-tmp-dir\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:59.504947 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:59.504883    5068 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/32f60f59-a98f-46ba-9fb4-8420f77998da-home\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:59.504947 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:59.504896    5068 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/32f60f59-a98f-46ba-9fb4-8420f77998da-model-cache\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:59.506483 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:59.506442    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32f60f59-a98f-46ba-9fb4-8420f77998da-tls-certs" pod "32f60f59-a98f-46ba-9fb4-8420f77998da" (UID: "32f60f59-a98f-46ba-9fb4-8420f77998da"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:33:59.506598 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:59.506494    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32f60f59-a98f-46ba-9fb4-8420f77998da-dshm" pod "32f60f59-a98f-46ba-9fb4-8420f77998da" (UID: "32f60f59-a98f-46ba-9fb4-8420f77998da"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:33:59.506845 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:59.506820    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32f60f59-a98f-46ba-9fb4-8420f77998da-kube-api-access-rsxlk" pod "32f60f59-a98f-46ba-9fb4-8420f77998da" (UID: "32f60f59-a98f-46ba-9fb4-8420f77998da"). InnerVolumeSpecName "kube-api-access-rsxlk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:33:59.606192 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:59.606152    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rsxlk\" (UniqueName: \"kubernetes.io/projected/32f60f59-a98f-46ba-9fb4-8420f77998da-kube-api-access-rsxlk\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:59.606192 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:59.606186    5068 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32f60f59-a98f-46ba-9fb4-8420f77998da-tls-certs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:59.606192 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:59.606196    5068 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/32f60f59-a98f-46ba-9fb4-8420f77998da-dshm\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:59.798223 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:59.798183    5068 patch_prober.go:29] interesting pod/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 01:33:59.798411 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:33:59.798249    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v" podUID="9b28aa81-39ec-4761-a076-0e5a2db936cd" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 01:34:00.041290 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:00.041256    5068 generic.go:356] "Generic (PLEG): container finished" podID="32f60f59-a98f-46ba-9fb4-8420f77998da" containerID="06300cf6bf05237f990c2cf9a7f6a09de989309df2b86f1ecfd8f967ddd31612" exitCode=0
Nov 04 01:34:00.041599 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:00.041307    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf"
Nov 04 01:34:00.041599 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:00.041340    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf" event={"ID":"32f60f59-a98f-46ba-9fb4-8420f77998da","Type":"ContainerDied","Data":"06300cf6bf05237f990c2cf9a7f6a09de989309df2b86f1ecfd8f967ddd31612"}
Nov 04 01:34:00.041599 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:00.041397    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf" event={"ID":"32f60f59-a98f-46ba-9fb4-8420f77998da","Type":"ContainerDied","Data":"fa23340c1e2eb7708174a662d9abafe9bf9910e61e55c8024f4e80f525dc67df"}
Nov 04 01:34:00.041599 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:00.041421    5068 scope.go:122] "RemoveContainer" containerID="06300cf6bf05237f990c2cf9a7f6a09de989309df2b86f1ecfd8f967ddd31612"
Nov 04 01:34:00.055107 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:00.055084    5068 scope.go:122] "RemoveContainer" containerID="06300cf6bf05237f990c2cf9a7f6a09de989309df2b86f1ecfd8f967ddd31612"
Nov 04 01:34:00.055528 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:34:00.055502    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06300cf6bf05237f990c2cf9a7f6a09de989309df2b86f1ecfd8f967ddd31612\": container with ID starting with 06300cf6bf05237f990c2cf9a7f6a09de989309df2b86f1ecfd8f967ddd31612 not found: ID does not exist" containerID="06300cf6bf05237f990c2cf9a7f6a09de989309df2b86f1ecfd8f967ddd31612"
Nov 04 01:34:00.055601 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:00.055542    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06300cf6bf05237f990c2cf9a7f6a09de989309df2b86f1ecfd8f967ddd31612"} err="failed to get container status \"06300cf6bf05237f990c2cf9a7f6a09de989309df2b86f1ecfd8f967ddd31612\": rpc error: code = NotFound desc = could not find container \"06300cf6bf05237f990c2cf9a7f6a09de989309df2b86f1ecfd8f967ddd31612\": container with ID starting with 06300cf6bf05237f990c2cf9a7f6a09de989309df2b86f1ecfd8f967ddd31612 not found: ID does not exist"
Nov 04 01:34:00.066845 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:00.066822    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf"]
Nov 04 01:34:00.072186 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:00.072159    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf"]
Nov 04 01:34:00.113432 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:00.113404    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="32f60f59-a98f-46ba-9fb4-8420f77998da" path="/var/lib/kubelet/pods/32f60f59-a98f-46ba-9fb4-8420f77998da/volumes"
Nov 04 01:34:00.331687 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:00.331581    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-1036e8/distinct-2-w1-1036e8-kserve-5c8bc49658-5frtf" podUID="32f60f59-a98f-46ba-9fb4-8420f77998da" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:34:01.457086 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.457019    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v"
Nov 04 01:34:01.495648 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.495582    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz"
Nov 04 01:34:01.499299 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.499275    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj"
Nov 04 01:34:01.526135 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.526104    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-istio-token\" (UniqueName: \"kubernetes.io/projected/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-istio-token\") pod \"239d3dec-a8fd-45e4-82ec-1ed4e1d01c72\" (UID: \"239d3dec-a8fd-45e4-82ec-1ed4e1d01c72\") "
Nov 04 01:34:01.526283 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.526166    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-workload-certs\") pod \"239d3dec-a8fd-45e4-82ec-1ed4e1d01c72\" (UID: \"239d3dec-a8fd-45e4-82ec-1ed4e1d01c72\") "
Nov 04 01:34:01.526283 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.526201    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-istio-data\") pod \"239d3dec-a8fd-45e4-82ec-1ed4e1d01c72\" (UID: \"239d3dec-a8fd-45e4-82ec-1ed4e1d01c72\") "
Nov 04 01:34:01.526283 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.526248    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-istio-envoy\") pod \"239d3dec-a8fd-45e4-82ec-1ed4e1d01c72\" (UID: \"239d3dec-a8fd-45e4-82ec-1ed4e1d01c72\") "
Nov 04 01:34:01.526461 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.526287    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9b28aa81-39ec-4761-a076-0e5a2db936cd-kube-api-access-p4f2q\" (UniqueName: \"kubernetes.io/projected/9b28aa81-39ec-4761-a076-0e5a2db936cd-kube-api-access-p4f2q\") pod \"9b28aa81-39ec-4761-a076-0e5a2db936cd\" (UID: \"9b28aa81-39ec-4761-a076-0e5a2db936cd\") "
Nov 04 01:34:01.526461 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.526328    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/52491e2f-36e5-4e6b-ae82-5795752be764-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52491e2f-36e5-4e6b-ae82-5795752be764-workload-socket\") pod \"52491e2f-36e5-4e6b-ae82-5795752be764\" (UID: \"52491e2f-36e5-4e6b-ae82-5795752be764\") "
Nov 04 01:34:01.526461 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.526361    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/52491e2f-36e5-4e6b-ae82-5795752be764-istio-token\" (UniqueName: \"kubernetes.io/projected/52491e2f-36e5-4e6b-ae82-5795752be764-istio-token\") pod \"52491e2f-36e5-4e6b-ae82-5795752be764\" (UID: \"52491e2f-36e5-4e6b-ae82-5795752be764\") "
Nov 04 01:34:01.526461 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.526387    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/52491e2f-36e5-4e6b-ae82-5795752be764-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52491e2f-36e5-4e6b-ae82-5795752be764-istio-envoy\") pod \"52491e2f-36e5-4e6b-ae82-5795752be764\" (UID: \"52491e2f-36e5-4e6b-ae82-5795752be764\") "
Nov 04 01:34:01.526461 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.526422    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9b28aa81-39ec-4761-a076-0e5a2db936cd-istio-token\" (UniqueName: \"kubernetes.io/projected/9b28aa81-39ec-4761-a076-0e5a2db936cd-istio-token\") pod \"9b28aa81-39ec-4761-a076-0e5a2db936cd\" (UID: \"9b28aa81-39ec-4761-a076-0e5a2db936cd\") "
Nov 04 01:34:01.526461 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.526434    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-workload-certs" pod "239d3dec-a8fd-45e4-82ec-1ed4e1d01c72" (UID: "239d3dec-a8fd-45e4-82ec-1ed4e1d01c72"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:34:01.526461 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.526452    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-istiod-ca-cert\") pod \"239d3dec-a8fd-45e4-82ec-1ed4e1d01c72\" (UID: \"239d3dec-a8fd-45e4-82ec-1ed4e1d01c72\") "
Nov 04 01:34:01.526840 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.526478    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9b28aa81-39ec-4761-a076-0e5a2db936cd-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b28aa81-39ec-4761-a076-0e5a2db936cd-workload-socket\") pod \"9b28aa81-39ec-4761-a076-0e5a2db936cd\" (UID: \"9b28aa81-39ec-4761-a076-0e5a2db936cd\") "
Nov 04 01:34:01.526840 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.526554    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9b28aa81-39ec-4761-a076-0e5a2db936cd-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b28aa81-39ec-4761-a076-0e5a2db936cd-credential-socket\") pod \"9b28aa81-39ec-4761-a076-0e5a2db936cd\" (UID: \"9b28aa81-39ec-4761-a076-0e5a2db936cd\") "
Nov 04 01:34:01.526840 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.526597    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-credential-socket\") pod \"239d3dec-a8fd-45e4-82ec-1ed4e1d01c72\" (UID: \"239d3dec-a8fd-45e4-82ec-1ed4e1d01c72\") "
Nov 04 01:34:01.526840 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.526659    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9b28aa81-39ec-4761-a076-0e5a2db936cd-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b28aa81-39ec-4761-a076-0e5a2db936cd-istio-podinfo\") pod \"9b28aa81-39ec-4761-a076-0e5a2db936cd\" (UID: \"9b28aa81-39ec-4761-a076-0e5a2db936cd\") "
Nov 04 01:34:01.526840 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.526664    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52491e2f-36e5-4e6b-ae82-5795752be764-workload-socket" pod "52491e2f-36e5-4e6b-ae82-5795752be764" (UID: "52491e2f-36e5-4e6b-ae82-5795752be764"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:34:01.526840 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.526694    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-kube-api-access-7pvp7\" (UniqueName: \"kubernetes.io/projected/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-kube-api-access-7pvp7\") pod \"239d3dec-a8fd-45e4-82ec-1ed4e1d01c72\" (UID: \"239d3dec-a8fd-45e4-82ec-1ed4e1d01c72\") "
Nov 04 01:34:01.526840 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.526725    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/52491e2f-36e5-4e6b-ae82-5795752be764-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52491e2f-36e5-4e6b-ae82-5795752be764-credential-socket\") pod \"52491e2f-36e5-4e6b-ae82-5795752be764\" (UID: \"52491e2f-36e5-4e6b-ae82-5795752be764\") "
Nov 04 01:34:01.526840 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.526751    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9b28aa81-39ec-4761-a076-0e5a2db936cd-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b28aa81-39ec-4761-a076-0e5a2db936cd-istio-data\") pod \"9b28aa81-39ec-4761-a076-0e5a2db936cd\" (UID: \"9b28aa81-39ec-4761-a076-0e5a2db936cd\") "
Nov 04 01:34:01.526840 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.526794    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-istio-podinfo\") pod \"239d3dec-a8fd-45e4-82ec-1ed4e1d01c72\" (UID: \"239d3dec-a8fd-45e4-82ec-1ed4e1d01c72\") "
Nov 04 01:34:01.526840 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.526822    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/52491e2f-36e5-4e6b-ae82-5795752be764-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52491e2f-36e5-4e6b-ae82-5795752be764-istio-podinfo\") pod \"52491e2f-36e5-4e6b-ae82-5795752be764\" (UID: \"52491e2f-36e5-4e6b-ae82-5795752be764\") "
Nov 04 01:34:01.527320 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.526864    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/52491e2f-36e5-4e6b-ae82-5795752be764-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52491e2f-36e5-4e6b-ae82-5795752be764-workload-certs\") pod \"52491e2f-36e5-4e6b-ae82-5795752be764\" (UID: \"52491e2f-36e5-4e6b-ae82-5795752be764\") "
Nov 04 01:34:01.527320 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.526954    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9b28aa81-39ec-4761-a076-0e5a2db936cd-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b28aa81-39ec-4761-a076-0e5a2db936cd-istio-envoy\") pod \"9b28aa81-39ec-4761-a076-0e5a2db936cd\" (UID: \"9b28aa81-39ec-4761-a076-0e5a2db936cd\") "
Nov 04 01:34:01.527320 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.526996    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9b28aa81-39ec-4761-a076-0e5a2db936cd-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b28aa81-39ec-4761-a076-0e5a2db936cd-workload-certs\") pod \"9b28aa81-39ec-4761-a076-0e5a2db936cd\" (UID: \"9b28aa81-39ec-4761-a076-0e5a2db936cd\") "
Nov 04 01:34:01.527320 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.527036    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9b28aa81-39ec-4761-a076-0e5a2db936cd-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b28aa81-39ec-4761-a076-0e5a2db936cd-istiod-ca-cert\") pod \"9b28aa81-39ec-4761-a076-0e5a2db936cd\" (UID: \"9b28aa81-39ec-4761-a076-0e5a2db936cd\") "
Nov 04 01:34:01.527320 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.527077    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/52491e2f-36e5-4e6b-ae82-5795752be764-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52491e2f-36e5-4e6b-ae82-5795752be764-istio-data\") pod \"52491e2f-36e5-4e6b-ae82-5795752be764\" (UID: \"52491e2f-36e5-4e6b-ae82-5795752be764\") "
Nov 04 01:34:01.527320 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.527104    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-workload-socket\") pod \"239d3dec-a8fd-45e4-82ec-1ed4e1d01c72\" (UID: \"239d3dec-a8fd-45e4-82ec-1ed4e1d01c72\") "
Nov 04 01:34:01.527320 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.527142    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/52491e2f-36e5-4e6b-ae82-5795752be764-kube-api-access-4lhl4\" (UniqueName: \"kubernetes.io/projected/52491e2f-36e5-4e6b-ae82-5795752be764-kube-api-access-4lhl4\") pod \"52491e2f-36e5-4e6b-ae82-5795752be764\" (UID: \"52491e2f-36e5-4e6b-ae82-5795752be764\") "
Nov 04 01:34:01.527320 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.527175    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/52491e2f-36e5-4e6b-ae82-5795752be764-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52491e2f-36e5-4e6b-ae82-5795752be764-istiod-ca-cert\") pod \"52491e2f-36e5-4e6b-ae82-5795752be764\" (UID: \"52491e2f-36e5-4e6b-ae82-5795752be764\") "
Nov 04 01:34:01.527826 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.527678    5068 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-workload-certs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:01.527826 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.527700    5068 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52491e2f-36e5-4e6b-ae82-5795752be764-workload-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:01.527826 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.527711    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-istio-data" pod "239d3dec-a8fd-45e4-82ec-1ed4e1d01c72" (UID: "239d3dec-a8fd-45e4-82ec-1ed4e1d01c72"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:34:01.528079 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.528054    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52491e2f-36e5-4e6b-ae82-5795752be764-istiod-ca-cert" pod "52491e2f-36e5-4e6b-ae82-5795752be764" (UID: "52491e2f-36e5-4e6b-ae82-5795752be764"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:34:01.529911 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.529567    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52491e2f-36e5-4e6b-ae82-5795752be764-workload-certs" pod "52491e2f-36e5-4e6b-ae82-5795752be764" (UID: "52491e2f-36e5-4e6b-ae82-5795752be764"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:34:01.532278 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.530955    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-credential-socket" pod "239d3dec-a8fd-45e4-82ec-1ed4e1d01c72" (UID: "239d3dec-a8fd-45e4-82ec-1ed4e1d01c72"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:34:01.532568 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.532541    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b28aa81-39ec-4761-a076-0e5a2db936cd-istiod-ca-cert" pod "9b28aa81-39ec-4761-a076-0e5a2db936cd" (UID: "9b28aa81-39ec-4761-a076-0e5a2db936cd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:34:01.532787 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.532650    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b28aa81-39ec-4761-a076-0e5a2db936cd-istio-token" pod "9b28aa81-39ec-4761-a076-0e5a2db936cd" (UID: "9b28aa81-39ec-4761-a076-0e5a2db936cd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:34:01.533103 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.533075    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-istiod-ca-cert" pod "239d3dec-a8fd-45e4-82ec-1ed4e1d01c72" (UID: "239d3dec-a8fd-45e4-82ec-1ed4e1d01c72"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:34:01.533524 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.533222    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b28aa81-39ec-4761-a076-0e5a2db936cd-workload-socket" pod "9b28aa81-39ec-4761-a076-0e5a2db936cd" (UID: "9b28aa81-39ec-4761-a076-0e5a2db936cd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:34:01.533524 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.533336    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b28aa81-39ec-4761-a076-0e5a2db936cd-credential-socket" pod "9b28aa81-39ec-4761-a076-0e5a2db936cd" (UID: "9b28aa81-39ec-4761-a076-0e5a2db936cd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:34:01.533524 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.533443    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b28aa81-39ec-4761-a076-0e5a2db936cd-workload-certs" pod "9b28aa81-39ec-4761-a076-0e5a2db936cd" (UID: "9b28aa81-39ec-4761-a076-0e5a2db936cd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:34:01.533762 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.533553    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-workload-socket" pod "239d3dec-a8fd-45e4-82ec-1ed4e1d01c72" (UID: "239d3dec-a8fd-45e4-82ec-1ed4e1d01c72"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:34:01.533826 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.533762    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b28aa81-39ec-4761-a076-0e5a2db936cd-istio-data" pod "9b28aa81-39ec-4761-a076-0e5a2db936cd" (UID: "9b28aa81-39ec-4761-a076-0e5a2db936cd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:34:01.534321 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.533941    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52491e2f-36e5-4e6b-ae82-5795752be764-credential-socket" pod "52491e2f-36e5-4e6b-ae82-5795752be764" (UID: "52491e2f-36e5-4e6b-ae82-5795752be764"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:34:01.534321 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.534061    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52491e2f-36e5-4e6b-ae82-5795752be764-istio-data" pod "52491e2f-36e5-4e6b-ae82-5795752be764" (UID: "52491e2f-36e5-4e6b-ae82-5795752be764"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:34:01.535757 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.535718    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52491e2f-36e5-4e6b-ae82-5795752be764-istio-token" pod "52491e2f-36e5-4e6b-ae82-5795752be764" (UID: "52491e2f-36e5-4e6b-ae82-5795752be764"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:34:01.537936 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.537915    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52491e2f-36e5-4e6b-ae82-5795752be764-istio-envoy" pod "52491e2f-36e5-4e6b-ae82-5795752be764" (UID: "52491e2f-36e5-4e6b-ae82-5795752be764"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:34:01.537936 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.537922    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52491e2f-36e5-4e6b-ae82-5795752be764-kube-api-access-4lhl4" pod "52491e2f-36e5-4e6b-ae82-5795752be764" (UID: "52491e2f-36e5-4e6b-ae82-5795752be764"). InnerVolumeSpecName "kube-api-access-4lhl4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:34:01.538109 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.537954    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-istio-podinfo" pod "239d3dec-a8fd-45e4-82ec-1ed4e1d01c72" (UID: "239d3dec-a8fd-45e4-82ec-1ed4e1d01c72"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:34:01.538109 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.537961    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-kube-api-access-7pvp7" pod "239d3dec-a8fd-45e4-82ec-1ed4e1d01c72" (UID: "239d3dec-a8fd-45e4-82ec-1ed4e1d01c72"). InnerVolumeSpecName "kube-api-access-7pvp7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:34:01.538109 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.538054    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9b28aa81-39ec-4761-a076-0e5a2db936cd-istio-podinfo" pod "9b28aa81-39ec-4761-a076-0e5a2db936cd" (UID: "9b28aa81-39ec-4761-a076-0e5a2db936cd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:34:01.538109 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.538064    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b28aa81-39ec-4761-a076-0e5a2db936cd-kube-api-access-p4f2q" pod "9b28aa81-39ec-4761-a076-0e5a2db936cd" (UID: "9b28aa81-39ec-4761-a076-0e5a2db936cd"). InnerVolumeSpecName "kube-api-access-p4f2q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:34:01.538319 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.538127    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-istio-envoy" pod "239d3dec-a8fd-45e4-82ec-1ed4e1d01c72" (UID: "239d3dec-a8fd-45e4-82ec-1ed4e1d01c72"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:34:01.538319 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.538197    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-istio-token" pod "239d3dec-a8fd-45e4-82ec-1ed4e1d01c72" (UID: "239d3dec-a8fd-45e4-82ec-1ed4e1d01c72"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:34:01.538520 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.538498    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b28aa81-39ec-4761-a076-0e5a2db936cd-istio-envoy" pod "9b28aa81-39ec-4761-a076-0e5a2db936cd" (UID: "9b28aa81-39ec-4761-a076-0e5a2db936cd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:34:01.540046 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.540025    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/52491e2f-36e5-4e6b-ae82-5795752be764-istio-podinfo" pod "52491e2f-36e5-4e6b-ae82-5795752be764" (UID: "52491e2f-36e5-4e6b-ae82-5795752be764"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:34:01.628390 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.628354    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7pvp7\" (UniqueName: \"kubernetes.io/projected/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-kube-api-access-7pvp7\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:01.628390 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.628385    5068 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52491e2f-36e5-4e6b-ae82-5795752be764-credential-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:01.628390 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.628396    5068 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b28aa81-39ec-4761-a076-0e5a2db936cd-istio-data\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:01.628645 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.628405    5068 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-istio-podinfo\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:01.628645 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.628414    5068 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52491e2f-36e5-4e6b-ae82-5795752be764-istio-podinfo\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:01.628645 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.628422    5068 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52491e2f-36e5-4e6b-ae82-5795752be764-workload-certs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:01.628645 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.628430    5068 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b28aa81-39ec-4761-a076-0e5a2db936cd-istio-envoy\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:01.628645 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.628439    5068 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b28aa81-39ec-4761-a076-0e5a2db936cd-workload-certs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:01.628645 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.628447    5068 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b28aa81-39ec-4761-a076-0e5a2db936cd-istiod-ca-cert\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:01.628645 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.628455    5068 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52491e2f-36e5-4e6b-ae82-5795752be764-istio-data\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:01.628645 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.628463    5068 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-workload-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:01.628645 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.628471    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4lhl4\" (UniqueName: \"kubernetes.io/projected/52491e2f-36e5-4e6b-ae82-5795752be764-kube-api-access-4lhl4\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:01.628645 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.628479    5068 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52491e2f-36e5-4e6b-ae82-5795752be764-istiod-ca-cert\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:01.628645 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.628487    5068 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-istio-token\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:01.628645 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.628494    5068 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-istio-data\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:01.628645 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.628504    5068 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-istio-envoy\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:01.628645 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.628513    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p4f2q\" (UniqueName: \"kubernetes.io/projected/9b28aa81-39ec-4761-a076-0e5a2db936cd-kube-api-access-p4f2q\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:01.628645 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.628521    5068 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52491e2f-36e5-4e6b-ae82-5795752be764-istio-token\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:01.628645 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.628529    5068 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52491e2f-36e5-4e6b-ae82-5795752be764-istio-envoy\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:01.628645 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.628539    5068 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b28aa81-39ec-4761-a076-0e5a2db936cd-istio-token\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:01.628645 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.628547    5068 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-istiod-ca-cert\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:01.628645 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.628555    5068 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b28aa81-39ec-4761-a076-0e5a2db936cd-workload-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:01.628645 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.628565    5068 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b28aa81-39ec-4761-a076-0e5a2db936cd-credential-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:01.628645 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.628573    5068 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72-credential-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:01.628645 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:01.628582    5068 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b28aa81-39ec-4761-a076-0e5a2db936cd-istio-podinfo\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:02.055401 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:02.055366    5068 generic.go:356] "Generic (PLEG): container finished" podID="239d3dec-a8fd-45e4-82ec-1ed4e1d01c72" containerID="9f3de262a8c7582a7320d9a0685f7ed27c0e5cdc196de4ffea24d76cbfdada64" exitCode=0
Nov 04 01:34:02.055557 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:02.055417    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj"
Nov 04 01:34:02.055557 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:02.055448    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj" event={"ID":"239d3dec-a8fd-45e4-82ec-1ed4e1d01c72","Type":"ContainerDied","Data":"9f3de262a8c7582a7320d9a0685f7ed27c0e5cdc196de4ffea24d76cbfdada64"}
Nov 04 01:34:02.055557 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:02.055517    5068 scope.go:122] "RemoveContainer" containerID="9f3de262a8c7582a7320d9a0685f7ed27c0e5cdc196de4ffea24d76cbfdada64"
Nov 04 01:34:02.055980 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:02.055930    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj" event={"ID":"239d3dec-a8fd-45e4-82ec-1ed4e1d01c72","Type":"ContainerDied","Data":"af0d7e2c1f0f7a0db311015326a5474c1b7ec72d11187d0f9c4c9b70921db89c"}
Nov 04 01:34:02.065719 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:02.065686    5068 generic.go:356] "Generic (PLEG): container finished" podID="52491e2f-36e5-4e6b-ae82-5795752be764" containerID="4fc8b329222454fe0d27fff9ece227fe9b9cf9bf10f190d65b6e90478bc9a6c3" exitCode=0
Nov 04 01:34:02.065862 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:02.065769    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz"
Nov 04 01:34:02.065862 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:02.065772    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz" event={"ID":"52491e2f-36e5-4e6b-ae82-5795752be764","Type":"ContainerDied","Data":"4fc8b329222454fe0d27fff9ece227fe9b9cf9bf10f190d65b6e90478bc9a6c3"}
Nov 04 01:34:02.065990 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:02.065915    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz" event={"ID":"52491e2f-36e5-4e6b-ae82-5795752be764","Type":"ContainerDied","Data":"d32201e4ccea8433294df57aaeba1df6c5a8a1a8630a2f120a1b43572c220674"}
Nov 04 01:34:02.067805 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:02.067780    5068 generic.go:356] "Generic (PLEG): container finished" podID="9b28aa81-39ec-4761-a076-0e5a2db936cd" containerID="192b0defb9cd1abc1b2ab2408f5e12dc2ee1612bc6af3470f72a482a2d34d0d2" exitCode=0
Nov 04 01:34:02.068036 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:02.068015    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v" event={"ID":"9b28aa81-39ec-4761-a076-0e5a2db936cd","Type":"ContainerDied","Data":"192b0defb9cd1abc1b2ab2408f5e12dc2ee1612bc6af3470f72a482a2d34d0d2"}
Nov 04 01:34:02.068173 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:02.068158    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v" event={"ID":"9b28aa81-39ec-4761-a076-0e5a2db936cd","Type":"ContainerDied","Data":"b2e29f63f14a2edaa795cfe7ef53f6c3e1f02ff7e46500bb6d5a6ceb3dcb05f1"}
Nov 04 01:34:02.068274 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:02.068064    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v"
Nov 04 01:34:02.078187 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:02.078158    5068 scope.go:122] "RemoveContainer" containerID="9f3de262a8c7582a7320d9a0685f7ed27c0e5cdc196de4ffea24d76cbfdada64"
Nov 04 01:34:02.078538 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:34:02.078504    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f3de262a8c7582a7320d9a0685f7ed27c0e5cdc196de4ffea24d76cbfdada64\": container with ID starting with 9f3de262a8c7582a7320d9a0685f7ed27c0e5cdc196de4ffea24d76cbfdada64 not found: ID does not exist" containerID="9f3de262a8c7582a7320d9a0685f7ed27c0e5cdc196de4ffea24d76cbfdada64"
Nov 04 01:34:02.078594 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:02.078551    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f3de262a8c7582a7320d9a0685f7ed27c0e5cdc196de4ffea24d76cbfdada64"} err="failed to get container status \"9f3de262a8c7582a7320d9a0685f7ed27c0e5cdc196de4ffea24d76cbfdada64\": rpc error: code = NotFound desc = could not find container \"9f3de262a8c7582a7320d9a0685f7ed27c0e5cdc196de4ffea24d76cbfdada64\": container with ID starting with 9f3de262a8c7582a7320d9a0685f7ed27c0e5cdc196de4ffea24d76cbfdada64 not found: ID does not exist"
Nov 04 01:34:02.078594 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:02.078568    5068 scope.go:122] "RemoveContainer" containerID="4fc8b329222454fe0d27fff9ece227fe9b9cf9bf10f190d65b6e90478bc9a6c3"
Nov 04 01:34:02.088009 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:02.087989    5068 scope.go:122] "RemoveContainer" containerID="4fc8b329222454fe0d27fff9ece227fe9b9cf9bf10f190d65b6e90478bc9a6c3"
Nov 04 01:34:02.088290 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:34:02.088272    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4fc8b329222454fe0d27fff9ece227fe9b9cf9bf10f190d65b6e90478bc9a6c3\": container with ID starting with 4fc8b329222454fe0d27fff9ece227fe9b9cf9bf10f190d65b6e90478bc9a6c3 not found: ID does not exist" containerID="4fc8b329222454fe0d27fff9ece227fe9b9cf9bf10f190d65b6e90478bc9a6c3"
Nov 04 01:34:02.088350 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:02.088297    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fc8b329222454fe0d27fff9ece227fe9b9cf9bf10f190d65b6e90478bc9a6c3"} err="failed to get container status \"4fc8b329222454fe0d27fff9ece227fe9b9cf9bf10f190d65b6e90478bc9a6c3\": rpc error: code = NotFound desc = could not find container \"4fc8b329222454fe0d27fff9ece227fe9b9cf9bf10f190d65b6e90478bc9a6c3\": container with ID starting with 4fc8b329222454fe0d27fff9ece227fe9b9cf9bf10f190d65b6e90478bc9a6c3 not found: ID does not exist"
Nov 04 01:34:02.088350 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:02.088311    5068 scope.go:122] "RemoveContainer" containerID="192b0defb9cd1abc1b2ab2408f5e12dc2ee1612bc6af3470f72a482a2d34d0d2"
Nov 04 01:34:02.094051 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:02.094030    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj"]
Nov 04 01:34:02.098103 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:02.098081    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-wp8bj"]
Nov 04 01:34:02.100028 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:02.100011    5068 scope.go:122] "RemoveContainer" containerID="192b0defb9cd1abc1b2ab2408f5e12dc2ee1612bc6af3470f72a482a2d34d0d2"
Nov 04 01:34:02.100302 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:34:02.100284    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"192b0defb9cd1abc1b2ab2408f5e12dc2ee1612bc6af3470f72a482a2d34d0d2\": container with ID starting with 192b0defb9cd1abc1b2ab2408f5e12dc2ee1612bc6af3470f72a482a2d34d0d2 not found: ID does not exist" containerID="192b0defb9cd1abc1b2ab2408f5e12dc2ee1612bc6af3470f72a482a2d34d0d2"
Nov 04 01:34:02.100364 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:02.100308    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"192b0defb9cd1abc1b2ab2408f5e12dc2ee1612bc6af3470f72a482a2d34d0d2"} err="failed to get container status \"192b0defb9cd1abc1b2ab2408f5e12dc2ee1612bc6af3470f72a482a2d34d0d2\": rpc error: code = NotFound desc = could not find container \"192b0defb9cd1abc1b2ab2408f5e12dc2ee1612bc6af3470f72a482a2d34d0d2\": container with ID starting with 192b0defb9cd1abc1b2ab2408f5e12dc2ee1612bc6af3470f72a482a2d34d0d2 not found: ID does not exist"
Nov 04 01:34:02.113958 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:02.113934    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="239d3dec-a8fd-45e4-82ec-1ed4e1d01c72" path="/var/lib/kubelet/pods/239d3dec-a8fd-45e4-82ec-1ed4e1d01c72/volumes"
Nov 04 01:34:02.114230 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:02.114217    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v"]
Nov 04 01:34:02.114371 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:02.114353    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-cnd9v"]
Nov 04 01:34:02.127725 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:02.127700    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz"]
Nov 04 01:34:02.133828 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:02.133801    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-6dac99-openshift-default-cbfb66ffc-5bdhz"]
Nov 04 01:34:04.112941 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:04.112904    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="52491e2f-36e5-4e6b-ae82-5795752be764" path="/var/lib/kubelet/pods/52491e2f-36e5-4e6b-ae82-5795752be764/volumes"
Nov 04 01:34:04.113490 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:04.113474    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9b28aa81-39ec-4761-a076-0e5a2db936cd" path="/var/lib/kubelet/pods/9b28aa81-39ec-4761-a076-0e5a2db936cd/volumes"
Nov 04 01:34:05.651798 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:05.651752    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:34:05.972475 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:05.972378    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-1036e8-555f8d74d7-shvjl"]
Nov 04 01:34:05.973230 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:05.973195    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-1036e8-555f8d74d7-shvjl" podUID="36b71cc4-21b7-42e0-86d7-1f25e215d068" containerName="maas-api" containerID="cri-o://7f42411b12a36dcfd60e2c5916d1bb33a425798248f35c29895d1b11e7d072f0" gracePeriod=30
Nov 04 01:34:06.097518 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:06.097486    5068 generic.go:356] "Generic (PLEG): container finished" podID="36b71cc4-21b7-42e0-86d7-1f25e215d068" containerID="7f42411b12a36dcfd60e2c5916d1bb33a425798248f35c29895d1b11e7d072f0" exitCode=0
Nov 04 01:34:06.097664 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:06.097558    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-1036e8-555f8d74d7-shvjl" event={"ID":"36b71cc4-21b7-42e0-86d7-1f25e215d068","Type":"ContainerDied","Data":"7f42411b12a36dcfd60e2c5916d1bb33a425798248f35c29895d1b11e7d072f0"}
Nov 04 01:34:06.289113 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:06.289069    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-1036e8-555f8d74d7-shvjl"
Nov 04 01:34:06.404640 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:06.404575    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/36b71cc4-21b7-42e0-86d7-1f25e215d068-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36b71cc4-21b7-42e0-86d7-1f25e215d068-maas-api-tls\") pod \"36b71cc4-21b7-42e0-86d7-1f25e215d068\" (UID: \"36b71cc4-21b7-42e0-86d7-1f25e215d068\") "
Nov 04 01:34:06.404640 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:06.404635    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/36b71cc4-21b7-42e0-86d7-1f25e215d068-metrics-certs\" (UniqueName: \"kubernetes.io/secret/36b71cc4-21b7-42e0-86d7-1f25e215d068-metrics-certs\") pod \"36b71cc4-21b7-42e0-86d7-1f25e215d068\" (UID: \"36b71cc4-21b7-42e0-86d7-1f25e215d068\") "
Nov 04 01:34:06.404907 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:06.404721    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/36b71cc4-21b7-42e0-86d7-1f25e215d068-kube-api-access-ss9hr\" (UniqueName: \"kubernetes.io/projected/36b71cc4-21b7-42e0-86d7-1f25e215d068-kube-api-access-ss9hr\") pod \"36b71cc4-21b7-42e0-86d7-1f25e215d068\" (UID: \"36b71cc4-21b7-42e0-86d7-1f25e215d068\") "
Nov 04 01:34:06.404907 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:06.404789    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/36b71cc4-21b7-42e0-86d7-1f25e215d068-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36b71cc4-21b7-42e0-86d7-1f25e215d068-openshift-service-ca\") pod \"36b71cc4-21b7-42e0-86d7-1f25e215d068\" (UID: \"36b71cc4-21b7-42e0-86d7-1f25e215d068\") "
Nov 04 01:34:06.405466 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:06.405424    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36b71cc4-21b7-42e0-86d7-1f25e215d068-openshift-service-ca" pod "36b71cc4-21b7-42e0-86d7-1f25e215d068" (UID: "36b71cc4-21b7-42e0-86d7-1f25e215d068"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:34:06.407894 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:06.407854    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36b71cc4-21b7-42e0-86d7-1f25e215d068-maas-api-tls" pod "36b71cc4-21b7-42e0-86d7-1f25e215d068" (UID: "36b71cc4-21b7-42e0-86d7-1f25e215d068"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:34:06.408029 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:06.407948    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36b71cc4-21b7-42e0-86d7-1f25e215d068-metrics-certs" pod "36b71cc4-21b7-42e0-86d7-1f25e215d068" (UID: "36b71cc4-21b7-42e0-86d7-1f25e215d068"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:34:06.408823 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:06.408796    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36b71cc4-21b7-42e0-86d7-1f25e215d068-kube-api-access-ss9hr" pod "36b71cc4-21b7-42e0-86d7-1f25e215d068" (UID: "36b71cc4-21b7-42e0-86d7-1f25e215d068"). InnerVolumeSpecName "kube-api-access-ss9hr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:34:06.506223 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:06.506180    5068 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36b71cc4-21b7-42e0-86d7-1f25e215d068-maas-api-tls\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:06.506223 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:06.506224    5068 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36b71cc4-21b7-42e0-86d7-1f25e215d068-metrics-certs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:06.506430 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:06.506242    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ss9hr\" (UniqueName: \"kubernetes.io/projected/36b71cc4-21b7-42e0-86d7-1f25e215d068-kube-api-access-ss9hr\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:06.506430 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:06.506260    5068 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36b71cc4-21b7-42e0-86d7-1f25e215d068-openshift-service-ca\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:07.116634 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:07.116503    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-1036e8-555f8d74d7-shvjl" event={"ID":"36b71cc4-21b7-42e0-86d7-1f25e215d068","Type":"ContainerDied","Data":"f38eab67db1396f3c1761e657b5934242866a1e4dd7116269cc9fda8c771de43"}
Nov 04 01:34:07.117225 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:07.117202    5068 scope.go:122] "RemoveContainer" containerID="7f42411b12a36dcfd60e2c5916d1bb33a425798248f35c29895d1b11e7d072f0"
Nov 04 01:34:07.117410 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:07.116752    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-1036e8-555f8d74d7-shvjl"
Nov 04 01:34:07.174749 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:07.174678    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-1036e8-555f8d74d7-shvjl"]
Nov 04 01:34:07.232794 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:07.228896    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-1036e8-555f8d74d7-shvjl"]
Nov 04 01:34:08.112455 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:08.112416    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="36b71cc4-21b7-42e0-86d7-1f25e215d068" path="/var/lib/kubelet/pods/36b71cc4-21b7-42e0-86d7-1f25e215d068/volumes"
Nov 04 01:34:11.605887 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:11.605850    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:34:13.450904 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:34:13.450862    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:34:22.175870 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:22.175832    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr"]
Nov 04 01:34:22.176520 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:22.176116    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr" podUID="b21786c9-e7fa-43eb-bb36-f2633486b8b5" containerName="istio-proxy" containerID="cri-o://a8e4220acdff9eb9a1893f30eadaf820fbf95dccbf293d3b9d1ca0aaa746d4d1" gracePeriod=30
Nov 04 01:34:22.179196 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:22.179169    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq"]
Nov 04 01:34:22.179419 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:22.179363    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq" podUID="cbbfd346-7a24-49e7-8853-b60e0dbf754c" containerName="istio-proxy" containerID="cri-o://b72360f764e028842daaf82dee7cb79611ee053f643c1dbccec8dd5f2344cebb" gracePeriod=30
Nov 04 01:34:22.202474 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:22.202443    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c"]
Nov 04 01:34:22.202667 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:22.202636    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c" podUID="5f777506-99a6-4278-a678-de005b0722e6" containerName="istio-proxy" containerID="cri-o://1a0fcf14d916e01ec286797ac9a9c930dcdb360aff9738c5bb38be96356d8502" gracePeriod=30
Nov 04 01:34:25.462010 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:25.461972    5068 patch_prober.go:29] interesting pod/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 01:34:25.462415 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:25.462037    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr" podUID="b21786c9-e7fa-43eb-bb36-f2633486b8b5" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 01:34:25.835340 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:25.835303    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:34:25.896020 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:25.895982    5068 patch_prober.go:29] interesting pod/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 01:34:25.896184 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:25.896041    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c" podUID="5f777506-99a6-4278-a678-de005b0722e6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 01:34:28.075763 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.075181    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq"
Nov 04 01:34:28.086642 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.085889    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr"
Nov 04 01:34:28.188281 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.188189    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cbbfd346-7a24-49e7-8853-b60e0dbf754c-istio-token\" (UniqueName: \"kubernetes.io/projected/cbbfd346-7a24-49e7-8853-b60e0dbf754c-istio-token\") pod \"cbbfd346-7a24-49e7-8853-b60e0dbf754c\" (UID: \"cbbfd346-7a24-49e7-8853-b60e0dbf754c\") "
Nov 04 01:34:28.226281 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.226254    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cbbfd346-7a24-49e7-8853-b60e0dbf754c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbbfd346-7a24-49e7-8853-b60e0dbf754c-credential-socket\") pod \"cbbfd346-7a24-49e7-8853-b60e0dbf754c\" (UID: \"cbbfd346-7a24-49e7-8853-b60e0dbf754c\") "
Nov 04 01:34:28.226522 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.226502    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b21786c9-e7fa-43eb-bb36-f2633486b8b5-istio-token\" (UniqueName: \"kubernetes.io/projected/b21786c9-e7fa-43eb-bb36-f2633486b8b5-istio-token\") pod \"b21786c9-e7fa-43eb-bb36-f2633486b8b5\" (UID: \"b21786c9-e7fa-43eb-bb36-f2633486b8b5\") "
Nov 04 01:34:28.235109 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.235083    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b21786c9-e7fa-43eb-bb36-f2633486b8b5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b21786c9-e7fa-43eb-bb36-f2633486b8b5-istio-podinfo\") pod \"b21786c9-e7fa-43eb-bb36-f2633486b8b5\" (UID: \"b21786c9-e7fa-43eb-bb36-f2633486b8b5\") "
Nov 04 01:34:28.235319 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.235300    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b21786c9-e7fa-43eb-bb36-f2633486b8b5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b21786c9-e7fa-43eb-bb36-f2633486b8b5-workload-socket\") pod \"b21786c9-e7fa-43eb-bb36-f2633486b8b5\" (UID: \"b21786c9-e7fa-43eb-bb36-f2633486b8b5\") "
Nov 04 01:34:28.235463 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.235447    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b21786c9-e7fa-43eb-bb36-f2633486b8b5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b21786c9-e7fa-43eb-bb36-f2633486b8b5-istiod-ca-cert\") pod \"b21786c9-e7fa-43eb-bb36-f2633486b8b5\" (UID: \"b21786c9-e7fa-43eb-bb36-f2633486b8b5\") "
Nov 04 01:34:28.235636 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.235591    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b21786c9-e7fa-43eb-bb36-f2633486b8b5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b21786c9-e7fa-43eb-bb36-f2633486b8b5-istio-envoy\") pod \"b21786c9-e7fa-43eb-bb36-f2633486b8b5\" (UID: \"b21786c9-e7fa-43eb-bb36-f2633486b8b5\") "
Nov 04 01:34:28.327879 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.327804    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cbbfd346-7a24-49e7-8853-b60e0dbf754c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cbbfd346-7a24-49e7-8853-b60e0dbf754c-istiod-ca-cert\") pod \"cbbfd346-7a24-49e7-8853-b60e0dbf754c\" (UID: \"cbbfd346-7a24-49e7-8853-b60e0dbf754c\") "
Nov 04 01:34:28.328952 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.328927    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/cbbfd346-7a24-49e7-8853-b60e0dbf754c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cbbfd346-7a24-49e7-8853-b60e0dbf754c-istio-podinfo\") pod \"cbbfd346-7a24-49e7-8853-b60e0dbf754c\" (UID: \"cbbfd346-7a24-49e7-8853-b60e0dbf754c\") "
Nov 04 01:34:28.329183 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.329167    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cbbfd346-7a24-49e7-8853-b60e0dbf754c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cbbfd346-7a24-49e7-8853-b60e0dbf754c-workload-certs\") pod \"cbbfd346-7a24-49e7-8853-b60e0dbf754c\" (UID: \"cbbfd346-7a24-49e7-8853-b60e0dbf754c\") "
Nov 04 01:34:28.330022 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.330005    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b21786c9-e7fa-43eb-bb36-f2633486b8b5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b21786c9-e7fa-43eb-bb36-f2633486b8b5-istio-data\") pod \"b21786c9-e7fa-43eb-bb36-f2633486b8b5\" (UID: \"b21786c9-e7fa-43eb-bb36-f2633486b8b5\") "
Nov 04 01:34:28.330185 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.330168    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cbbfd346-7a24-49e7-8853-b60e0dbf754c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cbbfd346-7a24-49e7-8853-b60e0dbf754c-istio-envoy\") pod \"cbbfd346-7a24-49e7-8853-b60e0dbf754c\" (UID: \"cbbfd346-7a24-49e7-8853-b60e0dbf754c\") "
Nov 04 01:34:28.330311 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.330295    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cbbfd346-7a24-49e7-8853-b60e0dbf754c-kube-api-access-clfmk\" (UniqueName: \"kubernetes.io/projected/cbbfd346-7a24-49e7-8853-b60e0dbf754c-kube-api-access-clfmk\") pod \"cbbfd346-7a24-49e7-8853-b60e0dbf754c\" (UID: \"cbbfd346-7a24-49e7-8853-b60e0dbf754c\") "
Nov 04 01:34:28.330436 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.330422    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b21786c9-e7fa-43eb-bb36-f2633486b8b5-kube-api-access-dnnrs\" (UniqueName: \"kubernetes.io/projected/b21786c9-e7fa-43eb-bb36-f2633486b8b5-kube-api-access-dnnrs\") pod \"b21786c9-e7fa-43eb-bb36-f2633486b8b5\" (UID: \"b21786c9-e7fa-43eb-bb36-f2633486b8b5\") "
Nov 04 01:34:28.330585 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.330568    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cbbfd346-7a24-49e7-8853-b60e0dbf754c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbbfd346-7a24-49e7-8853-b60e0dbf754c-workload-socket\") pod \"cbbfd346-7a24-49e7-8853-b60e0dbf754c\" (UID: \"cbbfd346-7a24-49e7-8853-b60e0dbf754c\") "
Nov 04 01:34:28.330765 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.330747    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b21786c9-e7fa-43eb-bb36-f2633486b8b5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b21786c9-e7fa-43eb-bb36-f2633486b8b5-credential-socket\") pod \"b21786c9-e7fa-43eb-bb36-f2633486b8b5\" (UID: \"b21786c9-e7fa-43eb-bb36-f2633486b8b5\") "
Nov 04 01:34:28.330890 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.330873    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cbbfd346-7a24-49e7-8853-b60e0dbf754c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cbbfd346-7a24-49e7-8853-b60e0dbf754c-istio-data\") pod \"cbbfd346-7a24-49e7-8853-b60e0dbf754c\" (UID: \"cbbfd346-7a24-49e7-8853-b60e0dbf754c\") "
Nov 04 01:34:28.331012 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.330995    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b21786c9-e7fa-43eb-bb36-f2633486b8b5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b21786c9-e7fa-43eb-bb36-f2633486b8b5-workload-certs\") pod \"b21786c9-e7fa-43eb-bb36-f2633486b8b5\" (UID: \"b21786c9-e7fa-43eb-bb36-f2633486b8b5\") "
Nov 04 01:34:28.332049 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.292869    5068 generic.go:356] "Generic (PLEG): container finished" podID="b21786c9-e7fa-43eb-bb36-f2633486b8b5" containerID="a8e4220acdff9eb9a1893f30eadaf820fbf95dccbf293d3b9d1ca0aaa746d4d1" exitCode=0
Nov 04 01:34:28.332049 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.292970    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr"
Nov 04 01:34:28.332049 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.235799    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbbfd346-7a24-49e7-8853-b60e0dbf754c-credential-socket" pod "cbbfd346-7a24-49e7-8853-b60e0dbf754c" (UID: "cbbfd346-7a24-49e7-8853-b60e0dbf754c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:34:28.332049 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.236959    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b21786c9-e7fa-43eb-bb36-f2633486b8b5-istiod-ca-cert" pod "b21786c9-e7fa-43eb-bb36-f2633486b8b5" (UID: "b21786c9-e7fa-43eb-bb36-f2633486b8b5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:34:28.332049 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.328135    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b21786c9-e7fa-43eb-bb36-f2633486b8b5-workload-socket" pod "b21786c9-e7fa-43eb-bb36-f2633486b8b5" (UID: "b21786c9-e7fa-43eb-bb36-f2633486b8b5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:34:28.332049 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.328855    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b21786c9-e7fa-43eb-bb36-f2633486b8b5-istio-podinfo" pod "b21786c9-e7fa-43eb-bb36-f2633486b8b5" (UID: "b21786c9-e7fa-43eb-bb36-f2633486b8b5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:34:28.332049 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.329423    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cbbfd346-7a24-49e7-8853-b60e0dbf754c-istiod-ca-cert" pod "cbbfd346-7a24-49e7-8853-b60e0dbf754c" (UID: "cbbfd346-7a24-49e7-8853-b60e0dbf754c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:34:28.332049 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.329935    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbbfd346-7a24-49e7-8853-b60e0dbf754c-workload-certs" pod "cbbfd346-7a24-49e7-8853-b60e0dbf754c" (UID: "cbbfd346-7a24-49e7-8853-b60e0dbf754c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:34:28.332049 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.330509    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b21786c9-e7fa-43eb-bb36-f2633486b8b5-istio-data" pod "b21786c9-e7fa-43eb-bb36-f2633486b8b5" (UID: "b21786c9-e7fa-43eb-bb36-f2633486b8b5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:34:28.332049 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.331347    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbbfd346-7a24-49e7-8853-b60e0dbf754c-workload-socket" pod "cbbfd346-7a24-49e7-8853-b60e0dbf754c" (UID: "cbbfd346-7a24-49e7-8853-b60e0dbf754c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:34:28.332049 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.292898    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr" event={"ID":"b21786c9-e7fa-43eb-bb36-f2633486b8b5","Type":"ContainerDied","Data":"a8e4220acdff9eb9a1893f30eadaf820fbf95dccbf293d3b9d1ca0aaa746d4d1"}
Nov 04 01:34:28.332049 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.331696    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr" event={"ID":"b21786c9-e7fa-43eb-bb36-f2633486b8b5","Type":"ContainerDied","Data":"b617e940f3bc13146ec95e4bb2f940dedc4d3979479aaef3d5fb4dd408ab6772"}
Nov 04 01:34:28.332049 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.331733    5068 scope.go:122] "RemoveContainer" containerID="a8e4220acdff9eb9a1893f30eadaf820fbf95dccbf293d3b9d1ca0aaa746d4d1"
Nov 04 01:34:28.337939 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.332838    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b21786c9-e7fa-43eb-bb36-f2633486b8b5-credential-socket" pod "b21786c9-e7fa-43eb-bb36-f2633486b8b5" (UID: "b21786c9-e7fa-43eb-bb36-f2633486b8b5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:34:28.351679 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.338100    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbbfd346-7a24-49e7-8853-b60e0dbf754c-istio-data" pod "cbbfd346-7a24-49e7-8853-b60e0dbf754c" (UID: "cbbfd346-7a24-49e7-8853-b60e0dbf754c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:34:28.352041 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.351993    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b21786c9-e7fa-43eb-bb36-f2633486b8b5-workload-certs" pod "b21786c9-e7fa-43eb-bb36-f2633486b8b5" (UID: "b21786c9-e7fa-43eb-bb36-f2633486b8b5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:34:28.372326 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.371891    5068 generic.go:356] "Generic (PLEG): container finished" podID="cbbfd346-7a24-49e7-8853-b60e0dbf754c" containerID="b72360f764e028842daaf82dee7cb79611ee053f643c1dbccec8dd5f2344cebb" exitCode=0
Nov 04 01:34:28.372326 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.372009    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq" event={"ID":"cbbfd346-7a24-49e7-8853-b60e0dbf754c","Type":"ContainerDied","Data":"b72360f764e028842daaf82dee7cb79611ee053f643c1dbccec8dd5f2344cebb"}
Nov 04 01:34:28.372326 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.372059    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq" event={"ID":"cbbfd346-7a24-49e7-8853-b60e0dbf754c","Type":"ContainerDied","Data":"a110569051177d1472234db77e9eb6d414e16bf54b2b23bf9b53baba14a80f01"}
Nov 04 01:34:28.372326 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.372155    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq"
Nov 04 01:34:28.377330 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.377056    5068 generic.go:356] "Generic (PLEG): container finished" podID="5f777506-99a6-4278-a678-de005b0722e6" containerID="1a0fcf14d916e01ec286797ac9a9c930dcdb360aff9738c5bb38be96356d8502" exitCode=0
Nov 04 01:34:28.377330 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.377177    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c" event={"ID":"5f777506-99a6-4278-a678-de005b0722e6","Type":"ContainerDied","Data":"1a0fcf14d916e01ec286797ac9a9c930dcdb360aff9738c5bb38be96356d8502"}
Nov 04 01:34:28.431809 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.431774    5068 scope.go:122] "RemoveContainer" containerID="a8e4220acdff9eb9a1893f30eadaf820fbf95dccbf293d3b9d1ca0aaa746d4d1"
Nov 04 01:34:28.432578 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.432550    5068 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbbfd346-7a24-49e7-8853-b60e0dbf754c-workload-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:28.433158 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.432581    5068 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b21786c9-e7fa-43eb-bb36-f2633486b8b5-credential-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:28.433158 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.432597    5068 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cbbfd346-7a24-49e7-8853-b60e0dbf754c-istio-data\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:28.433158 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.432628    5068 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b21786c9-e7fa-43eb-bb36-f2633486b8b5-workload-certs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:28.433158 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.432641    5068 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbbfd346-7a24-49e7-8853-b60e0dbf754c-credential-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:28.433158 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.432653    5068 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b21786c9-e7fa-43eb-bb36-f2633486b8b5-istio-podinfo\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:28.433158 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.432666    5068 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b21786c9-e7fa-43eb-bb36-f2633486b8b5-workload-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:28.433158 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.432678    5068 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b21786c9-e7fa-43eb-bb36-f2633486b8b5-istiod-ca-cert\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:28.433158 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.432692    5068 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cbbfd346-7a24-49e7-8853-b60e0dbf754c-istiod-ca-cert\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:28.433158 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.432706    5068 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cbbfd346-7a24-49e7-8853-b60e0dbf754c-workload-certs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:28.433158 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.432722    5068 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b21786c9-e7fa-43eb-bb36-f2633486b8b5-istio-data\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:28.435456 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:34:28.433952    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8e4220acdff9eb9a1893f30eadaf820fbf95dccbf293d3b9d1ca0aaa746d4d1\": container with ID starting with a8e4220acdff9eb9a1893f30eadaf820fbf95dccbf293d3b9d1ca0aaa746d4d1 not found: ID does not exist" containerID="a8e4220acdff9eb9a1893f30eadaf820fbf95dccbf293d3b9d1ca0aaa746d4d1"
Nov 04 01:34:28.435456 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.433998    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8e4220acdff9eb9a1893f30eadaf820fbf95dccbf293d3b9d1ca0aaa746d4d1"} err="failed to get container status \"a8e4220acdff9eb9a1893f30eadaf820fbf95dccbf293d3b9d1ca0aaa746d4d1\": rpc error: code = NotFound desc = could not find container \"a8e4220acdff9eb9a1893f30eadaf820fbf95dccbf293d3b9d1ca0aaa746d4d1\": container with ID starting with a8e4220acdff9eb9a1893f30eadaf820fbf95dccbf293d3b9d1ca0aaa746d4d1 not found: ID does not exist"
Nov 04 01:34:28.435456 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.434044    5068 scope.go:122] "RemoveContainer" containerID="b72360f764e028842daaf82dee7cb79611ee053f643c1dbccec8dd5f2344cebb"
Nov 04 01:34:28.435456 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.434359    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbbfd346-7a24-49e7-8853-b60e0dbf754c-istio-envoy" pod "cbbfd346-7a24-49e7-8853-b60e0dbf754c" (UID: "cbbfd346-7a24-49e7-8853-b60e0dbf754c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:34:28.435863 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.435579    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cbbfd346-7a24-49e7-8853-b60e0dbf754c-istio-podinfo" pod "cbbfd346-7a24-49e7-8853-b60e0dbf754c" (UID: "cbbfd346-7a24-49e7-8853-b60e0dbf754c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:34:28.435935 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.435906    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbbfd346-7a24-49e7-8853-b60e0dbf754c-istio-token" pod "cbbfd346-7a24-49e7-8853-b60e0dbf754c" (UID: "cbbfd346-7a24-49e7-8853-b60e0dbf754c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:34:28.436010 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.435977    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b21786c9-e7fa-43eb-bb36-f2633486b8b5-istio-envoy" pod "b21786c9-e7fa-43eb-bb36-f2633486b8b5" (UID: "b21786c9-e7fa-43eb-bb36-f2633486b8b5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:34:28.437813 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.437760    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbbfd346-7a24-49e7-8853-b60e0dbf754c-kube-api-access-clfmk" pod "cbbfd346-7a24-49e7-8853-b60e0dbf754c" (UID: "cbbfd346-7a24-49e7-8853-b60e0dbf754c"). InnerVolumeSpecName "kube-api-access-clfmk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:34:28.438003 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.437980    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b21786c9-e7fa-43eb-bb36-f2633486b8b5-istio-token" pod "b21786c9-e7fa-43eb-bb36-f2633486b8b5" (UID: "b21786c9-e7fa-43eb-bb36-f2633486b8b5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:34:28.446821 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.443723    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b21786c9-e7fa-43eb-bb36-f2633486b8b5-kube-api-access-dnnrs" pod "b21786c9-e7fa-43eb-bb36-f2633486b8b5" (UID: "b21786c9-e7fa-43eb-bb36-f2633486b8b5"). InnerVolumeSpecName "kube-api-access-dnnrs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:34:28.528978 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.528827    5068 scope.go:122] "RemoveContainer" containerID="b72360f764e028842daaf82dee7cb79611ee053f643c1dbccec8dd5f2344cebb"
Nov 04 01:34:28.529581 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:34:28.529481    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b72360f764e028842daaf82dee7cb79611ee053f643c1dbccec8dd5f2344cebb\": container with ID starting with b72360f764e028842daaf82dee7cb79611ee053f643c1dbccec8dd5f2344cebb not found: ID does not exist" containerID="b72360f764e028842daaf82dee7cb79611ee053f643c1dbccec8dd5f2344cebb"
Nov 04 01:34:28.529581 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.529526    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b72360f764e028842daaf82dee7cb79611ee053f643c1dbccec8dd5f2344cebb"} err="failed to get container status \"b72360f764e028842daaf82dee7cb79611ee053f643c1dbccec8dd5f2344cebb\": rpc error: code = NotFound desc = could not find container \"b72360f764e028842daaf82dee7cb79611ee053f643c1dbccec8dd5f2344cebb\": container with ID starting with b72360f764e028842daaf82dee7cb79611ee053f643c1dbccec8dd5f2344cebb not found: ID does not exist"
Nov 04 01:34:28.557723 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.534198    5068 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b21786c9-e7fa-43eb-bb36-f2633486b8b5-istio-token\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:28.557723 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.534225    5068 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b21786c9-e7fa-43eb-bb36-f2633486b8b5-istio-envoy\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:28.557723 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.534243    5068 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cbbfd346-7a24-49e7-8853-b60e0dbf754c-istio-podinfo\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:28.557723 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.534258    5068 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cbbfd346-7a24-49e7-8853-b60e0dbf754c-istio-envoy\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:28.557723 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.534274    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-clfmk\" (UniqueName: \"kubernetes.io/projected/cbbfd346-7a24-49e7-8853-b60e0dbf754c-kube-api-access-clfmk\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:28.557723 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.534289    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dnnrs\" (UniqueName: \"kubernetes.io/projected/b21786c9-e7fa-43eb-bb36-f2633486b8b5-kube-api-access-dnnrs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:28.557723 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.534303    5068 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cbbfd346-7a24-49e7-8853-b60e0dbf754c-istio-token\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:28.664362 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.664050    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c"
Nov 04 01:34:28.773432 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.773253    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5f777506-99a6-4278-a678-de005b0722e6-kube-api-access-qzjgg\" (UniqueName: \"kubernetes.io/projected/5f777506-99a6-4278-a678-de005b0722e6-kube-api-access-qzjgg\") pod \"5f777506-99a6-4278-a678-de005b0722e6\" (UID: \"5f777506-99a6-4278-a678-de005b0722e6\") "
Nov 04 01:34:28.773799 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.773414    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5f777506-99a6-4278-a678-de005b0722e6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f777506-99a6-4278-a678-de005b0722e6-workload-socket\") pod \"5f777506-99a6-4278-a678-de005b0722e6\" (UID: \"5f777506-99a6-4278-a678-de005b0722e6\") "
Nov 04 01:34:28.774104 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.773779    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5f777506-99a6-4278-a678-de005b0722e6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f777506-99a6-4278-a678-de005b0722e6-workload-certs\") pod \"5f777506-99a6-4278-a678-de005b0722e6\" (UID: \"5f777506-99a6-4278-a678-de005b0722e6\") "
Nov 04 01:34:28.774104 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.774067    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5f777506-99a6-4278-a678-de005b0722e6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f777506-99a6-4278-a678-de005b0722e6-istio-podinfo\") pod \"5f777506-99a6-4278-a678-de005b0722e6\" (UID: \"5f777506-99a6-4278-a678-de005b0722e6\") "
Nov 04 01:34:28.774441 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.774395    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5f777506-99a6-4278-a678-de005b0722e6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f777506-99a6-4278-a678-de005b0722e6-istiod-ca-cert\") pod \"5f777506-99a6-4278-a678-de005b0722e6\" (UID: \"5f777506-99a6-4278-a678-de005b0722e6\") "
Nov 04 01:34:28.796820 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.795550    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5f777506-99a6-4278-a678-de005b0722e6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f777506-99a6-4278-a678-de005b0722e6-istio-envoy\") pod \"5f777506-99a6-4278-a678-de005b0722e6\" (UID: \"5f777506-99a6-4278-a678-de005b0722e6\") "
Nov 04 01:34:28.796820 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.795787    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5f777506-99a6-4278-a678-de005b0722e6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f777506-99a6-4278-a678-de005b0722e6-credential-socket\") pod \"5f777506-99a6-4278-a678-de005b0722e6\" (UID: \"5f777506-99a6-4278-a678-de005b0722e6\") "
Nov 04 01:34:28.796820 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.795890    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5f777506-99a6-4278-a678-de005b0722e6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f777506-99a6-4278-a678-de005b0722e6-istio-data\") pod \"5f777506-99a6-4278-a678-de005b0722e6\" (UID: \"5f777506-99a6-4278-a678-de005b0722e6\") "
Nov 04 01:34:28.796820 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.795989    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5f777506-99a6-4278-a678-de005b0722e6-istio-token\" (UniqueName: \"kubernetes.io/projected/5f777506-99a6-4278-a678-de005b0722e6-istio-token\") pod \"5f777506-99a6-4278-a678-de005b0722e6\" (UID: \"5f777506-99a6-4278-a678-de005b0722e6\") "
Nov 04 01:34:28.797801 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.797750    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f777506-99a6-4278-a678-de005b0722e6-workload-certs" pod "5f777506-99a6-4278-a678-de005b0722e6" (UID: "5f777506-99a6-4278-a678-de005b0722e6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:34:28.798020 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.797701    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f777506-99a6-4278-a678-de005b0722e6-istiod-ca-cert" pod "5f777506-99a6-4278-a678-de005b0722e6" (UID: "5f777506-99a6-4278-a678-de005b0722e6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:34:28.815663 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.801346    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f777506-99a6-4278-a678-de005b0722e6-credential-socket" pod "5f777506-99a6-4278-a678-de005b0722e6" (UID: "5f777506-99a6-4278-a678-de005b0722e6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:34:28.815663 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.805684    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f777506-99a6-4278-a678-de005b0722e6-istio-data" pod "5f777506-99a6-4278-a678-de005b0722e6" (UID: "5f777506-99a6-4278-a678-de005b0722e6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:34:28.816054 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.815973    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f777506-99a6-4278-a678-de005b0722e6-workload-socket" pod "5f777506-99a6-4278-a678-de005b0722e6" (UID: "5f777506-99a6-4278-a678-de005b0722e6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:34:28.859354 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.841763    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f777506-99a6-4278-a678-de005b0722e6-istio-token" pod "5f777506-99a6-4278-a678-de005b0722e6" (UID: "5f777506-99a6-4278-a678-de005b0722e6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:34:28.859354 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.841849    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5f777506-99a6-4278-a678-de005b0722e6-istio-podinfo" pod "5f777506-99a6-4278-a678-de005b0722e6" (UID: "5f777506-99a6-4278-a678-de005b0722e6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:34:28.876941 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.876893    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f777506-99a6-4278-a678-de005b0722e6-istio-envoy" pod "5f777506-99a6-4278-a678-de005b0722e6" (UID: "5f777506-99a6-4278-a678-de005b0722e6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:34:28.877274 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.877239    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f777506-99a6-4278-a678-de005b0722e6-kube-api-access-qzjgg" pod "5f777506-99a6-4278-a678-de005b0722e6" (UID: "5f777506-99a6-4278-a678-de005b0722e6"). InnerVolumeSpecName "kube-api-access-qzjgg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:34:28.925633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.914036    5068 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f777506-99a6-4278-a678-de005b0722e6-istio-data\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:28.925633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.914071    5068 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f777506-99a6-4278-a678-de005b0722e6-istio-token\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:28.925633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.914088    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qzjgg\" (UniqueName: \"kubernetes.io/projected/5f777506-99a6-4278-a678-de005b0722e6-kube-api-access-qzjgg\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:28.925633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.914107    5068 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f777506-99a6-4278-a678-de005b0722e6-workload-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:28.925633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.914123    5068 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f777506-99a6-4278-a678-de005b0722e6-workload-certs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:28.925633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.914138    5068 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f777506-99a6-4278-a678-de005b0722e6-istio-podinfo\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:28.925633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.914154    5068 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f777506-99a6-4278-a678-de005b0722e6-istiod-ca-cert\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:28.925633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.914168    5068 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f777506-99a6-4278-a678-de005b0722e6-istio-envoy\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:28.925633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.914182    5068 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f777506-99a6-4278-a678-de005b0722e6-credential-socket\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:28.925633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.917885    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr"]
Nov 04 01:34:28.925633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:28.925642    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-qzlvr"]
Nov 04 01:34:29.015755 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:29.015692    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq"]
Nov 04 01:34:29.017839 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:29.017791    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-lqggq"]
Nov 04 01:34:29.392948 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:29.392910    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c"
Nov 04 01:34:29.392948 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:29.392931    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c" event={"ID":"5f777506-99a6-4278-a678-de005b0722e6","Type":"ContainerDied","Data":"9f4f70f1c6854ca2df53a4a5fd7dbb4b50e069f2acbcde6ae42c7af0ce025102"}
Nov 04 01:34:29.393526 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:29.392990    5068 scope.go:122] "RemoveContainer" containerID="1a0fcf14d916e01ec286797ac9a9c930dcdb360aff9738c5bb38be96356d8502"
Nov 04 01:34:29.421930 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:29.421897    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c"]
Nov 04 01:34:29.426373 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:29.426336    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-1036e8-openshift-default-5c566f4dbf-57w2c"]
Nov 04 01:34:30.112138 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:30.112105    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5f777506-99a6-4278-a678-de005b0722e6" path="/var/lib/kubelet/pods/5f777506-99a6-4278-a678-de005b0722e6/volumes"
Nov 04 01:34:30.112556 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:30.112543    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b21786c9-e7fa-43eb-bb36-f2633486b8b5" path="/var/lib/kubelet/pods/b21786c9-e7fa-43eb-bb36-f2633486b8b5/volumes"
Nov 04 01:34:30.112961 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:30.112947    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cbbfd346-7a24-49e7-8853-b60e0dbf754c" path="/var/lib/kubelet/pods/cbbfd346-7a24-49e7-8853-b60e0dbf754c/volumes"
Nov 04 01:34:43.450549 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:34:43.450511    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:34:49.804427 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:34:49.804341    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:35:13.448165 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:35:13.448125    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:35:37.112540 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:35:37.112502    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:35:43.448807 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:35:43.448762    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:35:53.023070 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:35:53.023031    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:35:56.837098 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:35:56.837051    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:36:09.746330 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:36:09.746293    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:36:13.518461 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:36:13.518374    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:36:22.512902 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:36:22.512811    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:36:25.863641 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:36:25.863484    5068 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-848f6bf8bc-mdxsf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:36:25.863641 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:36:25.863572    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-mdxsf" podUID="8fea1dc2-7dc8-44b2-ae40-e84791006ba0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:36:25.934840 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:36:25.934793    5068 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-848f6bf8bc-5g7t8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:36:25.935049 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:36:25.934859    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-5g7t8" podUID="5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:36:43.450102 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:36:43.450058    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:36:44.528417 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:36:44.528375    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:36:54.742722 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:36:54.742681    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:36:58.645924 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:36:58.645873    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:37:13.449488 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:37:13.449443    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:37:15.325356 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:37:15.325324    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:37:18.991827 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:37:18.991787    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:37:22.537514 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:37:22.537481    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:37:33.850787 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:37:33.850752    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:37:43.449430 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:37:43.449392    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:37:44.442044 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:37:44.442018    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tr8b2_21d1d23b-7b6e-4ab5-9593-cf7784fffea2/console-operator/2.log"
Nov 04 01:37:44.541098 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:37:44.541071    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tr8b2_21d1d23b-7b6e-4ab5-9593-cf7784fffea2/console-operator/2.log"
Nov 04 01:37:47.551075 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:37:47.551043    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:37:51.237209 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:37:51.237116    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:37:54.637153 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:37:54.637114    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:38:12.834899 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:38:12.834854    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:38:13.462546 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:38:13.462495    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:38:17.125681 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:38:17.125647    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:38:27.752005 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:38:27.751971    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:38:31.261780 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:38:31.261734    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:38:35.325442 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:38:35.325409    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:38:43.449226 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:38:43.449182    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:38:50.517253 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:38:50.517214    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:39:00.711452 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:39:00.711416    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:39:13.448094 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:39:13.448059    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:39:16.762075 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:39:16.762042    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:39:22.234350 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:39:22.234270    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:39:27.318555 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:39:27.318524    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:39:43.453460 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:39:43.453415    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:39:43.758013 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:39:43.757933    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:39:59.320067 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:39:59.320031    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:40:13.449874 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:40:13.449833    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:40:23.542265 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:40:23.542222    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:40:32.448066 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:40:32.447972    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:40:43.449325 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:40:43.449282    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:41:05.018182 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:41:05.018142    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:41:09.848480 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:41:09.848440    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:41:13.449978 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:41:13.449943    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:41:24.744221 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:41:24.744181    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:41:35.352123 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:41:35.352087    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:41:39.546377 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:41:39.546339    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:41:40.080853 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:41:40.080810    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-2l2q9"]
Nov 04 01:41:40.081067 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:41:40.081033    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-2l2q9" podUID="090cca38-c8e7-4a6f-96a7-e569fae6df5a" containerName="manager" containerID="cri-o://bb76d8f5b420fb00d8906b457d0326b1d999669c3787ff99d2ef0962576c2862" gracePeriod=10
Nov 04 01:41:40.945703 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:41:40.945670    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-2l2q9"
Nov 04 01:41:40.958494 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:41:40.958467    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/090cca38-c8e7-4a6f-96a7-e569fae6df5a-extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/090cca38-c8e7-4a6f-96a7-e569fae6df5a-extensions-socket-volume\") pod \"090cca38-c8e7-4a6f-96a7-e569fae6df5a\" (UID: \"090cca38-c8e7-4a6f-96a7-e569fae6df5a\") "
Nov 04 01:41:40.958650 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:41:40.958525    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/090cca38-c8e7-4a6f-96a7-e569fae6df5a-kube-api-access-kx9ww\" (UniqueName: \"kubernetes.io/projected/090cca38-c8e7-4a6f-96a7-e569fae6df5a-kube-api-access-kx9ww\") pod \"090cca38-c8e7-4a6f-96a7-e569fae6df5a\" (UID: \"090cca38-c8e7-4a6f-96a7-e569fae6df5a\") "
Nov 04 01:41:40.958884 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:41:40.958858    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/090cca38-c8e7-4a6f-96a7-e569fae6df5a-extensions-socket-volume" pod "090cca38-c8e7-4a6f-96a7-e569fae6df5a" (UID: "090cca38-c8e7-4a6f-96a7-e569fae6df5a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:41:40.960726 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:41:40.960700    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/090cca38-c8e7-4a6f-96a7-e569fae6df5a-kube-api-access-kx9ww" pod "090cca38-c8e7-4a6f-96a7-e569fae6df5a" (UID: "090cca38-c8e7-4a6f-96a7-e569fae6df5a"). InnerVolumeSpecName "kube-api-access-kx9ww". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:41:41.060265 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:41:41.060191    5068 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/090cca38-c8e7-4a6f-96a7-e569fae6df5a-extensions-socket-volume\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:41:41.060265 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:41:41.060217    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kx9ww\" (UniqueName: \"kubernetes.io/projected/090cca38-c8e7-4a6f-96a7-e569fae6df5a-kube-api-access-kx9ww\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:41:41.333503 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:41:41.333416    5068 generic.go:356] "Generic (PLEG): container finished" podID="090cca38-c8e7-4a6f-96a7-e569fae6df5a" containerID="bb76d8f5b420fb00d8906b457d0326b1d999669c3787ff99d2ef0962576c2862" exitCode=0
Nov 04 01:41:41.333503 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:41:41.333463    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-2l2q9"
Nov 04 01:41:41.333708 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:41:41.333500    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-2l2q9" event={"ID":"090cca38-c8e7-4a6f-96a7-e569fae6df5a","Type":"ContainerDied","Data":"bb76d8f5b420fb00d8906b457d0326b1d999669c3787ff99d2ef0962576c2862"}
Nov 04 01:41:41.333708 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:41:41.333554    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-2l2q9" event={"ID":"090cca38-c8e7-4a6f-96a7-e569fae6df5a","Type":"ContainerDied","Data":"63d1392501efe5797ca95c232ebace6cc8d34f846f9090d61544beb0c84835f8"}
Nov 04 01:41:41.333708 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:41:41.333577    5068 scope.go:122] "RemoveContainer" containerID="bb76d8f5b420fb00d8906b457d0326b1d999669c3787ff99d2ef0962576c2862"
Nov 04 01:41:41.348186 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:41:41.348171    5068 scope.go:122] "RemoveContainer" containerID="bb76d8f5b420fb00d8906b457d0326b1d999669c3787ff99d2ef0962576c2862"
Nov 04 01:41:41.348429 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:41:41.348414    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb76d8f5b420fb00d8906b457d0326b1d999669c3787ff99d2ef0962576c2862\": container with ID starting with bb76d8f5b420fb00d8906b457d0326b1d999669c3787ff99d2ef0962576c2862 not found: ID does not exist" containerID="bb76d8f5b420fb00d8906b457d0326b1d999669c3787ff99d2ef0962576c2862"
Nov 04 01:41:41.348483 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:41:41.348438    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb76d8f5b420fb00d8906b457d0326b1d999669c3787ff99d2ef0962576c2862"} err="failed to get container status \"bb76d8f5b420fb00d8906b457d0326b1d999669c3787ff99d2ef0962576c2862\": rpc error: code = NotFound desc = could not find container \"bb76d8f5b420fb00d8906b457d0326b1d999669c3787ff99d2ef0962576c2862\": container with ID starting with bb76d8f5b420fb00d8906b457d0326b1d999669c3787ff99d2ef0962576c2862 not found: ID does not exist"
Nov 04 01:41:41.358888 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:41:41.358867    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-2l2q9"]
Nov 04 01:41:41.362598 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:41:41.362577    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-2l2q9"]
Nov 04 01:41:42.111820 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:41:42.111787    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="090cca38-c8e7-4a6f-96a7-e569fae6df5a" path="/var/lib/kubelet/pods/090cca38-c8e7-4a6f-96a7-e569fae6df5a/volumes"
Nov 04 01:41:43.449418 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:41:43.449380    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:42:13.449639 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:42:13.449588    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:42:43.450347 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:42:43.450306    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:42:44.477026 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:42:44.476994    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tr8b2_21d1d23b-7b6e-4ab5-9593-cf7784fffea2/console-operator/2.log"
Nov 04 01:42:44.578326 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:42:44.578301    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tr8b2_21d1d23b-7b6e-4ab5-9593-cf7784fffea2/console-operator/2.log"
Nov 04 01:43:07.342512 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:43:07.342471    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:43:13.449374 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:43:13.449323    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:43:20.146162 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:43:20.146126    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:43:24.471633 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:43:24.470704    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:43:27.969272 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:43:27.969240    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:43:43.477825 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:43:43.477783    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:43:45.054377 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:43:45.054321    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:44:00.130586 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:44:00.130547    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:44:13.450049 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:44:13.450012    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:44:15.966452 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:44:15.966408    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:44:39.742695 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:44:39.742646    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:44:43.448598 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:44:43.448563    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:44:48.436409 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:44:48.436370    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:45:13.449441 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:45:13.449397    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:45:20.449336 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:45:20.449252    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:45:25.238696 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:45:25.238663    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:45:40.408288 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:45:40.408244    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:45:43.449911 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:45:43.449870    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:45:45.459179 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:45:45.459143    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:46:01.746763 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:46:01.746726    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:46:05.679847 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:46:05.679791    5068 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-848f6bf8bc-df8wd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:46:05.680499 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:46:05.680449    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-df8wd" podUID="70049bd7-ed93-4151-a7de-1938bbe0590d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:46:13.449780 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:46:13.449733    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:46:17.460187 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:46:17.460153    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:46:20.693770 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:46:20.693725    5068 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-848f6bf8bc-df8wd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:46:20.694409 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:46:20.694375    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-df8wd" podUID="70049bd7-ed93-4151-a7de-1938bbe0590d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:46:29.032653 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:46:29.032587    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:46:35.616254 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:46:35.616208    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:46:35.683890 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:46:35.683818    5068 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-848f6bf8bc-df8wd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:46:35.684224 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:46:35.684174    5068 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-848f6bf8bc-df8wd" podUID="70049bd7-ed93-4151-a7de-1938bbe0590d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:46:41.041061 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:46:41.041026    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:46:43.463794 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:46:43.463756    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:46:46.359358 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:46:46.359323    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:46:59.648813 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:46:59.648767    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:47:08.812536 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.812505    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-bd788d5f-spvdt"]
Nov 04 01:47:08.813053 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.812910    5068 cpu_manager.go:414] "containerMap: removing container" podUID="7c58619a-e60c-420d-a1b8-e1e54d7dce0e" containerName="main"
Nov 04 01:47:08.813053 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.812923    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7c58619a-e60c-420d-a1b8-e1e54d7dce0e" containerName="main"
Nov 04 01:47:08.813053 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.812936    5068 cpu_manager.go:414] "containerMap: removing container" podUID="d091cbea-bbd1-4242-a1c9-f72628c8e019" containerName="istio-proxy"
Nov 04 01:47:08.813053 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.812941    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d091cbea-bbd1-4242-a1c9-f72628c8e019" containerName="istio-proxy"
Nov 04 01:47:08.813053 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.812949    5068 cpu_manager.go:414] "containerMap: removing container" podUID="34c18742-f9bd-4f63-8e99-6a63af9c5999" containerName="payload-pre-processing"
Nov 04 01:47:08.813053 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.812982    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="34c18742-f9bd-4f63-8e99-6a63af9c5999" containerName="payload-pre-processing"
Nov 04 01:47:08.813053 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.812992    5068 cpu_manager.go:414] "containerMap: removing container" podUID="b21786c9-e7fa-43eb-bb36-f2633486b8b5" containerName="istio-proxy"
Nov 04 01:47:08.813053 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.812997    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b21786c9-e7fa-43eb-bb36-f2633486b8b5" containerName="istio-proxy"
Nov 04 01:47:08.813053 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813008    5068 cpu_manager.go:414] "containerMap: removing container" podUID="cbbfd346-7a24-49e7-8853-b60e0dbf754c" containerName="istio-proxy"
Nov 04 01:47:08.813053 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813026    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cbbfd346-7a24-49e7-8853-b60e0dbf754c" containerName="istio-proxy"
Nov 04 01:47:08.813053 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813035    5068 cpu_manager.go:414] "containerMap: removing container" podUID="4f6317cf-be73-46fc-bf2d-b694dc34e258" containerName="maas-api"
Nov 04 01:47:08.813053 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813039    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4f6317cf-be73-46fc-bf2d-b694dc34e258" containerName="maas-api"
Nov 04 01:47:08.813053 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813046    5068 cpu_manager.go:414] "containerMap: removing container" podUID="343b7e45-62f4-4533-a07c-0732c85a5b32" containerName="cleanup"
Nov 04 01:47:08.813053 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813050    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="343b7e45-62f4-4533-a07c-0732c85a5b32" containerName="cleanup"
Nov 04 01:47:08.813053 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813056    5068 cpu_manager.go:414] "containerMap: removing container" podUID="78c3da50-fc89-4b70-ae04-79c623854724" containerName="cleanup"
Nov 04 01:47:08.813053 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813061    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="78c3da50-fc89-4b70-ae04-79c623854724" containerName="cleanup"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813068    5068 cpu_manager.go:414] "containerMap: removing container" podUID="36b71cc4-21b7-42e0-86d7-1f25e215d068" containerName="maas-api"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813072    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="36b71cc4-21b7-42e0-86d7-1f25e215d068" containerName="maas-api"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813105    5068 cpu_manager.go:414] "containerMap: removing container" podUID="c6ddf85e-cd63-46b0-9f07-f6c93d158181" containerName="cleanup"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813110    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c6ddf85e-cd63-46b0-9f07-f6c93d158181" containerName="cleanup"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813116    5068 cpu_manager.go:414] "containerMap: removing container" podUID="343b7e45-62f4-4533-a07c-0732c85a5b32" containerName="cleanup"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813120    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="343b7e45-62f4-4533-a07c-0732c85a5b32" containerName="cleanup"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813128    5068 cpu_manager.go:414] "containerMap: removing container" podUID="88233bfe-a2e5-4dc2-b4ab-d6f9772f7219" containerName="payload-processing"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813132    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="88233bfe-a2e5-4dc2-b4ab-d6f9772f7219" containerName="payload-processing"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813140    5068 cpu_manager.go:414] "containerMap: removing container" podUID="1ac72a77-4112-4df4-ada5-ea100fe5dfb2" containerName="maas-api"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813145    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1ac72a77-4112-4df4-ada5-ea100fe5dfb2" containerName="maas-api"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813156    5068 cpu_manager.go:414] "containerMap: removing container" podUID="5f777506-99a6-4278-a678-de005b0722e6" containerName="istio-proxy"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813161    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5f777506-99a6-4278-a678-de005b0722e6" containerName="istio-proxy"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813176    5068 cpu_manager.go:414] "containerMap: removing container" podUID="52491e2f-36e5-4e6b-ae82-5795752be764" containerName="istio-proxy"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813182    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="52491e2f-36e5-4e6b-ae82-5795752be764" containerName="istio-proxy"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813191    5068 cpu_manager.go:414] "containerMap: removing container" podUID="239d3dec-a8fd-45e4-82ec-1ed4e1d01c72" containerName="istio-proxy"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813198    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="239d3dec-a8fd-45e4-82ec-1ed4e1d01c72" containerName="istio-proxy"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813210    5068 cpu_manager.go:414] "containerMap: removing container" podUID="090cca38-c8e7-4a6f-96a7-e569fae6df5a" containerName="manager"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813216    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="090cca38-c8e7-4a6f-96a7-e569fae6df5a" containerName="manager"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813240    5068 cpu_manager.go:414] "containerMap: removing container" podUID="332ac8c5-1996-489e-8a9c-3efcdc3e9ce5" containerName="istio-proxy"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813246    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="332ac8c5-1996-489e-8a9c-3efcdc3e9ce5" containerName="istio-proxy"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813261    5068 cpu_manager.go:414] "containerMap: removing container" podUID="c6ddf85e-cd63-46b0-9f07-f6c93d158181" containerName="cleanup"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813267    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c6ddf85e-cd63-46b0-9f07-f6c93d158181" containerName="cleanup"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813276    5068 cpu_manager.go:414] "containerMap: removing container" podUID="788ddd8c-788c-4575-8208-fd949c0bcc38" containerName="payload-processing"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813282    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="788ddd8c-788c-4575-8208-fd949c0bcc38" containerName="payload-processing"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813323    5068 cpu_manager.go:414] "containerMap: removing container" podUID="32f60f59-a98f-46ba-9fb4-8420f77998da" containerName="main"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813331    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="32f60f59-a98f-46ba-9fb4-8420f77998da" containerName="main"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813343    5068 cpu_manager.go:414] "containerMap: removing container" podUID="bda9909c-2480-4f83-9cef-3d14f19568ac" containerName="payload-pre-processing"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813351    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bda9909c-2480-4f83-9cef-3d14f19568ac" containerName="payload-pre-processing"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813361    5068 cpu_manager.go:414] "containerMap: removing container" podUID="343b7e45-62f4-4533-a07c-0732c85a5b32" containerName="cleanup"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813365    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="343b7e45-62f4-4533-a07c-0732c85a5b32" containerName="cleanup"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813371    5068 cpu_manager.go:414] "containerMap: removing container" podUID="4d2564ce-bab1-4bf5-b7dd-d2077596cf43" containerName="main"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813375    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4d2564ce-bab1-4bf5-b7dd-d2077596cf43" containerName="main"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813383    5068 cpu_manager.go:414] "containerMap: removing container" podUID="78c3da50-fc89-4b70-ae04-79c623854724" containerName="cleanup"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813388    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="78c3da50-fc89-4b70-ae04-79c623854724" containerName="cleanup"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813393    5068 cpu_manager.go:414] "containerMap: removing container" podUID="8a9762e4-1ac3-45c8-8324-c007bddda95f" containerName="main"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813398    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8a9762e4-1ac3-45c8-8324-c007bddda95f" containerName="main"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813404    5068 cpu_manager.go:414] "containerMap: removing container" podUID="9b28aa81-39ec-4761-a076-0e5a2db936cd" containerName="istio-proxy"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813408    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9b28aa81-39ec-4761-a076-0e5a2db936cd" containerName="istio-proxy"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813416    5068 cpu_manager.go:414] "containerMap: removing container" podUID="58b0b993-d096-479a-9cc8-907633a20652" containerName="istio-proxy"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813423    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="58b0b993-d096-479a-9cc8-907633a20652" containerName="istio-proxy"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813436    5068 cpu_manager.go:414] "containerMap: removing container" podUID="98e65cde-ca44-4ad9-9a57-ce81c0369a0f" containerName="payload-processing"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813443    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="98e65cde-ca44-4ad9-9a57-ce81c0369a0f" containerName="payload-processing"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813566    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="88233bfe-a2e5-4dc2-b4ab-d6f9772f7219" containerName="payload-processing"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813578    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="1ac72a77-4112-4df4-ada5-ea100fe5dfb2" containerName="maas-api"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813590    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="5f777506-99a6-4278-a678-de005b0722e6" containerName="istio-proxy"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813598    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="d091cbea-bbd1-4242-a1c9-f72628c8e019" containerName="istio-proxy"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813620    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="52491e2f-36e5-4e6b-ae82-5795752be764" containerName="istio-proxy"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813630    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="239d3dec-a8fd-45e4-82ec-1ed4e1d01c72" containerName="istio-proxy"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813636    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="090cca38-c8e7-4a6f-96a7-e569fae6df5a" containerName="manager"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813642    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="332ac8c5-1996-489e-8a9c-3efcdc3e9ce5" containerName="istio-proxy"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813648    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="c6ddf85e-cd63-46b0-9f07-f6c93d158181" containerName="cleanup"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813653    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="788ddd8c-788c-4575-8208-fd949c0bcc38" containerName="payload-processing"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813660    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="32f60f59-a98f-46ba-9fb4-8420f77998da" containerName="main"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813666    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="bda9909c-2480-4f83-9cef-3d14f19568ac" containerName="payload-pre-processing"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813673    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="78c3da50-fc89-4b70-ae04-79c623854724" containerName="cleanup"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813678    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="343b7e45-62f4-4533-a07c-0732c85a5b32" containerName="cleanup"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813683    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="4d2564ce-bab1-4bf5-b7dd-d2077596cf43" containerName="main"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813689    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="c6ddf85e-cd63-46b0-9f07-f6c93d158181" containerName="cleanup"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813697    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="78c3da50-fc89-4b70-ae04-79c623854724" containerName="cleanup"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813706    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="8a9762e4-1ac3-45c8-8324-c007bddda95f" containerName="main"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813713    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="9b28aa81-39ec-4761-a076-0e5a2db936cd" containerName="istio-proxy"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813723    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="58b0b993-d096-479a-9cc8-907633a20652" containerName="istio-proxy"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813730    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="98e65cde-ca44-4ad9-9a57-ce81c0369a0f" containerName="payload-processing"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813736    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="7c58619a-e60c-420d-a1b8-e1e54d7dce0e" containerName="main"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813741    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="d091cbea-bbd1-4242-a1c9-f72628c8e019" containerName="istio-proxy"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813747    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="34c18742-f9bd-4f63-8e99-6a63af9c5999" containerName="payload-pre-processing"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813755    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="b21786c9-e7fa-43eb-bb36-f2633486b8b5" containerName="istio-proxy"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813762    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="cbbfd346-7a24-49e7-8853-b60e0dbf754c" containerName="istio-proxy"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813766    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="4f6317cf-be73-46fc-bf2d-b694dc34e258" containerName="maas-api"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813771    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="36b71cc4-21b7-42e0-86d7-1f25e215d068" containerName="maas-api"
Nov 04 01:47:08.813838 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.813777    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="343b7e45-62f4-4533-a07c-0732c85a5b32" containerName="cleanup"
Nov 04 01:47:08.817139 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.817117    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-bd788d5f-spvdt"
Nov 04 01:47:08.832390 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.829180    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-bd788d5f-spvdt"]
Nov 04 01:47:08.951277 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.951240    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9767fc3-6433-4b05-94e3-9625b49a9aa7-openshift-service-ca\") pod \"maas-api-bd788d5f-spvdt\" (UID: \"c9767fc3-6433-4b05-94e3-9625b49a9aa7\") " pod="odh-ai-gateway-infra/maas-api-bd788d5f-spvdt"
Nov 04 01:47:08.951277 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.951280    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9767fc3-6433-4b05-94e3-9625b49a9aa7-metrics-certs\") pod \"maas-api-bd788d5f-spvdt\" (UID: \"c9767fc3-6433-4b05-94e3-9625b49a9aa7\") " pod="odh-ai-gateway-infra/maas-api-bd788d5f-spvdt"
Nov 04 01:47:08.951486 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.951379    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ct9zm\" (UniqueName: \"kubernetes.io/projected/c9767fc3-6433-4b05-94e3-9625b49a9aa7-kube-api-access-ct9zm\") pod \"maas-api-bd788d5f-spvdt\" (UID: \"c9767fc3-6433-4b05-94e3-9625b49a9aa7\") " pod="odh-ai-gateway-infra/maas-api-bd788d5f-spvdt"
Nov 04 01:47:08.951486 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:08.951419    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9767fc3-6433-4b05-94e3-9625b49a9aa7-maas-api-tls\") pod \"maas-api-bd788d5f-spvdt\" (UID: \"c9767fc3-6433-4b05-94e3-9625b49a9aa7\") " pod="odh-ai-gateway-infra/maas-api-bd788d5f-spvdt"
Nov 04 01:47:09.052631 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:09.052577    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9767fc3-6433-4b05-94e3-9625b49a9aa7-openshift-service-ca\") pod \"maas-api-bd788d5f-spvdt\" (UID: \"c9767fc3-6433-4b05-94e3-9625b49a9aa7\") " pod="odh-ai-gateway-infra/maas-api-bd788d5f-spvdt"
Nov 04 01:47:09.052846 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:09.052663    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9767fc3-6433-4b05-94e3-9625b49a9aa7-metrics-certs\") pod \"maas-api-bd788d5f-spvdt\" (UID: \"c9767fc3-6433-4b05-94e3-9625b49a9aa7\") " pod="odh-ai-gateway-infra/maas-api-bd788d5f-spvdt"
Nov 04 01:47:09.052846 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:09.052735    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ct9zm\" (UniqueName: \"kubernetes.io/projected/c9767fc3-6433-4b05-94e3-9625b49a9aa7-kube-api-access-ct9zm\") pod \"maas-api-bd788d5f-spvdt\" (UID: \"c9767fc3-6433-4b05-94e3-9625b49a9aa7\") " pod="odh-ai-gateway-infra/maas-api-bd788d5f-spvdt"
Nov 04 01:47:09.052846 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:09.052763    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9767fc3-6433-4b05-94e3-9625b49a9aa7-maas-api-tls\") pod \"maas-api-bd788d5f-spvdt\" (UID: \"c9767fc3-6433-4b05-94e3-9625b49a9aa7\") " pod="odh-ai-gateway-infra/maas-api-bd788d5f-spvdt"
Nov 04 01:47:09.053397 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:09.053370    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9767fc3-6433-4b05-94e3-9625b49a9aa7-openshift-service-ca\") pod \"maas-api-bd788d5f-spvdt\" (UID: \"c9767fc3-6433-4b05-94e3-9625b49a9aa7\") " pod="odh-ai-gateway-infra/maas-api-bd788d5f-spvdt"
Nov 04 01:47:09.056000 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:09.055970    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9767fc3-6433-4b05-94e3-9625b49a9aa7-metrics-certs\") pod \"maas-api-bd788d5f-spvdt\" (UID: \"c9767fc3-6433-4b05-94e3-9625b49a9aa7\") " pod="odh-ai-gateway-infra/maas-api-bd788d5f-spvdt"
Nov 04 01:47:09.056118 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:09.055970    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9767fc3-6433-4b05-94e3-9625b49a9aa7-maas-api-tls\") pod \"maas-api-bd788d5f-spvdt\" (UID: \"c9767fc3-6433-4b05-94e3-9625b49a9aa7\") " pod="odh-ai-gateway-infra/maas-api-bd788d5f-spvdt"
Nov 04 01:47:09.065192 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:09.065133    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ct9zm\" (UniqueName: \"kubernetes.io/projected/c9767fc3-6433-4b05-94e3-9625b49a9aa7-kube-api-access-ct9zm\") pod \"maas-api-bd788d5f-spvdt\" (UID: \"c9767fc3-6433-4b05-94e3-9625b49a9aa7\") " pod="odh-ai-gateway-infra/maas-api-bd788d5f-spvdt"
Nov 04 01:47:09.141492 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:09.141459    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-bd788d5f-spvdt"
Nov 04 01:47:09.422919 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:09.422895    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-bd788d5f-spvdt"]
Nov 04 01:47:09.425099 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:47:09.425071    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9767fc3_6433_4b05_94e3_9625b49a9aa7.slice/crio-88e924651bab9d8b496502eab507c89dc4a359ac5975df058db1efa24d57cbf2 WatchSource:0}: Error finding container 88e924651bab9d8b496502eab507c89dc4a359ac5975df058db1efa24d57cbf2: Status 404 returned error can't find the container with id 88e924651bab9d8b496502eab507c89dc4a359ac5975df058db1efa24d57cbf2
Nov 04 01:47:09.426459 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:09.426438    5068 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 01:47:10.247781 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:10.247695    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-bd788d5f-spvdt" event={"ID":"c9767fc3-6433-4b05-94e3-9625b49a9aa7","Type":"ContainerStarted","Data":"340f3184bbb571502a52bca2f81c968e52661f3af2171e2a3fe0ffa4bbfc3793"}
Nov 04 01:47:10.247781 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:10.247748    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-bd788d5f-spvdt" event={"ID":"c9767fc3-6433-4b05-94e3-9625b49a9aa7","Type":"ContainerStarted","Data":"88e924651bab9d8b496502eab507c89dc4a359ac5975df058db1efa24d57cbf2"}
Nov 04 01:47:10.248209 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:10.247842    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-bd788d5f-spvdt"
Nov 04 01:47:10.270111 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:10.270064    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-bd788d5f-spvdt" podStartSLOduration=1.7768998580000002 podStartE2EDuration="2.270051632s" totalImagesPullingTime="493.151774ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:47:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:47:10.267285293 +0000 UTC m=+2670.313846083" watchObservedRunningTime="2026-11-04 01:47:10.270051632 +0000 UTC m=+2670.316612421"
Nov 04 01:47:13.449227 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:47:13.449192    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:47:15.219692 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:15.219654    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:47:16.258275 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:16.258245    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-bd788d5f-spvdt"
Nov 04 01:47:16.318681 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:16.318646    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-668dc6579-zt6jf"]
Nov 04 01:47:16.318881 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:16.318853    5068 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-668dc6579-zt6jf" podUID="efb6ce93-18f8-44a6-9df0-90b96469447f" containerName="maas-api" containerID="cri-o://0554c45190fc62ff1515ceb96ec8d46313cb7f84e173a731cdf34ea9154abc85" gracePeriod=30
Nov 04 01:47:17.569216 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:17.569191    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-668dc6579-zt6jf"
Nov 04 01:47:17.651015 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:17.650937    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/efb6ce93-18f8-44a6-9df0-90b96469447f-kube-api-access-zx79b\" (UniqueName: \"kubernetes.io/projected/efb6ce93-18f8-44a6-9df0-90b96469447f-kube-api-access-zx79b\") pod \"efb6ce93-18f8-44a6-9df0-90b96469447f\" (UID: \"efb6ce93-18f8-44a6-9df0-90b96469447f\") "
Nov 04 01:47:17.651190 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:17.651018    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/efb6ce93-18f8-44a6-9df0-90b96469447f-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efb6ce93-18f8-44a6-9df0-90b96469447f-openshift-service-ca\") pod \"efb6ce93-18f8-44a6-9df0-90b96469447f\" (UID: \"efb6ce93-18f8-44a6-9df0-90b96469447f\") "
Nov 04 01:47:17.651190 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:17.651063    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/efb6ce93-18f8-44a6-9df0-90b96469447f-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efb6ce93-18f8-44a6-9df0-90b96469447f-maas-api-tls\") pod \"efb6ce93-18f8-44a6-9df0-90b96469447f\" (UID: \"efb6ce93-18f8-44a6-9df0-90b96469447f\") "
Nov 04 01:47:17.651190 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:17.651107    5068 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/efb6ce93-18f8-44a6-9df0-90b96469447f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/efb6ce93-18f8-44a6-9df0-90b96469447f-metrics-certs\") pod \"efb6ce93-18f8-44a6-9df0-90b96469447f\" (UID: \"efb6ce93-18f8-44a6-9df0-90b96469447f\") "
Nov 04 01:47:17.651440 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:17.651417    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/efb6ce93-18f8-44a6-9df0-90b96469447f-openshift-service-ca" pod "efb6ce93-18f8-44a6-9df0-90b96469447f" (UID: "efb6ce93-18f8-44a6-9df0-90b96469447f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:47:17.653213 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:17.653183    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efb6ce93-18f8-44a6-9df0-90b96469447f-maas-api-tls" pod "efb6ce93-18f8-44a6-9df0-90b96469447f" (UID: "efb6ce93-18f8-44a6-9df0-90b96469447f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:47:17.653324 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:17.653206    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efb6ce93-18f8-44a6-9df0-90b96469447f-metrics-certs" pod "efb6ce93-18f8-44a6-9df0-90b96469447f" (UID: "efb6ce93-18f8-44a6-9df0-90b96469447f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:47:17.653324 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:17.653261    5068 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efb6ce93-18f8-44a6-9df0-90b96469447f-kube-api-access-zx79b" pod "efb6ce93-18f8-44a6-9df0-90b96469447f" (UID: "efb6ce93-18f8-44a6-9df0-90b96469447f"). InnerVolumeSpecName "kube-api-access-zx79b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:47:17.751858 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:17.751820    5068 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zx79b\" (UniqueName: \"kubernetes.io/projected/efb6ce93-18f8-44a6-9df0-90b96469447f-kube-api-access-zx79b\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:47:17.751858 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:17.751849    5068 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efb6ce93-18f8-44a6-9df0-90b96469447f-openshift-service-ca\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:47:17.751858 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:17.751860    5068 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efb6ce93-18f8-44a6-9df0-90b96469447f-maas-api-tls\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:47:17.752098 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:17.751869    5068 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efb6ce93-18f8-44a6-9df0-90b96469447f-metrics-certs\") on node \"ip-10-0-132-67.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 01:47:18.290578 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:18.290548    5068 generic.go:356] "Generic (PLEG): container finished" podID="efb6ce93-18f8-44a6-9df0-90b96469447f" containerID="0554c45190fc62ff1515ceb96ec8d46313cb7f84e173a731cdf34ea9154abc85" exitCode=0
Nov 04 01:47:18.290777 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:18.290595    5068 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-668dc6579-zt6jf"
Nov 04 01:47:18.290777 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:18.290635    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-668dc6579-zt6jf" event={"ID":"efb6ce93-18f8-44a6-9df0-90b96469447f","Type":"ContainerDied","Data":"0554c45190fc62ff1515ceb96ec8d46313cb7f84e173a731cdf34ea9154abc85"}
Nov 04 01:47:18.290777 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:18.290686    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-668dc6579-zt6jf" event={"ID":"efb6ce93-18f8-44a6-9df0-90b96469447f","Type":"ContainerDied","Data":"de7a3173f23325cab060c3267c38666b2f97951bfef1b27bbe3eb48af8b077ad"}
Nov 04 01:47:18.290777 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:18.290707    5068 scope.go:122] "RemoveContainer" containerID="0554c45190fc62ff1515ceb96ec8d46313cb7f84e173a731cdf34ea9154abc85"
Nov 04 01:47:18.300020 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:18.300002    5068 scope.go:122] "RemoveContainer" containerID="0554c45190fc62ff1515ceb96ec8d46313cb7f84e173a731cdf34ea9154abc85"
Nov 04 01:47:18.300294 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:47:18.300275    5068 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0554c45190fc62ff1515ceb96ec8d46313cb7f84e173a731cdf34ea9154abc85\": container with ID starting with 0554c45190fc62ff1515ceb96ec8d46313cb7f84e173a731cdf34ea9154abc85 not found: ID does not exist" containerID="0554c45190fc62ff1515ceb96ec8d46313cb7f84e173a731cdf34ea9154abc85"
Nov 04 01:47:18.300353 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:18.300306    5068 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0554c45190fc62ff1515ceb96ec8d46313cb7f84e173a731cdf34ea9154abc85"} err="failed to get container status \"0554c45190fc62ff1515ceb96ec8d46313cb7f84e173a731cdf34ea9154abc85\": rpc error: code = NotFound desc = could not find container \"0554c45190fc62ff1515ceb96ec8d46313cb7f84e173a731cdf34ea9154abc85\": container with ID starting with 0554c45190fc62ff1515ceb96ec8d46313cb7f84e173a731cdf34ea9154abc85 not found: ID does not exist"
Nov 04 01:47:18.309274 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:18.309251    5068 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-668dc6579-zt6jf"]
Nov 04 01:47:18.314175 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:18.314150    5068 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-668dc6579-zt6jf"]
Nov 04 01:47:20.112014 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:20.111985    5068 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="efb6ce93-18f8-44a6-9df0-90b96469447f" path="/var/lib/kubelet/pods/efb6ce93-18f8-44a6-9df0-90b96469447f/volumes"
Nov 04 01:47:39.430270 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:39.430235    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:47:43.453548 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:47:43.453489    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:47:44.515524 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:44.515485    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tr8b2_21d1d23b-7b6e-4ab5-9593-cf7784fffea2/console-operator/2.log"
Nov 04 01:47:44.614727 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:44.614698    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tr8b2_21d1d23b-7b6e-4ab5-9593-cf7784fffea2/console-operator/2.log"
Nov 04 01:47:47.639683 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:47:47.639636    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:48:13.458829 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:48:13.458790    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:48:43.462283 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:48:43.462248    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:49:13.449126 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:49:13.449090    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:49:27.035688 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:49:27.028770    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:49:34.519832 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:49:34.519796    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:49:40.219598 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:49:40.219566    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:49:43.499939 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:49:43.499863    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:49:55.545783 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:49:55.545748    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:50:13.449337 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:50:13.449298    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:50:22.454550 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:50:22.454507    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gqfds"]
Nov 04 01:50:34.945336 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:50:34.945298    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-q9rwc_578bfcab-9f61-467e-89b2-911084fbe80f/manager/0.log"
Nov 04 01:50:35.146056 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:50:35.146011    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-ntcsr_1d70f32b-d928-4c1f-a5c3-fd4a55455107/manager/0.log"
Nov 04 01:50:37.753454 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:50:37.753416    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-bd788d5f-spvdt_c9767fc3-6433-4b05-94e3-9625b49a9aa7/maas-api/0.log"
Nov 04 01:50:41.827764 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:50:41.827728    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-gqfds_fd577f45-13c7-4247-9621-256b73bcf243/limitador/0.log"
Nov 04 01:50:42.015039 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:50:42.015009    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-vcbhc_65e1cd08-d934-467d-a5cb-3fcbdff09f95/manager/0.log"
Nov 04 01:50:43.441502 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:50:43.441464    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c_96d34b7a-d945-4c03-b858-88ef5c11fe03/istio-proxy/0.log"
Nov 04 01:50:43.449831 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:50:43.449801    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:50:43.630773 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:50:43.630744    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-gs6sr_9fe2e235-4232-4929-b61c-e31045e8e3f0/discovery/0.log"
Nov 04 01:50:44.390406 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:50:44.390379    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-848f6bf8bc-5g7t8_5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c/istio-proxy/0.log"
Nov 04 01:50:44.750892 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:50:44.750808    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-848f6bf8bc-df8wd_70049bd7-ed93-4151-a7de-1938bbe0590d/istio-proxy/1.log"
Nov 04 01:50:45.487948 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:50:45.487905    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-848f6bf8bc-mdxsf_8fea1dc2-7dc8-44b2-ae40-e84791006ba0/istio-proxy/0.log"
Nov 04 01:50:46.577354 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:50:46.577317    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5569c8658c-v7txl_ae02b148-c5ba-4db0-b817-9be4a488c708/router/0.log"
Nov 04 01:50:47.192287 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:50:47.192238    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-848f6bf8bc-df8wd_70049bd7-ed93-4151-a7de-1938bbe0590d/istio-proxy/0.log"
Nov 04 01:50:59.830232 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:50:59.830196    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-q9rwc_578bfcab-9f61-467e-89b2-911084fbe80f/manager/0.log"
Nov 04 01:51:00.041272 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:51:00.041239    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-ntcsr_1d70f32b-d928-4c1f-a5c3-fd4a55455107/manager/0.log"
Nov 04 01:51:01.667533 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:51:01.667487    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-bd788d5f-spvdt_c9767fc3-6433-4b05-94e3-9625b49a9aa7/maas-api/0.log"
Nov 04 01:51:05.163294 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:51:05.163266    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-gqfds_fd577f45-13c7-4247-9621-256b73bcf243/limitador/0.log"
Nov 04 01:51:05.353875 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:51:05.353840    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-vcbhc_65e1cd08-d934-467d-a5cb-3fcbdff09f95/manager/0.log"
Nov 04 01:51:05.925014 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:51:05.924980    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c_96d34b7a-d945-4c03-b858-88ef5c11fe03/istio-proxy/0.log"
Nov 04 01:51:06.110104 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:51:06.110080    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-gs6sr_9fe2e235-4232-4929-b61c-e31045e8e3f0/discovery/0.log"
Nov 04 01:51:06.889423 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:51:06.889391    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-848f6bf8bc-5g7t8_5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c/istio-proxy/0.log"
Nov 04 01:51:07.280157 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:51:07.280128    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-848f6bf8bc-df8wd_70049bd7-ed93-4151-a7de-1938bbe0590d/istio-proxy/1.log"
Nov 04 01:51:08.069000 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:51:08.068966    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-848f6bf8bc-mdxsf_8fea1dc2-7dc8-44b2-ae40-e84791006ba0/istio-proxy/0.log"
Nov 04 01:51:09.205078 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:51:09.205049    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5569c8658c-v7txl_ae02b148-c5ba-4db0-b817-9be4a488c708/router/0.log"
Nov 04 01:51:13.449290 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:51:13.449248    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:51:43.449943 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:51:43.449866    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:52:13.448725 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:52:13.448691    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:52:28.094791 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:28.094756    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-gnrzw_6be44550-8625-4296-aeb9-da833aaa4b2f/global-pull-secret-syncer/0.log"
Nov 04 01:52:28.259902 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:28.259875    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-6lcfv_23cb8b1a-61f6-4045-9f5a-3ccc1dc346a5/connection-checker/0.log"
Nov 04 01:52:28.285921 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:28.285895    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-hhfmk_a0c9abfd-2f47-4cb9-9d3c-479a6231be97/connection-checker/0.log"
Nov 04 01:52:28.314590 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:28.314566    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-rff54_4a79348d-8cf7-4429-8c5a-4084cf85fcc5/connection-checker/0.log"
Nov 04 01:52:28.480566 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:28.480492    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-sjcm7_3f74f502-9d11-4533-8bdf-cf880a0fbe0c/konnectivity-agent/0.log"
Nov 04 01:52:28.507507 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:28.507484    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-67.us-east-2.compute.internal_9589190378fb255196269e8a3bdf442c/haproxy/0.log"
Nov 04 01:52:35.406831 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:35.406798    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-gqfds_fd577f45-13c7-4247-9621-256b73bcf243/limitador/0.log"
Nov 04 01:52:35.482771 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:35.482739    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-vcbhc_65e1cd08-d934-467d-a5cb-3fcbdff09f95/manager/0.log"
Nov 04 01:52:37.142368 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:37.142339    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-884576c57-6vxbv_28a8304b-9226-422d-9044-b50678d52971/cluster-monitoring-operator/0.log"
Nov 04 01:52:37.612181 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:37.612150    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5gf2h_a623766e-ac81-46ee-8643-cb92eaaf61cb/node-exporter/0.log"
Nov 04 01:52:37.636308 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:37.636284    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5gf2h_a623766e-ac81-46ee-8643-cb92eaaf61cb/kube-rbac-proxy/0.log"
Nov 04 01:52:37.660528 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:37.660509    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5gf2h_a623766e-ac81-46ee-8643-cb92eaaf61cb/init-textfile/0.log"
Nov 04 01:52:38.267690 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:38.267657    5068 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-lsdgx/perf-node-gather-daemonset-p8wzf"]
Nov 04 01:52:38.268251 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:38.268217    5068 cpu_manager.go:414] "containerMap: removing container" podUID="c6ddf85e-cd63-46b0-9f07-f6c93d158181" containerName="cleanup"
Nov 04 01:52:38.268404 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:38.268259    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c6ddf85e-cd63-46b0-9f07-f6c93d158181" containerName="cleanup"
Nov 04 01:52:38.268404 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:38.268319    5068 cpu_manager.go:414] "containerMap: removing container" podUID="78c3da50-fc89-4b70-ae04-79c623854724" containerName="cleanup"
Nov 04 01:52:38.268404 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:38.268328    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="78c3da50-fc89-4b70-ae04-79c623854724" containerName="cleanup"
Nov 04 01:52:38.268404 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:38.268350    5068 cpu_manager.go:414] "containerMap: removing container" podUID="d091cbea-bbd1-4242-a1c9-f72628c8e019" containerName="istio-proxy"
Nov 04 01:52:38.268404 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:38.268357    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d091cbea-bbd1-4242-a1c9-f72628c8e019" containerName="istio-proxy"
Nov 04 01:52:38.268404 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:38.268375    5068 cpu_manager.go:414] "containerMap: removing container" podUID="efb6ce93-18f8-44a6-9df0-90b96469447f" containerName="maas-api"
Nov 04 01:52:38.268404 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:38.268384    5068 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="efb6ce93-18f8-44a6-9df0-90b96469447f" containerName="maas-api"
Nov 04 01:52:38.268795 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:38.268508    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="c6ddf85e-cd63-46b0-9f07-f6c93d158181" containerName="cleanup"
Nov 04 01:52:38.268795 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:38.268524    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="78c3da50-fc89-4b70-ae04-79c623854724" containerName="cleanup"
Nov 04 01:52:38.268795 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:38.268540    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="efb6ce93-18f8-44a6-9df0-90b96469447f" containerName="maas-api"
Nov 04 01:52:38.268795 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:38.268551    5068 memory_manager.go:365] "RemoveStaleState removing state" podUID="343b7e45-62f4-4533-a07c-0732c85a5b32" containerName="cleanup"
Nov 04 01:52:38.271983 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:38.271962    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lsdgx/perf-node-gather-daemonset-p8wzf"
Nov 04 01:52:38.281096 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:38.281072    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-lsdgx\"/\"kube-root-ca.crt\""
Nov 04 01:52:38.281300 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:38.281154    5068 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-lsdgx\"/\"openshift-service-ca.crt\""
Nov 04 01:52:38.281300 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:38.281251    5068 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-lsdgx\"/\"default-dockercfg-d5q5k\""
Nov 04 01:52:38.283360 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:38.283339    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lsdgx/perf-node-gather-daemonset-p8wzf"]
Nov 04 01:52:38.409894 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:38.409858    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c77f8bb0-a565-4881-88ea-8216a1b0555e-sys\") pod \"perf-node-gather-daemonset-p8wzf\" (UID: \"c77f8bb0-a565-4881-88ea-8216a1b0555e\") " pod="openshift-must-gather-lsdgx/perf-node-gather-daemonset-p8wzf"
Nov 04 01:52:38.409894 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:38.409896    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnw5q\" (UniqueName: \"kubernetes.io/projected/c77f8bb0-a565-4881-88ea-8216a1b0555e-kube-api-access-nnw5q\") pod \"perf-node-gather-daemonset-p8wzf\" (UID: \"c77f8bb0-a565-4881-88ea-8216a1b0555e\") " pod="openshift-must-gather-lsdgx/perf-node-gather-daemonset-p8wzf"
Nov 04 01:52:38.410141 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:38.409927    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c77f8bb0-a565-4881-88ea-8216a1b0555e-podres\") pod \"perf-node-gather-daemonset-p8wzf\" (UID: \"c77f8bb0-a565-4881-88ea-8216a1b0555e\") " pod="openshift-must-gather-lsdgx/perf-node-gather-daemonset-p8wzf"
Nov 04 01:52:38.410141 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:38.410007    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c77f8bb0-a565-4881-88ea-8216a1b0555e-lib-modules\") pod \"perf-node-gather-daemonset-p8wzf\" (UID: \"c77f8bb0-a565-4881-88ea-8216a1b0555e\") " pod="openshift-must-gather-lsdgx/perf-node-gather-daemonset-p8wzf"
Nov 04 01:52:38.410141 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:38.410074    5068 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c77f8bb0-a565-4881-88ea-8216a1b0555e-proc\") pod \"perf-node-gather-daemonset-p8wzf\" (UID: \"c77f8bb0-a565-4881-88ea-8216a1b0555e\") " pod="openshift-must-gather-lsdgx/perf-node-gather-daemonset-p8wzf"
Nov 04 01:52:38.511504 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:38.511470    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c77f8bb0-a565-4881-88ea-8216a1b0555e-proc\") pod \"perf-node-gather-daemonset-p8wzf\" (UID: \"c77f8bb0-a565-4881-88ea-8216a1b0555e\") " pod="openshift-must-gather-lsdgx/perf-node-gather-daemonset-p8wzf"
Nov 04 01:52:38.511742 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:38.511526    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c77f8bb0-a565-4881-88ea-8216a1b0555e-sys\") pod \"perf-node-gather-daemonset-p8wzf\" (UID: \"c77f8bb0-a565-4881-88ea-8216a1b0555e\") " pod="openshift-must-gather-lsdgx/perf-node-gather-daemonset-p8wzf"
Nov 04 01:52:38.511742 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:38.511545    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnw5q\" (UniqueName: \"kubernetes.io/projected/c77f8bb0-a565-4881-88ea-8216a1b0555e-kube-api-access-nnw5q\") pod \"perf-node-gather-daemonset-p8wzf\" (UID: \"c77f8bb0-a565-4881-88ea-8216a1b0555e\") " pod="openshift-must-gather-lsdgx/perf-node-gather-daemonset-p8wzf"
Nov 04 01:52:38.511742 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:38.511565    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c77f8bb0-a565-4881-88ea-8216a1b0555e-podres\") pod \"perf-node-gather-daemonset-p8wzf\" (UID: \"c77f8bb0-a565-4881-88ea-8216a1b0555e\") " pod="openshift-must-gather-lsdgx/perf-node-gather-daemonset-p8wzf"
Nov 04 01:52:38.511742 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:38.511594    5068 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c77f8bb0-a565-4881-88ea-8216a1b0555e-lib-modules\") pod \"perf-node-gather-daemonset-p8wzf\" (UID: \"c77f8bb0-a565-4881-88ea-8216a1b0555e\") " pod="openshift-must-gather-lsdgx/perf-node-gather-daemonset-p8wzf"
Nov 04 01:52:38.511742 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:38.511647    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c77f8bb0-a565-4881-88ea-8216a1b0555e-proc\") pod \"perf-node-gather-daemonset-p8wzf\" (UID: \"c77f8bb0-a565-4881-88ea-8216a1b0555e\") " pod="openshift-must-gather-lsdgx/perf-node-gather-daemonset-p8wzf"
Nov 04 01:52:38.511742 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:38.511651    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c77f8bb0-a565-4881-88ea-8216a1b0555e-sys\") pod \"perf-node-gather-daemonset-p8wzf\" (UID: \"c77f8bb0-a565-4881-88ea-8216a1b0555e\") " pod="openshift-must-gather-lsdgx/perf-node-gather-daemonset-p8wzf"
Nov 04 01:52:38.511742 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:38.511723    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c77f8bb0-a565-4881-88ea-8216a1b0555e-podres\") pod \"perf-node-gather-daemonset-p8wzf\" (UID: \"c77f8bb0-a565-4881-88ea-8216a1b0555e\") " pod="openshift-must-gather-lsdgx/perf-node-gather-daemonset-p8wzf"
Nov 04 01:52:38.512014 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:38.511768    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c77f8bb0-a565-4881-88ea-8216a1b0555e-lib-modules\") pod \"perf-node-gather-daemonset-p8wzf\" (UID: \"c77f8bb0-a565-4881-88ea-8216a1b0555e\") " pod="openshift-must-gather-lsdgx/perf-node-gather-daemonset-p8wzf"
Nov 04 01:52:38.523475 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:38.523408    5068 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnw5q\" (UniqueName: \"kubernetes.io/projected/c77f8bb0-a565-4881-88ea-8216a1b0555e-kube-api-access-nnw5q\") pod \"perf-node-gather-daemonset-p8wzf\" (UID: \"c77f8bb0-a565-4881-88ea-8216a1b0555e\") " pod="openshift-must-gather-lsdgx/perf-node-gather-daemonset-p8wzf"
Nov 04 01:52:38.586192 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:38.586160    5068 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lsdgx/perf-node-gather-daemonset-p8wzf"
Nov 04 01:52:38.862694 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:38.862668    5068 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lsdgx/perf-node-gather-daemonset-p8wzf"]
Nov 04 01:52:38.864923 ip-10-0-132-67 kubenswrapper[5068]: W1104 01:52:38.864897    5068 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc77f8bb0_a565_4881_88ea_8216a1b0555e.slice/crio-1ce32bdad850ec32f3fb851ef5efb6fdb187e9423e4d9bcc0283e9e9196ec007 WatchSource:0}: Error finding container 1ce32bdad850ec32f3fb851ef5efb6fdb187e9423e4d9bcc0283e9e9196ec007: Status 404 returned error can't find the container with id 1ce32bdad850ec32f3fb851ef5efb6fdb187e9423e4d9bcc0283e9e9196ec007
Nov 04 01:52:39.067899 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:39.067861    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lsdgx/perf-node-gather-daemonset-p8wzf" event={"ID":"c77f8bb0-a565-4881-88ea-8216a1b0555e","Type":"ContainerStarted","Data":"355cb941fbe73e9f16a90bef77255cbf7ac05b3b88db167d83c11668c4ae0cde"}
Nov 04 01:52:39.068086 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:39.067922    5068 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lsdgx/perf-node-gather-daemonset-p8wzf" event={"ID":"c77f8bb0-a565-4881-88ea-8216a1b0555e","Type":"ContainerStarted","Data":"1ce32bdad850ec32f3fb851ef5efb6fdb187e9423e4d9bcc0283e9e9196ec007"}
Nov 04 01:52:39.068086 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:39.068059    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-lsdgx/perf-node-gather-daemonset-p8wzf"
Nov 04 01:52:39.086696 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:39.086650    5068 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-lsdgx/perf-node-gather-daemonset-p8wzf" podStartSLOduration=1.086637943 podStartE2EDuration="1.086637943s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:52:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:52:39.08518572 +0000 UTC m=+2999.131746538" watchObservedRunningTime="2026-11-04 01:52:39.086637943 +0000 UTC m=+2999.133198733"
Nov 04 01:52:39.651304 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:39.651274    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-86c478bb88-27rmf_3739befd-6689-4197-b793-6f5b4b796880/networking-console-plugin/0.log"
Nov 04 01:52:40.276017 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:40.275988    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tr8b2_21d1d23b-7b6e-4ab5-9593-cf7784fffea2/console-operator/2.log"
Nov 04 01:52:40.280429 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:40.280407    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tr8b2_21d1d23b-7b6e-4ab5-9593-cf7784fffea2/console-operator/3.log"
Nov 04 01:52:41.468012 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:41.467981    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-58c9f5f46d-gdqjb_d4efaef2-c145-4839-b32f-ecf7a1c76a99/volume-data-source-validator/0.log"
Nov 04 01:52:42.678782 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:42.678755    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-klq54_2f68cb5a-3e22-4dd6-98d0-f28988ea86a2/dns/0.log"
Nov 04 01:52:42.703081 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:42.703058    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-klq54_2f68cb5a-3e22-4dd6-98d0-f28988ea86a2/kube-rbac-proxy/0.log"
Nov 04 01:52:42.811938 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:42.811915    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8gvq8_d6a4c158-e115-4d2f-af0d-1d9249dc7384/dns-node-resolver/0.log"
Nov 04 01:52:43.448792 ip-10-0-132-67 kubenswrapper[5068]: E1104 01:52:43.448756    5068 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-sg4cw" containerName="extractor"
Nov 04 01:52:43.513879 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:43.513848    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-fvjq7_7b044258-a241-4bd2-a6eb-9ed586a0f2cc/node-ca/0.log"
Nov 04 01:52:44.538894 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:44.538864    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c_96d34b7a-d945-4c03-b858-88ef5c11fe03/istio-proxy/0.log"
Nov 04 01:52:44.551739 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:44.551716    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tr8b2_21d1d23b-7b6e-4ab5-9593-cf7784fffea2/console-operator/2.log"
Nov 04 01:52:44.656078 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:44.656050    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tr8b2_21d1d23b-7b6e-4ab5-9593-cf7784fffea2/console-operator/2.log"
Nov 04 01:52:45.084804 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:45.084771    5068 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-lsdgx/perf-node-gather-daemonset-p8wzf"
Nov 04 01:52:45.489636 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:45.489548    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-gs6sr_9fe2e235-4232-4929-b61c-e31045e8e3f0/discovery/0.log"
Nov 04 01:52:45.610414 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:45.610386    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-848f6bf8bc-5g7t8_5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c/istio-proxy/0.log"
Nov 04 01:52:45.678190 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:45.678156    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-848f6bf8bc-df8wd_70049bd7-ed93-4151-a7de-1938bbe0590d/istio-proxy/0.log"
Nov 04 01:52:45.679592 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:45.679567    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-848f6bf8bc-df8wd_70049bd7-ed93-4151-a7de-1938bbe0590d/istio-proxy/1.log"
Nov 04 01:52:45.805758 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:45.805731    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-848f6bf8bc-mdxsf_8fea1dc2-7dc8-44b2-ae40-e84791006ba0/istio-proxy/0.log"
Nov 04 01:52:45.998134 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:45.998104    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5569c8658c-v7txl_ae02b148-c5ba-4db0-b817-9be4a488c708/router/0.log"
Nov 04 01:52:48.106812 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:48.106778    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4827b_82ac4b70-1c01-478c-ae1b-4a60dad06677/serve-healthcheck-canary/0.log"
Nov 04 01:52:48.909757 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:48.909724    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-7c4986d449-sbjxr_de97ada4-9cae-4775-a8f8-014004e17d25/insights-operator/0.log"
Nov 04 01:52:48.910599 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:48.910582    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-7c4986d449-sbjxr_de97ada4-9cae-4775-a8f8-014004e17d25/insights-operator/1.log"
Nov 04 01:52:49.155136 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:49.155109    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-sg4cw_ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774/kube-rbac-proxy/0.log"
Nov 04 01:52:49.182592 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:49.182526    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-sg4cw_ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774/exporter/0.log"
Nov 04 01:52:49.207237 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:49.207213    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-sg4cw_ad05b9b1-4ef6-4fec-83fb-b2b10d9d8774/extractor/0.log"
Nov 04 01:52:49.246411 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:49.246388    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_periodic-gathering-ntz4t-kjj6c_ccdd8744-3f01-4e10-84fe-cd936e6132fa/insights-gathering/0.log"
Nov 04 01:52:49.314009 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:49.313978    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c_96d34b7a-d945-4c03-b858-88ef5c11fe03/istio-proxy/0.log"
Nov 04 01:52:50.274829 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:50.274753    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-gs6sr_9fe2e235-4232-4929-b61c-e31045e8e3f0/discovery/0.log"
Nov 04 01:52:50.346945 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:50.346916    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-848f6bf8bc-5g7t8_5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c/istio-proxy/0.log"
Nov 04 01:52:50.383719 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:50.383692    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-848f6bf8bc-df8wd_70049bd7-ed93-4151-a7de-1938bbe0590d/istio-proxy/0.log"
Nov 04 01:52:50.384949 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:50.384925    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-848f6bf8bc-df8wd_70049bd7-ed93-4151-a7de-1938bbe0590d/istio-proxy/1.log"
Nov 04 01:52:50.459925 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:50.459903    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-848f6bf8bc-mdxsf_8fea1dc2-7dc8-44b2-ae40-e84791006ba0/istio-proxy/0.log"
Nov 04 01:52:50.584153 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:50.584091    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5569c8658c-v7txl_ae02b148-c5ba-4db0-b817-9be4a488c708/router/0.log"
Nov 04 01:52:52.131482 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:52.131452    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub-operator-system_opendatahub-operator-controller-manager-8ff7fd4ff-65jvg_781d0e29-3d65-4a97-ac09-aa8f3973e32a/manager/0.log"
Nov 04 01:52:52.159879 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:52.159856    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub-operator-system_opendatahub-operator-controller-manager-8ff7fd4ff-65jvg_781d0e29-3d65-4a97-ac09-aa8f3973e32a/copy-manifests/0.log"
Nov 04 01:52:53.718081 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:53.718049    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6bc49dd89-flz2c_96d34b7a-d945-4c03-b858-88ef5c11fe03/istio-proxy/0.log"
Nov 04 01:52:54.176976 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:54.176944    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-q9rwc_578bfcab-9f61-467e-89b2-911084fbe80f/manager/0.log"
Nov 04 01:52:54.734380 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:54.734338    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-ntcsr_1d70f32b-d928-4c1f-a5c3-fd4a55455107/manager/0.log"
Nov 04 01:52:54.901810 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:54.901776    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-gs6sr_9fe2e235-4232-4929-b61c-e31045e8e3f0/discovery/0.log"
Nov 04 01:52:55.004282 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:55.004199    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-848f6bf8bc-5g7t8_5cc6cc9a-2ce1-4505-bbd8-5eba3a93237c/istio-proxy/0.log"
Nov 04 01:52:55.050068 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:55.050021    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-848f6bf8bc-df8wd_70049bd7-ed93-4151-a7de-1938bbe0590d/istio-proxy/0.log"
Nov 04 01:52:55.051113 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:55.051095    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-848f6bf8bc-df8wd_70049bd7-ed93-4151-a7de-1938bbe0590d/istio-proxy/1.log"
Nov 04 01:52:55.139470 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:55.139435    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-848f6bf8bc-mdxsf_8fea1dc2-7dc8-44b2-ae40-e84791006ba0/istio-proxy/0.log"
Nov 04 01:52:55.269221 ip-10-0-132-67 kubenswrapper[5068]: I1104 01:52:55.269194    5068 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5569c8658c-v7txl_ae02b148-c5ba-4db0-b817-9be4a488c708/router/0.log"