Jul 30 14:46:25.130927 ip-10-0-133-117 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 30 14:46:25.130937 ip-10-0-133-117 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 30 14:46:25.130943 ip-10-0-133-117 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 30 14:46:25.131152 ip-10-0-133-117 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 30 14:46:35.243349 ip-10-0-133-117 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 30 14:46:35.243372 ip-10-0-133-117 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 9be5f83182bd499fac0929abb8d0c6c5 --
Jul 30 14:48:08.346434 ip-10-0-133-117 systemd[1]: Starting Kubernetes Kubelet...
Jul 30 14:48:08.778562 ip-10-0-133-117 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 30 14:48:08.778562 ip-10-0-133-117 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 30 14:48:08.778562 ip-10-0-133-117 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 30 14:48:08.778562 ip-10-0-133-117 kubenswrapper[2578]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 30 14:48:08.778562 ip-10-0-133-117 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 30 14:48:08.781607 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.781519    2578 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 30 14:48:08.788199 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788184    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 14:48:08.788199 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788198    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 14:48:08.788261 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788202    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 14:48:08.788261 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788206    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 14:48:08.788261 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788208    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 14:48:08.788261 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788211    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 14:48:08.788261 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788214    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 14:48:08.788261 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788217    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 14:48:08.788261 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788220    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 14:48:08.788261 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788223    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 14:48:08.788261 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788225    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 14:48:08.788261 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788228    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 14:48:08.788261 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788231    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 14:48:08.788261 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788234    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 14:48:08.788261 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788236    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 14:48:08.788261 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788240    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 14:48:08.788261 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788242    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 14:48:08.788261 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788245    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 14:48:08.788261 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788248    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 14:48:08.788261 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788253    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 14:48:08.788261 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788256    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 14:48:08.788261 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788258    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 14:48:08.788773 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788261    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 14:48:08.788773 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788264    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 14:48:08.788773 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788267    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 14:48:08.788773 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788270    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 14:48:08.788773 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788272    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 14:48:08.788773 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788275    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 14:48:08.788773 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788278    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 14:48:08.788773 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788281    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 14:48:08.788773 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788283    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 14:48:08.788773 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788286    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 14:48:08.788773 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788288    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 14:48:08.788773 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788291    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 14:48:08.788773 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788293    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 14:48:08.788773 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788296    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 14:48:08.788773 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788299    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 14:48:08.788773 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788301    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 14:48:08.788773 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788304    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 14:48:08.788773 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788307    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 14:48:08.788773 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788309    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 14:48:08.788773 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788312    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 14:48:08.789279 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788315    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 14:48:08.789279 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788317    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 14:48:08.789279 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788319    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 14:48:08.789279 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788322    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 14:48:08.789279 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788326    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 14:48:08.789279 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788330    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 14:48:08.789279 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788334    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 14:48:08.789279 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788338    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 14:48:08.789279 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788341    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 14:48:08.789279 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788343    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 14:48:08.789279 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788346    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 14:48:08.789279 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788348    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 14:48:08.789279 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788351    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 14:48:08.789279 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788354    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 14:48:08.789279 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788357    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 14:48:08.789279 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788360    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 14:48:08.789279 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788362    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 14:48:08.789279 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788365    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 14:48:08.789279 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788367    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 14:48:08.789279 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788369    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 14:48:08.789772 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788372    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 14:48:08.789772 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788376    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 14:48:08.789772 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788378    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 14:48:08.789772 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788381    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 14:48:08.789772 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788383    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 14:48:08.789772 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788386    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 14:48:08.789772 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788388    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 14:48:08.789772 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788391    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 14:48:08.789772 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788393    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 14:48:08.789772 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788397    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 14:48:08.789772 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788399    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 14:48:08.789772 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788402    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 14:48:08.789772 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788404    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 14:48:08.789772 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788407    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 14:48:08.789772 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788409    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 14:48:08.789772 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788412    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 14:48:08.789772 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788414    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 14:48:08.789772 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788416    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 14:48:08.789772 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788419    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 14:48:08.790256 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788423    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 14:48:08.790256 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788426    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 14:48:08.790256 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788429    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 14:48:08.790256 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788431    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 14:48:08.790256 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788434    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 14:48:08.790256 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788436    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 14:48:08.790256 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788440    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 14:48:08.790256 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788443    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 14:48:08.790256 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788445    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 14:48:08.790256 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788448    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 14:48:08.790256 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788450    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 14:48:08.790256 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788453    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 14:48:08.790256 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788455    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 14:48:08.790256 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788458    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 14:48:08.790256 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788460    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 14:48:08.790256 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788463    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 14:48:08.790256 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788465    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 14:48:08.790256 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.788468    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 14:48:08.790256 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789167    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 14:48:08.790725 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789173    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 14:48:08.790725 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789176    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 14:48:08.790725 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789179    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 14:48:08.790725 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789181    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 14:48:08.790725 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789185    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 14:48:08.790725 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789187    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 14:48:08.790725 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789190    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 14:48:08.790725 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789193    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 14:48:08.790725 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789195    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 14:48:08.790725 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789198    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 14:48:08.790725 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789200    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 14:48:08.790725 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789203    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 14:48:08.790725 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789205    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 14:48:08.790725 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789208    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 14:48:08.790725 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789210    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 14:48:08.790725 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789213    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 14:48:08.790725 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789216    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 14:48:08.790725 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789219    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 14:48:08.790725 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789221    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 14:48:08.791320 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789225    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 14:48:08.791320 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789228    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 14:48:08.791320 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789231    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 14:48:08.791320 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789233    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 14:48:08.791320 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789238    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 14:48:08.791320 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789242    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 14:48:08.791320 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789245    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 14:48:08.791320 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789247    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 14:48:08.791320 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789250    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 14:48:08.791320 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789252    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 14:48:08.791320 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789255    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 14:48:08.791320 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789257    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 14:48:08.791320 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789260    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 14:48:08.791320 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789262    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 14:48:08.791320 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789265    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 14:48:08.791320 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789268    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 14:48:08.791320 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789271    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 14:48:08.791320 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789273    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 14:48:08.791320 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789277    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 14:48:08.791798 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789279    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 14:48:08.791798 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789282    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 14:48:08.791798 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789284    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 14:48:08.791798 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789287    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 14:48:08.791798 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789289    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 14:48:08.791798 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789292    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 14:48:08.791798 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789296    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 14:48:08.791798 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789298    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 14:48:08.791798 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789301    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 14:48:08.791798 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789303    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 14:48:08.791798 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789306    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 14:48:08.791798 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789308    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 14:48:08.791798 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789311    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 14:48:08.791798 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789314    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 14:48:08.791798 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789317    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 14:48:08.791798 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789319    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 14:48:08.791798 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789322    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 14:48:08.791798 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789324    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 14:48:08.791798 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789329    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 14:48:08.792285 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789332    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 14:48:08.792285 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789335    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 14:48:08.792285 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789338    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 14:48:08.792285 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789340    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 14:48:08.792285 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789343    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 14:48:08.792285 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789346    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 14:48:08.792285 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789349    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 14:48:08.792285 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789351    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 14:48:08.792285 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789354    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 14:48:08.792285 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789356    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 14:48:08.792285 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789359    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 14:48:08.792285 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789361    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 14:48:08.792285 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789364    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 14:48:08.792285 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789367    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 14:48:08.792285 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789369    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 14:48:08.792285 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789372    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 14:48:08.792285 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789374    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 14:48:08.792285 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789376    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 14:48:08.792285 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789379    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 14:48:08.792285 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789381    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 14:48:08.792783 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789384    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 14:48:08.792783 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789386    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 14:48:08.792783 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789389    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 14:48:08.792783 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789392    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 14:48:08.792783 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789394    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 14:48:08.792783 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789397    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 14:48:08.792783 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789399    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 14:48:08.792783 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789403    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 14:48:08.792783 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789405    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 14:48:08.792783 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789408    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 14:48:08.792783 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789411    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 14:48:08.792783 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789413    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 14:48:08.792783 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789416    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 14:48:08.792783 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789418    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 14:48:08.792783 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789420    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 14:48:08.792783 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789423    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 14:48:08.792783 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789426    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 14:48:08.792783 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789428    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 14:48:08.792783 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789431    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 14:48:08.792783 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789434    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 14:48:08.793290 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789437    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 14:48:08.793290 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789792    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 14:48:08.793290 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789797    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 14:48:08.793290 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789799    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 14:48:08.793290 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789802    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 14:48:08.793290 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789805    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 14:48:08.793290 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789808    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 14:48:08.793290 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789811    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 14:48:08.793290 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789814    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 14:48:08.793290 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789816    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 14:48:08.793290 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789819    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 14:48:08.793290 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789822    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 14:48:08.793290 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789824    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 14:48:08.793290 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789827    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 14:48:08.793290 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789830    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 14:48:08.793290 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789832    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 14:48:08.793290 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789835    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 14:48:08.793290 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789837    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 14:48:08.793290 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789840    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 14:48:08.793759 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789843    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 14:48:08.793759 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789846    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 14:48:08.793759 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789848    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 14:48:08.793759 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789851    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 14:48:08.793759 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789855    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 14:48:08.793759 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789858    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 14:48:08.793759 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789862    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 14:48:08.793759 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789865    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 14:48:08.793759 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789868    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 14:48:08.793759 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789870    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 14:48:08.793759 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789873    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 14:48:08.793759 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789876    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 14:48:08.793759 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789878    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 14:48:08.793759 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789881    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 14:48:08.793759 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789886    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 14:48:08.793759 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789889    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 14:48:08.793759 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789891    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 14:48:08.793759 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789894    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 14:48:08.793759 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789897    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 14:48:08.794225 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789900    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 14:48:08.794225 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789904    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 14:48:08.794225 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789907    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 14:48:08.794225 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789909    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 14:48:08.794225 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789912    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 14:48:08.794225 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789914    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 14:48:08.794225 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789917    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 14:48:08.794225 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789920    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 14:48:08.794225 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789923    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 14:48:08.794225 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789926    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 14:48:08.794225 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789928    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 14:48:08.794225 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789931    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 14:48:08.794225 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789933    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 14:48:08.794225 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789936    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 14:48:08.794225 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789939    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 14:48:08.794225 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789941    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 14:48:08.794225 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789944    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 14:48:08.794225 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789947    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 14:48:08.794225 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789949    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 14:48:08.794685 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789952    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 14:48:08.794685 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789954    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 14:48:08.794685 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789957    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 14:48:08.794685 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789959    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 14:48:08.794685 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789961    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 14:48:08.794685 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789964    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 14:48:08.794685 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789966    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 14:48:08.794685 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789969    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 14:48:08.794685 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789971    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 14:48:08.794685 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789974    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 14:48:08.794685 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789976    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 14:48:08.794685 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789979    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 14:48:08.794685 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789981    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 14:48:08.794685 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789985    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 14:48:08.794685 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789988    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 14:48:08.794685 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789991    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 14:48:08.794685 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789993    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 14:48:08.794685 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789996    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 14:48:08.794685 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.789998    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 14:48:08.794685 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790001    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 14:48:08.795202 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790003    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 14:48:08.795202 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790005    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 14:48:08.795202 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790008    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 14:48:08.795202 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790010    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 14:48:08.795202 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790013    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 14:48:08.795202 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790016    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 14:48:08.795202 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790018    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 14:48:08.795202 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790021    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 14:48:08.795202 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790023    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 14:48:08.795202 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790026    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 14:48:08.795202 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790028    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 14:48:08.795202 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790031    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 14:48:08.795202 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790033    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 14:48:08.795202 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790036    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 14:48:08.795202 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790039    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 14:48:08.795202 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790041    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 14:48:08.795202 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790044    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 14:48:08.795202 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790046    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 14:48:08.795202 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790049    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 14:48:08.795202 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790051    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 14:48:08.795694 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790054    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 14:48:08.795694 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790056    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 14:48:08.795694 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790059    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 14:48:08.795694 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790132    2578 flags.go:64] FLAG: --address="0.0.0.0"
Jul 30 14:48:08.795694 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790139    2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 30 14:48:08.795694 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790159    2578 flags.go:64] FLAG: --anonymous-auth="true"
Jul 30 14:48:08.795694 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790164    2578 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 30 14:48:08.795694 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790168    2578 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 30 14:48:08.795694 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790172    2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 30 14:48:08.795694 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790176    2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 30 14:48:08.795694 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790181    2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 30 14:48:08.795694 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790184    2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 30 14:48:08.795694 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790187    2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 30 14:48:08.795694 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790193    2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 30 14:48:08.795694 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790198    2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 30 14:48:08.795694 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790201    2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 30 14:48:08.795694 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790204    2578 flags.go:64] FLAG: --cgroup-root=""
Jul 30 14:48:08.795694 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790207    2578 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 30 14:48:08.795694 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790210    2578 flags.go:64] FLAG: --client-ca-file=""
Jul 30 14:48:08.795694 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790213    2578 flags.go:64] FLAG: --cloud-provider="external"
Jul 30 14:48:08.795694 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790216    2578 flags.go:64] FLAG: --cluster-dns="[]"
Jul 30 14:48:08.795694 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790220    2578 flags.go:64] FLAG: --cluster-domain=""
Jul 30 14:48:08.795694 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790223    2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 30 14:48:08.795694 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790225    2578 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 30 14:48:08.796292 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790229    2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 30 14:48:08.796292 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790232    2578 flags.go:64] FLAG: --container-log-max-files="5"
Jul 30 14:48:08.796292 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790236    2578 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 30 14:48:08.796292 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790239    2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 30 14:48:08.796292 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790242    2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 30 14:48:08.796292 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790245    2578 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 30 14:48:08.796292 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790248    2578 flags.go:64] FLAG: --contention-profiling="false"
Jul 30 14:48:08.796292 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790250    2578 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 30 14:48:08.796292 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790254    2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 30 14:48:08.796292 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790257    2578 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 30 14:48:08.796292 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790260    2578 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 30 14:48:08.796292 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790264    2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 30 14:48:08.796292 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790267    2578 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 30 14:48:08.796292 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790270    2578 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 30 14:48:08.796292 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790273    2578 flags.go:64] FLAG: --enable-load-reader="false"
Jul 30 14:48:08.796292 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790276    2578 flags.go:64] FLAG: --enable-server="true"
Jul 30 14:48:08.796292 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790279    2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 30 14:48:08.796292 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790284    2578 flags.go:64] FLAG: --event-burst="100"
Jul 30 14:48:08.796292 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790287    2578 flags.go:64] FLAG: --event-qps="50"
Jul 30 14:48:08.796292 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790290    2578 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 30 14:48:08.796292 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790293    2578 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 30 14:48:08.796292 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790297    2578 flags.go:64] FLAG: --eviction-hard=""
Jul 30 14:48:08.796292 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790302    2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 30 14:48:08.796292 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790304    2578 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 30 14:48:08.796908 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790307    2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 30 14:48:08.796908 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790310    2578 flags.go:64] FLAG: --eviction-soft=""
Jul 30 14:48:08.796908 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790313    2578 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 30 14:48:08.796908 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790316    2578 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 30 14:48:08.796908 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790319    2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 30 14:48:08.796908 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790322    2578 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 30 14:48:08.796908 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790324    2578 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 30 14:48:08.796908 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790327    2578 flags.go:64] FLAG: --fail-swap-on="true"
Jul 30 14:48:08.796908 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790330    2578 flags.go:64] FLAG: --feature-gates=""
Jul 30 14:48:08.796908 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790333    2578 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 30 14:48:08.796908 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790336    2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 30 14:48:08.796908 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790339    2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 30 14:48:08.796908 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790342    2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 30 14:48:08.796908 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790345    2578 flags.go:64] FLAG: --healthz-port="10248"
Jul 30 14:48:08.796908 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790348    2578 flags.go:64] FLAG: --help="false"
Jul 30 14:48:08.796908 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790351    2578 flags.go:64] FLAG: --hostname-override="ip-10-0-133-117.ec2.internal"
Jul 30 14:48:08.796908 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790354    2578 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 30 14:48:08.796908 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790357    2578 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 30 14:48:08.796908 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790360    2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 30 14:48:08.796908 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790363    2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 30 14:48:08.796908 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790366    2578 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 30 14:48:08.796908 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790369    2578 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 30 14:48:08.796908 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790372    2578 flags.go:64] FLAG: --image-service-endpoint=""
Jul 30 14:48:08.797493 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790375    2578 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 30 14:48:08.797493 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790378    2578 flags.go:64] FLAG: --kube-api-burst="100"
Jul 30 14:48:08.797493 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790381    2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 30 14:48:08.797493 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790384    2578 flags.go:64] FLAG: --kube-api-qps="50"
Jul 30 14:48:08.797493 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790387    2578 flags.go:64] FLAG: --kube-reserved=""
Jul 30 14:48:08.797493 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790390    2578 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 30 14:48:08.797493 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790393    2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 30 14:48:08.797493 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790397    2578 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 30 14:48:08.797493 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790400    2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 30 14:48:08.797493 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790402    2578 flags.go:64] FLAG: --lock-file=""
Jul 30 14:48:08.797493 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790405    2578 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 30 14:48:08.797493 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790408    2578 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 30 14:48:08.797493 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790411    2578 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 30 14:48:08.797493 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790416    2578 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 30 14:48:08.797493 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790419    2578 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 30 14:48:08.797493 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790422    2578 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 30 14:48:08.797493 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790425    2578 flags.go:64] FLAG: --logging-format="text"
Jul 30 14:48:08.797493 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790428    2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 30 14:48:08.797493 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790431    2578 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 30 14:48:08.797493 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790434    2578 flags.go:64] FLAG: --manifest-url=""
Jul 30 14:48:08.797493 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790436    2578 flags.go:64] FLAG: --manifest-url-header=""
Jul 30 14:48:08.797493 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790441    2578 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 30 14:48:08.797493 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790444    2578 flags.go:64] FLAG: --max-open-files="1000000"
Jul 30 14:48:08.797493 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790452    2578 flags.go:64] FLAG: --max-pods="110"
Jul 30 14:48:08.797493 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790455    2578 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 30 14:48:08.798102 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790458    2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 30 14:48:08.798102 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790461    2578 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 30 14:48:08.798102 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790464    2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 30 14:48:08.798102 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790466    2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 30 14:48:08.798102 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790469    2578 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 30 14:48:08.798102 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790472    2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 30 14:48:08.798102 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790480    2578 flags.go:64] FLAG: --node-status-max-images="50"
Jul 30 14:48:08.798102 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790483    2578 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 30 14:48:08.798102 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790486    2578 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 30 14:48:08.798102 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790489    2578 flags.go:64] FLAG: --pod-cidr=""
Jul 30 14:48:08.798102 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790492    2578 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 30 14:48:08.798102 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790497    2578 flags.go:64] FLAG: --pod-manifest-path=""
Jul 30 14:48:08.798102 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790500    2578 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 30 14:48:08.798102 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790503    2578 flags.go:64] FLAG: --pods-per-core="0"
Jul 30 14:48:08.798102 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790508    2578 flags.go:64] FLAG: --port="10250"
Jul 30 14:48:08.798102 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790511    2578 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 30 14:48:08.798102 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790514    2578 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ddf7d15e42ecb29d"
Jul 30 14:48:08.798102 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790517    2578 flags.go:64] FLAG: --qos-reserved=""
Jul 30 14:48:08.798102 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790520    2578 flags.go:64] FLAG: --read-only-port="10255"
Jul 30 14:48:08.798102 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790522    2578 flags.go:64] FLAG: --register-node="true"
Jul 30 14:48:08.798102 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790525    2578 flags.go:64] FLAG: --register-with-taints=""
Jul 30 14:48:08.798102 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790529    2578 flags.go:64] FLAG: --registry-burst="10"
Jul 30 14:48:08.798102 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790532    2578 flags.go:64] FLAG: --registry-qps="5"
Jul 30 14:48:08.798102 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790535    2578 flags.go:64] FLAG: --reserved-cpus=""
Jul 30 14:48:08.798704 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790537    2578 flags.go:64] FLAG: --reserved-memory=""
Jul 30 14:48:08.798704 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790541    2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 30 14:48:08.798704 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790544    2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 30 14:48:08.798704 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790547    2578 flags.go:64] FLAG: --rotate-certificates="false"
Jul 30 14:48:08.798704 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790550    2578 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 30 14:48:08.798704 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790552    2578 flags.go:64] FLAG: --runonce="false"
Jul 30 14:48:08.798704 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790555    2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 30 14:48:08.798704 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790558    2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 30 14:48:08.798704 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790561    2578 flags.go:64] FLAG: --seccomp-default="false"
Jul 30 14:48:08.798704 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790564    2578 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 30 14:48:08.798704 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790567    2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 30 14:48:08.798704 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790570    2578 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 30 14:48:08.798704 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790573    2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 30 14:48:08.798704 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790576    2578 flags.go:64] FLAG: --storage-driver-password="root"
Jul 30 14:48:08.798704 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790579    2578 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 30 14:48:08.798704 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790582    2578 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 30 14:48:08.798704 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790584    2578 flags.go:64] FLAG: --storage-driver-user="root"
Jul 30 14:48:08.798704 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790587    2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 30 14:48:08.798704 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790591    2578 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 30 14:48:08.798704 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790594    2578 flags.go:64] FLAG: --system-cgroups=""
Jul 30 14:48:08.798704 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790597    2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 30 14:48:08.798704 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790602    2578 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 30 14:48:08.798704 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790606    2578 flags.go:64] FLAG: --tls-cert-file=""
Jul 30 14:48:08.798704 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790609    2578 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 30 14:48:08.798704 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790613    2578 flags.go:64] FLAG: --tls-min-version=""
Jul 30 14:48:08.799332 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790616    2578 flags.go:64] FLAG: --tls-private-key-file=""
Jul 30 14:48:08.799332 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790619    2578 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 30 14:48:08.799332 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790621    2578 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 30 14:48:08.799332 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790624    2578 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 30 14:48:08.799332 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790627    2578 flags.go:64] FLAG: --v="2"
Jul 30 14:48:08.799332 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790631    2578 flags.go:64] FLAG: --version="false"
Jul 30 14:48:08.799332 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790635    2578 flags.go:64] FLAG: --vmodule=""
Jul 30 14:48:08.799332 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790638    2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 30 14:48:08.799332 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.790641    2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 30 14:48:08.799332 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790766    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 14:48:08.799332 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790770    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 14:48:08.799332 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790773    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 14:48:08.799332 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790776    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 14:48:08.799332 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790779    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 14:48:08.799332 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790781    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 14:48:08.799332 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790784    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 14:48:08.799332 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790786    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 14:48:08.799332 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790789    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 14:48:08.799332 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790791    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 14:48:08.799332 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790794    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 14:48:08.799332 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790797    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 14:48:08.799332 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790799    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 14:48:08.799873 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790802    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 14:48:08.799873 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790805    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 14:48:08.799873 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790807    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 14:48:08.799873 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790810    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 14:48:08.799873 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790813    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 14:48:08.799873 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790816    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 14:48:08.799873 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790818    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 14:48:08.799873 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790823    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 14:48:08.799873 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790826    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 14:48:08.799873 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790829    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 14:48:08.799873 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790831    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 14:48:08.799873 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790834    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 14:48:08.799873 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790836    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 14:48:08.799873 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790839    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 14:48:08.799873 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790841    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 14:48:08.799873 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790844    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 14:48:08.799873 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790846    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 14:48:08.799873 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790848    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 14:48:08.799873 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790851    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 14:48:08.799873 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790853    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 14:48:08.800434 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790856    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 14:48:08.800434 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790858    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 14:48:08.800434 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790862    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 14:48:08.800434 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790866    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 14:48:08.800434 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790869    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 14:48:08.800434 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790871    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 14:48:08.800434 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790874    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 14:48:08.800434 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790876    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 14:48:08.800434 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790879    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 14:48:08.800434 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790881    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 14:48:08.800434 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790883    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 14:48:08.800434 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790886    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 14:48:08.800434 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790888    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 14:48:08.800434 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790891    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 14:48:08.800434 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790894    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 14:48:08.800434 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790896    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 14:48:08.800434 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790899    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 14:48:08.800434 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790902    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 14:48:08.800434 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790905    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 14:48:08.800916 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790909    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 14:48:08.800916 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790912    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 14:48:08.800916 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790914    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 14:48:08.800916 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790917    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 14:48:08.800916 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790919    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 14:48:08.800916 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790922    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 14:48:08.800916 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790924    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 14:48:08.800916 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790927    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 14:48:08.800916 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790929    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 14:48:08.800916 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790932    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 14:48:08.800916 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790935    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 14:48:08.800916 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790938    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 14:48:08.800916 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790940    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 14:48:08.800916 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790943    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 14:48:08.800916 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790945    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 14:48:08.800916 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790947    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 14:48:08.800916 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790952    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 14:48:08.800916 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790955    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 14:48:08.800916 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790958    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 14:48:08.801388 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790961    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 14:48:08.801388 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790964    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 14:48:08.801388 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790966    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 14:48:08.801388 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790969    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 14:48:08.801388 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790972    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 14:48:08.801388 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790974    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 14:48:08.801388 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790977    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 14:48:08.801388 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790979    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 14:48:08.801388 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790982    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 14:48:08.801388 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790984    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 14:48:08.801388 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790986    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 14:48:08.801388 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790989    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 14:48:08.801388 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790992    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 14:48:08.801388 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790996    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 14:48:08.801388 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.790999    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 14:48:08.801388 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.791001    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 14:48:08.801388 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.791004    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 14:48:08.801388 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.791008    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 14:48:08.801388 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.791011    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 14:48:08.801849 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.791014    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 14:48:08.801849 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.791017    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 14:48:08.801849 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.791020    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 14:48:08.801849 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.791023    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 14:48:08.801849 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.791025    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 14:48:08.801849 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.791028    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 14:48:08.801849 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.791030    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 14:48:08.801849 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.791032    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 14:48:08.801849 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.791035    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 14:48:08.801849 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.791937    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 30 14:48:08.801849 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.799018    2578 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 30 14:48:08.801849 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.799032    2578 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 30 14:48:08.801849 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.799996    2578 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 30 14:48:08.801849 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.800004    2578 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 30 14:48:08.802216 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800061    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 14:48:08.802216 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800066    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 14:48:08.802216 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800070    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 14:48:08.802216 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800073    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 14:48:08.802216 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800076    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 14:48:08.802216 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800079    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 14:48:08.802216 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800081    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 14:48:08.802216 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800084    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 14:48:08.802216 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800087    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 14:48:08.802216 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800090    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 14:48:08.802216 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800092    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 14:48:08.802216 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800095    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 14:48:08.802216 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800099    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 14:48:08.802216 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800102    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 14:48:08.802216 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800105    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 14:48:08.802216 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800108    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 14:48:08.802216 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800112    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 14:48:08.802216 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800115    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 14:48:08.802216 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800118    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 14:48:08.802676 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800121    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 14:48:08.802676 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800124    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 14:48:08.802676 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800126    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 14:48:08.802676 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800129    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 14:48:08.802676 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800132    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 14:48:08.802676 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800134    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 14:48:08.802676 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800137    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 14:48:08.802676 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800139    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 14:48:08.802676 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800156    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 14:48:08.802676 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800161    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 14:48:08.802676 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800165    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 14:48:08.802676 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800168    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 14:48:08.802676 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800171    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 14:48:08.802676 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800174    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 14:48:08.802676 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800179    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 14:48:08.802676 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800182    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 14:48:08.802676 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800185    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 14:48:08.802676 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800187    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 14:48:08.802676 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800190    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 14:48:08.802676 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800192    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 14:48:08.803181 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800195    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 14:48:08.803181 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800198    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 14:48:08.803181 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800200    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 14:48:08.803181 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800203    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 14:48:08.803181 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800205    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 14:48:08.803181 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800208    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 14:48:08.803181 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800211    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 14:48:08.803181 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800214    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 14:48:08.803181 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800216    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 14:48:08.803181 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800218    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 14:48:08.803181 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800221    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 14:48:08.803181 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800224    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 14:48:08.803181 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800226    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 14:48:08.803181 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800228    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 14:48:08.803181 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800231    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 14:48:08.803181 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800233    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 14:48:08.803181 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800236    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 14:48:08.803181 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800238    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 14:48:08.803181 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800241    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 14:48:08.803181 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800243    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 14:48:08.803656 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800246    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 14:48:08.803656 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800248    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 14:48:08.803656 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800251    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 14:48:08.803656 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800254    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 14:48:08.803656 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800256    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 14:48:08.803656 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800259    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 14:48:08.803656 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800261    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 14:48:08.803656 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800264    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 14:48:08.803656 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800267    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 14:48:08.803656 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800269    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 14:48:08.803656 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800272    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 14:48:08.803656 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800274    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 14:48:08.803656 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800277    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 14:48:08.803656 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800279    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 14:48:08.803656 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800281    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 14:48:08.803656 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800284    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 14:48:08.803656 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800286    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 14:48:08.803656 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800289    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 14:48:08.803656 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800291    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 14:48:08.803656 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800294    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 14:48:08.804280 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800296    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 14:48:08.804280 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800299    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 14:48:08.804280 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800302    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 14:48:08.804280 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800304    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 14:48:08.804280 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800306    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 14:48:08.804280 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800309    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 14:48:08.804280 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800311    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 14:48:08.804280 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800314    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 14:48:08.804280 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800317    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 14:48:08.804280 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800319    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 14:48:08.804280 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800322    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 14:48:08.804280 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800324    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 14:48:08.804280 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800327    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 14:48:08.804280 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800329    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 14:48:08.804280 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800332    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 14:48:08.804280 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800335    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 14:48:08.804280 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800338    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 14:48:08.804280 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800340    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 14:48:08.804280 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800343    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 14:48:08.804280 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800345    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 14:48:08.804760 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.800350    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 30 14:48:08.804760 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800475    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 14:48:08.804760 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800480    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 14:48:08.804760 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800483    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 14:48:08.804760 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800486    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 14:48:08.804760 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800488    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 14:48:08.804760 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800491    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 14:48:08.804760 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800494    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 14:48:08.804760 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800496    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 14:48:08.804760 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800499    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 14:48:08.804760 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800502    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 14:48:08.804760 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800504    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 14:48:08.804760 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800507    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 14:48:08.804760 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800509    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 14:48:08.804760 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800512    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 14:48:08.805127 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800514    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 14:48:08.805127 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800517    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 14:48:08.805127 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800519    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 14:48:08.805127 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800522    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 14:48:08.805127 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800525    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 14:48:08.805127 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800527    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 14:48:08.805127 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800530    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 14:48:08.805127 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800532    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 14:48:08.805127 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800535    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 14:48:08.805127 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800538    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 14:48:08.805127 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800540    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 14:48:08.805127 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800542    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 14:48:08.805127 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800545    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 14:48:08.805127 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800548    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 14:48:08.805127 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800550    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 14:48:08.805127 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800553    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 14:48:08.805127 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800555    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 14:48:08.805127 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800558    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 14:48:08.805127 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800560    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 14:48:08.805127 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800563    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 14:48:08.805626 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800565    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 14:48:08.805626 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800568    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 14:48:08.805626 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800570    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 14:48:08.805626 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800573    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 14:48:08.805626 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800575    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 14:48:08.805626 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800578    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 14:48:08.805626 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800580    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 14:48:08.805626 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800583    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 14:48:08.805626 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800585    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 14:48:08.805626 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800587    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 14:48:08.805626 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800591    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 14:48:08.805626 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800593    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 14:48:08.805626 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800596    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 14:48:08.805626 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800598    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 14:48:08.805626 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800601    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 14:48:08.805626 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800603    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 14:48:08.805626 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800606    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 14:48:08.805626 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800608    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 14:48:08.805626 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800612    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 14:48:08.806085 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800615    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 14:48:08.806085 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800618    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 14:48:08.806085 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800620    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 14:48:08.806085 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800623    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 14:48:08.806085 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800625    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 14:48:08.806085 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800628    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 14:48:08.806085 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800631    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 14:48:08.806085 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800633    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 14:48:08.806085 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800636    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 14:48:08.806085 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800638    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 14:48:08.806085 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800641    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 14:48:08.806085 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800643    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 14:48:08.806085 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800646    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 14:48:08.806085 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800648    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 14:48:08.806085 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800650    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 14:48:08.806085 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800653    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 14:48:08.806085 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800655    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 14:48:08.806085 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800658    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 14:48:08.806085 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800660    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 14:48:08.806085 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800663    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 14:48:08.806829 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800665    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 14:48:08.806829 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800667    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 14:48:08.806829 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800670    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 14:48:08.806829 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800674    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 14:48:08.806829 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800677    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 14:48:08.806829 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800681    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 14:48:08.806829 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800684    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 14:48:08.806829 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800687    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 14:48:08.806829 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800690    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 14:48:08.806829 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800693    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 14:48:08.806829 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800695    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 14:48:08.806829 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800698    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 14:48:08.806829 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800700    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 14:48:08.806829 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800703    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 14:48:08.806829 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800705    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 14:48:08.806829 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800707    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 14:48:08.806829 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800710    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 14:48:08.806829 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800713    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 14:48:08.806829 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800715    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 14:48:08.807512 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800718    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 14:48:08.807512 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800720    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 14:48:08.807512 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800723    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 14:48:08.807512 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800725    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 14:48:08.807512 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800727    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 14:48:08.807512 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800730    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 14:48:08.807512 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:08.800732    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 14:48:08.807512 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.800737    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 30 14:48:08.807512 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.800833    2578 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 30 14:48:08.807512 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.805513    2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 30 14:48:08.808307 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.808290    2578 server.go:1019] "Starting client certificate rotation"
Jul 30 14:48:08.808414 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.808396    2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 30 14:48:08.808450 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.808441    2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 30 14:48:08.811815 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.811801    2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 30 14:48:08.814675 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.814657    2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 30 14:48:08.826609 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.826588    2578 log.go:25] "Validated CRI v1 runtime API"
Jul 30 14:48:08.828914 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.828898    2578 log.go:25] "Validated CRI v1 image API"
Jul 30 14:48:08.830440 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.830424    2578 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 30 14:48:08.833674 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.833657    2578 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 9da9e219-a9d8-4638-9f7d-54f61a1a3287:/dev/nvme0n1p3 9e9a14cd-1639-4f34-b904-aab41b069dde:/dev/nvme0n1p4]
Jul 30 14:48:08.833727 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.833674    2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 30 14:48:08.837888 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.837772    2578 manager.go:216] Machine: {Timestamp:2026-07-30 14:48:08.837296456 +0000 UTC m=+0.366234920 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3116118 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec264865b36c510052887017bbf904ac SystemUUID:ec264865-b36c-5100-5288-7017bbf904ac BootID:9be5f831-82bd-499f-ac09-29abb8d0c6c5 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:0a:c4:ef:4e:87 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:0a:c4:ef:4e:87 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:fe:6b:48:78:7f:14 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 30 14:48:08.837888 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.837880    2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 30 14:48:08.837992 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.837981    2578 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 30 14:48:08.841372 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.841354    2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 30 14:48:08.841454 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.841429    2578 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 30 14:48:08.841633 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.841463    2578 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-117.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 30 14:48:08.842347 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.842335    2578 topology_manager.go:138] "Creating topology manager with none policy"
Jul 30 14:48:08.842396 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.842353    2578 container_manager_linux.go:309] "Creating device plugin manager"
Jul 30 14:48:08.842396 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.842363    2578 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 30 14:48:08.842396 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.842377    2578 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 30 14:48:08.843540 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.843529    2578 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 30 14:48:08.846287 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.846274    2578 state_mem.go:36] "Initialized new in-memory state store"
Jul 30 14:48:08.846416 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.846406    2578 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 30 14:48:08.846478 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.846444    2578 kubelet.go:486] "Attempting to sync node with API server"
Jul 30 14:48:08.846478 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.846459    2578 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 30 14:48:08.846478 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.846478    2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 30 14:48:08.846599 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.846492    2578 kubelet.go:398] "Adding apiserver pod source"
Jul 30 14:48:08.846599 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.846505    2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 30 14:48:08.846861 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.846847    2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 30 14:48:08.846920 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.846884    2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 30 14:48:08.849298 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.849281    2578 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 30 14:48:08.851557 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.851544    2578 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 30 14:48:08.851600 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.851574    2578 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 30 14:48:08.853605 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.853594    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 30 14:48:08.853651 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.853614    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 30 14:48:08.853651 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.853623    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 30 14:48:08.853651 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.853631    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 30 14:48:08.853651 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.853640    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 30 14:48:08.853651 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.853645    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 30 14:48:08.853651 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.853653    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 30 14:48:08.853804 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.853658    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 30 14:48:08.853804 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.853665    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 30 14:48:08.853804 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.853670    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 30 14:48:08.853804 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.853678    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 30 14:48:08.853804 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.853683    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 30 14:48:08.854760 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.854744    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 30 14:48:08.854810 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.854765    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 30 14:48:08.861059 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.861043    2578 server.go:1268] "Started kubelet"
Jul 30 14:48:08.861322 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.861273    2578 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 30 14:48:08.861410 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.861344    2578 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 30 14:48:08.861577 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.861551    2578 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 30 14:48:08.861853 ip-10-0-133-117 systemd[1]: Started Kubernetes Kubelet.
Jul 30 14:48:08.862866 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:08.862712    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 30 14:48:08.862866 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.862722    2578 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 30 14:48:08.862971 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:08.862889    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-117.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 30 14:48:08.864323 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.863455    2578 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 30 14:48:08.864323 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.863882    2578 server.go:310] "Adding debug handlers to kubelet server"
Jul 30 14:48:08.867649 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:08.867623    2578 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 30 14:48:08.869017 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:08.867951    2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-117.ec2.internal.18c718a86869b60e  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-117.ec2.internal,UID:ip-10-0-133-117.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-117.ec2.internal,},FirstTimestamp:2026-07-30 14:48:08.86102171 +0000 UTC m=+0.389960175,LastTimestamp:2026-07-30 14:48:08.86102171 +0000 UTC m=+0.389960175,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-117.ec2.internal,}"
Jul 30 14:48:08.872419 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.872400    2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-87tx6"
Jul 30 14:48:08.872806 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.872786    2578 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 30 14:48:08.872806 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.872795    2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 30 14:48:08.872957 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:08.872875    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-117.ec2.internal\" not found"
Jul 30 14:48:08.873510 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.873489    2578 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 30 14:48:08.873510 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.873507    2578 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 30 14:48:08.873646 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.873494    2578 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 30 14:48:08.873646 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.873612    2578 reconstruct.go:97] "Volume reconstruction finished"
Jul 30 14:48:08.873646 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.873621    2578 reconciler.go:29] "Reconciler: start to sync state"
Jul 30 14:48:08.873896 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.873881    2578 factory.go:153] Registering CRI-O factory
Jul 30 14:48:08.873968 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.873899    2578 factory.go:223] Registration of the crio container factory successfully
Jul 30 14:48:08.873968 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.873948    2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 30 14:48:08.873968 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.873957    2578 factory.go:55] Registering systemd factory
Jul 30 14:48:08.873968 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.873965    2578 factory.go:223] Registration of the systemd container factory successfully
Jul 30 14:48:08.874173 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.873988    2578 factory.go:103] Registering Raw factory
Jul 30 14:48:08.874173 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.874001    2578 manager.go:1196] Started watching for new ooms in manager
Jul 30 14:48:08.874370 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.874357    2578 manager.go:349] Starting recovery of all containers
Jul 30 14:48:08.881738 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.881713    2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-87tx6"
Jul 30 14:48:08.883232 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:08.883205    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 30 14:48:08.883232 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:08.883208    2578 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-117.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 30 14:48:08.883651 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.883635    2578 manager.go:354] Recovery completed
Jul 30 14:48:08.889305 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.889291    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 14:48:08.892197 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.892181    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-117.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 14:48:08.892314 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.892204    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-117.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 14:48:08.892314 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.892219    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-117.ec2.internal" event="NodeHasSufficientPID"
Jul 30 14:48:08.892797 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.892783    2578 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 30 14:48:08.892797 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.892792    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 14:48:08.892905 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.892796    2578 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 30 14:48:08.892953 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.892907    2578 state_mem.go:36] "Initialized new in-memory state store"
Jul 30 14:48:08.893709 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.893689    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-117.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 14:48:08.893788 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.893723    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-117.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 14:48:08.893788 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.893739    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-117.ec2.internal" event="NodeHasSufficientPID"
Jul 30 14:48:08.893788 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:08.893690    2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-117.ec2.internal.18c718a86a4563f1  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-117.ec2.internal,UID:ip-10-0-133-117.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-117.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-117.ec2.internal,},FirstTimestamp:2026-07-30 14:48:08.892195825 +0000 UTC m=+0.421134289,LastTimestamp:2026-07-30 14:48:08.892195825 +0000 UTC m=+0.421134289,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-117.ec2.internal,}"
Jul 30 14:48:08.895665 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.895646    2578 policy_none.go:49] "None policy: Start"
Jul 30 14:48:08.895665 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.895664    2578 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 30 14:48:08.895782 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.895677    2578 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 30 14:48:08.897777 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.897765    2578 policy_none.go:47] "Start"
Jul 30 14:48:08.920884 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.920754    2578 manager.go:337] "Starting Device Plugin manager"
Jul 30 14:48:08.947474 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:08.920909    2578 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 30 14:48:08.947474 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.920925    2578 server.go:87] "Starting device plugin registration server"
Jul 30 14:48:08.947474 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.921109    2578 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 30 14:48:08.947474 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.921117    2578 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 30 14:48:08.947474 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.921208    2578 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 30 14:48:08.947474 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.921291    2578 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 30 14:48:08.947474 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.921298    2578 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 30 14:48:08.947474 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:08.921777    2578 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 30 14:48:08.947474 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:08.921821    2578 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-117.ec2.internal\" not found"
Jul 30 14:48:08.970845 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.970822    2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 30 14:48:08.971931 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.971896    2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 30 14:48:08.972032 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.971943    2578 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 30 14:48:08.972032 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.971965    2578 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 30 14:48:08.972032 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:08.971997    2578 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 30 14:48:08.974798 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:08.974782    2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 14:48:09.021865 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:09.021829    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 14:48:09.022724 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:09.022708    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-117.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 14:48:09.022822 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:09.022740    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-117.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 14:48:09.022822 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:09.022753    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-117.ec2.internal" event="NodeHasSufficientPID"
Jul 30 14:48:09.022822 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:09.022786    2578 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-117.ec2.internal"
Jul 30 14:48:09.031644 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:09.031598    2578 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-117.ec2.internal"
Jul 30 14:48:09.031644 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:09.031618    2578 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-117.ec2.internal\": node \"ip-10-0-133-117.ec2.internal\" not found"
Jul 30 14:48:09.045295 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:09.045275    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-117.ec2.internal\" not found"
Jul 30 14:48:09.072503 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:09.072467    2578 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-117.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-117.ec2.internal"]
Jul 30 14:48:09.072554 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:09.072543    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 14:48:09.074012 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:09.073997    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-117.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 14:48:09.074078 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:09.074023    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-117.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 14:48:09.074078 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:09.074038    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-117.ec2.internal" event="NodeHasSufficientPID"
Jul 30 14:48:09.076323 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:09.076311    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 14:48:09.076456 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:09.076440    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-117.ec2.internal"
Jul 30 14:48:09.076543 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:09.076468    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 14:48:09.077280 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:09.077263    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-117.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 14:48:09.077280 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:09.077276    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-117.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 14:48:09.077369 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:09.077289    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-117.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 14:48:09.077369 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:09.077293    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-117.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 14:48:09.077369 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:09.077305    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-117.ec2.internal" event="NodeHasSufficientPID"
Jul 30 14:48:09.077369 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:09.077307    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-117.ec2.internal" event="NodeHasSufficientPID"
Jul 30 14:48:09.079524 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:09.079508    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-117.ec2.internal"
Jul 30 14:48:09.079572 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:09.079540    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 14:48:09.080609 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:09.080594    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-117.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 14:48:09.080680 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:09.080620    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-117.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 14:48:09.080680 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:09.080634    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-117.ec2.internal" event="NodeHasSufficientPID"
Jul 30 14:48:09.106743 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:09.106723    2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-117.ec2.internal\" not found" node="ip-10-0-133-117.ec2.internal"
Jul 30 14:48:09.111018 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:09.111002    2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-117.ec2.internal\" not found" node="ip-10-0-133-117.ec2.internal"
Jul 30 14:48:09.145559 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:09.145540    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-117.ec2.internal\" not found"
Jul 30 14:48:09.173890 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:09.173870    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d0f02a53bf99ae56c5e1962a8f62bc87-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-117.ec2.internal\" (UID: \"d0f02a53bf99ae56c5e1962a8f62bc87\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-117.ec2.internal"
Jul 30 14:48:09.173967 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:09.173893    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d0f02a53bf99ae56c5e1962a8f62bc87-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-117.ec2.internal\" (UID: \"d0f02a53bf99ae56c5e1962a8f62bc87\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-117.ec2.internal"
Jul 30 14:48:09.246360 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:09.246339    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-117.ec2.internal\" not found"
Jul 30 14:48:09.274644 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:09.274612    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d0f02a53bf99ae56c5e1962a8f62bc87-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-117.ec2.internal\" (UID: \"d0f02a53bf99ae56c5e1962a8f62bc87\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-117.ec2.internal"
Jul 30 14:48:09.274734 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:09.274653    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d0f02a53bf99ae56c5e1962a8f62bc87-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-117.ec2.internal\" (UID: \"d0f02a53bf99ae56c5e1962a8f62bc87\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-117.ec2.internal"
Jul 30 14:48:09.274734 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:09.274682    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a5efaf84ea37e1b2796825887b9db661-config\") pod \"kube-apiserver-proxy-ip-10-0-133-117.ec2.internal\" (UID: \"a5efaf84ea37e1b2796825887b9db661\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-117.ec2.internal"
Jul 30 14:48:09.274734 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:09.274709    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d0f02a53bf99ae56c5e1962a8f62bc87-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-117.ec2.internal\" (UID: \"d0f02a53bf99ae56c5e1962a8f62bc87\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-117.ec2.internal"
Jul 30 14:48:09.274847 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:09.274729    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d0f02a53bf99ae56c5e1962a8f62bc87-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-117.ec2.internal\" (UID: \"d0f02a53bf99ae56c5e1962a8f62bc87\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-117.ec2.internal"
Jul 30 14:48:09.346936 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:09.346886    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-117.ec2.internal\" not found"
Jul 30 14:48:09.375307 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:09.375287    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a5efaf84ea37e1b2796825887b9db661-config\") pod \"kube-apiserver-proxy-ip-10-0-133-117.ec2.internal\" (UID: \"a5efaf84ea37e1b2796825887b9db661\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-117.ec2.internal"
Jul 30 14:48:09.375367 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:09.375352    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a5efaf84ea37e1b2796825887b9db661-config\") pod \"kube-apiserver-proxy-ip-10-0-133-117.ec2.internal\" (UID: \"a5efaf84ea37e1b2796825887b9db661\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-117.ec2.internal"
Jul 30 14:48:09.410438 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:09.410416    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-117.ec2.internal"
Jul 30 14:48:09.415408 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:09.415289    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-117.ec2.internal"
Jul 30 14:48:09.447999 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:09.447969    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-117.ec2.internal\" not found"
Jul 30 14:48:09.548537 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:09.548510    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-117.ec2.internal\" not found"
Jul 30 14:48:09.649546 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:09.649522    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-117.ec2.internal\" not found"
Jul 30 14:48:09.750441 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:09.750411    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-117.ec2.internal\" not found"
Jul 30 14:48:09.808209 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:09.808183    2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 30 14:48:09.808648 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:09.808354    2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 30 14:48:09.850791 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:09.850768    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-117.ec2.internal\" not found"
Jul 30 14:48:09.874593 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:09.874535    2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 14:48:09.877536 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:09.877521    2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 14:48:09.889592 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:09.889565    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-29 14:43:09 +0000 UTC" deadline="2028-03-24 04:33:40.719370322 +0000 UTC"
Jul 30 14:48:09.889592 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:09.889588    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14461h45m30.829785418s"
Jul 30 14:48:09.951291 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:09.951266    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-117.ec2.internal\" not found"
Jul 30 14:48:10.052271 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:10.052231    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-117.ec2.internal\" not found"
Jul 30 14:48:10.117098 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:10.117057    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5efaf84ea37e1b2796825887b9db661.slice/crio-89052ff4af0606be740ea8d4656c1a238517635d3a7c0d5dcc239aade6e9319b WatchSource:0}: Error finding container 89052ff4af0606be740ea8d4656c1a238517635d3a7c0d5dcc239aade6e9319b: Status 404 returned error can't find the container with id 89052ff4af0606be740ea8d4656c1a238517635d3a7c0d5dcc239aade6e9319b
Jul 30 14:48:10.117595 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:10.117578    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0f02a53bf99ae56c5e1962a8f62bc87.slice/crio-e2ca9edf3b8fa5c4981455c7de24bf48d4aac4941abc983c6119d7fe2a5e620f WatchSource:0}: Error finding container e2ca9edf3b8fa5c4981455c7de24bf48d4aac4941abc983c6119d7fe2a5e620f: Status 404 returned error can't find the container with id e2ca9edf3b8fa5c4981455c7de24bf48d4aac4941abc983c6119d7fe2a5e620f
Jul 30 14:48:10.123007 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.122987    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 14:48:10.152545 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:10.152516    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-117.ec2.internal\" not found"
Jul 30 14:48:10.257919 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:10.257892    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-117.ec2.internal\" not found"
Jul 30 14:48:10.345698 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.345668    2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 14:48:10.373010 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.372981    2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-117.ec2.internal"
Jul 30 14:48:10.384541 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.384497    2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 30 14:48:10.385668 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.385654    2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-117.ec2.internal"
Jul 30 14:48:10.394214 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.394199    2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 30 14:48:10.849602 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.849496    2578 apiserver.go:52] "Watching apiserver"
Jul 30 14:48:10.862749 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.862721    2578 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 30 14:48:10.864532 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.864495    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-117.ec2.internal","openshift-multus/multus-additional-cni-plugins-dkdwk","openshift-multus/network-metrics-daemon-rj8gr","openshift-network-diagnostics/network-check-target-j6hld","openshift-network-operator/iptables-alerter-vx6bx","kube-system/global-pull-secret-syncer-9lgq9","kube-system/konnectivity-agent-t65nt","kube-system/kube-apiserver-proxy-ip-10-0-133-117.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hs9dl","openshift-dns/node-resolver-8jj48","openshift-image-registry/node-ca-9q72z","openshift-multus/multus-cp7nq","openshift-ovn-kubernetes/ovnkube-node-xllmv","openshift-cluster-node-tuning-operator/tuned-sgpbt"]
Jul 30 14:48:10.867482 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.867458    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-t65nt"
Jul 30 14:48:10.870693 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.869648    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 30 14:48:10.870693 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.869648    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-4cdzm\""
Jul 30 14:48:10.870693 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.870068    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 30 14:48:10.870693 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.870341    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.872076 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.872054    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 30 14:48:10.872393 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.872373    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 30 14:48:10.872688 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.872669    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 30 14:48:10.872765 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.872717    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-tl2xx\""
Jul 30 14:48:10.872826 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.872803    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 30 14:48:10.873087 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.873061    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rj8gr"
Jul 30 14:48:10.873170 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.873127    2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 30 14:48:10.873245 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:10.873143    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rj8gr" podUID="be2b73fa-10f0-4ea3-a68e-fa452e30914b"
Jul 30 14:48:10.873245 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.873180    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 30 14:48:10.873369 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.873064    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 30 14:48:10.875343 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.875322    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j6hld"
Jul 30 14:48:10.875467 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:10.875406    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j6hld" podUID="78565b9e-9521-4e9f-811e-6cc4cd76a4d3"
Jul 30 14:48:10.879001 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.878980    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vx6bx"
Jul 30 14:48:10.881975 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.881955    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-b7j5d\""
Jul 30 14:48:10.882210 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.882193    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 30 14:48:10.882438 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.882421    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 30 14:48:10.882625 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.882610    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 30 14:48:10.883061 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.883046    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lgq9"
Jul 30 14:48:10.883124 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:10.883109    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lgq9" podUID="963eac2b-b8af-497c-a206-0a1c976a0e2c"
Jul 30 14:48:10.885119 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.885098    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/89cc8684-f168-40cf-93ec-1143c2cb0784-run-ovn\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.885258 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.885150    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/89cc8684-f168-40cf-93ec-1143c2cb0784-node-log\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.885258 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.885178    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/89cc8684-f168-40cf-93ec-1143c2cb0784-host-cni-netd\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.885258 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.885219    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/89cc8684-f168-40cf-93ec-1143c2cb0784-env-overrides\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.885258 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.885246    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwlvw\" (UniqueName: \"kubernetes.io/projected/89cc8684-f168-40cf-93ec-1143c2cb0784-kube-api-access-cwlvw\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.885489 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.885273    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nn2gl\" (UniqueName: \"kubernetes.io/projected/be2b73fa-10f0-4ea3-a68e-fa452e30914b-kube-api-access-nn2gl\") pod \"network-metrics-daemon-rj8gr\" (UID: \"be2b73fa-10f0-4ea3-a68e-fa452e30914b\") " pod="openshift-multus/network-metrics-daemon-rj8gr"
Jul 30 14:48:10.885489 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.885297    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/89cc8684-f168-40cf-93ec-1143c2cb0784-host-kubelet\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.885489 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.885321    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/89cc8684-f168-40cf-93ec-1143c2cb0784-run-systemd\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.885489 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.885351    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/89cc8684-f168-40cf-93ec-1143c2cb0784-log-socket\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.885489 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.885393    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/89cc8684-f168-40cf-93ec-1143c2cb0784-host-cni-bin\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.885489 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.885420    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/89cc8684-f168-40cf-93ec-1143c2cb0784-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.885489 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.885447    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/89cc8684-f168-40cf-93ec-1143c2cb0784-ovn-node-metrics-cert\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.885489 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.885471    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be2b73fa-10f0-4ea3-a68e-fa452e30914b-metrics-certs\") pod \"network-metrics-daemon-rj8gr\" (UID: \"be2b73fa-10f0-4ea3-a68e-fa452e30914b\") " pod="openshift-multus/network-metrics-daemon-rj8gr"
Jul 30 14:48:10.885895 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.885498    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b5b201dc-6dcd-44c8-9c70-9946315da836-konnectivity-ca\") pod \"konnectivity-agent-t65nt\" (UID: \"b5b201dc-6dcd-44c8-9c70-9946315da836\") " pod="kube-system/konnectivity-agent-t65nt"
Jul 30 14:48:10.885895 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.885521    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/89cc8684-f168-40cf-93ec-1143c2cb0784-systemd-units\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.885895 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.885544    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/89cc8684-f168-40cf-93ec-1143c2cb0784-host-run-netns\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.885895 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.885570    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/89cc8684-f168-40cf-93ec-1143c2cb0784-var-lib-openvswitch\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.885895 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.885595    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/89cc8684-f168-40cf-93ec-1143c2cb0784-etc-openvswitch\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.885895 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.885618    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/89cc8684-f168-40cf-93ec-1143c2cb0784-run-openvswitch\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.885895 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.885645    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/89cc8684-f168-40cf-93ec-1143c2cb0784-host-run-ovn-kubernetes\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.885895 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.885674    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b5b201dc-6dcd-44c8-9c70-9946315da836-agent-certs\") pod \"konnectivity-agent-t65nt\" (UID: \"b5b201dc-6dcd-44c8-9c70-9946315da836\") " pod="kube-system/konnectivity-agent-t65nt"
Jul 30 14:48:10.885895 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.885699    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/89cc8684-f168-40cf-93ec-1143c2cb0784-host-slash\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.885895 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.885735    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/89cc8684-f168-40cf-93ec-1143c2cb0784-ovnkube-config\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.885895 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.885772    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/89cc8684-f168-40cf-93ec-1143c2cb0784-ovnkube-script-lib\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.885895 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.885808    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rbmt\" (UniqueName: \"kubernetes.io/projected/78565b9e-9521-4e9f-811e-6cc4cd76a4d3-kube-api-access-9rbmt\") pod \"network-check-target-j6hld\" (UID: \"78565b9e-9521-4e9f-811e-6cc4cd76a4d3\") " pod="openshift-network-diagnostics/network-check-target-j6hld"
Jul 30 14:48:10.886722 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.886699    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hs9dl"
Jul 30 14:48:10.888572 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.888551    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-sgpbt"
Jul 30 14:48:10.889466 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.889446    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 30 14:48:10.889597 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.889505    2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 30 14:48:10.889664 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.889450    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 30 14:48:10.890028 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.890010    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 30 14:48:10.890469 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.890446    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 30 14:48:10.890663 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.890646    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 30 14:48:10.890924 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.890909    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-zdrxt\""
Jul 30 14:48:10.891081 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.891066    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-jvgs6\""
Jul 30 14:48:10.892983 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.892964    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dkdwk"
Jul 30 14:48:10.894796 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.894776    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 30 14:48:10.894963 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.894945    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 30 14:48:10.895020 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.894979    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 30 14:48:10.895068 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.895036    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 30 14:48:10.896041 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.896017    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-hpwxn\""
Jul 30 14:48:10.896541 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.896519    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 30 14:48:10.897884 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.897864    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8jj48"
Jul 30 14:48:10.899991 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.899966    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-lvf8r\""
Jul 30 14:48:10.901102 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.901081    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 30 14:48:10.901199 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.901138    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 30 14:48:10.902195 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.902174    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9q72z"
Jul 30 14:48:10.903927 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.903889    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 30 14:48:10.904265 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.904248    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 30 14:48:10.904359 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.904326    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-p97th\""
Jul 30 14:48:10.904474 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.904456    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 30 14:48:10.905039 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.905021    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cp7nq"
Jul 30 14:48:10.906614 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.906594    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 30 14:48:10.906882 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.906862    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-nggbp\""
Jul 30 14:48:10.912411 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.912387    2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-fr529"
Jul 30 14:48:10.918867 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.918846    2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 14:48:10.921048 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.921006    2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-fr529"
Jul 30 14:48:10.974986 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.974955    2578 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 30 14:48:10.976900 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.976845    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-117.ec2.internal" event={"ID":"a5efaf84ea37e1b2796825887b9db661","Type":"ContainerStarted","Data":"89052ff4af0606be740ea8d4656c1a238517635d3a7c0d5dcc239aade6e9319b"}
Jul 30 14:48:10.978320 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.978293    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-117.ec2.internal" event={"ID":"d0f02a53bf99ae56c5e1962a8f62bc87","Type":"ContainerStarted","Data":"e2ca9edf3b8fa5c4981455c7de24bf48d4aac4941abc983c6119d7fe2a5e620f"}
Jul 30 14:48:10.986630 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.986603    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rbmt\" (UniqueName: \"kubernetes.io/projected/78565b9e-9521-4e9f-811e-6cc4cd76a4d3-kube-api-access-9rbmt\") pod \"network-check-target-j6hld\" (UID: \"78565b9e-9521-4e9f-811e-6cc4cd76a4d3\") " pod="openshift-network-diagnostics/network-check-target-j6hld"
Jul 30 14:48:10.986759 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.986675    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/061ec575-e1af-4d7f-b40e-57258eaecf71-run\") pod \"tuned-sgpbt\" (UID: \"061ec575-e1af-4d7f-b40e-57258eaecf71\") " pod="openshift-cluster-node-tuning-operator/tuned-sgpbt"
Jul 30 14:48:10.986759 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.986726    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/061ec575-e1af-4d7f-b40e-57258eaecf71-host\") pod \"tuned-sgpbt\" (UID: \"061ec575-e1af-4d7f-b40e-57258eaecf71\") " pod="openshift-cluster-node-tuning-operator/tuned-sgpbt"
Jul 30 14:48:10.986759 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.986754    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/963eac2b-b8af-497c-a206-0a1c976a0e2c-dbus\") pod \"global-pull-secret-syncer-9lgq9\" (UID: \"963eac2b-b8af-497c-a206-0a1c976a0e2c\") " pod="kube-system/global-pull-secret-syncer-9lgq9"
Jul 30 14:48:10.986980 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.986782    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/89cc8684-f168-40cf-93ec-1143c2cb0784-host-slash\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.986980 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.986806    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/061ec575-e1af-4d7f-b40e-57258eaecf71-etc-modprobe-d\") pod \"tuned-sgpbt\" (UID: \"061ec575-e1af-4d7f-b40e-57258eaecf71\") " pod="openshift-cluster-node-tuning-operator/tuned-sgpbt"
Jul 30 14:48:10.986980 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.986831    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/061ec575-e1af-4d7f-b40e-57258eaecf71-etc-tuned\") pod \"tuned-sgpbt\" (UID: \"061ec575-e1af-4d7f-b40e-57258eaecf71\") " pod="openshift-cluster-node-tuning-operator/tuned-sgpbt"
Jul 30 14:48:10.986980 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.986856    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b75a5fc6-d32b-4d96-a669-ff83a1abd706-system-cni-dir\") pod \"multus-additional-cni-plugins-dkdwk\" (UID: \"b75a5fc6-d32b-4d96-a669-ff83a1abd706\") " pod="openshift-multus/multus-additional-cni-plugins-dkdwk"
Jul 30 14:48:10.986980 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.986883    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b75a5fc6-d32b-4d96-a669-ff83a1abd706-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dkdwk\" (UID: \"b75a5fc6-d32b-4d96-a669-ff83a1abd706\") " pod="openshift-multus/multus-additional-cni-plugins-dkdwk"
Jul 30 14:48:10.986980 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.986908    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vssdg\" (UniqueName: \"kubernetes.io/projected/d9b6c536-3a43-49a8-853d-f9532668f149-kube-api-access-vssdg\") pod \"node-ca-9q72z\" (UID: \"d9b6c536-3a43-49a8-853d-f9532668f149\") " pod="openshift-image-registry/node-ca-9q72z"
Jul 30 14:48:10.986980 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.986936    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/89cc8684-f168-40cf-93ec-1143c2cb0784-env-overrides\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.986980 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.986961    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/05f0cae0-8999-4a51-ae71-9e9dad066804-registration-dir\") pod \"aws-ebs-csi-driver-node-hs9dl\" (UID: \"05f0cae0-8999-4a51-ae71-9e9dad066804\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hs9dl"
Jul 30 14:48:10.987394 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.986989    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/061ec575-e1af-4d7f-b40e-57258eaecf71-var-lib-kubelet\") pod \"tuned-sgpbt\" (UID: \"061ec575-e1af-4d7f-b40e-57258eaecf71\") " pod="openshift-cluster-node-tuning-operator/tuned-sgpbt"
Jul 30 14:48:10.987394 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.987044    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/89cc8684-f168-40cf-93ec-1143c2cb0784-ovn-node-metrics-cert\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.987394 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.987071    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d9b6c536-3a43-49a8-853d-f9532668f149-host\") pod \"node-ca-9q72z\" (UID: \"d9b6c536-3a43-49a8-853d-f9532668f149\") " pod="openshift-image-registry/node-ca-9q72z"
Jul 30 14:48:10.987394 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.987099    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/541aeca7-768f-41c8-8c0f-70663db887b2-host-var-lib-cni-bin\") pod \"multus-cp7nq\" (UID: \"541aeca7-768f-41c8-8c0f-70663db887b2\") " pod="openshift-multus/multus-cp7nq"
Jul 30 14:48:10.987394 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.987161    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/89cc8684-f168-40cf-93ec-1143c2cb0784-run-openvswitch\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.987394 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.987188    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/061ec575-e1af-4d7f-b40e-57258eaecf71-etc-sysconfig\") pod \"tuned-sgpbt\" (UID: \"061ec575-e1af-4d7f-b40e-57258eaecf71\") " pod="openshift-cluster-node-tuning-operator/tuned-sgpbt"
Jul 30 14:48:10.987394 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.987243    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d9b6c536-3a43-49a8-853d-f9532668f149-serviceca\") pod \"node-ca-9q72z\" (UID: \"d9b6c536-3a43-49a8-853d-f9532668f149\") " pod="openshift-image-registry/node-ca-9q72z"
Jul 30 14:48:10.987394 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.987278    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/541aeca7-768f-41c8-8c0f-70663db887b2-cni-binary-copy\") pod \"multus-cp7nq\" (UID: \"541aeca7-768f-41c8-8c0f-70663db887b2\") " pod="openshift-multus/multus-cp7nq"
Jul 30 14:48:10.987394 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.987305    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b5b201dc-6dcd-44c8-9c70-9946315da836-agent-certs\") pod \"konnectivity-agent-t65nt\" (UID: \"b5b201dc-6dcd-44c8-9c70-9946315da836\") " pod="kube-system/konnectivity-agent-t65nt"
Jul 30 14:48:10.987394 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.987330    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b9def45f-210b-4ceb-989c-6c696881a6f9-host-slash\") pod \"iptables-alerter-vx6bx\" (UID: \"b9def45f-210b-4ceb-989c-6c696881a6f9\") " pod="openshift-network-operator/iptables-alerter-vx6bx"
Jul 30 14:48:10.987394 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.987357    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/05f0cae0-8999-4a51-ae71-9e9dad066804-etc-selinux\") pod \"aws-ebs-csi-driver-node-hs9dl\" (UID: \"05f0cae0-8999-4a51-ae71-9e9dad066804\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hs9dl"
Jul 30 14:48:10.987871 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.987444    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/963eac2b-b8af-497c-a206-0a1c976a0e2c-kubelet-config\") pod \"global-pull-secret-syncer-9lgq9\" (UID: \"963eac2b-b8af-497c-a206-0a1c976a0e2c\") " pod="kube-system/global-pull-secret-syncer-9lgq9"
Jul 30 14:48:10.987871 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.987473    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b75a5fc6-d32b-4d96-a669-ff83a1abd706-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dkdwk\" (UID: \"b75a5fc6-d32b-4d96-a669-ff83a1abd706\") " pod="openshift-multus/multus-additional-cni-plugins-dkdwk"
Jul 30 14:48:10.987871 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.987502    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fc042c9a-bc2a-4cf4-8606-e9e20e462cb3-hosts-file\") pod \"node-resolver-8jj48\" (UID: \"fc042c9a-bc2a-4cf4-8606-e9e20e462cb3\") " pod="openshift-dns/node-resolver-8jj48"
Jul 30 14:48:10.987871 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.987528    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/541aeca7-768f-41c8-8c0f-70663db887b2-etc-kubernetes\") pod \"multus-cp7nq\" (UID: \"541aeca7-768f-41c8-8c0f-70663db887b2\") " pod="openshift-multus/multus-cp7nq"
Jul 30 14:48:10.987871 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.987556    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/89cc8684-f168-40cf-93ec-1143c2cb0784-host-cni-netd\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.987871 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.987582    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwlvw\" (UniqueName: \"kubernetes.io/projected/89cc8684-f168-40cf-93ec-1143c2cb0784-kube-api-access-cwlvw\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.987871 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.987574    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/89cc8684-f168-40cf-93ec-1143c2cb0784-env-overrides\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.987871 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.987624    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc042c9a-bc2a-4cf4-8606-e9e20e462cb3-tmp-dir\") pod \"node-resolver-8jj48\" (UID: \"fc042c9a-bc2a-4cf4-8606-e9e20e462cb3\") " pod="openshift-dns/node-resolver-8jj48"
Jul 30 14:48:10.987871 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.987650    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/541aeca7-768f-41c8-8c0f-70663db887b2-multus-socket-dir-parent\") pod \"multus-cp7nq\" (UID: \"541aeca7-768f-41c8-8c0f-70663db887b2\") " pod="openshift-multus/multus-cp7nq"
Jul 30 14:48:10.987871 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.987659    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/89cc8684-f168-40cf-93ec-1143c2cb0784-host-slash\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.987871 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.987679    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/89cc8684-f168-40cf-93ec-1143c2cb0784-host-kubelet\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.987871 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.987762    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/89cc8684-f168-40cf-93ec-1143c2cb0784-host-cni-netd\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.988438 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.988000    2578 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 30 14:48:10.988438 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.988044    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/89cc8684-f168-40cf-93ec-1143c2cb0784-run-openvswitch\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.988438 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.988040    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/89cc8684-f168-40cf-93ec-1143c2cb0784-host-kubelet\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.988438 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.988079    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/061ec575-e1af-4d7f-b40e-57258eaecf71-etc-sysctl-d\") pod \"tuned-sgpbt\" (UID: \"061ec575-e1af-4d7f-b40e-57258eaecf71\") " pod="openshift-cluster-node-tuning-operator/tuned-sgpbt"
Jul 30 14:48:10.988438 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.988129    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/541aeca7-768f-41c8-8c0f-70663db887b2-multus-cni-dir\") pod \"multus-cp7nq\" (UID: \"541aeca7-768f-41c8-8c0f-70663db887b2\") " pod="openshift-multus/multus-cp7nq"
Jul 30 14:48:10.988438 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.988184    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/541aeca7-768f-41c8-8c0f-70663db887b2-cnibin\") pod \"multus-cp7nq\" (UID: \"541aeca7-768f-41c8-8c0f-70663db887b2\") " pod="openshift-multus/multus-cp7nq"
Jul 30 14:48:10.988438 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.988219    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/541aeca7-768f-41c8-8c0f-70663db887b2-multus-conf-dir\") pod \"multus-cp7nq\" (UID: \"541aeca7-768f-41c8-8c0f-70663db887b2\") " pod="openshift-multus/multus-cp7nq"
Jul 30 14:48:10.988438 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.988251    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/541aeca7-768f-41c8-8c0f-70663db887b2-host-run-multus-certs\") pod \"multus-cp7nq\" (UID: \"541aeca7-768f-41c8-8c0f-70663db887b2\") " pod="openshift-multus/multus-cp7nq"
Jul 30 14:48:10.988438 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.988311    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/89cc8684-f168-40cf-93ec-1143c2cb0784-systemd-units\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.988438 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.988358    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/89cc8684-f168-40cf-93ec-1143c2cb0784-etc-openvswitch\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.988438 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.988386    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/89cc8684-f168-40cf-93ec-1143c2cb0784-systemd-units\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.988438 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.988431    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/89cc8684-f168-40cf-93ec-1143c2cb0784-host-run-ovn-kubernetes\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.989042 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.988481    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/89cc8684-f168-40cf-93ec-1143c2cb0784-etc-openvswitch\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.989042 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.988534    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/061ec575-e1af-4d7f-b40e-57258eaecf71-etc-sysctl-conf\") pod \"tuned-sgpbt\" (UID: \"061ec575-e1af-4d7f-b40e-57258eaecf71\") " pod="openshift-cluster-node-tuning-operator/tuned-sgpbt"
Jul 30 14:48:10.989042 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.988565    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/061ec575-e1af-4d7f-b40e-57258eaecf71-tmp\") pod \"tuned-sgpbt\" (UID: \"061ec575-e1af-4d7f-b40e-57258eaecf71\") " pod="openshift-cluster-node-tuning-operator/tuned-sgpbt"
Jul 30 14:48:10.989042 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.988610    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/89cc8684-f168-40cf-93ec-1143c2cb0784-host-run-ovn-kubernetes\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.989042 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.988639    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/541aeca7-768f-41c8-8c0f-70663db887b2-system-cni-dir\") pod \"multus-cp7nq\" (UID: \"541aeca7-768f-41c8-8c0f-70663db887b2\") " pod="openshift-multus/multus-cp7nq"
Jul 30 14:48:10.989042 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.988665    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/541aeca7-768f-41c8-8c0f-70663db887b2-host-var-lib-kubelet\") pod \"multus-cp7nq\" (UID: \"541aeca7-768f-41c8-8c0f-70663db887b2\") " pod="openshift-multus/multus-cp7nq"
Jul 30 14:48:10.989042 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.988704    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kk4qs\" (UniqueName: \"kubernetes.io/projected/b9def45f-210b-4ceb-989c-6c696881a6f9-kube-api-access-kk4qs\") pod \"iptables-alerter-vx6bx\" (UID: \"b9def45f-210b-4ceb-989c-6c696881a6f9\") " pod="openshift-network-operator/iptables-alerter-vx6bx"
Jul 30 14:48:10.989042 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.988779    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/963eac2b-b8af-497c-a206-0a1c976a0e2c-original-pull-secret\") pod \"global-pull-secret-syncer-9lgq9\" (UID: \"963eac2b-b8af-497c-a206-0a1c976a0e2c\") " pod="kube-system/global-pull-secret-syncer-9lgq9"
Jul 30 14:48:10.989042 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.988814    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b75a5fc6-d32b-4d96-a669-ff83a1abd706-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dkdwk\" (UID: \"b75a5fc6-d32b-4d96-a669-ff83a1abd706\") " pod="openshift-multus/multus-additional-cni-plugins-dkdwk"
Jul 30 14:48:10.989042 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.988840    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dl4t2\" (UniqueName: \"kubernetes.io/projected/fc042c9a-bc2a-4cf4-8606-e9e20e462cb3-kube-api-access-dl4t2\") pod \"node-resolver-8jj48\" (UID: \"fc042c9a-bc2a-4cf4-8606-e9e20e462cb3\") " pod="openshift-dns/node-resolver-8jj48"
Jul 30 14:48:10.989042 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.988865    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/541aeca7-768f-41c8-8c0f-70663db887b2-host-var-lib-cni-multus\") pod \"multus-cp7nq\" (UID: \"541aeca7-768f-41c8-8c0f-70663db887b2\") " pod="openshift-multus/multus-cp7nq"
Jul 30 14:48:10.989042 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.988907    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/89cc8684-f168-40cf-93ec-1143c2cb0784-run-ovn\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.989042 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.988950    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/89cc8684-f168-40cf-93ec-1143c2cb0784-node-log\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.989042 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.988981    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/061ec575-e1af-4d7f-b40e-57258eaecf71-etc-kubernetes\") pod \"tuned-sgpbt\" (UID: \"061ec575-e1af-4d7f-b40e-57258eaecf71\") " pod="openshift-cluster-node-tuning-operator/tuned-sgpbt"
Jul 30 14:48:10.989042 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.988994    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/89cc8684-f168-40cf-93ec-1143c2cb0784-run-ovn\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.989042 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.989009    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b75a5fc6-d32b-4d96-a669-ff83a1abd706-cnibin\") pod \"multus-additional-cni-plugins-dkdwk\" (UID: \"b75a5fc6-d32b-4d96-a669-ff83a1abd706\") " pod="openshift-multus/multus-additional-cni-plugins-dkdwk"
Jul 30 14:48:10.989042 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.989037    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/89cc8684-f168-40cf-93ec-1143c2cb0784-node-log\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.989898 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.989034    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/541aeca7-768f-41c8-8c0f-70663db887b2-host-run-k8s-cni-cncf-io\") pod \"multus-cp7nq\" (UID: \"541aeca7-768f-41c8-8c0f-70663db887b2\") " pod="openshift-multus/multus-cp7nq"
Jul 30 14:48:10.989898 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.989074    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/89cc8684-f168-40cf-93ec-1143c2cb0784-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.989898 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.989111    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/89cc8684-f168-40cf-93ec-1143c2cb0784-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.989898 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.989143    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be2b73fa-10f0-4ea3-a68e-fa452e30914b-metrics-certs\") pod \"network-metrics-daemon-rj8gr\" (UID: \"be2b73fa-10f0-4ea3-a68e-fa452e30914b\") " pod="openshift-multus/network-metrics-daemon-rj8gr"
Jul 30 14:48:10.989898 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.989221    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/05f0cae0-8999-4a51-ae71-9e9dad066804-device-dir\") pod \"aws-ebs-csi-driver-node-hs9dl\" (UID: \"05f0cae0-8999-4a51-ae71-9e9dad066804\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hs9dl"
Jul 30 14:48:10.989898 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.989239    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/061ec575-e1af-4d7f-b40e-57258eaecf71-etc-systemd\") pod \"tuned-sgpbt\" (UID: \"061ec575-e1af-4d7f-b40e-57258eaecf71\") " pod="openshift-cluster-node-tuning-operator/tuned-sgpbt"
Jul 30 14:48:10.989898 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:10.989336    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 14:48:10.989898 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:10.989418    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be2b73fa-10f0-4ea3-a68e-fa452e30914b-metrics-certs podName:be2b73fa-10f0-4ea3-a68e-fa452e30914b nodeName:}" failed. No retries permitted until 2026-07-30 14:48:11.489394807 +0000 UTC m=+3.018333279 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/be2b73fa-10f0-4ea3-a68e-fa452e30914b-metrics-certs") pod "network-metrics-daemon-rj8gr" (UID: "be2b73fa-10f0-4ea3-a68e-fa452e30914b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 14:48:10.989898 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.989439    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/061ec575-e1af-4d7f-b40e-57258eaecf71-lib-modules\") pod \"tuned-sgpbt\" (UID: \"061ec575-e1af-4d7f-b40e-57258eaecf71\") " pod="openshift-cluster-node-tuning-operator/tuned-sgpbt"
Jul 30 14:48:10.989898 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.989466    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b75a5fc6-d32b-4d96-a669-ff83a1abd706-os-release\") pod \"multus-additional-cni-plugins-dkdwk\" (UID: \"b75a5fc6-d32b-4d96-a669-ff83a1abd706\") " pod="openshift-multus/multus-additional-cni-plugins-dkdwk"
Jul 30 14:48:10.989898 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.989494    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/541aeca7-768f-41c8-8c0f-70663db887b2-host-run-netns\") pod \"multus-cp7nq\" (UID: \"541aeca7-768f-41c8-8c0f-70663db887b2\") " pod="openshift-multus/multus-cp7nq"
Jul 30 14:48:10.989898 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.989521    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jd5d\" (UniqueName: \"kubernetes.io/projected/541aeca7-768f-41c8-8c0f-70663db887b2-kube-api-access-8jd5d\") pod \"multus-cp7nq\" (UID: \"541aeca7-768f-41c8-8c0f-70663db887b2\") " pod="openshift-multus/multus-cp7nq"
Jul 30 14:48:10.989898 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.989548    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/89cc8684-f168-40cf-93ec-1143c2cb0784-host-run-netns\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.989898 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.989569    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/89cc8684-f168-40cf-93ec-1143c2cb0784-ovnkube-script-lib\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.989898 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.989591    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/05f0cae0-8999-4a51-ae71-9e9dad066804-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hs9dl\" (UID: \"05f0cae0-8999-4a51-ae71-9e9dad066804\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hs9dl"
Jul 30 14:48:10.989898 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.989612    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/05f0cae0-8999-4a51-ae71-9e9dad066804-sys-fs\") pod \"aws-ebs-csi-driver-node-hs9dl\" (UID: \"05f0cae0-8999-4a51-ae71-9e9dad066804\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hs9dl"
Jul 30 14:48:10.990612 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.989632    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jk2hg\" (UniqueName: \"kubernetes.io/projected/05f0cae0-8999-4a51-ae71-9e9dad066804-kube-api-access-jk2hg\") pod \"aws-ebs-csi-driver-node-hs9dl\" (UID: \"05f0cae0-8999-4a51-ae71-9e9dad066804\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hs9dl"
Jul 30 14:48:10.990612 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.989655    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b75a5fc6-d32b-4d96-a669-ff83a1abd706-cni-binary-copy\") pod \"multus-additional-cni-plugins-dkdwk\" (UID: \"b75a5fc6-d32b-4d96-a669-ff83a1abd706\") " pod="openshift-multus/multus-additional-cni-plugins-dkdwk"
Jul 30 14:48:10.990612 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.989687    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/541aeca7-768f-41c8-8c0f-70663db887b2-os-release\") pod \"multus-cp7nq\" (UID: \"541aeca7-768f-41c8-8c0f-70663db887b2\") " pod="openshift-multus/multus-cp7nq"
Jul 30 14:48:10.990612 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.989711    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/541aeca7-768f-41c8-8c0f-70663db887b2-hostroot\") pod \"multus-cp7nq\" (UID: \"541aeca7-768f-41c8-8c0f-70663db887b2\") " pod="openshift-multus/multus-cp7nq"
Jul 30 14:48:10.990612 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.989735    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/89cc8684-f168-40cf-93ec-1143c2cb0784-ovnkube-config\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.990612 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.989763    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/061ec575-e1af-4d7f-b40e-57258eaecf71-sys\") pod \"tuned-sgpbt\" (UID: \"061ec575-e1af-4d7f-b40e-57258eaecf71\") " pod="openshift-cluster-node-tuning-operator/tuned-sgpbt"
Jul 30 14:48:10.990612 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.989820    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzqzp\" (UniqueName: \"kubernetes.io/projected/061ec575-e1af-4d7f-b40e-57258eaecf71-kube-api-access-bzqzp\") pod \"tuned-sgpbt\" (UID: \"061ec575-e1af-4d7f-b40e-57258eaecf71\") " pod="openshift-cluster-node-tuning-operator/tuned-sgpbt"
Jul 30 14:48:10.990612 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.989872    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xn56w\" (UniqueName: \"kubernetes.io/projected/b75a5fc6-d32b-4d96-a669-ff83a1abd706-kube-api-access-xn56w\") pod \"multus-additional-cni-plugins-dkdwk\" (UID: \"b75a5fc6-d32b-4d96-a669-ff83a1abd706\") " pod="openshift-multus/multus-additional-cni-plugins-dkdwk"
Jul 30 14:48:10.990612 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.989900    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/541aeca7-768f-41c8-8c0f-70663db887b2-multus-daemon-config\") pod \"multus-cp7nq\" (UID: \"541aeca7-768f-41c8-8c0f-70663db887b2\") " pod="openshift-multus/multus-cp7nq"
Jul 30 14:48:10.990612 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.989933    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nn2gl\" (UniqueName: \"kubernetes.io/projected/be2b73fa-10f0-4ea3-a68e-fa452e30914b-kube-api-access-nn2gl\") pod \"network-metrics-daemon-rj8gr\" (UID: \"be2b73fa-10f0-4ea3-a68e-fa452e30914b\") " pod="openshift-multus/network-metrics-daemon-rj8gr"
Jul 30 14:48:10.990612 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.989960    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b9def45f-210b-4ceb-989c-6c696881a6f9-iptables-alerter-script\") pod \"iptables-alerter-vx6bx\" (UID: \"b9def45f-210b-4ceb-989c-6c696881a6f9\") " pod="openshift-network-operator/iptables-alerter-vx6bx"
Jul 30 14:48:10.990612 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.989988    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/05f0cae0-8999-4a51-ae71-9e9dad066804-socket-dir\") pod \"aws-ebs-csi-driver-node-hs9dl\" (UID: \"05f0cae0-8999-4a51-ae71-9e9dad066804\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hs9dl"
Jul 30 14:48:10.990612 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.990074    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/89cc8684-f168-40cf-93ec-1143c2cb0784-run-systemd\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.990612 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.990114    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/89cc8684-f168-40cf-93ec-1143c2cb0784-log-socket\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.990612 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.990142    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/89cc8684-f168-40cf-93ec-1143c2cb0784-host-cni-bin\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.990612 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.990170    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b5b201dc-6dcd-44c8-9c70-9946315da836-konnectivity-ca\") pod \"konnectivity-agent-t65nt\" (UID: \"b5b201dc-6dcd-44c8-9c70-9946315da836\") " pod="kube-system/konnectivity-agent-t65nt"
Jul 30 14:48:10.991359 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.990194    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/89cc8684-f168-40cf-93ec-1143c2cb0784-var-lib-openvswitch\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.991359 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.990242    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/89cc8684-f168-40cf-93ec-1143c2cb0784-ovnkube-script-lib\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.991359 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.990309    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/89cc8684-f168-40cf-93ec-1143c2cb0784-ovnkube-config\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.991359 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.990319    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/89cc8684-f168-40cf-93ec-1143c2cb0784-host-run-netns\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.991359 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.990366    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/89cc8684-f168-40cf-93ec-1143c2cb0784-var-lib-openvswitch\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.991359 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.990369    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/89cc8684-f168-40cf-93ec-1143c2cb0784-log-socket\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.991359 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.990416    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/89cc8684-f168-40cf-93ec-1143c2cb0784-run-systemd\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.991359 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.990470    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/89cc8684-f168-40cf-93ec-1143c2cb0784-host-cni-bin\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.991359 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.991028    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b5b201dc-6dcd-44c8-9c70-9946315da836-konnectivity-ca\") pod \"konnectivity-agent-t65nt\" (UID: \"b5b201dc-6dcd-44c8-9c70-9946315da836\") " pod="kube-system/konnectivity-agent-t65nt"
Jul 30 14:48:10.992447 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.992386    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/89cc8684-f168-40cf-93ec-1143c2cb0784-ovn-node-metrics-cert\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.992563 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.992548    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b5b201dc-6dcd-44c8-9c70-9946315da836-agent-certs\") pod \"konnectivity-agent-t65nt\" (UID: \"b5b201dc-6dcd-44c8-9c70-9946315da836\") " pod="kube-system/konnectivity-agent-t65nt"
Jul 30 14:48:10.993725 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:10.993704    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 14:48:10.993829 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:10.993731    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 14:48:10.993829 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:10.993745    2578 projected.go:196] Error preparing data for projected volume kube-api-access-9rbmt for pod openshift-network-diagnostics/network-check-target-j6hld: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 14:48:10.993829 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:10.993809    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/78565b9e-9521-4e9f-811e-6cc4cd76a4d3-kube-api-access-9rbmt podName:78565b9e-9521-4e9f-811e-6cc4cd76a4d3 nodeName:}" failed. No retries permitted until 2026-07-30 14:48:11.493790628 +0000 UTC m=+3.022729106 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-9rbmt" (UniqueName: "kubernetes.io/projected/78565b9e-9521-4e9f-811e-6cc4cd76a4d3-kube-api-access-9rbmt") pod "network-check-target-j6hld" (UID: "78565b9e-9521-4e9f-811e-6cc4cd76a4d3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 14:48:10.997068 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.997048    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwlvw\" (UniqueName: \"kubernetes.io/projected/89cc8684-f168-40cf-93ec-1143c2cb0784-kube-api-access-cwlvw\") pod \"ovnkube-node-xllmv\" (UID: \"89cc8684-f168-40cf-93ec-1143c2cb0784\") " pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:10.999461 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:10.999412    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nn2gl\" (UniqueName: \"kubernetes.io/projected/be2b73fa-10f0-4ea3-a68e-fa452e30914b-kube-api-access-nn2gl\") pod \"network-metrics-daemon-rj8gr\" (UID: \"be2b73fa-10f0-4ea3-a68e-fa452e30914b\") " pod="openshift-multus/network-metrics-daemon-rj8gr"
Jul 30 14:48:11.091488 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.091459    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/061ec575-e1af-4d7f-b40e-57258eaecf71-run\") pod \"tuned-sgpbt\" (UID: \"061ec575-e1af-4d7f-b40e-57258eaecf71\") " pod="openshift-cluster-node-tuning-operator/tuned-sgpbt"
Jul 30 14:48:11.091488 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.091490    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/061ec575-e1af-4d7f-b40e-57258eaecf71-host\") pod \"tuned-sgpbt\" (UID: \"061ec575-e1af-4d7f-b40e-57258eaecf71\") " pod="openshift-cluster-node-tuning-operator/tuned-sgpbt"
Jul 30 14:48:11.091736 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.091506    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/963eac2b-b8af-497c-a206-0a1c976a0e2c-dbus\") pod \"global-pull-secret-syncer-9lgq9\" (UID: \"963eac2b-b8af-497c-a206-0a1c976a0e2c\") " pod="kube-system/global-pull-secret-syncer-9lgq9"
Jul 30 14:48:11.091736 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.091530    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/061ec575-e1af-4d7f-b40e-57258eaecf71-etc-modprobe-d\") pod \"tuned-sgpbt\" (UID: \"061ec575-e1af-4d7f-b40e-57258eaecf71\") " pod="openshift-cluster-node-tuning-operator/tuned-sgpbt"
Jul 30 14:48:11.091736 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.091560    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/061ec575-e1af-4d7f-b40e-57258eaecf71-etc-tuned\") pod \"tuned-sgpbt\" (UID: \"061ec575-e1af-4d7f-b40e-57258eaecf71\") " pod="openshift-cluster-node-tuning-operator/tuned-sgpbt"
Jul 30 14:48:11.091736 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.091586    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b75a5fc6-d32b-4d96-a669-ff83a1abd706-system-cni-dir\") pod \"multus-additional-cni-plugins-dkdwk\" (UID: \"b75a5fc6-d32b-4d96-a669-ff83a1abd706\") " pod="openshift-multus/multus-additional-cni-plugins-dkdwk"
Jul 30 14:48:11.091736 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.091612    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b75a5fc6-d32b-4d96-a669-ff83a1abd706-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dkdwk\" (UID: \"b75a5fc6-d32b-4d96-a669-ff83a1abd706\") " pod="openshift-multus/multus-additional-cni-plugins-dkdwk"
Jul 30 14:48:11.091736 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.091621    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/061ec575-e1af-4d7f-b40e-57258eaecf71-run\") pod \"tuned-sgpbt\" (UID: \"061ec575-e1af-4d7f-b40e-57258eaecf71\") " pod="openshift-cluster-node-tuning-operator/tuned-sgpbt"
Jul 30 14:48:11.091736 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.091638    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vssdg\" (UniqueName: \"kubernetes.io/projected/d9b6c536-3a43-49a8-853d-f9532668f149-kube-api-access-vssdg\") pod \"node-ca-9q72z\" (UID: \"d9b6c536-3a43-49a8-853d-f9532668f149\") " pod="openshift-image-registry/node-ca-9q72z"
Jul 30 14:48:11.091736 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.091658    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/05f0cae0-8999-4a51-ae71-9e9dad066804-registration-dir\") pod \"aws-ebs-csi-driver-node-hs9dl\" (UID: \"05f0cae0-8999-4a51-ae71-9e9dad066804\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hs9dl"
Jul 30 14:48:11.091736 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.091685    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/061ec575-e1af-4d7f-b40e-57258eaecf71-var-lib-kubelet\") pod \"tuned-sgpbt\" (UID: \"061ec575-e1af-4d7f-b40e-57258eaecf71\") " pod="openshift-cluster-node-tuning-operator/tuned-sgpbt"
Jul 30 14:48:11.091736 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.091710    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d9b6c536-3a43-49a8-853d-f9532668f149-host\") pod \"node-ca-9q72z\" (UID: \"d9b6c536-3a43-49a8-853d-f9532668f149\") " pod="openshift-image-registry/node-ca-9q72z"
Jul 30 14:48:11.091736 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.091735    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/541aeca7-768f-41c8-8c0f-70663db887b2-host-var-lib-cni-bin\") pod \"multus-cp7nq\" (UID: \"541aeca7-768f-41c8-8c0f-70663db887b2\") " pod="openshift-multus/multus-cp7nq"
Jul 30 14:48:11.092334 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.091760    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/061ec575-e1af-4d7f-b40e-57258eaecf71-etc-sysconfig\") pod \"tuned-sgpbt\" (UID: \"061ec575-e1af-4d7f-b40e-57258eaecf71\") " pod="openshift-cluster-node-tuning-operator/tuned-sgpbt"
Jul 30 14:48:11.092334 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.091785    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d9b6c536-3a43-49a8-853d-f9532668f149-serviceca\") pod \"node-ca-9q72z\" (UID: \"d9b6c536-3a43-49a8-853d-f9532668f149\") " pod="openshift-image-registry/node-ca-9q72z"
Jul 30 14:48:11.092334 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.091824    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/541aeca7-768f-41c8-8c0f-70663db887b2-cni-binary-copy\") pod \"multus-cp7nq\" (UID: \"541aeca7-768f-41c8-8c0f-70663db887b2\") " pod="openshift-multus/multus-cp7nq"
Jul 30 14:48:11.092334 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.091849    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b9def45f-210b-4ceb-989c-6c696881a6f9-host-slash\") pod \"iptables-alerter-vx6bx\" (UID: \"b9def45f-210b-4ceb-989c-6c696881a6f9\") " pod="openshift-network-operator/iptables-alerter-vx6bx"
Jul 30 14:48:11.092334 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.091872    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/05f0cae0-8999-4a51-ae71-9e9dad066804-etc-selinux\") pod \"aws-ebs-csi-driver-node-hs9dl\" (UID: \"05f0cae0-8999-4a51-ae71-9e9dad066804\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hs9dl"
Jul 30 14:48:11.092334 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.091900    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/963eac2b-b8af-497c-a206-0a1c976a0e2c-kubelet-config\") pod \"global-pull-secret-syncer-9lgq9\" (UID: \"963eac2b-b8af-497c-a206-0a1c976a0e2c\") " pod="kube-system/global-pull-secret-syncer-9lgq9"
Jul 30 14:48:11.092334 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.091929    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b75a5fc6-d32b-4d96-a669-ff83a1abd706-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dkdwk\" (UID: \"b75a5fc6-d32b-4d96-a669-ff83a1abd706\") " pod="openshift-multus/multus-additional-cni-plugins-dkdwk"
Jul 30 14:48:11.092334 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.091956    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fc042c9a-bc2a-4cf4-8606-e9e20e462cb3-hosts-file\") pod \"node-resolver-8jj48\" (UID: \"fc042c9a-bc2a-4cf4-8606-e9e20e462cb3\") " pod="openshift-dns/node-resolver-8jj48"
Jul 30 14:48:11.092334 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.091972    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b75a5fc6-d32b-4d96-a669-ff83a1abd706-system-cni-dir\") pod \"multus-additional-cni-plugins-dkdwk\" (UID: \"b75a5fc6-d32b-4d96-a669-ff83a1abd706\") " pod="openshift-multus/multus-additional-cni-plugins-dkdwk"
Jul 30 14:48:11.092740 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.092539    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d9b6c536-3a43-49a8-853d-f9532668f149-host\") pod \"node-ca-9q72z\" (UID: \"d9b6c536-3a43-49a8-853d-f9532668f149\") " pod="openshift-image-registry/node-ca-9q72z"
Jul 30 14:48:11.092740 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.092629    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/963eac2b-b8af-497c-a206-0a1c976a0e2c-dbus\") pod \"global-pull-secret-syncer-9lgq9\" (UID: \"963eac2b-b8af-497c-a206-0a1c976a0e2c\") " pod="kube-system/global-pull-secret-syncer-9lgq9"
Jul 30 14:48:11.092740 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.092692    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fc042c9a-bc2a-4cf4-8606-e9e20e462cb3-hosts-file\") pod \"node-resolver-8jj48\" (UID: \"fc042c9a-bc2a-4cf4-8606-e9e20e462cb3\") " pod="openshift-dns/node-resolver-8jj48"
Jul 30 14:48:11.092882 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.092750    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/541aeca7-768f-41c8-8c0f-70663db887b2-host-var-lib-cni-bin\") pod \"multus-cp7nq\" (UID: \"541aeca7-768f-41c8-8c0f-70663db887b2\") " pod="openshift-multus/multus-cp7nq"
Jul 30 14:48:11.092988 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.092964    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/05f0cae0-8999-4a51-ae71-9e9dad066804-registration-dir\") pod \"aws-ebs-csi-driver-node-hs9dl\" (UID: \"05f0cae0-8999-4a51-ae71-9e9dad066804\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hs9dl"
Jul 30 14:48:11.093042 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.093030    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b9def45f-210b-4ceb-989c-6c696881a6f9-host-slash\") pod \"iptables-alerter-vx6bx\" (UID: \"b9def45f-210b-4ceb-989c-6c696881a6f9\") " pod="openshift-network-operator/iptables-alerter-vx6bx"
Jul 30 14:48:11.093250 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.093216    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/061ec575-e1af-4d7f-b40e-57258eaecf71-var-lib-kubelet\") pod \"tuned-sgpbt\" (UID: \"061ec575-e1af-4d7f-b40e-57258eaecf71\") " pod="openshift-cluster-node-tuning-operator/tuned-sgpbt"
Jul 30 14:48:11.093334 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.093292    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/061ec575-e1af-4d7f-b40e-57258eaecf71-host\") pod \"tuned-sgpbt\" (UID: \"061ec575-e1af-4d7f-b40e-57258eaecf71\") " pod="openshift-cluster-node-tuning-operator/tuned-sgpbt"
Jul 30 14:48:11.093391 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.093363    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/061ec575-e1af-4d7f-b40e-57258eaecf71-etc-sysconfig\") pod \"tuned-sgpbt\" (UID: \"061ec575-e1af-4d7f-b40e-57258eaecf71\") " pod="openshift-cluster-node-tuning-operator/tuned-sgpbt"
Jul 30 14:48:11.093391 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.093367    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b75a5fc6-d32b-4d96-a669-ff83a1abd706-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dkdwk\" (UID: \"b75a5fc6-d32b-4d96-a669-ff83a1abd706\") " pod="openshift-multus/multus-additional-cni-plugins-dkdwk"
Jul 30 14:48:11.093503 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.093405    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/541aeca7-768f-41c8-8c0f-70663db887b2-etc-kubernetes\") pod \"multus-cp7nq\" (UID: \"541aeca7-768f-41c8-8c0f-70663db887b2\") " pod="openshift-multus/multus-cp7nq"
Jul 30 14:48:11.093557 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.093510    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc042c9a-bc2a-4cf4-8606-e9e20e462cb3-tmp-dir\") pod \"node-resolver-8jj48\" (UID: \"fc042c9a-bc2a-4cf4-8606-e9e20e462cb3\") " pod="openshift-dns/node-resolver-8jj48"
Jul 30 14:48:11.093557 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.093550    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/541aeca7-768f-41c8-8c0f-70663db887b2-multus-socket-dir-parent\") pod \"multus-cp7nq\" (UID: \"541aeca7-768f-41c8-8c0f-70663db887b2\") " pod="openshift-multus/multus-cp7nq"
Jul 30 14:48:11.093652 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.093581    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d9b6c536-3a43-49a8-853d-f9532668f149-serviceca\") pod \"node-ca-9q72z\" (UID: \"d9b6c536-3a43-49a8-853d-f9532668f149\") " pod="openshift-image-registry/node-ca-9q72z"
Jul 30 14:48:11.093708 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.093688    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b75a5fc6-d32b-4d96-a669-ff83a1abd706-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dkdwk\" (UID: \"b75a5fc6-d32b-4d96-a669-ff83a1abd706\") " pod="openshift-multus/multus-additional-cni-plugins-dkdwk"
Jul 30 14:48:11.093758 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.093707    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/061ec575-e1af-4d7f-b40e-57258eaecf71-etc-sysctl-d\") pod \"tuned-sgpbt\" (UID: \"061ec575-e1af-4d7f-b40e-57258eaecf71\") " pod="openshift-cluster-node-tuning-operator/tuned-sgpbt"
Jul 30 14:48:11.093758 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.093747    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/541aeca7-768f-41c8-8c0f-70663db887b2-multus-cni-dir\") pod \"multus-cp7nq\" (UID: \"541aeca7-768f-41c8-8c0f-70663db887b2\") " pod="openshift-multus/multus-cp7nq"
Jul 30 14:48:11.093851 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.093758    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/541aeca7-768f-41c8-8c0f-70663db887b2-etc-kubernetes\") pod \"multus-cp7nq\" (UID: \"541aeca7-768f-41c8-8c0f-70663db887b2\") " pod="openshift-multus/multus-cp7nq"
Jul 30 14:48:11.094306 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.093974    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/541aeca7-768f-41c8-8c0f-70663db887b2-cnibin\") pod \"multus-cp7nq\" (UID: \"541aeca7-768f-41c8-8c0f-70663db887b2\") " pod="openshift-multus/multus-cp7nq"
Jul 30 14:48:11.094306 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.094003    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc042c9a-bc2a-4cf4-8606-e9e20e462cb3-tmp-dir\") pod \"node-resolver-8jj48\" (UID: \"fc042c9a-bc2a-4cf4-8606-e9e20e462cb3\") " pod="openshift-dns/node-resolver-8jj48"
Jul 30 14:48:11.094306 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.094021    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/541aeca7-768f-41c8-8c0f-70663db887b2-multus-conf-dir\") pod \"multus-cp7nq\" (UID: \"541aeca7-768f-41c8-8c0f-70663db887b2\") " pod="openshift-multus/multus-cp7nq"
Jul 30 14:48:11.094306 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.094031    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/963eac2b-b8af-497c-a206-0a1c976a0e2c-kubelet-config\") pod \"global-pull-secret-syncer-9lgq9\" (UID: \"963eac2b-b8af-497c-a206-0a1c976a0e2c\") " pod="kube-system/global-pull-secret-syncer-9lgq9"
Jul 30 14:48:11.094306 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.094070    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/541aeca7-768f-41c8-8c0f-70663db887b2-host-run-multus-certs\") pod \"multus-cp7nq\" (UID: \"541aeca7-768f-41c8-8c0f-70663db887b2\") " pod="openshift-multus/multus-cp7nq"
Jul 30 14:48:11.094306 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.094110    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/541aeca7-768f-41c8-8c0f-70663db887b2-multus-socket-dir-parent\") pod \"multus-cp7nq\" (UID: \"541aeca7-768f-41c8-8c0f-70663db887b2\") " pod="openshift-multus/multus-cp7nq"
Jul 30 14:48:11.094306 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.094115    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/061ec575-e1af-4d7f-b40e-57258eaecf71-etc-modprobe-d\") pod \"tuned-sgpbt\" (UID: \"061ec575-e1af-4d7f-b40e-57258eaecf71\") " pod="openshift-cluster-node-tuning-operator/tuned-sgpbt"
Jul 30 14:48:11.094306 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.093972    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/541aeca7-768f-41c8-8c0f-70663db887b2-cni-binary-copy\") pod \"multus-cp7nq\" (UID: \"541aeca7-768f-41c8-8c0f-70663db887b2\") " pod="openshift-multus/multus-cp7nq"
Jul 30 14:48:11.094306 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.094137    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/05f0cae0-8999-4a51-ae71-9e9dad066804-etc-selinux\") pod \"aws-ebs-csi-driver-node-hs9dl\" (UID: \"05f0cae0-8999-4a51-ae71-9e9dad066804\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hs9dl"
Jul 30 14:48:11.094306 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.094193    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/541aeca7-768f-41c8-8c0f-70663db887b2-multus-cni-dir\") pod \"multus-cp7nq\" (UID: \"541aeca7-768f-41c8-8c0f-70663db887b2\") " pod="openshift-multus/multus-cp7nq"
Jul 30 14:48:11.094306 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.094189    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/541aeca7-768f-41c8-8c0f-70663db887b2-host-run-multus-certs\") pod \"multus-cp7nq\" (UID: \"541aeca7-768f-41c8-8c0f-70663db887b2\") " pod="openshift-multus/multus-cp7nq"
Jul 30 14:48:11.094306 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.094201    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/061ec575-e1af-4d7f-b40e-57258eaecf71-etc-sysctl-d\") pod \"tuned-sgpbt\" (UID: \"061ec575-e1af-4d7f-b40e-57258eaecf71\") " pod="openshift-cluster-node-tuning-operator/tuned-sgpbt"
Jul 30 14:48:11.094306 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.094280    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/541aeca7-768f-41c8-8c0f-70663db887b2-multus-conf-dir\") pod \"multus-cp7nq\" (UID: \"541aeca7-768f-41c8-8c0f-70663db887b2\") " pod="openshift-multus/multus-cp7nq"
Jul 30 14:48:11.094306 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.094283    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/541aeca7-768f-41c8-8c0f-70663db887b2-cnibin\") pod \"multus-cp7nq\" (UID: \"541aeca7-768f-41c8-8c0f-70663db887b2\") " pod="openshift-multus/multus-cp7nq"
Jul 30 14:48:11.094306 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.094304    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/061ec575-e1af-4d7f-b40e-57258eaecf71-etc-sysctl-conf\") pod \"tuned-sgpbt\" (UID: \"061ec575-e1af-4d7f-b40e-57258eaecf71\") " pod="openshift-cluster-node-tuning-operator/tuned-sgpbt"
Jul 30 14:48:11.096412 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.094344    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/061ec575-e1af-4d7f-b40e-57258eaecf71-tmp\") pod \"tuned-sgpbt\" (UID: \"061ec575-e1af-4d7f-b40e-57258eaecf71\") " pod="openshift-cluster-node-tuning-operator/tuned-sgpbt"
Jul 30 14:48:11.096412 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.094372    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/541aeca7-768f-41c8-8c0f-70663db887b2-system-cni-dir\") pod \"multus-cp7nq\" (UID: \"541aeca7-768f-41c8-8c0f-70663db887b2\") " pod="openshift-multus/multus-cp7nq"
Jul 30 14:48:11.096412 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.094431    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/541aeca7-768f-41c8-8c0f-70663db887b2-host-var-lib-kubelet\") pod \"multus-cp7nq\" (UID: \"541aeca7-768f-41c8-8c0f-70663db887b2\") " pod="openshift-multus/multus-cp7nq"
Jul 30 14:48:11.096412 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.094461    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kk4qs\" (UniqueName: \"kubernetes.io/projected/b9def45f-210b-4ceb-989c-6c696881a6f9-kube-api-access-kk4qs\") pod \"iptables-alerter-vx6bx\" (UID: \"b9def45f-210b-4ceb-989c-6c696881a6f9\") " pod="openshift-network-operator/iptables-alerter-vx6bx"
Jul 30 14:48:11.096412 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.094499    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/061ec575-e1af-4d7f-b40e-57258eaecf71-etc-sysctl-conf\") pod \"tuned-sgpbt\" (UID: \"061ec575-e1af-4d7f-b40e-57258eaecf71\") " pod="openshift-cluster-node-tuning-operator/tuned-sgpbt"
Jul 30 14:48:11.096412 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.094520    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/963eac2b-b8af-497c-a206-0a1c976a0e2c-original-pull-secret\") pod \"global-pull-secret-syncer-9lgq9\" (UID: \"963eac2b-b8af-497c-a206-0a1c976a0e2c\") " pod="kube-system/global-pull-secret-syncer-9lgq9"
Jul 30 14:48:11.096412 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.094553    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b75a5fc6-d32b-4d96-a669-ff83a1abd706-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dkdwk\" (UID: \"b75a5fc6-d32b-4d96-a669-ff83a1abd706\") " pod="openshift-multus/multus-additional-cni-plugins-dkdwk"
Jul 30 14:48:11.096412 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.094587    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dl4t2\" (UniqueName: \"kubernetes.io/projected/fc042c9a-bc2a-4cf4-8606-e9e20e462cb3-kube-api-access-dl4t2\") pod \"node-resolver-8jj48\" (UID: \"fc042c9a-bc2a-4cf4-8606-e9e20e462cb3\") " pod="openshift-dns/node-resolver-8jj48"
Jul 30 14:48:11.096412 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.094620    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/541aeca7-768f-41c8-8c0f-70663db887b2-host-var-lib-cni-multus\") pod \"multus-cp7nq\" (UID: \"541aeca7-768f-41c8-8c0f-70663db887b2\") " pod="openshift-multus/multus-cp7nq"
Jul 30 14:48:11.096412 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.094693    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/061ec575-e1af-4d7f-b40e-57258eaecf71-etc-kubernetes\") pod \"tuned-sgpbt\" (UID: \"061ec575-e1af-4d7f-b40e-57258eaecf71\") " pod="openshift-cluster-node-tuning-operator/tuned-sgpbt"
Jul 30 14:48:11.096412 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.094730    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b75a5fc6-d32b-4d96-a669-ff83a1abd706-cnibin\") pod \"multus-additional-cni-plugins-dkdwk\" (UID: \"b75a5fc6-d32b-4d96-a669-ff83a1abd706\") " pod="openshift-multus/multus-additional-cni-plugins-dkdwk"
Jul 30 14:48:11.096412 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.094762    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/541aeca7-768f-41c8-8c0f-70663db887b2-host-run-k8s-cni-cncf-io\") pod \"multus-cp7nq\" (UID: \"541aeca7-768f-41c8-8c0f-70663db887b2\") " pod="openshift-multus/multus-cp7nq"
Jul 30 14:48:11.096412 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:11.094854    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 14:48:11.096412 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.094917    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b75a5fc6-d32b-4d96-a669-ff83a1abd706-cnibin\") pod \"multus-additional-cni-plugins-dkdwk\" (UID: \"b75a5fc6-d32b-4d96-a669-ff83a1abd706\") " pod="openshift-multus/multus-additional-cni-plugins-dkdwk"
Jul 30 14:48:11.096412 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.094923    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/541aeca7-768f-41c8-8c0f-70663db887b2-host-var-lib-cni-multus\") pod \"multus-cp7nq\" (UID: \"541aeca7-768f-41c8-8c0f-70663db887b2\") " pod="openshift-multus/multus-cp7nq"
Jul 30 14:48:11.096412 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:11.094951    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/963eac2b-b8af-497c-a206-0a1c976a0e2c-original-pull-secret podName:963eac2b-b8af-497c-a206-0a1c976a0e2c nodeName:}" failed. No retries permitted until 2026-07-30 14:48:11.5949246 +0000 UTC m=+3.123863070 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/963eac2b-b8af-497c-a206-0a1c976a0e2c-original-pull-secret") pod "global-pull-secret-syncer-9lgq9" (UID: "963eac2b-b8af-497c-a206-0a1c976a0e2c") : object "kube-system"/"original-pull-secret" not registered
Jul 30 14:48:11.096412 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.094982    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/541aeca7-768f-41c8-8c0f-70663db887b2-system-cni-dir\") pod \"multus-cp7nq\" (UID: \"541aeca7-768f-41c8-8c0f-70663db887b2\") " pod="openshift-multus/multus-cp7nq"
Jul 30 14:48:11.097349 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.094982    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/061ec575-e1af-4d7f-b40e-57258eaecf71-etc-kubernetes\") pod \"tuned-sgpbt\" (UID: \"061ec575-e1af-4d7f-b40e-57258eaecf71\") " pod="openshift-cluster-node-tuning-operator/tuned-sgpbt"
Jul 30 14:48:11.097349 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.095027    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/541aeca7-768f-41c8-8c0f-70663db887b2-host-run-k8s-cni-cncf-io\") pod \"multus-cp7nq\" (UID: \"541aeca7-768f-41c8-8c0f-70663db887b2\") " pod="openshift-multus/multus-cp7nq"
Jul 30 14:48:11.097349 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.095063    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/05f0cae0-8999-4a51-ae71-9e9dad066804-device-dir\") pod \"aws-ebs-csi-driver-node-hs9dl\" (UID: \"05f0cae0-8999-4a51-ae71-9e9dad066804\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hs9dl"
Jul 30 14:48:11.097349 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.095067    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/541aeca7-768f-41c8-8c0f-70663db887b2-host-var-lib-kubelet\") pod \"multus-cp7nq\" (UID: \"541aeca7-768f-41c8-8c0f-70663db887b2\") " pod="openshift-multus/multus-cp7nq"
Jul 30 14:48:11.097349 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.095084    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/061ec575-e1af-4d7f-b40e-57258eaecf71-etc-tuned\") pod \"tuned-sgpbt\" (UID: \"061ec575-e1af-4d7f-b40e-57258eaecf71\") " pod="openshift-cluster-node-tuning-operator/tuned-sgpbt"
Jul 30 14:48:11.097349 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.095100    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/061ec575-e1af-4d7f-b40e-57258eaecf71-etc-systemd\") pod \"tuned-sgpbt\" (UID: \"061ec575-e1af-4d7f-b40e-57258eaecf71\") " pod="openshift-cluster-node-tuning-operator/tuned-sgpbt"
Jul 30 14:48:11.097349 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.095134    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/061ec575-e1af-4d7f-b40e-57258eaecf71-lib-modules\") pod \"tuned-sgpbt\" (UID: \"061ec575-e1af-4d7f-b40e-57258eaecf71\") " pod="openshift-cluster-node-tuning-operator/tuned-sgpbt"
Jul 30 14:48:11.097349 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.095198    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b75a5fc6-d32b-4d96-a669-ff83a1abd706-os-release\") pod \"multus-additional-cni-plugins-dkdwk\" (UID: \"b75a5fc6-d32b-4d96-a669-ff83a1abd706\") " pod="openshift-multus/multus-additional-cni-plugins-dkdwk"
Jul 30 14:48:11.097349 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.095231    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/541aeca7-768f-41c8-8c0f-70663db887b2-host-run-netns\") pod \"multus-cp7nq\" (UID: \"541aeca7-768f-41c8-8c0f-70663db887b2\") " pod="openshift-multus/multus-cp7nq"
Jul 30 14:48:11.097349 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.095264    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jd5d\" (UniqueName: \"kubernetes.io/projected/541aeca7-768f-41c8-8c0f-70663db887b2-kube-api-access-8jd5d\") pod \"multus-cp7nq\" (UID: \"541aeca7-768f-41c8-8c0f-70663db887b2\") " pod="openshift-multus/multus-cp7nq"
Jul 30 14:48:11.097349 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.095300    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/05f0cae0-8999-4a51-ae71-9e9dad066804-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hs9dl\" (UID: \"05f0cae0-8999-4a51-ae71-9e9dad066804\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hs9dl"
Jul 30 14:48:11.097349 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.095325    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/05f0cae0-8999-4a51-ae71-9e9dad066804-sys-fs\") pod \"aws-ebs-csi-driver-node-hs9dl\" (UID: \"05f0cae0-8999-4a51-ae71-9e9dad066804\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hs9dl"
Jul 30 14:48:11.097349 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.095354    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jk2hg\" (UniqueName: \"kubernetes.io/projected/05f0cae0-8999-4a51-ae71-9e9dad066804-kube-api-access-jk2hg\") pod \"aws-ebs-csi-driver-node-hs9dl\" (UID: \"05f0cae0-8999-4a51-ae71-9e9dad066804\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hs9dl"
Jul 30 14:48:11.097349 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.095386    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b75a5fc6-d32b-4d96-a669-ff83a1abd706-cni-binary-copy\") pod \"multus-additional-cni-plugins-dkdwk\" (UID: \"b75a5fc6-d32b-4d96-a669-ff83a1abd706\") " pod="openshift-multus/multus-additional-cni-plugins-dkdwk"
Jul 30 14:48:11.097349 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.095399    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/061ec575-e1af-4d7f-b40e-57258eaecf71-lib-modules\") pod \"tuned-sgpbt\" (UID: \"061ec575-e1af-4d7f-b40e-57258eaecf71\") " pod="openshift-cluster-node-tuning-operator/tuned-sgpbt"
Jul 30 14:48:11.097349 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.095417    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/541aeca7-768f-41c8-8c0f-70663db887b2-os-release\") pod \"multus-cp7nq\" (UID: \"541aeca7-768f-41c8-8c0f-70663db887b2\") " pod="openshift-multus/multus-cp7nq"
Jul 30 14:48:11.097349 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.095450    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/541aeca7-768f-41c8-8c0f-70663db887b2-hostroot\") pod \"multus-cp7nq\" (UID: \"541aeca7-768f-41c8-8c0f-70663db887b2\") " pod="openshift-multus/multus-cp7nq"
Jul 30 14:48:11.098302 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.095478    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/061ec575-e1af-4d7f-b40e-57258eaecf71-sys\") pod \"tuned-sgpbt\" (UID: \"061ec575-e1af-4d7f-b40e-57258eaecf71\") " pod="openshift-cluster-node-tuning-operator/tuned-sgpbt"
Jul 30 14:48:11.098302 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.095551    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/05f0cae0-8999-4a51-ae71-9e9dad066804-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hs9dl\" (UID: \"05f0cae0-8999-4a51-ae71-9e9dad066804\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hs9dl"
Jul 30 14:48:11.098302 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.095545    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/05f0cae0-8999-4a51-ae71-9e9dad066804-device-dir\") pod \"aws-ebs-csi-driver-node-hs9dl\" (UID: \"05f0cae0-8999-4a51-ae71-9e9dad066804\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hs9dl"
Jul 30 14:48:11.098302 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.095553    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b75a5fc6-d32b-4d96-a669-ff83a1abd706-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dkdwk\" (UID: \"b75a5fc6-d32b-4d96-a669-ff83a1abd706\") " pod="openshift-multus/multus-additional-cni-plugins-dkdwk"
Jul 30 14:48:11.098302 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.095605    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/061ec575-e1af-4d7f-b40e-57258eaecf71-etc-systemd\") pod \"tuned-sgpbt\" (UID: \"061ec575-e1af-4d7f-b40e-57258eaecf71\") " pod="openshift-cluster-node-tuning-operator/tuned-sgpbt"
Jul 30 14:48:11.098302 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.095610    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/05f0cae0-8999-4a51-ae71-9e9dad066804-sys-fs\") pod \"aws-ebs-csi-driver-node-hs9dl\" (UID: \"05f0cae0-8999-4a51-ae71-9e9dad066804\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hs9dl"
Jul 30 14:48:11.098302 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.095618    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzqzp\" (UniqueName: \"kubernetes.io/projected/061ec575-e1af-4d7f-b40e-57258eaecf71-kube-api-access-bzqzp\") pod \"tuned-sgpbt\" (UID: \"061ec575-e1af-4d7f-b40e-57258eaecf71\") " pod="openshift-cluster-node-tuning-operator/tuned-sgpbt"
Jul 30 14:48:11.098302 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.095670    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/541aeca7-768f-41c8-8c0f-70663db887b2-hostroot\") pod \"multus-cp7nq\" (UID: \"541aeca7-768f-41c8-8c0f-70663db887b2\") " pod="openshift-multus/multus-cp7nq"
Jul 30 14:48:11.098302 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.095732    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b75a5fc6-d32b-4d96-a669-ff83a1abd706-os-release\") pod \"multus-additional-cni-plugins-dkdwk\" (UID: \"b75a5fc6-d32b-4d96-a669-ff83a1abd706\") " pod="openshift-multus/multus-additional-cni-plugins-dkdwk"
Jul 30 14:48:11.098302 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.096986    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/541aeca7-768f-41c8-8c0f-70663db887b2-os-release\") pod \"multus-cp7nq\" (UID: \"541aeca7-768f-41c8-8c0f-70663db887b2\") " pod="openshift-multus/multus-cp7nq"
Jul 30 14:48:11.098302 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.097051    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/541aeca7-768f-41c8-8c0f-70663db887b2-host-run-netns\") pod \"multus-cp7nq\" (UID: \"541aeca7-768f-41c8-8c0f-70663db887b2\") " pod="openshift-multus/multus-cp7nq"
Jul 30 14:48:11.098302 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.097114    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/061ec575-e1af-4d7f-b40e-57258eaecf71-sys\") pod \"tuned-sgpbt\" (UID: \"061ec575-e1af-4d7f-b40e-57258eaecf71\") " pod="openshift-cluster-node-tuning-operator/tuned-sgpbt"
Jul 30 14:48:11.098302 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.097148    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b75a5fc6-d32b-4d96-a669-ff83a1abd706-cni-binary-copy\") pod \"multus-additional-cni-plugins-dkdwk\" (UID: \"b75a5fc6-d32b-4d96-a669-ff83a1abd706\") " pod="openshift-multus/multus-additional-cni-plugins-dkdwk"
Jul 30 14:48:11.098302 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.097160    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xn56w\" (UniqueName: \"kubernetes.io/projected/b75a5fc6-d32b-4d96-a669-ff83a1abd706-kube-api-access-xn56w\") pod \"multus-additional-cni-plugins-dkdwk\" (UID: \"b75a5fc6-d32b-4d96-a669-ff83a1abd706\") " pod="openshift-multus/multus-additional-cni-plugins-dkdwk"
Jul 30 14:48:11.098302 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.097201    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/541aeca7-768f-41c8-8c0f-70663db887b2-multus-daemon-config\") pod \"multus-cp7nq\" (UID: \"541aeca7-768f-41c8-8c0f-70663db887b2\") " pod="openshift-multus/multus-cp7nq"
Jul 30 14:48:11.098302 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.097258    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b9def45f-210b-4ceb-989c-6c696881a6f9-iptables-alerter-script\") pod \"iptables-alerter-vx6bx\" (UID: \"b9def45f-210b-4ceb-989c-6c696881a6f9\") " pod="openshift-network-operator/iptables-alerter-vx6bx"
Jul 30 14:48:11.098302 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.097293    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/05f0cae0-8999-4a51-ae71-9e9dad066804-socket-dir\") pod \"aws-ebs-csi-driver-node-hs9dl\" (UID: \"05f0cae0-8999-4a51-ae71-9e9dad066804\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hs9dl"
Jul 30 14:48:11.099095 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.097443    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/05f0cae0-8999-4a51-ae71-9e9dad066804-socket-dir\") pod \"aws-ebs-csi-driver-node-hs9dl\" (UID: \"05f0cae0-8999-4a51-ae71-9e9dad066804\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hs9dl"
Jul 30 14:48:11.099095 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.097977    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/541aeca7-768f-41c8-8c0f-70663db887b2-multus-daemon-config\") pod \"multus-cp7nq\" (UID: \"541aeca7-768f-41c8-8c0f-70663db887b2\") " pod="openshift-multus/multus-cp7nq"
Jul 30 14:48:11.099095 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.098062    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b9def45f-210b-4ceb-989c-6c696881a6f9-iptables-alerter-script\") pod \"iptables-alerter-vx6bx\" (UID: \"b9def45f-210b-4ceb-989c-6c696881a6f9\") " pod="openshift-network-operator/iptables-alerter-vx6bx"
Jul 30 14:48:11.100385 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.100335    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/061ec575-e1af-4d7f-b40e-57258eaecf71-tmp\") pod \"tuned-sgpbt\" (UID: \"061ec575-e1af-4d7f-b40e-57258eaecf71\") " pod="openshift-cluster-node-tuning-operator/tuned-sgpbt"
Jul 30 14:48:11.102155 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.102132    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vssdg\" (UniqueName: \"kubernetes.io/projected/d9b6c536-3a43-49a8-853d-f9532668f149-kube-api-access-vssdg\") pod \"node-ca-9q72z\" (UID: \"d9b6c536-3a43-49a8-853d-f9532668f149\") " pod="openshift-image-registry/node-ca-9q72z"
Jul 30 14:48:11.120735 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.120701    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzqzp\" (UniqueName: \"kubernetes.io/projected/061ec575-e1af-4d7f-b40e-57258eaecf71-kube-api-access-bzqzp\") pod \"tuned-sgpbt\" (UID: \"061ec575-e1af-4d7f-b40e-57258eaecf71\") " pod="openshift-cluster-node-tuning-operator/tuned-sgpbt"
Jul 30 14:48:11.121186 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.121136    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kk4qs\" (UniqueName: \"kubernetes.io/projected/b9def45f-210b-4ceb-989c-6c696881a6f9-kube-api-access-kk4qs\") pod \"iptables-alerter-vx6bx\" (UID: \"b9def45f-210b-4ceb-989c-6c696881a6f9\") " pod="openshift-network-operator/iptables-alerter-vx6bx"
Jul 30 14:48:11.121395 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.121368    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jk2hg\" (UniqueName: \"kubernetes.io/projected/05f0cae0-8999-4a51-ae71-9e9dad066804-kube-api-access-jk2hg\") pod \"aws-ebs-csi-driver-node-hs9dl\" (UID: \"05f0cae0-8999-4a51-ae71-9e9dad066804\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hs9dl"
Jul 30 14:48:11.121963 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.121940    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dl4t2\" (UniqueName: \"kubernetes.io/projected/fc042c9a-bc2a-4cf4-8606-e9e20e462cb3-kube-api-access-dl4t2\") pod \"node-resolver-8jj48\" (UID: \"fc042c9a-bc2a-4cf4-8606-e9e20e462cb3\") " pod="openshift-dns/node-resolver-8jj48"
Jul 30 14:48:11.122238 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.122195    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xn56w\" (UniqueName: \"kubernetes.io/projected/b75a5fc6-d32b-4d96-a669-ff83a1abd706-kube-api-access-xn56w\") pod \"multus-additional-cni-plugins-dkdwk\" (UID: \"b75a5fc6-d32b-4d96-a669-ff83a1abd706\") " pod="openshift-multus/multus-additional-cni-plugins-dkdwk"
Jul 30 14:48:11.122740 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.122719    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jd5d\" (UniqueName: \"kubernetes.io/projected/541aeca7-768f-41c8-8c0f-70663db887b2-kube-api-access-8jd5d\") pod \"multus-cp7nq\" (UID: \"541aeca7-768f-41c8-8c0f-70663db887b2\") " pod="openshift-multus/multus-cp7nq"
Jul 30 14:48:11.181880 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.181842    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-t65nt"
Jul 30 14:48:11.197816 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.197751    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:11.209781 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.209752    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vx6bx"
Jul 30 14:48:11.218670 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.218649    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8jj48"
Jul 30 14:48:11.226299 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.226277    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hs9dl"
Jul 30 14:48:11.236224 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.236204    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-sgpbt"
Jul 30 14:48:11.244142 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.244122    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dkdwk"
Jul 30 14:48:11.258202 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.258177    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9q72z"
Jul 30 14:48:11.263139 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.263121    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cp7nq"
Jul 30 14:48:11.500287 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.500198    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be2b73fa-10f0-4ea3-a68e-fa452e30914b-metrics-certs\") pod \"network-metrics-daemon-rj8gr\" (UID: \"be2b73fa-10f0-4ea3-a68e-fa452e30914b\") " pod="openshift-multus/network-metrics-daemon-rj8gr"
Jul 30 14:48:11.500287 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.500254    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rbmt\" (UniqueName: \"kubernetes.io/projected/78565b9e-9521-4e9f-811e-6cc4cd76a4d3-kube-api-access-9rbmt\") pod \"network-check-target-j6hld\" (UID: \"78565b9e-9521-4e9f-811e-6cc4cd76a4d3\") " pod="openshift-network-diagnostics/network-check-target-j6hld"
Jul 30 14:48:11.500515 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:11.500386    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 14:48:11.500515 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:11.500457    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be2b73fa-10f0-4ea3-a68e-fa452e30914b-metrics-certs podName:be2b73fa-10f0-4ea3-a68e-fa452e30914b nodeName:}" failed. No retries permitted until 2026-07-30 14:48:12.50043735 +0000 UTC m=+4.029375824 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/be2b73fa-10f0-4ea3-a68e-fa452e30914b-metrics-certs") pod "network-metrics-daemon-rj8gr" (UID: "be2b73fa-10f0-4ea3-a68e-fa452e30914b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 14:48:11.500515 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:11.500395    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 14:48:11.500515 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:11.500499    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 14:48:11.500515 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:11.500514    2578 projected.go:196] Error preparing data for projected volume kube-api-access-9rbmt for pod openshift-network-diagnostics/network-check-target-j6hld: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 14:48:11.500742 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:11.500578    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/78565b9e-9521-4e9f-811e-6cc4cd76a4d3-kube-api-access-9rbmt podName:78565b9e-9521-4e9f-811e-6cc4cd76a4d3 nodeName:}" failed. No retries permitted until 2026-07-30 14:48:12.500559575 +0000 UTC m=+4.029498044 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-9rbmt" (UniqueName: "kubernetes.io/projected/78565b9e-9521-4e9f-811e-6cc4cd76a4d3-kube-api-access-9rbmt") pod "network-check-target-j6hld" (UID: "78565b9e-9521-4e9f-811e-6cc4cd76a4d3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 14:48:11.600933 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.600897    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/963eac2b-b8af-497c-a206-0a1c976a0e2c-original-pull-secret\") pod \"global-pull-secret-syncer-9lgq9\" (UID: \"963eac2b-b8af-497c-a206-0a1c976a0e2c\") " pod="kube-system/global-pull-secret-syncer-9lgq9"
Jul 30 14:48:11.601140 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:11.601036    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 14:48:11.601140 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:11.601108    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/963eac2b-b8af-497c-a206-0a1c976a0e2c-original-pull-secret podName:963eac2b-b8af-497c-a206-0a1c976a0e2c nodeName:}" failed. No retries permitted until 2026-07-30 14:48:12.601089281 +0000 UTC m=+4.130027739 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/963eac2b-b8af-497c-a206-0a1c976a0e2c-original-pull-secret") pod "global-pull-secret-syncer-9lgq9" (UID: "963eac2b-b8af-497c-a206-0a1c976a0e2c") : object "kube-system"/"original-pull-secret" not registered
Jul 30 14:48:11.925044 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.924950    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-29 14:43:11 +0000 UTC" deadline="2028-03-14 19:41:29.839301904 +0000 UTC"
Jul 30 14:48:11.925044 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.924986    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14236h53m17.914319848s"
Jul 30 14:48:11.972247 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.972211    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rj8gr"
Jul 30 14:48:11.972448 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:11.972361    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rj8gr" podUID="be2b73fa-10f0-4ea3-a68e-fa452e30914b"
Jul 30 14:48:11.972448 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:11.972417    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lgq9"
Jul 30 14:48:11.972541 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:11.972524    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lgq9" podUID="963eac2b-b8af-497c-a206-0a1c976a0e2c"
Jul 30 14:48:12.144254 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:12.144215    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05f0cae0_8999_4a51_ae71_9e9dad066804.slice/crio-dda76d69e81e8effdd5ef50ae765ee46a9abe6f5ae6fa55cb292a0a5119121cc WatchSource:0}: Error finding container dda76d69e81e8effdd5ef50ae765ee46a9abe6f5ae6fa55cb292a0a5119121cc: Status 404 returned error can't find the container with id dda76d69e81e8effdd5ef50ae765ee46a9abe6f5ae6fa55cb292a0a5119121cc
Jul 30 14:48:12.145842 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:12.145800    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89cc8684_f168_40cf_93ec_1143c2cb0784.slice/crio-c8b754c74ca15e72cb63224beddd6a64e10f87f4e1593d3fbb08522df4ebb5ad WatchSource:0}: Error finding container c8b754c74ca15e72cb63224beddd6a64e10f87f4e1593d3fbb08522df4ebb5ad: Status 404 returned error can't find the container with id c8b754c74ca15e72cb63224beddd6a64e10f87f4e1593d3fbb08522df4ebb5ad
Jul 30 14:48:12.146450 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:12.146382    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9def45f_210b_4ceb_989c_6c696881a6f9.slice/crio-2cf56f269f1d2da6f2c62a3d3453a6f6ffcbe5211272b757873a40dfe38e5ef2 WatchSource:0}: Error finding container 2cf56f269f1d2da6f2c62a3d3453a6f6ffcbe5211272b757873a40dfe38e5ef2: Status 404 returned error can't find the container with id 2cf56f269f1d2da6f2c62a3d3453a6f6ffcbe5211272b757873a40dfe38e5ef2
Jul 30 14:48:12.151354 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:12.151156    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5b201dc_6dcd_44c8_9c70_9946315da836.slice/crio-41492e732c62469b1b25f8f267bec215081ee4122f21c46dbb66c50e71cec49e WatchSource:0}: Error finding container 41492e732c62469b1b25f8f267bec215081ee4122f21c46dbb66c50e71cec49e: Status 404 returned error can't find the container with id 41492e732c62469b1b25f8f267bec215081ee4122f21c46dbb66c50e71cec49e
Jul 30 14:48:12.152103 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:12.152078    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9b6c536_3a43_49a8_853d_f9532668f149.slice/crio-b13e6eb81d85da956897ea0092f839f7f2af9c506dbbea9b60e65e8adffa7ffe WatchSource:0}: Error finding container b13e6eb81d85da956897ea0092f839f7f2af9c506dbbea9b60e65e8adffa7ffe: Status 404 returned error can't find the container with id b13e6eb81d85da956897ea0092f839f7f2af9c506dbbea9b60e65e8adffa7ffe
Jul 30 14:48:12.154422 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:12.154162    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb75a5fc6_d32b_4d96_a669_ff83a1abd706.slice/crio-3640bf7283a4c6ad956d8807fb334f5e24b4a71a2355563875ca578729b8beb3 WatchSource:0}: Error finding container 3640bf7283a4c6ad956d8807fb334f5e24b4a71a2355563875ca578729b8beb3: Status 404 returned error can't find the container with id 3640bf7283a4c6ad956d8807fb334f5e24b4a71a2355563875ca578729b8beb3
Jul 30 14:48:12.155702 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:12.154834    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod541aeca7_768f_41c8_8c0f_70663db887b2.slice/crio-985398fda75f0aa87b0a51bcf625b7f971f1e61219d93819b08167ecf8c6c262 WatchSource:0}: Error finding container 985398fda75f0aa87b0a51bcf625b7f971f1e61219d93819b08167ecf8c6c262: Status 404 returned error can't find the container with id 985398fda75f0aa87b0a51bcf625b7f971f1e61219d93819b08167ecf8c6c262
Jul 30 14:48:12.157055 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:12.157016    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc042c9a_bc2a_4cf4_8606_e9e20e462cb3.slice/crio-67d615a43da0e7bbdac5445a09772ce7f05f814d96d06d6f47f64d456ccc3d85 WatchSource:0}: Error finding container 67d615a43da0e7bbdac5445a09772ce7f05f814d96d06d6f47f64d456ccc3d85: Status 404 returned error can't find the container with id 67d615a43da0e7bbdac5445a09772ce7f05f814d96d06d6f47f64d456ccc3d85
Jul 30 14:48:12.509479 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:12.509201    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rbmt\" (UniqueName: \"kubernetes.io/projected/78565b9e-9521-4e9f-811e-6cc4cd76a4d3-kube-api-access-9rbmt\") pod \"network-check-target-j6hld\" (UID: \"78565b9e-9521-4e9f-811e-6cc4cd76a4d3\") " pod="openshift-network-diagnostics/network-check-target-j6hld"
Jul 30 14:48:12.509628 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:12.509480    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be2b73fa-10f0-4ea3-a68e-fa452e30914b-metrics-certs\") pod \"network-metrics-daemon-rj8gr\" (UID: \"be2b73fa-10f0-4ea3-a68e-fa452e30914b\") " pod="openshift-multus/network-metrics-daemon-rj8gr"
Jul 30 14:48:12.509628 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:12.509367    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 14:48:12.509628 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:12.509514    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 14:48:12.509628 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:12.509525    2578 projected.go:196] Error preparing data for projected volume kube-api-access-9rbmt for pod openshift-network-diagnostics/network-check-target-j6hld: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 14:48:12.509628 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:12.509573    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/78565b9e-9521-4e9f-811e-6cc4cd76a4d3-kube-api-access-9rbmt podName:78565b9e-9521-4e9f-811e-6cc4cd76a4d3 nodeName:}" failed. No retries permitted until 2026-07-30 14:48:14.509557612 +0000 UTC m=+6.038496065 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-9rbmt" (UniqueName: "kubernetes.io/projected/78565b9e-9521-4e9f-811e-6cc4cd76a4d3-kube-api-access-9rbmt") pod "network-check-target-j6hld" (UID: "78565b9e-9521-4e9f-811e-6cc4cd76a4d3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 14:48:12.509628 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:12.509611    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 14:48:12.509837 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:12.509665    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be2b73fa-10f0-4ea3-a68e-fa452e30914b-metrics-certs podName:be2b73fa-10f0-4ea3-a68e-fa452e30914b nodeName:}" failed. No retries permitted until 2026-07-30 14:48:14.509652282 +0000 UTC m=+6.038590735 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/be2b73fa-10f0-4ea3-a68e-fa452e30914b-metrics-certs") pod "network-metrics-daemon-rj8gr" (UID: "be2b73fa-10f0-4ea3-a68e-fa452e30914b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 14:48:12.610410 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:12.610374    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/963eac2b-b8af-497c-a206-0a1c976a0e2c-original-pull-secret\") pod \"global-pull-secret-syncer-9lgq9\" (UID: \"963eac2b-b8af-497c-a206-0a1c976a0e2c\") " pod="kube-system/global-pull-secret-syncer-9lgq9"
Jul 30 14:48:12.610580 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:12.610510    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 14:48:12.610580 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:12.610574    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/963eac2b-b8af-497c-a206-0a1c976a0e2c-original-pull-secret podName:963eac2b-b8af-497c-a206-0a1c976a0e2c nodeName:}" failed. No retries permitted until 2026-07-30 14:48:14.610558288 +0000 UTC m=+6.139496741 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/963eac2b-b8af-497c-a206-0a1c976a0e2c-original-pull-secret") pod "global-pull-secret-syncer-9lgq9" (UID: "963eac2b-b8af-497c-a206-0a1c976a0e2c") : object "kube-system"/"original-pull-secret" not registered
Jul 30 14:48:12.926187 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:12.926075    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-29 14:43:11 +0000 UTC" deadline="2028-01-08 16:26:32.157802214 +0000 UTC"
Jul 30 14:48:12.926187 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:12.926121    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12649h38m19.231686905s"
Jul 30 14:48:12.974056 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:12.974021    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j6hld"
Jul 30 14:48:12.974227 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:12.974159    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j6hld" podUID="78565b9e-9521-4e9f-811e-6cc4cd76a4d3"
Jul 30 14:48:12.984375 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:12.984341    2578 generic.go:356] "Generic (PLEG): container finished" podID="d0f02a53bf99ae56c5e1962a8f62bc87" containerID="e0baf406159093a3b4328837178d24132600116ad2a45b241c710cff2e6217a2" exitCode=0
Jul 30 14:48:12.984559 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:12.984426    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-117.ec2.internal" event={"ID":"d0f02a53bf99ae56c5e1962a8f62bc87","Type":"ContainerDied","Data":"e0baf406159093a3b4328837178d24132600116ad2a45b241c710cff2e6217a2"}
Jul 30 14:48:12.993159 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:12.990968    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dkdwk" event={"ID":"b75a5fc6-d32b-4d96-a669-ff83a1abd706","Type":"ContainerStarted","Data":"3640bf7283a4c6ad956d8807fb334f5e24b4a71a2355563875ca578729b8beb3"}
Jul 30 14:48:12.995355 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:12.993762    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vx6bx" event={"ID":"b9def45f-210b-4ceb-989c-6c696881a6f9","Type":"ContainerStarted","Data":"2cf56f269f1d2da6f2c62a3d3453a6f6ffcbe5211272b757873a40dfe38e5ef2"}
Jul 30 14:48:12.996567 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:12.996479    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xllmv" event={"ID":"89cc8684-f168-40cf-93ec-1143c2cb0784","Type":"ContainerStarted","Data":"c8b754c74ca15e72cb63224beddd6a64e10f87f4e1593d3fbb08522df4ebb5ad"}
Jul 30 14:48:12.998079 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:12.998053    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hs9dl" event={"ID":"05f0cae0-8999-4a51-ae71-9e9dad066804","Type":"ContainerStarted","Data":"dda76d69e81e8effdd5ef50ae765ee46a9abe6f5ae6fa55cb292a0a5119121cc"}
Jul 30 14:48:13.000389 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:13.000364    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-117.ec2.internal" event={"ID":"a5efaf84ea37e1b2796825887b9db661","Type":"ContainerStarted","Data":"d968354b3848d3cf25ff461015f90c18dafe7713d52299e66eb1c779701154a8"}
Jul 30 14:48:13.002353 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:13.002328    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-sgpbt" event={"ID":"061ec575-e1af-4d7f-b40e-57258eaecf71","Type":"ContainerStarted","Data":"bdbe30e0e0b7c0b2541995e4054041c5192d6cbb1f4b4d24bfa97cbc6e95a9af"}
Jul 30 14:48:13.004412 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:13.004096    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8jj48" event={"ID":"fc042c9a-bc2a-4cf4-8606-e9e20e462cb3","Type":"ContainerStarted","Data":"67d615a43da0e7bbdac5445a09772ce7f05f814d96d06d6f47f64d456ccc3d85"}
Jul 30 14:48:13.005823 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:13.005794    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cp7nq" event={"ID":"541aeca7-768f-41c8-8c0f-70663db887b2","Type":"ContainerStarted","Data":"985398fda75f0aa87b0a51bcf625b7f971f1e61219d93819b08167ecf8c6c262"}
Jul 30 14:48:13.007258 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:13.007233    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9q72z" event={"ID":"d9b6c536-3a43-49a8-853d-f9532668f149","Type":"ContainerStarted","Data":"b13e6eb81d85da956897ea0092f839f7f2af9c506dbbea9b60e65e8adffa7ffe"}
Jul 30 14:48:13.008767 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:13.008730    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-t65nt" event={"ID":"b5b201dc-6dcd-44c8-9c70-9946315da836","Type":"ContainerStarted","Data":"41492e732c62469b1b25f8f267bec215081ee4122f21c46dbb66c50e71cec49e"}
Jul 30 14:48:13.973033 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:13.972267    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lgq9"
Jul 30 14:48:13.973033 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:13.972409    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lgq9" podUID="963eac2b-b8af-497c-a206-0a1c976a0e2c"
Jul 30 14:48:13.973033 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:13.972848    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rj8gr"
Jul 30 14:48:13.973033 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:13.972963    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rj8gr" podUID="be2b73fa-10f0-4ea3-a68e-fa452e30914b"
Jul 30 14:48:14.019463 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:14.019327    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-117.ec2.internal" event={"ID":"d0f02a53bf99ae56c5e1962a8f62bc87","Type":"ContainerStarted","Data":"0e48885b4585cc883c1b796d07756dff7d593ccf6c41ad7dfc3e739c5ec7b6bf"}
Jul 30 14:48:14.034276 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:14.033865    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-117.ec2.internal" podStartSLOduration=3.0338422290000002 podStartE2EDuration="3.033842229s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 14:48:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:48:13.015149786 +0000 UTC m=+4.544088266" watchObservedRunningTime="2026-07-30 14:48:14.033842229 +0000 UTC m=+5.562780712"
Jul 30 14:48:14.530583 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:14.529685    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be2b73fa-10f0-4ea3-a68e-fa452e30914b-metrics-certs\") pod \"network-metrics-daemon-rj8gr\" (UID: \"be2b73fa-10f0-4ea3-a68e-fa452e30914b\") " pod="openshift-multus/network-metrics-daemon-rj8gr"
Jul 30 14:48:14.530583 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:14.529747    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rbmt\" (UniqueName: \"kubernetes.io/projected/78565b9e-9521-4e9f-811e-6cc4cd76a4d3-kube-api-access-9rbmt\") pod \"network-check-target-j6hld\" (UID: \"78565b9e-9521-4e9f-811e-6cc4cd76a4d3\") " pod="openshift-network-diagnostics/network-check-target-j6hld"
Jul 30 14:48:14.530583 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:14.529899    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 14:48:14.530583 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:14.529921    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 14:48:14.530583 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:14.529933    2578 projected.go:196] Error preparing data for projected volume kube-api-access-9rbmt for pod openshift-network-diagnostics/network-check-target-j6hld: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 14:48:14.530583 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:14.529995    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/78565b9e-9521-4e9f-811e-6cc4cd76a4d3-kube-api-access-9rbmt podName:78565b9e-9521-4e9f-811e-6cc4cd76a4d3 nodeName:}" failed. No retries permitted until 2026-07-30 14:48:18.529974952 +0000 UTC m=+10.058913411 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-9rbmt" (UniqueName: "kubernetes.io/projected/78565b9e-9521-4e9f-811e-6cc4cd76a4d3-kube-api-access-9rbmt") pod "network-check-target-j6hld" (UID: "78565b9e-9521-4e9f-811e-6cc4cd76a4d3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 14:48:14.530583 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:14.530447    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 14:48:14.530583 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:14.530500    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be2b73fa-10f0-4ea3-a68e-fa452e30914b-metrics-certs podName:be2b73fa-10f0-4ea3-a68e-fa452e30914b nodeName:}" failed. No retries permitted until 2026-07-30 14:48:18.530482671 +0000 UTC m=+10.059421129 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/be2b73fa-10f0-4ea3-a68e-fa452e30914b-metrics-certs") pod "network-metrics-daemon-rj8gr" (UID: "be2b73fa-10f0-4ea3-a68e-fa452e30914b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 14:48:14.631201 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:14.630582    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/963eac2b-b8af-497c-a206-0a1c976a0e2c-original-pull-secret\") pod \"global-pull-secret-syncer-9lgq9\" (UID: \"963eac2b-b8af-497c-a206-0a1c976a0e2c\") " pod="kube-system/global-pull-secret-syncer-9lgq9"
Jul 30 14:48:14.631201 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:14.630769    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 14:48:14.631201 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:14.630834    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/963eac2b-b8af-497c-a206-0a1c976a0e2c-original-pull-secret podName:963eac2b-b8af-497c-a206-0a1c976a0e2c nodeName:}" failed. No retries permitted until 2026-07-30 14:48:18.630815094 +0000 UTC m=+10.159753550 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/963eac2b-b8af-497c-a206-0a1c976a0e2c-original-pull-secret") pod "global-pull-secret-syncer-9lgq9" (UID: "963eac2b-b8af-497c-a206-0a1c976a0e2c") : object "kube-system"/"original-pull-secret" not registered
Jul 30 14:48:14.976526 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:14.976446    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j6hld"
Jul 30 14:48:14.977031 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:14.976574    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j6hld" podUID="78565b9e-9521-4e9f-811e-6cc4cd76a4d3"
Jul 30 14:48:15.972817 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:15.972779    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rj8gr"
Jul 30 14:48:15.973029 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:15.972944    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rj8gr" podUID="be2b73fa-10f0-4ea3-a68e-fa452e30914b"
Jul 30 14:48:15.973029 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:15.973007    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lgq9"
Jul 30 14:48:15.973152 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:15.973132    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lgq9" podUID="963eac2b-b8af-497c-a206-0a1c976a0e2c"
Jul 30 14:48:16.972834 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:16.972749    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j6hld"
Jul 30 14:48:16.973352 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:16.972888    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j6hld" podUID="78565b9e-9521-4e9f-811e-6cc4cd76a4d3"
Jul 30 14:48:17.972389 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:17.972247    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rj8gr"
Jul 30 14:48:17.972389 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:17.972377    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rj8gr" podUID="be2b73fa-10f0-4ea3-a68e-fa452e30914b"
Jul 30 14:48:17.972815 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:17.972646    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lgq9"
Jul 30 14:48:17.972815 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:17.972776    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lgq9" podUID="963eac2b-b8af-497c-a206-0a1c976a0e2c"
Jul 30 14:48:18.564478 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:18.564436    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be2b73fa-10f0-4ea3-a68e-fa452e30914b-metrics-certs\") pod \"network-metrics-daemon-rj8gr\" (UID: \"be2b73fa-10f0-4ea3-a68e-fa452e30914b\") " pod="openshift-multus/network-metrics-daemon-rj8gr"
Jul 30 14:48:18.564699 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:18.564516    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rbmt\" (UniqueName: \"kubernetes.io/projected/78565b9e-9521-4e9f-811e-6cc4cd76a4d3-kube-api-access-9rbmt\") pod \"network-check-target-j6hld\" (UID: \"78565b9e-9521-4e9f-811e-6cc4cd76a4d3\") " pod="openshift-network-diagnostics/network-check-target-j6hld"
Jul 30 14:48:18.564699 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:18.564668    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 14:48:18.564699 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:18.564690    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 14:48:18.564699 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:18.564701    2578 projected.go:196] Error preparing data for projected volume kube-api-access-9rbmt for pod openshift-network-diagnostics/network-check-target-j6hld: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 14:48:18.564986 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:18.564763    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/78565b9e-9521-4e9f-811e-6cc4cd76a4d3-kube-api-access-9rbmt podName:78565b9e-9521-4e9f-811e-6cc4cd76a4d3 nodeName:}" failed. No retries permitted until 2026-07-30 14:48:26.564741967 +0000 UTC m=+18.093680421 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-9rbmt" (UniqueName: "kubernetes.io/projected/78565b9e-9521-4e9f-811e-6cc4cd76a4d3-kube-api-access-9rbmt") pod "network-check-target-j6hld" (UID: "78565b9e-9521-4e9f-811e-6cc4cd76a4d3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 14:48:18.565218 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:18.565127    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 14:48:18.565218 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:18.565188    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be2b73fa-10f0-4ea3-a68e-fa452e30914b-metrics-certs podName:be2b73fa-10f0-4ea3-a68e-fa452e30914b nodeName:}" failed. No retries permitted until 2026-07-30 14:48:26.565169295 +0000 UTC m=+18.094107770 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/be2b73fa-10f0-4ea3-a68e-fa452e30914b-metrics-certs") pod "network-metrics-daemon-rj8gr" (UID: "be2b73fa-10f0-4ea3-a68e-fa452e30914b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 14:48:18.665870 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:18.665826    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/963eac2b-b8af-497c-a206-0a1c976a0e2c-original-pull-secret\") pod \"global-pull-secret-syncer-9lgq9\" (UID: \"963eac2b-b8af-497c-a206-0a1c976a0e2c\") " pod="kube-system/global-pull-secret-syncer-9lgq9"
Jul 30 14:48:18.666065 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:18.665976    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 14:48:18.666065 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:18.666042    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/963eac2b-b8af-497c-a206-0a1c976a0e2c-original-pull-secret podName:963eac2b-b8af-497c-a206-0a1c976a0e2c nodeName:}" failed. No retries permitted until 2026-07-30 14:48:26.666022481 +0000 UTC m=+18.194960939 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/963eac2b-b8af-497c-a206-0a1c976a0e2c-original-pull-secret") pod "global-pull-secret-syncer-9lgq9" (UID: "963eac2b-b8af-497c-a206-0a1c976a0e2c") : object "kube-system"/"original-pull-secret" not registered
Jul 30 14:48:18.974292 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:18.974205    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j6hld"
Jul 30 14:48:18.974766 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:18.974352    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j6hld" podUID="78565b9e-9521-4e9f-811e-6cc4cd76a4d3"
Jul 30 14:48:19.972845 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:19.972810    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lgq9"
Jul 30 14:48:19.973051 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:19.972855    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rj8gr"
Jul 30 14:48:19.973051 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:19.972937    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lgq9" podUID="963eac2b-b8af-497c-a206-0a1c976a0e2c"
Jul 30 14:48:19.973129 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:19.973071    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rj8gr" podUID="be2b73fa-10f0-4ea3-a68e-fa452e30914b"
Jul 30 14:48:20.972708 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:20.972614    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j6hld"
Jul 30 14:48:20.973296 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:20.973235    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j6hld" podUID="78565b9e-9521-4e9f-811e-6cc4cd76a4d3"
Jul 30 14:48:21.972286 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:21.972240    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lgq9"
Jul 30 14:48:21.972479 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:21.972386    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lgq9" podUID="963eac2b-b8af-497c-a206-0a1c976a0e2c"
Jul 30 14:48:21.972560 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:21.972503    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rj8gr"
Jul 30 14:48:21.972686 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:21.972662    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rj8gr" podUID="be2b73fa-10f0-4ea3-a68e-fa452e30914b"
Jul 30 14:48:22.972572 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:22.972417    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j6hld"
Jul 30 14:48:22.972572 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:22.972551    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j6hld" podUID="78565b9e-9521-4e9f-811e-6cc4cd76a4d3"
Jul 30 14:48:23.972161 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:23.972125    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rj8gr"
Jul 30 14:48:23.972327 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:23.972169    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lgq9"
Jul 30 14:48:23.972327 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:23.972269    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rj8gr" podUID="be2b73fa-10f0-4ea3-a68e-fa452e30914b"
Jul 30 14:48:23.972433 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:23.972405    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lgq9" podUID="963eac2b-b8af-497c-a206-0a1c976a0e2c"
Jul 30 14:48:24.972969 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:24.972939    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j6hld"
Jul 30 14:48:24.973396 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:24.973057    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j6hld" podUID="78565b9e-9521-4e9f-811e-6cc4cd76a4d3"
Jul 30 14:48:25.972673 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:25.972640    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lgq9"
Jul 30 14:48:25.972897 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:25.972742    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rj8gr"
Jul 30 14:48:25.972897 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:25.972760    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lgq9" podUID="963eac2b-b8af-497c-a206-0a1c976a0e2c"
Jul 30 14:48:25.972897 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:25.972877    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rj8gr" podUID="be2b73fa-10f0-4ea3-a68e-fa452e30914b"
Jul 30 14:48:26.629133 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:26.629102    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rbmt\" (UniqueName: \"kubernetes.io/projected/78565b9e-9521-4e9f-811e-6cc4cd76a4d3-kube-api-access-9rbmt\") pod \"network-check-target-j6hld\" (UID: \"78565b9e-9521-4e9f-811e-6cc4cd76a4d3\") " pod="openshift-network-diagnostics/network-check-target-j6hld"
Jul 30 14:48:26.629411 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:26.629187    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be2b73fa-10f0-4ea3-a68e-fa452e30914b-metrics-certs\") pod \"network-metrics-daemon-rj8gr\" (UID: \"be2b73fa-10f0-4ea3-a68e-fa452e30914b\") " pod="openshift-multus/network-metrics-daemon-rj8gr"
Jul 30 14:48:26.629411 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:26.629262    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 14:48:26.629411 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:26.629284    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 14:48:26.629411 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:26.629284    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 14:48:26.629411 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:26.629294    2578 projected.go:196] Error preparing data for projected volume kube-api-access-9rbmt for pod openshift-network-diagnostics/network-check-target-j6hld: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 14:48:26.629411 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:26.629341    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be2b73fa-10f0-4ea3-a68e-fa452e30914b-metrics-certs podName:be2b73fa-10f0-4ea3-a68e-fa452e30914b nodeName:}" failed. No retries permitted until 2026-07-30 14:48:42.629325543 +0000 UTC m=+34.158264010 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/be2b73fa-10f0-4ea3-a68e-fa452e30914b-metrics-certs") pod "network-metrics-daemon-rj8gr" (UID: "be2b73fa-10f0-4ea3-a68e-fa452e30914b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 14:48:26.629411 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:26.629355    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/78565b9e-9521-4e9f-811e-6cc4cd76a4d3-kube-api-access-9rbmt podName:78565b9e-9521-4e9f-811e-6cc4cd76a4d3 nodeName:}" failed. No retries permitted until 2026-07-30 14:48:42.62934907 +0000 UTC m=+34.158287523 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-9rbmt" (UniqueName: "kubernetes.io/projected/78565b9e-9521-4e9f-811e-6cc4cd76a4d3-kube-api-access-9rbmt") pod "network-check-target-j6hld" (UID: "78565b9e-9521-4e9f-811e-6cc4cd76a4d3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 14:48:26.730184 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:26.730156    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/963eac2b-b8af-497c-a206-0a1c976a0e2c-original-pull-secret\") pod \"global-pull-secret-syncer-9lgq9\" (UID: \"963eac2b-b8af-497c-a206-0a1c976a0e2c\") " pod="kube-system/global-pull-secret-syncer-9lgq9"
Jul 30 14:48:26.730295 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:26.730278    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 14:48:26.730346 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:26.730338    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/963eac2b-b8af-497c-a206-0a1c976a0e2c-original-pull-secret podName:963eac2b-b8af-497c-a206-0a1c976a0e2c nodeName:}" failed. No retries permitted until 2026-07-30 14:48:42.730323654 +0000 UTC m=+34.259262105 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/963eac2b-b8af-497c-a206-0a1c976a0e2c-original-pull-secret") pod "global-pull-secret-syncer-9lgq9" (UID: "963eac2b-b8af-497c-a206-0a1c976a0e2c") : object "kube-system"/"original-pull-secret" not registered
Jul 30 14:48:26.972592 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:26.972510    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j6hld"
Jul 30 14:48:26.972755 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:26.972641    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j6hld" podUID="78565b9e-9521-4e9f-811e-6cc4cd76a4d3"
Jul 30 14:48:27.972333 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:27.972297    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lgq9"
Jul 30 14:48:27.972767 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:27.972297    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rj8gr"
Jul 30 14:48:27.972767 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:27.972442    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lgq9" podUID="963eac2b-b8af-497c-a206-0a1c976a0e2c"
Jul 30 14:48:27.972767 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:27.972526    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rj8gr" podUID="be2b73fa-10f0-4ea3-a68e-fa452e30914b"
Jul 30 14:48:28.973450 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:28.973414    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j6hld"
Jul 30 14:48:28.973893 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:28.973512    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j6hld" podUID="78565b9e-9521-4e9f-811e-6cc4cd76a4d3"
Jul 30 14:48:29.972727 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:29.972691    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lgq9"
Jul 30 14:48:29.972907 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:29.972819    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lgq9" podUID="963eac2b-b8af-497c-a206-0a1c976a0e2c"
Jul 30 14:48:29.972907 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:29.972880    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rj8gr"
Jul 30 14:48:29.973028 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:29.973004    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rj8gr" podUID="be2b73fa-10f0-4ea3-a68e-fa452e30914b"
Jul 30 14:48:30.973162 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:30.972845    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j6hld"
Jul 30 14:48:30.973937 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:30.973239    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j6hld" podUID="78565b9e-9521-4e9f-811e-6cc4cd76a4d3"
Jul 30 14:48:31.057683 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:31.057657    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xllmv" event={"ID":"89cc8684-f168-40cf-93ec-1143c2cb0784","Type":"ContainerStarted","Data":"cd63f2f5de985395a4fb07fa9c791c1f5366062b8f3f1c9c8ccdaf0d37d33790"}
Jul 30 14:48:31.057790 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:31.057765    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xllmv" event={"ID":"89cc8684-f168-40cf-93ec-1143c2cb0784","Type":"ContainerStarted","Data":"b944a188152387fca3baef2e30b0f0519ecdb2e5a33c97d190918b8b582abd0b"}
Jul 30 14:48:31.057857 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:31.057815    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xllmv" event={"ID":"89cc8684-f168-40cf-93ec-1143c2cb0784","Type":"ContainerStarted","Data":"418860beb0f231c214da6dc104724d1d3672029ee714e2b4088dafd4f679f7d4"}
Jul 30 14:48:31.057913 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:31.057863    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xllmv" event={"ID":"89cc8684-f168-40cf-93ec-1143c2cb0784","Type":"ContainerStarted","Data":"fa3368628f4e6c746051a2d5524c43ca18a34b23932c007ae3133977ca8731f9"}
Jul 30 14:48:31.057966 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:31.057914    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xllmv" event={"ID":"89cc8684-f168-40cf-93ec-1143c2cb0784","Type":"ContainerStarted","Data":"9221e1707ab701b9f7435493079a95da96be6ba68e40eeaa3bc9843ca79b4020"}
Jul 30 14:48:31.059161 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:31.059120    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hs9dl" event={"ID":"05f0cae0-8999-4a51-ae71-9e9dad066804","Type":"ContainerStarted","Data":"ebbf7636430cfe6e2f0c3ae5cf04c046b994f16c0dcbf9ccd4eee3e8cac016eb"}
Jul 30 14:48:31.060536 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:31.060507    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-sgpbt" event={"ID":"061ec575-e1af-4d7f-b40e-57258eaecf71","Type":"ContainerStarted","Data":"80c1865722a9f93ab44f5f0740e034a73a32e70853f7373d568fbedf37915c3f"}
Jul 30 14:48:31.061893 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:31.061869    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8jj48" event={"ID":"fc042c9a-bc2a-4cf4-8606-e9e20e462cb3","Type":"ContainerStarted","Data":"351583e24760ab4a7324e66acbae1be0296d7b0c4b571c15fb0dd777e64c2b4b"}
Jul 30 14:48:31.063196 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:31.063172    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cp7nq" event={"ID":"541aeca7-768f-41c8-8c0f-70663db887b2","Type":"ContainerStarted","Data":"41b7836a5c3b206e0265d23f2856ca623e46961d23f2bd158e4621d673c6a867"}
Jul 30 14:48:31.064636 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:31.064616    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9q72z" event={"ID":"d9b6c536-3a43-49a8-853d-f9532668f149","Type":"ContainerStarted","Data":"c945ca3bab0bd5d7588a9619829b51ab3f6851d6304e89a7d5da5a3d2177426c"}
Jul 30 14:48:31.066336 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:31.066317    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-t65nt" event={"ID":"b5b201dc-6dcd-44c8-9c70-9946315da836","Type":"ContainerStarted","Data":"d21fe7db37e6aac640cb034f17ef034d704919f86632477c61c977371e50fcfc"}
Jul 30 14:48:31.067671 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:31.067652    2578 generic.go:356] "Generic (PLEG): container finished" podID="b75a5fc6-d32b-4d96-a669-ff83a1abd706" containerID="cf87b24d003b1a3121c9fbc6aa885df0d9f702d346e52b29d04d38b61434c1e0" exitCode=0
Jul 30 14:48:31.067750 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:31.067681    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dkdwk" event={"ID":"b75a5fc6-d32b-4d96-a669-ff83a1abd706","Type":"ContainerDied","Data":"cf87b24d003b1a3121c9fbc6aa885df0d9f702d346e52b29d04d38b61434c1e0"}
Jul 30 14:48:31.077672 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:31.077634    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-sgpbt" podStartSLOduration=3.893922318 podStartE2EDuration="22.077621721s" totalImagesPullingTime="18.183699403s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 14:48:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:48:31.077279666 +0000 UTC m=+22.606218140" watchObservedRunningTime="2026-07-30 14:48:31.077621721 +0000 UTC m=+22.606560403"
Jul 30 14:48:31.077750 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:31.077734    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-117.ec2.internal" podStartSLOduration=20.077730286 podStartE2EDuration="20.077730286s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 14:48:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:48:14.034142658 +0000 UTC m=+5.563081136" watchObservedRunningTime="2026-07-30 14:48:31.077730286 +0000 UTC m=+22.606668759"
Jul 30 14:48:31.112366 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:31.112263    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-9q72z" podStartSLOduration=3.924899092 podStartE2EDuration="22.11224264s" totalImagesPullingTime="18.187343548s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 14:48:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:48:31.112119256 +0000 UTC m=+22.641057730" watchObservedRunningTime="2026-07-30 14:48:31.11224264 +0000 UTC m=+22.641181126"
Jul 30 14:48:31.126114 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:31.126058    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-t65nt" podStartSLOduration=3.937525971 podStartE2EDuration="22.126037092s" totalImagesPullingTime="18.188511121s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 14:48:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:48:31.125565492 +0000 UTC m=+22.654503967" watchObservedRunningTime="2026-07-30 14:48:31.126037092 +0000 UTC m=+22.654975568"
Jul 30 14:48:31.138909 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:31.138871    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-8jj48" podStartSLOduration=3.940736926 podStartE2EDuration="22.138857375s" totalImagesPullingTime="18.198120449s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 14:48:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:48:31.13832511 +0000 UTC m=+22.667263576" watchObservedRunningTime="2026-07-30 14:48:31.138857375 +0000 UTC m=+22.667795848"
Jul 30 14:48:31.153741 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:31.153704    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-cp7nq" podStartSLOduration=3.890367745 podStartE2EDuration="22.153690039s" totalImagesPullingTime="18.263322294s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 14:48:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:48:31.153315784 +0000 UTC m=+22.682254259" watchObservedRunningTime="2026-07-30 14:48:31.153690039 +0000 UTC m=+22.682628513"
Jul 30 14:48:31.548904 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:31.548864    2578 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 30 14:48:31.937704 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:31.937622    2578 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-30T14:48:31.548899845Z","UUID":"cc1fa972-d658-4959-8c3c-2d1f9d352b59","Handler":null,"Name":"","Endpoint":""}
Jul 30 14:48:31.938956 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:31.938937    2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 30 14:48:31.939086 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:31.938974    2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 30 14:48:31.972983 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:31.972959    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rj8gr"
Jul 30 14:48:31.973107 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:31.972959    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lgq9"
Jul 30 14:48:31.973107 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:31.973061    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rj8gr" podUID="be2b73fa-10f0-4ea3-a68e-fa452e30914b"
Jul 30 14:48:31.973199 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:31.973117    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lgq9" podUID="963eac2b-b8af-497c-a206-0a1c976a0e2c"
Jul 30 14:48:32.071345 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:32.071315    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vx6bx" event={"ID":"b9def45f-210b-4ceb-989c-6c696881a6f9","Type":"ContainerStarted","Data":"a712bc91a76a0c97b4bac5a0f9f0832334deb8fb3b1ed4bcdcb00fc6aae76919"}
Jul 30 14:48:32.074035 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:32.074007    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xllmv" event={"ID":"89cc8684-f168-40cf-93ec-1143c2cb0784","Type":"ContainerStarted","Data":"2522ec34e9a1adbc385e3f3f40b9746f06ae3e8492f6cffa3ac7fdd323838134"}
Jul 30 14:48:32.075644 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:32.075617    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hs9dl" event={"ID":"05f0cae0-8999-4a51-ae71-9e9dad066804","Type":"ContainerStarted","Data":"d2e2f73f646a0f0748fa5ed00e489f6f674a1d4f7741a7296ff4f7436cc721b7"}
Jul 30 14:48:32.086446 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:32.086404    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-vx6bx" podStartSLOduration=4.88943035 podStartE2EDuration="23.08639136s" totalImagesPullingTime="18.19696101s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 14:48:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:48:32.085836494 +0000 UTC m=+23.614774967" watchObservedRunningTime="2026-07-30 14:48:32.08639136 +0000 UTC m=+23.615329834"
Jul 30 14:48:32.094021 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:32.094000    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-t65nt"
Jul 30 14:48:32.094555 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:32.094538    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-t65nt"
Jul 30 14:48:32.972471 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:32.972438    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j6hld"
Jul 30 14:48:32.972722 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:32.972561    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j6hld" podUID="78565b9e-9521-4e9f-811e-6cc4cd76a4d3"
Jul 30 14:48:33.077658 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:33.077606    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-t65nt"
Jul 30 14:48:33.078134 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:33.077921    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-t65nt"
Jul 30 14:48:33.972642 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:33.972620    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lgq9"
Jul 30 14:48:33.972642 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:33.972637    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rj8gr"
Jul 30 14:48:33.972806 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:33.972729    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lgq9" podUID="963eac2b-b8af-497c-a206-0a1c976a0e2c"
Jul 30 14:48:33.972866 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:33.972838    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rj8gr" podUID="be2b73fa-10f0-4ea3-a68e-fa452e30914b"
Jul 30 14:48:34.083675 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:34.083642    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xllmv" event={"ID":"89cc8684-f168-40cf-93ec-1143c2cb0784","Type":"ContainerStarted","Data":"10ad98955a15a6ed7ad37d35ea20b30ffcb9219e2cd1883102572d4a679dcd86"}
Jul 30 14:48:34.086010 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:34.085978    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hs9dl" event={"ID":"05f0cae0-8999-4a51-ae71-9e9dad066804","Type":"ContainerStarted","Data":"36305ffa1bf9617ee43f7f84807ca9447b777d1c1792b65027163f3471afa3db"}
Jul 30 14:48:34.105031 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:34.104993    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hs9dl" podStartSLOduration=3.6176267749999997 podStartE2EDuration="25.104979123s" totalImagesPullingTime="21.487352348s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 14:48:09 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:48:34.104744026 +0000 UTC m=+25.633682512" watchObservedRunningTime="2026-07-30 14:48:34.104979123 +0000 UTC m=+25.633917597"
Jul 30 14:48:34.972754 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:34.972721    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j6hld"
Jul 30 14:48:34.972962 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:34.972842    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j6hld" podUID="78565b9e-9521-4e9f-811e-6cc4cd76a4d3"
Jul 30 14:48:35.973192 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:35.973142    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lgq9"
Jul 30 14:48:35.973804 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:35.973287    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lgq9" podUID="963eac2b-b8af-497c-a206-0a1c976a0e2c"
Jul 30 14:48:35.973804 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:35.973306    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rj8gr"
Jul 30 14:48:35.973804 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:35.973421    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rj8gr" podUID="be2b73fa-10f0-4ea3-a68e-fa452e30914b"
Jul 30 14:48:36.973419 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:36.973203    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j6hld"
Jul 30 14:48:36.973915 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:36.973435    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j6hld" podUID="78565b9e-9521-4e9f-811e-6cc4cd76a4d3"
Jul 30 14:48:37.095574 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:37.095547    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xllmv" event={"ID":"89cc8684-f168-40cf-93ec-1143c2cb0784","Type":"ContainerStarted","Data":"294075f0e28668705380459eb6b50ca5ce2ecbb0922844921957e0910e7f8a96"}
Jul 30 14:48:37.095857 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:37.095837    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:37.095951 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:37.095899    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:37.110490 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:37.110470    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:37.120383 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:37.120346    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-xllmv" podStartSLOduration=9.856030602 podStartE2EDuration="28.120334411s" totalImagesPullingTime="18.264303809s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 14:48:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:48:37.119575761 +0000 UTC m=+28.648514234" watchObservedRunningTime="2026-07-30 14:48:37.120334411 +0000 UTC m=+28.649272884"
Jul 30 14:48:37.973945 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:37.973206    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lgq9"
Jul 30 14:48:37.973945 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:37.973343    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lgq9" podUID="963eac2b-b8af-497c-a206-0a1c976a0e2c"
Jul 30 14:48:37.973945 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:37.973793    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rj8gr"
Jul 30 14:48:37.973945 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:37.973903    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rj8gr" podUID="be2b73fa-10f0-4ea3-a68e-fa452e30914b"
Jul 30 14:48:38.099431 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:38.099391    2578 generic.go:356] "Generic (PLEG): container finished" podID="b75a5fc6-d32b-4d96-a669-ff83a1abd706" containerID="1fed14f77784680ad06780060039fccfebed2fec820457def724b9919fd47be5" exitCode=0
Jul 30 14:48:38.099580 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:38.099453    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dkdwk" event={"ID":"b75a5fc6-d32b-4d96-a669-ff83a1abd706","Type":"ContainerDied","Data":"1fed14f77784680ad06780060039fccfebed2fec820457def724b9919fd47be5"}
Jul 30 14:48:38.099905 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:38.099884    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:38.114406 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:38.114384    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:48:38.815945 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:38.815907    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9lgq9"]
Jul 30 14:48:38.816133 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:38.816019    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lgq9"
Jul 30 14:48:38.816133 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:38.816108    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lgq9" podUID="963eac2b-b8af-497c-a206-0a1c976a0e2c"
Jul 30 14:48:38.819592 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:38.819565    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rj8gr"]
Jul 30 14:48:38.819710 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:38.819662    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rj8gr"
Jul 30 14:48:38.819778 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:38.819757    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rj8gr" podUID="be2b73fa-10f0-4ea3-a68e-fa452e30914b"
Jul 30 14:48:38.820432 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:38.820405    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-j6hld"]
Jul 30 14:48:38.820665 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:38.820648    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j6hld"
Jul 30 14:48:38.820860 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:38.820839    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j6hld" podUID="78565b9e-9521-4e9f-811e-6cc4cd76a4d3"
Jul 30 14:48:39.972613 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:39.972461    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j6hld"
Jul 30 14:48:39.972995 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:39.972689    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j6hld" podUID="78565b9e-9521-4e9f-811e-6cc4cd76a4d3"
Jul 30 14:48:40.105378 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:40.105349    2578 generic.go:356] "Generic (PLEG): container finished" podID="b75a5fc6-d32b-4d96-a669-ff83a1abd706" containerID="bfa50e3360fa4eae57d01f1c0a45ee3e66917bcf0bc9db2f17dc64170e3a56e6" exitCode=0
Jul 30 14:48:40.105505 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:40.105418    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dkdwk" event={"ID":"b75a5fc6-d32b-4d96-a669-ff83a1abd706","Type":"ContainerDied","Data":"bfa50e3360fa4eae57d01f1c0a45ee3e66917bcf0bc9db2f17dc64170e3a56e6"}
Jul 30 14:48:40.972475 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:40.972441    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rj8gr"
Jul 30 14:48:40.972644 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:40.972550    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rj8gr" podUID="be2b73fa-10f0-4ea3-a68e-fa452e30914b"
Jul 30 14:48:40.972644 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:40.972606    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lgq9"
Jul 30 14:48:40.973309 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:40.972696    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lgq9" podUID="963eac2b-b8af-497c-a206-0a1c976a0e2c"
Jul 30 14:48:41.973142 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:41.973112    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j6hld"
Jul 30 14:48:41.973486 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:41.973218    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j6hld" podUID="78565b9e-9521-4e9f-811e-6cc4cd76a4d3"
Jul 30 14:48:42.112202 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:42.112163    2578 generic.go:356] "Generic (PLEG): container finished" podID="b75a5fc6-d32b-4d96-a669-ff83a1abd706" containerID="92a5dbf751cfdd49b0cc4fc90f1f40c85c7f20f27f2279dd7137220c28f87277" exitCode=0
Jul 30 14:48:42.112202 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:42.112204    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dkdwk" event={"ID":"b75a5fc6-d32b-4d96-a669-ff83a1abd706","Type":"ContainerDied","Data":"92a5dbf751cfdd49b0cc4fc90f1f40c85c7f20f27f2279dd7137220c28f87277"}
Jul 30 14:48:42.650321 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:42.650291    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rbmt\" (UniqueName: \"kubernetes.io/projected/78565b9e-9521-4e9f-811e-6cc4cd76a4d3-kube-api-access-9rbmt\") pod \"network-check-target-j6hld\" (UID: \"78565b9e-9521-4e9f-811e-6cc4cd76a4d3\") " pod="openshift-network-diagnostics/network-check-target-j6hld"
Jul 30 14:48:42.650490 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:42.650343    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be2b73fa-10f0-4ea3-a68e-fa452e30914b-metrics-certs\") pod \"network-metrics-daemon-rj8gr\" (UID: \"be2b73fa-10f0-4ea3-a68e-fa452e30914b\") " pod="openshift-multus/network-metrics-daemon-rj8gr"
Jul 30 14:48:42.650490 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:42.650423    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 14:48:42.650490 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:42.650441    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 14:48:42.650490 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:42.650468    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 14:48:42.650490 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:42.650478    2578 projected.go:196] Error preparing data for projected volume kube-api-access-9rbmt for pod openshift-network-diagnostics/network-check-target-j6hld: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 14:48:42.650665 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:42.650469    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be2b73fa-10f0-4ea3-a68e-fa452e30914b-metrics-certs podName:be2b73fa-10f0-4ea3-a68e-fa452e30914b nodeName:}" failed. No retries permitted until 2026-07-30 14:49:14.650456354 +0000 UTC m=+66.179394805 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/be2b73fa-10f0-4ea3-a68e-fa452e30914b-metrics-certs") pod "network-metrics-daemon-rj8gr" (UID: "be2b73fa-10f0-4ea3-a68e-fa452e30914b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 14:48:42.650665 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:42.650523    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/78565b9e-9521-4e9f-811e-6cc4cd76a4d3-kube-api-access-9rbmt podName:78565b9e-9521-4e9f-811e-6cc4cd76a4d3 nodeName:}" failed. No retries permitted until 2026-07-30 14:49:14.650511889 +0000 UTC m=+66.179450341 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-9rbmt" (UniqueName: "kubernetes.io/projected/78565b9e-9521-4e9f-811e-6cc4cd76a4d3-kube-api-access-9rbmt") pod "network-check-target-j6hld" (UID: "78565b9e-9521-4e9f-811e-6cc4cd76a4d3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 14:48:42.750759 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:42.750731    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/963eac2b-b8af-497c-a206-0a1c976a0e2c-original-pull-secret\") pod \"global-pull-secret-syncer-9lgq9\" (UID: \"963eac2b-b8af-497c-a206-0a1c976a0e2c\") " pod="kube-system/global-pull-secret-syncer-9lgq9"
Jul 30 14:48:42.750913 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:42.750863    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 14:48:42.750953 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:42.750913    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/963eac2b-b8af-497c-a206-0a1c976a0e2c-original-pull-secret podName:963eac2b-b8af-497c-a206-0a1c976a0e2c nodeName:}" failed. No retries permitted until 2026-07-30 14:49:14.750900577 +0000 UTC m=+66.279839033 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/963eac2b-b8af-497c-a206-0a1c976a0e2c-original-pull-secret") pod "global-pull-secret-syncer-9lgq9" (UID: "963eac2b-b8af-497c-a206-0a1c976a0e2c") : object "kube-system"/"original-pull-secret" not registered
Jul 30 14:48:42.973032 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:42.972953    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rj8gr"
Jul 30 14:48:42.973032 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:42.972974    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lgq9"
Jul 30 14:48:42.973587 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:42.973073    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rj8gr" podUID="be2b73fa-10f0-4ea3-a68e-fa452e30914b"
Jul 30 14:48:42.973587 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:42.973230    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lgq9" podUID="963eac2b-b8af-497c-a206-0a1c976a0e2c"
Jul 30 14:48:43.972370 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:43.972339    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j6hld"
Jul 30 14:48:43.972539 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:43.972450    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j6hld" podUID="78565b9e-9521-4e9f-811e-6cc4cd76a4d3"
Jul 30 14:48:44.245238 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:44.245163    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-117.ec2.internal" event="NodeReady"
Jul 30 14:48:44.245818 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:44.245324    2578 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 30 14:48:44.294313 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:44.294281    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-nzzqf"]
Jul 30 14:48:44.317548 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:44.317521    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-frg92"]
Jul 30 14:48:44.317724 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:44.317664    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nzzqf"
Jul 30 14:48:44.319552 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:44.319527    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 30 14:48:44.319552 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:44.319549    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 30 14:48:44.319905 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:44.319889    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vrbw2\""
Jul 30 14:48:44.332060 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:44.332035    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nzzqf"]
Jul 30 14:48:44.332196 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:44.332079    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-frg92"]
Jul 30 14:48:44.332196 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:44.332160    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-frg92"
Jul 30 14:48:44.336463 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:44.336442    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 30 14:48:44.336707 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:44.336691    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 30 14:48:44.336977 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:44.336954    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-dcx9h\""
Jul 30 14:48:44.337100 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:44.337004    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 30 14:48:44.463475 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:44.463444    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8nvz\" (UniqueName: \"kubernetes.io/projected/8ce55aa3-7792-4ea0-83ed-64890b613632-kube-api-access-r8nvz\") pod \"ingress-canary-frg92\" (UID: \"8ce55aa3-7792-4ea0-83ed-64890b613632\") " pod="openshift-ingress-canary/ingress-canary-frg92"
Jul 30 14:48:44.463646 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:44.463507    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1-config-volume\") pod \"dns-default-nzzqf\" (UID: \"64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1\") " pod="openshift-dns/dns-default-nzzqf"
Jul 30 14:48:44.463646 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:44.463598    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8ce55aa3-7792-4ea0-83ed-64890b613632-cert\") pod \"ingress-canary-frg92\" (UID: \"8ce55aa3-7792-4ea0-83ed-64890b613632\") " pod="openshift-ingress-canary/ingress-canary-frg92"
Jul 30 14:48:44.463646 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:44.463632    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1-metrics-tls\") pod \"dns-default-nzzqf\" (UID: \"64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1\") " pod="openshift-dns/dns-default-nzzqf"
Jul 30 14:48:44.463813 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:44.463661    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zc2nj\" (UniqueName: \"kubernetes.io/projected/64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1-kube-api-access-zc2nj\") pod \"dns-default-nzzqf\" (UID: \"64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1\") " pod="openshift-dns/dns-default-nzzqf"
Jul 30 14:48:44.463813 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:44.463703    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1-tmp-dir\") pod \"dns-default-nzzqf\" (UID: \"64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1\") " pod="openshift-dns/dns-default-nzzqf"
Jul 30 14:48:44.564764 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:44.564733    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8nvz\" (UniqueName: \"kubernetes.io/projected/8ce55aa3-7792-4ea0-83ed-64890b613632-kube-api-access-r8nvz\") pod \"ingress-canary-frg92\" (UID: \"8ce55aa3-7792-4ea0-83ed-64890b613632\") " pod="openshift-ingress-canary/ingress-canary-frg92"
Jul 30 14:48:44.565011 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:44.564804    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1-config-volume\") pod \"dns-default-nzzqf\" (UID: \"64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1\") " pod="openshift-dns/dns-default-nzzqf"
Jul 30 14:48:44.565011 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:44.564845    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8ce55aa3-7792-4ea0-83ed-64890b613632-cert\") pod \"ingress-canary-frg92\" (UID: \"8ce55aa3-7792-4ea0-83ed-64890b613632\") " pod="openshift-ingress-canary/ingress-canary-frg92"
Jul 30 14:48:44.565011 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:44.564867    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1-metrics-tls\") pod \"dns-default-nzzqf\" (UID: \"64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1\") " pod="openshift-dns/dns-default-nzzqf"
Jul 30 14:48:44.565011 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:44.564894    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zc2nj\" (UniqueName: \"kubernetes.io/projected/64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1-kube-api-access-zc2nj\") pod \"dns-default-nzzqf\" (UID: \"64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1\") " pod="openshift-dns/dns-default-nzzqf"
Jul 30 14:48:44.565011 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:44.564925    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1-tmp-dir\") pod \"dns-default-nzzqf\" (UID: \"64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1\") " pod="openshift-dns/dns-default-nzzqf"
Jul 30 14:48:44.565358 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:44.565306    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1-tmp-dir\") pod \"dns-default-nzzqf\" (UID: \"64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1\") " pod="openshift-dns/dns-default-nzzqf"
Jul 30 14:48:44.565569 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:44.565543    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 14:48:44.565687 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:44.565637    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1-metrics-tls podName:64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1 nodeName:}" failed. No retries permitted until 2026-07-30 14:48:45.065614942 +0000 UTC m=+36.594553397 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1-metrics-tls") pod "dns-default-nzzqf" (UID: "64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1") : secret "dns-default-metrics-tls" not found
Jul 30 14:48:44.566041 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:44.565993    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1-config-volume\") pod \"dns-default-nzzqf\" (UID: \"64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1\") " pod="openshift-dns/dns-default-nzzqf"
Jul 30 14:48:44.570223 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:44.570196    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8ce55aa3-7792-4ea0-83ed-64890b613632-cert\") pod \"ingress-canary-frg92\" (UID: \"8ce55aa3-7792-4ea0-83ed-64890b613632\") " pod="openshift-ingress-canary/ingress-canary-frg92"
Jul 30 14:48:44.573836 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:44.573817    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zc2nj\" (UniqueName: \"kubernetes.io/projected/64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1-kube-api-access-zc2nj\") pod \"dns-default-nzzqf\" (UID: \"64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1\") " pod="openshift-dns/dns-default-nzzqf"
Jul 30 14:48:44.574019 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:44.574002    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8nvz\" (UniqueName: \"kubernetes.io/projected/8ce55aa3-7792-4ea0-83ed-64890b613632-kube-api-access-r8nvz\") pod \"ingress-canary-frg92\" (UID: \"8ce55aa3-7792-4ea0-83ed-64890b613632\") " pod="openshift-ingress-canary/ingress-canary-frg92"
Jul 30 14:48:44.641982 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:44.641948    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-frg92"
Jul 30 14:48:44.866120 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:44.865399    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-frg92"]
Jul 30 14:48:44.868765 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:48:44.868734    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ce55aa3_7792_4ea0_83ed_64890b613632.slice/crio-ef50636a5d72d89db142418bc72747dbbd17766ee7c4a397230c3551661aee04 WatchSource:0}: Error finding container ef50636a5d72d89db142418bc72747dbbd17766ee7c4a397230c3551661aee04: Status 404 returned error can't find the container with id ef50636a5d72d89db142418bc72747dbbd17766ee7c4a397230c3551661aee04
Jul 30 14:48:44.972776 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:44.972732    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rj8gr"
Jul 30 14:48:44.972998 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:44.972945    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lgq9"
Jul 30 14:48:44.975201 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:44.975181    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 30 14:48:44.975201 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:44.975200    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 30 14:48:44.975445 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:44.975429    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-82hs8\""
Jul 30 14:48:45.070765 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:45.070727    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1-metrics-tls\") pod \"dns-default-nzzqf\" (UID: \"64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1\") " pod="openshift-dns/dns-default-nzzqf"
Jul 30 14:48:45.070927 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:45.070903    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 14:48:45.070992 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:45.070980    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1-metrics-tls podName:64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1 nodeName:}" failed. No retries permitted until 2026-07-30 14:48:46.07095928 +0000 UTC m=+37.599897739 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1-metrics-tls") pod "dns-default-nzzqf" (UID: "64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1") : secret "dns-default-metrics-tls" not found
Jul 30 14:48:45.121094 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:45.121019    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-frg92" event={"ID":"8ce55aa3-7792-4ea0-83ed-64890b613632","Type":"ContainerStarted","Data":"ef50636a5d72d89db142418bc72747dbbd17766ee7c4a397230c3551661aee04"}
Jul 30 14:48:45.972551 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:45.972515    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j6hld"
Jul 30 14:48:45.974898 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:45.974874    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 30 14:48:45.975257 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:45.975233    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 30 14:48:45.975363 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:45.975245    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dd6sm\""
Jul 30 14:48:46.077772 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:46.077737    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1-metrics-tls\") pod \"dns-default-nzzqf\" (UID: \"64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1\") " pod="openshift-dns/dns-default-nzzqf"
Jul 30 14:48:46.077943 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:46.077916    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 14:48:46.078014 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:46.078003    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1-metrics-tls podName:64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1 nodeName:}" failed. No retries permitted until 2026-07-30 14:48:48.077982065 +0000 UTC m=+39.606920521 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1-metrics-tls") pod "dns-default-nzzqf" (UID: "64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1") : secret "dns-default-metrics-tls" not found
Jul 30 14:48:47.132482 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:47.132446    2578 generic.go:356] "Generic (PLEG): container finished" podID="b75a5fc6-d32b-4d96-a669-ff83a1abd706" containerID="f013b59709ded1f1248a2f0cfa0dd49e6d804ba11f0ea61761e6ff2a4acdba83" exitCode=0
Jul 30 14:48:47.133016 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:47.132512    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dkdwk" event={"ID":"b75a5fc6-d32b-4d96-a669-ff83a1abd706","Type":"ContainerDied","Data":"f013b59709ded1f1248a2f0cfa0dd49e6d804ba11f0ea61761e6ff2a4acdba83"}
Jul 30 14:48:48.093686 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:48.093649    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1-metrics-tls\") pod \"dns-default-nzzqf\" (UID: \"64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1\") " pod="openshift-dns/dns-default-nzzqf"
Jul 30 14:48:48.093846 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:48.093819    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 14:48:48.093905 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:48.093894    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1-metrics-tls podName:64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1 nodeName:}" failed. No retries permitted until 2026-07-30 14:48:52.093877194 +0000 UTC m=+43.622815664 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1-metrics-tls") pod "dns-default-nzzqf" (UID: "64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1") : secret "dns-default-metrics-tls" not found
Jul 30 14:48:48.138360 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:48.138326    2578 generic.go:356] "Generic (PLEG): container finished" podID="b75a5fc6-d32b-4d96-a669-ff83a1abd706" containerID="0c5d0572911ff5697d50842969aba74ac4ae0ecfe3fae44a433675742de0df24" exitCode=0
Jul 30 14:48:48.138802 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:48.138389    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dkdwk" event={"ID":"b75a5fc6-d32b-4d96-a669-ff83a1abd706","Type":"ContainerDied","Data":"0c5d0572911ff5697d50842969aba74ac4ae0ecfe3fae44a433675742de0df24"}
Jul 30 14:48:49.144115 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:49.143864    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-frg92" event={"ID":"8ce55aa3-7792-4ea0-83ed-64890b613632","Type":"ContainerStarted","Data":"bdd8f9eedecc7acd5762ef1f37d520745525ac4688ed527001abcbac57329f18"}
Jul 30 14:48:49.147479 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:49.147454    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dkdwk" event={"ID":"b75a5fc6-d32b-4d96-a669-ff83a1abd706","Type":"ContainerStarted","Data":"4480c058cff06ae09533ee5ff5bc8fcf014ec8c42902ad818fa3ad4fe846c788"}
Jul 30 14:48:49.159224 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:49.159133    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-frg92" podStartSLOduration=1.132358749 podStartE2EDuration="5.159118313s" totalImagesPullingTime="4.026759564s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 14:48:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:48:49.158277059 +0000 UTC m=+40.687215532" watchObservedRunningTime="2026-07-30 14:48:49.159118313 +0000 UTC m=+40.688056800"
Jul 30 14:48:49.180195 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:49.180117    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-dkdwk" podStartSLOduration=9.099335094 podStartE2EDuration="40.180101533s" totalImagesPullingTime="31.080766439s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 14:48:09 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:48:49.178714261 +0000 UTC m=+40.707652736" watchObservedRunningTime="2026-07-30 14:48:49.180101533 +0000 UTC m=+40.709040006"
Jul 30 14:48:52.121058 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:48:52.121018    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1-metrics-tls\") pod \"dns-default-nzzqf\" (UID: \"64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1\") " pod="openshift-dns/dns-default-nzzqf"
Jul 30 14:48:52.121478 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:52.121182    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 14:48:52.121478 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:48:52.121247    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1-metrics-tls podName:64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1 nodeName:}" failed. No retries permitted until 2026-07-30 14:49:00.121231614 +0000 UTC m=+51.650170065 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1-metrics-tls") pod "dns-default-nzzqf" (UID: "64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1") : secret "dns-default-metrics-tls" not found
Jul 30 14:49:00.176528 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:00.176497    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1-metrics-tls\") pod \"dns-default-nzzqf\" (UID: \"64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1\") " pod="openshift-dns/dns-default-nzzqf"
Jul 30 14:49:00.176974 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:49:00.176610    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 14:49:00.176974 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:49:00.176663    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1-metrics-tls podName:64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1 nodeName:}" failed. No retries permitted until 2026-07-30 14:49:16.176649754 +0000 UTC m=+67.705588205 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1-metrics-tls") pod "dns-default-nzzqf" (UID: "64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1") : secret "dns-default-metrics-tls" not found
Jul 30 14:49:01.058222 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.058187    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-9shzj"]
Jul 30 14:49:01.060184 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.060168    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-4mjmp"]
Jul 30 14:49:01.060299 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.060282    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-9shzj"
Jul 30 14:49:01.061802 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.061782    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-4mjmp"
Jul 30 14:49:01.062401 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.062378    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-z6lxz\""
Jul 30 14:49:01.063662 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.063646    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 30 14:49:01.064242 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.064225    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 30 14:49:01.064523 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.064503    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 30 14:49:01.064700 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.064684    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 30 14:49:01.064936 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.064895    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-5f2n2\""
Jul 30 14:49:01.069382 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.069366    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 30 14:49:01.072354 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.072334    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-9shzj"]
Jul 30 14:49:01.073253 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.073231    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-4mjmp"]
Jul 30 14:49:01.084138 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.084118    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58qs8\" (UniqueName: \"kubernetes.io/projected/de241220-5f27-4b0b-b7c6-6d4b350e24e1-kube-api-access-58qs8\") pod \"console-operator-584dcb6b54-4mjmp\" (UID: \"de241220-5f27-4b0b-b7c6-6d4b350e24e1\") " pod="openshift-console-operator/console-operator-584dcb6b54-4mjmp"
Jul 30 14:49:01.084239 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.084203    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sm2g8\" (UniqueName: \"kubernetes.io/projected/ea7a4fd9-ee55-43ed-b848-d6bd1f0e5e3a-kube-api-access-sm2g8\") pod \"network-check-source-7d88cb55d8-9shzj\" (UID: \"ea7a4fd9-ee55-43ed-b848-d6bd1f0e5e3a\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-9shzj"
Jul 30 14:49:01.084239 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.084225    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/de241220-5f27-4b0b-b7c6-6d4b350e24e1-config\") pod \"console-operator-584dcb6b54-4mjmp\" (UID: \"de241220-5f27-4b0b-b7c6-6d4b350e24e1\") " pod="openshift-console-operator/console-operator-584dcb6b54-4mjmp"
Jul 30 14:49:01.084312 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.084243    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/de241220-5f27-4b0b-b7c6-6d4b350e24e1-serving-cert\") pod \"console-operator-584dcb6b54-4mjmp\" (UID: \"de241220-5f27-4b0b-b7c6-6d4b350e24e1\") " pod="openshift-console-operator/console-operator-584dcb6b54-4mjmp"
Jul 30 14:49:01.084312 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.084276    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/de241220-5f27-4b0b-b7c6-6d4b350e24e1-trusted-ca\") pod \"console-operator-584dcb6b54-4mjmp\" (UID: \"de241220-5f27-4b0b-b7c6-6d4b350e24e1\") " pod="openshift-console-operator/console-operator-584dcb6b54-4mjmp"
Jul 30 14:49:01.160764 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.160731    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-684cf4bf4d-2j296"]
Jul 30 14:49:01.162707 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.162693    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-684cf4bf4d-2j296"
Jul 30 14:49:01.166494 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.166468    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 30 14:49:01.166615 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.166515    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 30 14:49:01.166615 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.166556    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 30 14:49:01.166983 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.166963    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-fxhsd\""
Jul 30 14:49:01.178893 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.178876    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 30 14:49:01.185624 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.185605    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58qs8\" (UniqueName: \"kubernetes.io/projected/de241220-5f27-4b0b-b7c6-6d4b350e24e1-kube-api-access-58qs8\") pod \"console-operator-584dcb6b54-4mjmp\" (UID: \"de241220-5f27-4b0b-b7c6-6d4b350e24e1\") " pod="openshift-console-operator/console-operator-584dcb6b54-4mjmp"
Jul 30 14:49:01.185699 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.185639    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-trusted-ca\") pod \"image-registry-684cf4bf4d-2j296\" (UID: \"6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7\") " pod="openshift-image-registry/image-registry-684cf4bf4d-2j296"
Jul 30 14:49:01.185699 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.185680    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sm2g8\" (UniqueName: \"kubernetes.io/projected/ea7a4fd9-ee55-43ed-b848-d6bd1f0e5e3a-kube-api-access-sm2g8\") pod \"network-check-source-7d88cb55d8-9shzj\" (UID: \"ea7a4fd9-ee55-43ed-b848-d6bd1f0e5e3a\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-9shzj"
Jul 30 14:49:01.185699 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.185696    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/de241220-5f27-4b0b-b7c6-6d4b350e24e1-config\") pod \"console-operator-584dcb6b54-4mjmp\" (UID: \"de241220-5f27-4b0b-b7c6-6d4b350e24e1\") " pod="openshift-console-operator/console-operator-584dcb6b54-4mjmp"
Jul 30 14:49:01.185884 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.185711    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/de241220-5f27-4b0b-b7c6-6d4b350e24e1-serving-cert\") pod \"console-operator-584dcb6b54-4mjmp\" (UID: \"de241220-5f27-4b0b-b7c6-6d4b350e24e1\") " pod="openshift-console-operator/console-operator-584dcb6b54-4mjmp"
Jul 30 14:49:01.185884 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.185726    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-installation-pull-secrets\") pod \"image-registry-684cf4bf4d-2j296\" (UID: \"6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7\") " pod="openshift-image-registry/image-registry-684cf4bf4d-2j296"
Jul 30 14:49:01.185884 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.185743    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/de241220-5f27-4b0b-b7c6-6d4b350e24e1-trusted-ca\") pod \"console-operator-584dcb6b54-4mjmp\" (UID: \"de241220-5f27-4b0b-b7c6-6d4b350e24e1\") " pod="openshift-console-operator/console-operator-584dcb6b54-4mjmp"
Jul 30 14:49:01.185884 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.185766    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-ca-trust-extracted\") pod \"image-registry-684cf4bf4d-2j296\" (UID: \"6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7\") " pod="openshift-image-registry/image-registry-684cf4bf4d-2j296"
Jul 30 14:49:01.185884 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.185787    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-image-registry-private-configuration\") pod \"image-registry-684cf4bf4d-2j296\" (UID: \"6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7\") " pod="openshift-image-registry/image-registry-684cf4bf4d-2j296"
Jul 30 14:49:01.185884 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.185811    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-bound-sa-token\") pod \"image-registry-684cf4bf4d-2j296\" (UID: \"6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7\") " pod="openshift-image-registry/image-registry-684cf4bf4d-2j296"
Jul 30 14:49:01.185884 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.185834    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-registry-certificates\") pod \"image-registry-684cf4bf4d-2j296\" (UID: \"6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7\") " pod="openshift-image-registry/image-registry-684cf4bf4d-2j296"
Jul 30 14:49:01.185884 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.185852    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-registry-tls\") pod \"image-registry-684cf4bf4d-2j296\" (UID: \"6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7\") " pod="openshift-image-registry/image-registry-684cf4bf4d-2j296"
Jul 30 14:49:01.185884 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.185871    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqjs4\" (UniqueName: \"kubernetes.io/projected/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-kube-api-access-wqjs4\") pod \"image-registry-684cf4bf4d-2j296\" (UID: \"6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7\") " pod="openshift-image-registry/image-registry-684cf4bf4d-2j296"
Jul 30 14:49:01.186452 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.186432    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/de241220-5f27-4b0b-b7c6-6d4b350e24e1-config\") pod \"console-operator-584dcb6b54-4mjmp\" (UID: \"de241220-5f27-4b0b-b7c6-6d4b350e24e1\") " pod="openshift-console-operator/console-operator-584dcb6b54-4mjmp"
Jul 30 14:49:01.186645 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.186624    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/de241220-5f27-4b0b-b7c6-6d4b350e24e1-trusted-ca\") pod \"console-operator-584dcb6b54-4mjmp\" (UID: \"de241220-5f27-4b0b-b7c6-6d4b350e24e1\") " pod="openshift-console-operator/console-operator-584dcb6b54-4mjmp"
Jul 30 14:49:01.189373 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.189357    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/de241220-5f27-4b0b-b7c6-6d4b350e24e1-serving-cert\") pod \"console-operator-584dcb6b54-4mjmp\" (UID: \"de241220-5f27-4b0b-b7c6-6d4b350e24e1\") " pod="openshift-console-operator/console-operator-584dcb6b54-4mjmp"
Jul 30 14:49:01.191506 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.191483    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-684cf4bf4d-2j296"]
Jul 30 14:49:01.203707 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.203684    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-58qs8\" (UniqueName: \"kubernetes.io/projected/de241220-5f27-4b0b-b7c6-6d4b350e24e1-kube-api-access-58qs8\") pod \"console-operator-584dcb6b54-4mjmp\" (UID: \"de241220-5f27-4b0b-b7c6-6d4b350e24e1\") " pod="openshift-console-operator/console-operator-584dcb6b54-4mjmp"
Jul 30 14:49:01.204385 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.204359    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sm2g8\" (UniqueName: \"kubernetes.io/projected/ea7a4fd9-ee55-43ed-b848-d6bd1f0e5e3a-kube-api-access-sm2g8\") pod \"network-check-source-7d88cb55d8-9shzj\" (UID: \"ea7a4fd9-ee55-43ed-b848-d6bd1f0e5e3a\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-9shzj"
Jul 30 14:49:01.256619 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.256590    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-zqcwj"]
Jul 30 14:49:01.258497 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.258483    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-zqcwj"
Jul 30 14:49:01.260511 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.260489    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 30 14:49:01.260612 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.260530    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 30 14:49:01.260612 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.260498    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 30 14:49:01.260612 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.260498    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-6n5w5\""
Jul 30 14:49:01.260612 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.260569    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 30 14:49:01.265643 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.265620    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 30 14:49:01.268739 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.268717    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-zqcwj"]
Jul 30 14:49:01.286268 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.286247    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-trusted-ca\") pod \"image-registry-684cf4bf4d-2j296\" (UID: \"6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7\") " pod="openshift-image-registry/image-registry-684cf4bf4d-2j296"
Jul 30 14:49:01.286375 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.286314    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-installation-pull-secrets\") pod \"image-registry-684cf4bf4d-2j296\" (UID: \"6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7\") " pod="openshift-image-registry/image-registry-684cf4bf4d-2j296"
Jul 30 14:49:01.286375 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.286358    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-ca-trust-extracted\") pod \"image-registry-684cf4bf4d-2j296\" (UID: \"6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7\") " pod="openshift-image-registry/image-registry-684cf4bf4d-2j296"
Jul 30 14:49:01.286488 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.286394    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f07334b-dd72-4771-b26f-bb23fc9a0ada-service-ca-bundle\") pod \"insights-operator-5885fdb759-zqcwj\" (UID: \"1f07334b-dd72-4771-b26f-bb23fc9a0ada\") " pod="openshift-insights/insights-operator-5885fdb759-zqcwj"
Jul 30 14:49:01.286488 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.286424    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-image-registry-private-configuration\") pod \"image-registry-684cf4bf4d-2j296\" (UID: \"6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7\") " pod="openshift-image-registry/image-registry-684cf4bf4d-2j296"
Jul 30 14:49:01.286488 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.286449    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-bound-sa-token\") pod \"image-registry-684cf4bf4d-2j296\" (UID: \"6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7\") " pod="openshift-image-registry/image-registry-684cf4bf4d-2j296"
Jul 30 14:49:01.286488 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.286475    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1f07334b-dd72-4771-b26f-bb23fc9a0ada-tmp\") pod \"insights-operator-5885fdb759-zqcwj\" (UID: \"1f07334b-dd72-4771-b26f-bb23fc9a0ada\") " pod="openshift-insights/insights-operator-5885fdb759-zqcwj"
Jul 30 14:49:01.286682 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.286498    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f07334b-dd72-4771-b26f-bb23fc9a0ada-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-zqcwj\" (UID: \"1f07334b-dd72-4771-b26f-bb23fc9a0ada\") " pod="openshift-insights/insights-operator-5885fdb759-zqcwj"
Jul 30 14:49:01.286682 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.286524    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pdzm\" (UniqueName: \"kubernetes.io/projected/1f07334b-dd72-4771-b26f-bb23fc9a0ada-kube-api-access-2pdzm\") pod \"insights-operator-5885fdb759-zqcwj\" (UID: \"1f07334b-dd72-4771-b26f-bb23fc9a0ada\") " pod="openshift-insights/insights-operator-5885fdb759-zqcwj"
Jul 30 14:49:01.286682 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.286572    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-registry-certificates\") pod \"image-registry-684cf4bf4d-2j296\" (UID: \"6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7\") " pod="openshift-image-registry/image-registry-684cf4bf4d-2j296"
Jul 30 14:49:01.286682 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.286602    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/1f07334b-dd72-4771-b26f-bb23fc9a0ada-snapshots\") pod \"insights-operator-5885fdb759-zqcwj\" (UID: \"1f07334b-dd72-4771-b26f-bb23fc9a0ada\") " pod="openshift-insights/insights-operator-5885fdb759-zqcwj"
Jul 30 14:49:01.286682 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.286631    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-registry-tls\") pod \"image-registry-684cf4bf4d-2j296\" (UID: \"6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7\") " pod="openshift-image-registry/image-registry-684cf4bf4d-2j296"
Jul 30 14:49:01.286682 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.286672    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqjs4\" (UniqueName: \"kubernetes.io/projected/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-kube-api-access-wqjs4\") pod \"image-registry-684cf4bf4d-2j296\" (UID: \"6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7\") " pod="openshift-image-registry/image-registry-684cf4bf4d-2j296"
Jul 30 14:49:01.286936 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.286757    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-ca-trust-extracted\") pod \"image-registry-684cf4bf4d-2j296\" (UID: \"6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7\") " pod="openshift-image-registry/image-registry-684cf4bf4d-2j296"
Jul 30 14:49:01.286936 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.286759    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1f07334b-dd72-4771-b26f-bb23fc9a0ada-serving-cert\") pod \"insights-operator-5885fdb759-zqcwj\" (UID: \"1f07334b-dd72-4771-b26f-bb23fc9a0ada\") " pod="openshift-insights/insights-operator-5885fdb759-zqcwj"
Jul 30 14:49:01.286936 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:49:01.286805    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 30 14:49:01.286936 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:49:01.286821    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-684cf4bf4d-2j296: secret "image-registry-tls" not found
Jul 30 14:49:01.286936 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:49:01.286878    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-registry-tls podName:6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7 nodeName:}" failed. No retries permitted until 2026-07-30 14:49:01.786860656 +0000 UTC m=+53.315799114 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-registry-tls") pod "image-registry-684cf4bf4d-2j296" (UID: "6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7") : secret "image-registry-tls" not found
Jul 30 14:49:01.287383 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.287345    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-registry-certificates\") pod \"image-registry-684cf4bf4d-2j296\" (UID: \"6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7\") " pod="openshift-image-registry/image-registry-684cf4bf4d-2j296"
Jul 30 14:49:01.287428 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.287396    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-trusted-ca\") pod \"image-registry-684cf4bf4d-2j296\" (UID: \"6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7\") " pod="openshift-image-registry/image-registry-684cf4bf4d-2j296"
Jul 30 14:49:01.288778 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.288762    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-image-registry-private-configuration\") pod \"image-registry-684cf4bf4d-2j296\" (UID: \"6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7\") " pod="openshift-image-registry/image-registry-684cf4bf4d-2j296"
Jul 30 14:49:01.288822 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.288790    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-installation-pull-secrets\") pod \"image-registry-684cf4bf4d-2j296\" (UID: \"6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7\") " pod="openshift-image-registry/image-registry-684cf4bf4d-2j296"
Jul 30 14:49:01.299255 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.299228    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-bound-sa-token\") pod \"image-registry-684cf4bf4d-2j296\" (UID: \"6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7\") " pod="openshift-image-registry/image-registry-684cf4bf4d-2j296"
Jul 30 14:49:01.299398 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.299380    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqjs4\" (UniqueName: \"kubernetes.io/projected/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-kube-api-access-wqjs4\") pod \"image-registry-684cf4bf4d-2j296\" (UID: \"6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7\") " pod="openshift-image-registry/image-registry-684cf4bf4d-2j296"
Jul 30 14:49:01.370544 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.370492    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-9shzj"
Jul 30 14:49:01.375662 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.375618    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-4mjmp"
Jul 30 14:49:01.387620 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.387597    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f07334b-dd72-4771-b26f-bb23fc9a0ada-service-ca-bundle\") pod \"insights-operator-5885fdb759-zqcwj\" (UID: \"1f07334b-dd72-4771-b26f-bb23fc9a0ada\") " pod="openshift-insights/insights-operator-5885fdb759-zqcwj"
Jul 30 14:49:01.387724 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.387635    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1f07334b-dd72-4771-b26f-bb23fc9a0ada-tmp\") pod \"insights-operator-5885fdb759-zqcwj\" (UID: \"1f07334b-dd72-4771-b26f-bb23fc9a0ada\") " pod="openshift-insights/insights-operator-5885fdb759-zqcwj"
Jul 30 14:49:01.387724 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.387658    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f07334b-dd72-4771-b26f-bb23fc9a0ada-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-zqcwj\" (UID: \"1f07334b-dd72-4771-b26f-bb23fc9a0ada\") " pod="openshift-insights/insights-operator-5885fdb759-zqcwj"
Jul 30 14:49:01.387724 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.387674    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pdzm\" (UniqueName: \"kubernetes.io/projected/1f07334b-dd72-4771-b26f-bb23fc9a0ada-kube-api-access-2pdzm\") pod \"insights-operator-5885fdb759-zqcwj\" (UID: \"1f07334b-dd72-4771-b26f-bb23fc9a0ada\") " pod="openshift-insights/insights-operator-5885fdb759-zqcwj"
Jul 30 14:49:01.387724 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.387703    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/1f07334b-dd72-4771-b26f-bb23fc9a0ada-snapshots\") pod \"insights-operator-5885fdb759-zqcwj\" (UID: \"1f07334b-dd72-4771-b26f-bb23fc9a0ada\") " pod="openshift-insights/insights-operator-5885fdb759-zqcwj"
Jul 30 14:49:01.387920 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.387777    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1f07334b-dd72-4771-b26f-bb23fc9a0ada-serving-cert\") pod \"insights-operator-5885fdb759-zqcwj\" (UID: \"1f07334b-dd72-4771-b26f-bb23fc9a0ada\") " pod="openshift-insights/insights-operator-5885fdb759-zqcwj"
Jul 30 14:49:01.388367 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.388232    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1f07334b-dd72-4771-b26f-bb23fc9a0ada-tmp\") pod \"insights-operator-5885fdb759-zqcwj\" (UID: \"1f07334b-dd72-4771-b26f-bb23fc9a0ada\") " pod="openshift-insights/insights-operator-5885fdb759-zqcwj"
Jul 30 14:49:01.388470 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.388366    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f07334b-dd72-4771-b26f-bb23fc9a0ada-service-ca-bundle\") pod \"insights-operator-5885fdb759-zqcwj\" (UID: \"1f07334b-dd72-4771-b26f-bb23fc9a0ada\") " pod="openshift-insights/insights-operator-5885fdb759-zqcwj"
Jul 30 14:49:01.388550 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.388528    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/1f07334b-dd72-4771-b26f-bb23fc9a0ada-snapshots\") pod \"insights-operator-5885fdb759-zqcwj\" (UID: \"1f07334b-dd72-4771-b26f-bb23fc9a0ada\") " pod="openshift-insights/insights-operator-5885fdb759-zqcwj"
Jul 30 14:49:01.388628 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.388577    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f07334b-dd72-4771-b26f-bb23fc9a0ada-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-zqcwj\" (UID: \"1f07334b-dd72-4771-b26f-bb23fc9a0ada\") " pod="openshift-insights/insights-operator-5885fdb759-zqcwj"
Jul 30 14:49:01.390265 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.390249    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1f07334b-dd72-4771-b26f-bb23fc9a0ada-serving-cert\") pod \"insights-operator-5885fdb759-zqcwj\" (UID: \"1f07334b-dd72-4771-b26f-bb23fc9a0ada\") " pod="openshift-insights/insights-operator-5885fdb759-zqcwj"
Jul 30 14:49:01.396256 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.396231    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pdzm\" (UniqueName: \"kubernetes.io/projected/1f07334b-dd72-4771-b26f-bb23fc9a0ada-kube-api-access-2pdzm\") pod \"insights-operator-5885fdb759-zqcwj\" (UID: \"1f07334b-dd72-4771-b26f-bb23fc9a0ada\") " pod="openshift-insights/insights-operator-5885fdb759-zqcwj"
Jul 30 14:49:01.496407 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.496375    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-4mjmp"]
Jul 30 14:49:01.500214 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:49:01.500185    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde241220_5f27_4b0b_b7c6_6d4b350e24e1.slice/crio-0715083b89b33d0aceb91d3c5abc178b7570d1c2d32f7ae7ea53466cad2d7e23 WatchSource:0}: Error finding container 0715083b89b33d0aceb91d3c5abc178b7570d1c2d32f7ae7ea53466cad2d7e23: Status 404 returned error can't find the container with id 0715083b89b33d0aceb91d3c5abc178b7570d1c2d32f7ae7ea53466cad2d7e23
Jul 30 14:49:01.514262 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.514239    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-9shzj"]
Jul 30 14:49:01.516575 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:49:01.516553    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea7a4fd9_ee55_43ed_b848_d6bd1f0e5e3a.slice/crio-7ad87efe287e270e7915880fd2d23e83d85139162f860ee099eb8fba271d001f WatchSource:0}: Error finding container 7ad87efe287e270e7915880fd2d23e83d85139162f860ee099eb8fba271d001f: Status 404 returned error can't find the container with id 7ad87efe287e270e7915880fd2d23e83d85139162f860ee099eb8fba271d001f
Jul 30 14:49:01.566983 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.566954    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-zqcwj"
Jul 30 14:49:01.684561 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.684528    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-zqcwj"]
Jul 30 14:49:01.687522 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:49:01.687487    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f07334b_dd72_4771_b26f_bb23fc9a0ada.slice/crio-f5c6d5adf8a64b26c2aa5dec8f866a7d0b97552287484cca0abe947570ef4041 WatchSource:0}: Error finding container f5c6d5adf8a64b26c2aa5dec8f866a7d0b97552287484cca0abe947570ef4041: Status 404 returned error can't find the container with id f5c6d5adf8a64b26c2aa5dec8f866a7d0b97552287484cca0abe947570ef4041
Jul 30 14:49:01.791557 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:01.791529    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-registry-tls\") pod \"image-registry-684cf4bf4d-2j296\" (UID: \"6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7\") " pod="openshift-image-registry/image-registry-684cf4bf4d-2j296"
Jul 30 14:49:01.791701 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:49:01.791672    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 30 14:49:01.791701 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:49:01.791691    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-684cf4bf4d-2j296: secret "image-registry-tls" not found
Jul 30 14:49:01.791780 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:49:01.791743    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-registry-tls podName:6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7 nodeName:}" failed. No retries permitted until 2026-07-30 14:49:02.791726826 +0000 UTC m=+54.320665278 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-registry-tls") pod "image-registry-684cf4bf4d-2j296" (UID: "6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7") : secret "image-registry-tls" not found
Jul 30 14:49:02.180178 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:02.180123    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-zqcwj" event={"ID":"1f07334b-dd72-4771-b26f-bb23fc9a0ada","Type":"ContainerStarted","Data":"f5c6d5adf8a64b26c2aa5dec8f866a7d0b97552287484cca0abe947570ef4041"}
Jul 30 14:49:02.181415 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:02.181389    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-4mjmp" event={"ID":"de241220-5f27-4b0b-b7c6-6d4b350e24e1","Type":"ContainerStarted","Data":"0715083b89b33d0aceb91d3c5abc178b7570d1c2d32f7ae7ea53466cad2d7e23"}
Jul 30 14:49:02.182524 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:02.182503    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-9shzj" event={"ID":"ea7a4fd9-ee55-43ed-b848-d6bd1f0e5e3a","Type":"ContainerStarted","Data":"7ad87efe287e270e7915880fd2d23e83d85139162f860ee099eb8fba271d001f"}
Jul 30 14:49:02.803920 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:02.803882    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-registry-tls\") pod \"image-registry-684cf4bf4d-2j296\" (UID: \"6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7\") " pod="openshift-image-registry/image-registry-684cf4bf4d-2j296"
Jul 30 14:49:02.804108 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:49:02.804018    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 30 14:49:02.804108 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:49:02.804034    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-684cf4bf4d-2j296: secret "image-registry-tls" not found
Jul 30 14:49:02.804108 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:49:02.804099    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-registry-tls podName:6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7 nodeName:}" failed. No retries permitted until 2026-07-30 14:49:04.804082085 +0000 UTC m=+56.333020539 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-registry-tls") pod "image-registry-684cf4bf4d-2j296" (UID: "6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7") : secret "image-registry-tls" not found
Jul 30 14:49:04.819341 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:04.819299    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-registry-tls\") pod \"image-registry-684cf4bf4d-2j296\" (UID: \"6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7\") " pod="openshift-image-registry/image-registry-684cf4bf4d-2j296"
Jul 30 14:49:04.819859 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:49:04.819459    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 30 14:49:04.819859 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:49:04.819484    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-684cf4bf4d-2j296: secret "image-registry-tls" not found
Jul 30 14:49:04.819859 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:49:04.819561    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-registry-tls podName:6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7 nodeName:}" failed. No retries permitted until 2026-07-30 14:49:08.81954431 +0000 UTC m=+60.348482762 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-registry-tls") pod "image-registry-684cf4bf4d-2j296" (UID: "6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7") : secret "image-registry-tls" not found
Jul 30 14:49:06.196776 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:06.196496    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-9lk5c"]
Jul 30 14:49:06.199467 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:06.199442    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4mjmp_de241220-5f27-4b0b-b7c6-6d4b350e24e1/console-operator/0.log"
Jul 30 14:49:06.199585 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:06.199485    2578 generic.go:356] "Generic (PLEG): container finished" podID="de241220-5f27-4b0b-b7c6-6d4b350e24e1" containerID="24b60f64a20b71f7013fac6330d811c58d66a7ecb79b378a633626aac21f11ef" exitCode=255
Jul 30 14:49:06.199851 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:06.199824    2578 scope.go:117] "RemoveContainer" containerID="24b60f64a20b71f7013fac6330d811c58d66a7ecb79b378a633626aac21f11ef"
Jul 30 14:49:06.201165 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:06.200668    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-zqcwj" event={"ID":"1f07334b-dd72-4771-b26f-bb23fc9a0ada","Type":"ContainerStarted","Data":"deefbaa84c623159f2ddb1ad752b5e45b3c480ad09a676cd0f7afbefcaf44d75"}
Jul 30 14:49:06.201165 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:06.200701    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-4mjmp" event={"ID":"de241220-5f27-4b0b-b7c6-6d4b350e24e1","Type":"ContainerDied","Data":"24b60f64a20b71f7013fac6330d811c58d66a7ecb79b378a633626aac21f11ef"}
Jul 30 14:49:06.201165 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:06.200789    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9lk5c"
Jul 30 14:49:06.202059 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:06.202026    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-9shzj" event={"ID":"ea7a4fd9-ee55-43ed-b848-d6bd1f0e5e3a","Type":"ContainerStarted","Data":"4588b7a1f0a1c97dd21655881e28e6e4484accd8442da1079aaecb743e03bcb8"}
Jul 30 14:49:06.203196 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:06.202525    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9mfqr\""
Jul 30 14:49:06.203196 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:06.202826    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 30 14:49:06.203196 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:06.203015    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 30 14:49:06.211435 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:06.211346    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9lk5c"]
Jul 30 14:49:06.214931 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:06.214878    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-zqcwj" podStartSLOduration=1.473258492 podStartE2EDuration="5.214860473s" totalImagesPullingTime="3.741601981s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 14:49:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:49:06.213578242 +0000 UTC m=+57.742516738" watchObservedRunningTime="2026-07-30 14:49:06.214860473 +0000 UTC m=+57.743798950"
Jul 30 14:49:06.233125 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:06.232294    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/623272af-807b-4e8c-b308-3c2737b1fe31-crio-socket\") pod \"insights-runtime-extractor-9lk5c\" (UID: \"623272af-807b-4e8c-b308-3c2737b1fe31\") " pod="openshift-insights/insights-runtime-extractor-9lk5c"
Jul 30 14:49:06.233125 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:06.232570    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/623272af-807b-4e8c-b308-3c2737b1fe31-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9lk5c\" (UID: \"623272af-807b-4e8c-b308-3c2737b1fe31\") " pod="openshift-insights/insights-runtime-extractor-9lk5c"
Jul 30 14:49:06.233125 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:06.232696    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wjss\" (UniqueName: \"kubernetes.io/projected/623272af-807b-4e8c-b308-3c2737b1fe31-kube-api-access-5wjss\") pod \"insights-runtime-extractor-9lk5c\" (UID: \"623272af-807b-4e8c-b308-3c2737b1fe31\") " pod="openshift-insights/insights-runtime-extractor-9lk5c"
Jul 30 14:49:06.233125 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:06.232810    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/623272af-807b-4e8c-b308-3c2737b1fe31-data-volume\") pod \"insights-runtime-extractor-9lk5c\" (UID: \"623272af-807b-4e8c-b308-3c2737b1fe31\") " pod="openshift-insights/insights-runtime-extractor-9lk5c"
Jul 30 14:49:06.233125 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:06.232931    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/623272af-807b-4e8c-b308-3c2737b1fe31-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9lk5c\" (UID: \"623272af-807b-4e8c-b308-3c2737b1fe31\") " pod="openshift-insights/insights-runtime-extractor-9lk5c"
Jul 30 14:49:06.253977 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:06.253921    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-9shzj" podStartSLOduration=1.339634016 podStartE2EDuration="5.253901597s" totalImagesPullingTime="3.914267581s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 14:49:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:49:06.235019812 +0000 UTC m=+57.763958279" watchObservedRunningTime="2026-07-30 14:49:06.253901597 +0000 UTC m=+57.782840086"
Jul 30 14:49:06.334480 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:06.334452    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/623272af-807b-4e8c-b308-3c2737b1fe31-data-volume\") pod \"insights-runtime-extractor-9lk5c\" (UID: \"623272af-807b-4e8c-b308-3c2737b1fe31\") " pod="openshift-insights/insights-runtime-extractor-9lk5c"
Jul 30 14:49:06.334599 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:06.334499    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/623272af-807b-4e8c-b308-3c2737b1fe31-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9lk5c\" (UID: \"623272af-807b-4e8c-b308-3c2737b1fe31\") " pod="openshift-insights/insights-runtime-extractor-9lk5c"
Jul 30 14:49:06.334599 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:06.334528    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/623272af-807b-4e8c-b308-3c2737b1fe31-crio-socket\") pod \"insights-runtime-extractor-9lk5c\" (UID: \"623272af-807b-4e8c-b308-3c2737b1fe31\") " pod="openshift-insights/insights-runtime-extractor-9lk5c"
Jul 30 14:49:06.334599 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:06.334553    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/623272af-807b-4e8c-b308-3c2737b1fe31-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9lk5c\" (UID: \"623272af-807b-4e8c-b308-3c2737b1fe31\") " pod="openshift-insights/insights-runtime-extractor-9lk5c"
Jul 30 14:49:06.334599 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:06.334592    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wjss\" (UniqueName: \"kubernetes.io/projected/623272af-807b-4e8c-b308-3c2737b1fe31-kube-api-access-5wjss\") pod \"insights-runtime-extractor-9lk5c\" (UID: \"623272af-807b-4e8c-b308-3c2737b1fe31\") " pod="openshift-insights/insights-runtime-extractor-9lk5c"
Jul 30 14:49:06.334813 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:49:06.334656    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 14:49:06.334813 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:06.334664    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/623272af-807b-4e8c-b308-3c2737b1fe31-crio-socket\") pod \"insights-runtime-extractor-9lk5c\" (UID: \"623272af-807b-4e8c-b308-3c2737b1fe31\") " pod="openshift-insights/insights-runtime-extractor-9lk5c"
Jul 30 14:49:06.334813 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:49:06.334730    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/623272af-807b-4e8c-b308-3c2737b1fe31-kube-rbac-proxy-cm podName:623272af-807b-4e8c-b308-3c2737b1fe31 nodeName:}" failed. No retries permitted until 2026-07-30 14:49:06.83471187 +0000 UTC m=+58.363650325 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/623272af-807b-4e8c-b308-3c2737b1fe31-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9lk5c" (UID: "623272af-807b-4e8c-b308-3c2737b1fe31") : configmap "kube-rbac-proxy" not found
Jul 30 14:49:06.334813 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:49:06.334727    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 14:49:06.334813 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:49:06.334803    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/623272af-807b-4e8c-b308-3c2737b1fe31-insights-runtime-extractor-tls podName:623272af-807b-4e8c-b308-3c2737b1fe31 nodeName:}" failed. No retries permitted until 2026-07-30 14:49:06.834785373 +0000 UTC m=+58.363723831 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/623272af-807b-4e8c-b308-3c2737b1fe31-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9lk5c" (UID: "623272af-807b-4e8c-b308-3c2737b1fe31") : secret "insights-runtime-extractor-tls" not found
Jul 30 14:49:06.334988 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:06.334822    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/623272af-807b-4e8c-b308-3c2737b1fe31-data-volume\") pod \"insights-runtime-extractor-9lk5c\" (UID: \"623272af-807b-4e8c-b308-3c2737b1fe31\") " pod="openshift-insights/insights-runtime-extractor-9lk5c"
Jul 30 14:49:06.342176 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:06.342136    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wjss\" (UniqueName: \"kubernetes.io/projected/623272af-807b-4e8c-b308-3c2737b1fe31-kube-api-access-5wjss\") pod \"insights-runtime-extractor-9lk5c\" (UID: \"623272af-807b-4e8c-b308-3c2737b1fe31\") " pod="openshift-insights/insights-runtime-extractor-9lk5c"
Jul 30 14:49:06.838839 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:06.838805    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/623272af-807b-4e8c-b308-3c2737b1fe31-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9lk5c\" (UID: \"623272af-807b-4e8c-b308-3c2737b1fe31\") " pod="openshift-insights/insights-runtime-extractor-9lk5c"
Jul 30 14:49:06.838996 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:06.838862    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/623272af-807b-4e8c-b308-3c2737b1fe31-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9lk5c\" (UID: \"623272af-807b-4e8c-b308-3c2737b1fe31\") " pod="openshift-insights/insights-runtime-extractor-9lk5c"
Jul 30 14:49:06.838996 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:49:06.838923    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 14:49:06.838996 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:49:06.838996    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/623272af-807b-4e8c-b308-3c2737b1fe31-kube-rbac-proxy-cm podName:623272af-807b-4e8c-b308-3c2737b1fe31 nodeName:}" failed. No retries permitted until 2026-07-30 14:49:07.838979963 +0000 UTC m=+59.367918415 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/623272af-807b-4e8c-b308-3c2737b1fe31-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9lk5c" (UID: "623272af-807b-4e8c-b308-3c2737b1fe31") : configmap "kube-rbac-proxy" not found
Jul 30 14:49:06.839134 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:49:06.839005    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 14:49:06.839134 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:49:06.839049    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/623272af-807b-4e8c-b308-3c2737b1fe31-insights-runtime-extractor-tls podName:623272af-807b-4e8c-b308-3c2737b1fe31 nodeName:}" failed. No retries permitted until 2026-07-30 14:49:07.839033452 +0000 UTC m=+59.367971905 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/623272af-807b-4e8c-b308-3c2737b1fe31-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9lk5c" (UID: "623272af-807b-4e8c-b308-3c2737b1fe31") : secret "insights-runtime-extractor-tls" not found
Jul 30 14:49:07.206436 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:07.206367    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4mjmp_de241220-5f27-4b0b-b7c6-6d4b350e24e1/console-operator/1.log"
Jul 30 14:49:07.206778 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:07.206762    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4mjmp_de241220-5f27-4b0b-b7c6-6d4b350e24e1/console-operator/0.log"
Jul 30 14:49:07.206823 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:07.206795    2578 generic.go:356] "Generic (PLEG): container finished" podID="de241220-5f27-4b0b-b7c6-6d4b350e24e1" containerID="940b95bb60645ce7ec92cf7e0f7946851746b4f0822ba7d4814145ce446a3186" exitCode=255
Jul 30 14:49:07.206883 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:07.206868    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-4mjmp" event={"ID":"de241220-5f27-4b0b-b7c6-6d4b350e24e1","Type":"ContainerDied","Data":"940b95bb60645ce7ec92cf7e0f7946851746b4f0822ba7d4814145ce446a3186"}
Jul 30 14:49:07.206935 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:07.206926    2578 scope.go:117] "RemoveContainer" containerID="24b60f64a20b71f7013fac6330d811c58d66a7ecb79b378a633626aac21f11ef"
Jul 30 14:49:07.207137 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:07.207115    2578 scope.go:117] "RemoveContainer" containerID="940b95bb60645ce7ec92cf7e0f7946851746b4f0822ba7d4814145ce446a3186"
Jul 30 14:49:07.207336 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:49:07.207315    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-4mjmp_openshift-console-operator(de241220-5f27-4b0b-b7c6-6d4b350e24e1)\"" pod="openshift-console-operator/console-operator-584dcb6b54-4mjmp" podUID="de241220-5f27-4b0b-b7c6-6d4b350e24e1"
Jul 30 14:49:07.849218 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:07.849180    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/623272af-807b-4e8c-b308-3c2737b1fe31-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9lk5c\" (UID: \"623272af-807b-4e8c-b308-3c2737b1fe31\") " pod="openshift-insights/insights-runtime-extractor-9lk5c"
Jul 30 14:49:07.849386 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:07.849283    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/623272af-807b-4e8c-b308-3c2737b1fe31-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9lk5c\" (UID: \"623272af-807b-4e8c-b308-3c2737b1fe31\") " pod="openshift-insights/insights-runtime-extractor-9lk5c"
Jul 30 14:49:07.849386 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:49:07.849324    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 14:49:07.849386 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:49:07.849352    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 14:49:07.849386 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:49:07.849385    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/623272af-807b-4e8c-b308-3c2737b1fe31-insights-runtime-extractor-tls podName:623272af-807b-4e8c-b308-3c2737b1fe31 nodeName:}" failed. No retries permitted until 2026-07-30 14:49:09.849369862 +0000 UTC m=+61.378308314 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/623272af-807b-4e8c-b308-3c2737b1fe31-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9lk5c" (UID: "623272af-807b-4e8c-b308-3c2737b1fe31") : secret "insights-runtime-extractor-tls" not found
Jul 30 14:49:07.849521 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:49:07.849404    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/623272af-807b-4e8c-b308-3c2737b1fe31-kube-rbac-proxy-cm podName:623272af-807b-4e8c-b308-3c2737b1fe31 nodeName:}" failed. No retries permitted until 2026-07-30 14:49:09.849391548 +0000 UTC m=+61.378330000 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/623272af-807b-4e8c-b308-3c2737b1fe31-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9lk5c" (UID: "623272af-807b-4e8c-b308-3c2737b1fe31") : configmap "kube-rbac-proxy" not found
Jul 30 14:49:08.210640 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:08.210576    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4mjmp_de241220-5f27-4b0b-b7c6-6d4b350e24e1/console-operator/1.log"
Jul 30 14:49:08.210958 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:08.210871    2578 scope.go:117] "RemoveContainer" containerID="940b95bb60645ce7ec92cf7e0f7946851746b4f0822ba7d4814145ce446a3186"
Jul 30 14:49:08.211033 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:49:08.211014    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-4mjmp_openshift-console-operator(de241220-5f27-4b0b-b7c6-6d4b350e24e1)\"" pod="openshift-console-operator/console-operator-584dcb6b54-4mjmp" podUID="de241220-5f27-4b0b-b7c6-6d4b350e24e1"
Jul 30 14:49:08.789781 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:08.789756    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8jj48_fc042c9a-bc2a-4cf4-8606-e9e20e462cb3/dns-node-resolver/0.log"
Jul 30 14:49:08.855568 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:08.855539    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-registry-tls\") pod \"image-registry-684cf4bf4d-2j296\" (UID: \"6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7\") " pod="openshift-image-registry/image-registry-684cf4bf4d-2j296"
Jul 30 14:49:08.855714 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:49:08.855656    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 30 14:49:08.855714 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:49:08.855668    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-684cf4bf4d-2j296: secret "image-registry-tls" not found
Jul 30 14:49:08.855714 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:49:08.855710    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-registry-tls podName:6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7 nodeName:}" failed. No retries permitted until 2026-07-30 14:49:16.855698063 +0000 UTC m=+68.384636515 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-registry-tls") pod "image-registry-684cf4bf4d-2j296" (UID: "6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7") : secret "image-registry-tls" not found
Jul 30 14:49:09.789863 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:09.789835    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9q72z_d9b6c536-3a43-49a8-853d-f9532668f149/node-ca/0.log"
Jul 30 14:49:09.863133 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:09.863104    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/623272af-807b-4e8c-b308-3c2737b1fe31-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9lk5c\" (UID: \"623272af-807b-4e8c-b308-3c2737b1fe31\") " pod="openshift-insights/insights-runtime-extractor-9lk5c"
Jul 30 14:49:09.863331 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:09.863197    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/623272af-807b-4e8c-b308-3c2737b1fe31-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9lk5c\" (UID: \"623272af-807b-4e8c-b308-3c2737b1fe31\") " pod="openshift-insights/insights-runtime-extractor-9lk5c"
Jul 30 14:49:09.863331 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:49:09.863258    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 14:49:09.863331 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:49:09.863270    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 14:49:09.863331 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:49:09.863326    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/623272af-807b-4e8c-b308-3c2737b1fe31-insights-runtime-extractor-tls podName:623272af-807b-4e8c-b308-3c2737b1fe31 nodeName:}" failed. No retries permitted until 2026-07-30 14:49:13.863306487 +0000 UTC m=+65.392244938 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/623272af-807b-4e8c-b308-3c2737b1fe31-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9lk5c" (UID: "623272af-807b-4e8c-b308-3c2737b1fe31") : secret "insights-runtime-extractor-tls" not found
Jul 30 14:49:09.863485 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:49:09.863346    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/623272af-807b-4e8c-b308-3c2737b1fe31-kube-rbac-proxy-cm podName:623272af-807b-4e8c-b308-3c2737b1fe31 nodeName:}" failed. No retries permitted until 2026-07-30 14:49:13.863336914 +0000 UTC m=+65.392275401 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/623272af-807b-4e8c-b308-3c2737b1fe31-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9lk5c" (UID: "623272af-807b-4e8c-b308-3c2737b1fe31") : configmap "kube-rbac-proxy" not found
Jul 30 14:49:10.115825 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:10.115803    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xllmv"
Jul 30 14:49:10.552684 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:10.552658    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-jrblr"]
Jul 30 14:49:10.555281 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:10.555265    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-jrblr"
Jul 30 14:49:10.557527 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:10.557495    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 30 14:49:10.557527 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:10.557511    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 30 14:49:10.557697 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:10.557540    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 30 14:49:10.557697 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:10.557559    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-gqnm6\""
Jul 30 14:49:10.557697 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:10.557570    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 30 14:49:10.563683 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:10.563664    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-jrblr"]
Jul 30 14:49:10.670390 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:10.670361    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d41b3168-0581-4969-99db-4c2649053aaf-signing-key\") pod \"service-ca-658d7b7c78-jrblr\" (UID: \"d41b3168-0581-4969-99db-4c2649053aaf\") " pod="openshift-service-ca/service-ca-658d7b7c78-jrblr"
Jul 30 14:49:10.670558 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:10.670428    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5sz4\" (UniqueName: \"kubernetes.io/projected/d41b3168-0581-4969-99db-4c2649053aaf-kube-api-access-q5sz4\") pod \"service-ca-658d7b7c78-jrblr\" (UID: \"d41b3168-0581-4969-99db-4c2649053aaf\") " pod="openshift-service-ca/service-ca-658d7b7c78-jrblr"
Jul 30 14:49:10.670558 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:10.670525    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d41b3168-0581-4969-99db-4c2649053aaf-signing-cabundle\") pod \"service-ca-658d7b7c78-jrblr\" (UID: \"d41b3168-0581-4969-99db-4c2649053aaf\") " pod="openshift-service-ca/service-ca-658d7b7c78-jrblr"
Jul 30 14:49:10.771596 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:10.771562    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d41b3168-0581-4969-99db-4c2649053aaf-signing-cabundle\") pod \"service-ca-658d7b7c78-jrblr\" (UID: \"d41b3168-0581-4969-99db-4c2649053aaf\") " pod="openshift-service-ca/service-ca-658d7b7c78-jrblr"
Jul 30 14:49:10.771851 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:10.771604    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d41b3168-0581-4969-99db-4c2649053aaf-signing-key\") pod \"service-ca-658d7b7c78-jrblr\" (UID: \"d41b3168-0581-4969-99db-4c2649053aaf\") " pod="openshift-service-ca/service-ca-658d7b7c78-jrblr"
Jul 30 14:49:10.771851 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:10.771653    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5sz4\" (UniqueName: \"kubernetes.io/projected/d41b3168-0581-4969-99db-4c2649053aaf-kube-api-access-q5sz4\") pod \"service-ca-658d7b7c78-jrblr\" (UID: \"d41b3168-0581-4969-99db-4c2649053aaf\") " pod="openshift-service-ca/service-ca-658d7b7c78-jrblr"
Jul 30 14:49:10.772331 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:10.772305    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d41b3168-0581-4969-99db-4c2649053aaf-signing-cabundle\") pod \"service-ca-658d7b7c78-jrblr\" (UID: \"d41b3168-0581-4969-99db-4c2649053aaf\") " pod="openshift-service-ca/service-ca-658d7b7c78-jrblr"
Jul 30 14:49:10.774015 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:10.773995    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d41b3168-0581-4969-99db-4c2649053aaf-signing-key\") pod \"service-ca-658d7b7c78-jrblr\" (UID: \"d41b3168-0581-4969-99db-4c2649053aaf\") " pod="openshift-service-ca/service-ca-658d7b7c78-jrblr"
Jul 30 14:49:10.778798 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:10.778776    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5sz4\" (UniqueName: \"kubernetes.io/projected/d41b3168-0581-4969-99db-4c2649053aaf-kube-api-access-q5sz4\") pod \"service-ca-658d7b7c78-jrblr\" (UID: \"d41b3168-0581-4969-99db-4c2649053aaf\") " pod="openshift-service-ca/service-ca-658d7b7c78-jrblr"
Jul 30 14:49:10.789135 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:10.789116    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-frg92_8ce55aa3-7792-4ea0-83ed-64890b613632/serve-healthcheck-canary/0.log"
Jul 30 14:49:10.864672 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:10.864612    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-jrblr"
Jul 30 14:49:10.981135 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:10.981107    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-jrblr"]
Jul 30 14:49:10.984318 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:49:10.984293    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd41b3168_0581_4969_99db_4c2649053aaf.slice/crio-3d20bd0e884c9eaa886c1f191ce7c5bef6ed3dc1adb0396abebc8be64cbfc177 WatchSource:0}: Error finding container 3d20bd0e884c9eaa886c1f191ce7c5bef6ed3dc1adb0396abebc8be64cbfc177: Status 404 returned error can't find the container with id 3d20bd0e884c9eaa886c1f191ce7c5bef6ed3dc1adb0396abebc8be64cbfc177
Jul 30 14:49:11.219045 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:11.218971    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-jrblr" event={"ID":"d41b3168-0581-4969-99db-4c2649053aaf","Type":"ContainerStarted","Data":"3d20bd0e884c9eaa886c1f191ce7c5bef6ed3dc1adb0396abebc8be64cbfc177"}
Jul 30 14:49:11.376043 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:11.376010    2578 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-4mjmp"
Jul 30 14:49:11.376225 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:11.376060    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-4mjmp"
Jul 30 14:49:11.376380 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:11.376369    2578 scope.go:117] "RemoveContainer" containerID="940b95bb60645ce7ec92cf7e0f7946851746b4f0822ba7d4814145ce446a3186"
Jul 30 14:49:11.376520 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:49:11.376503    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-4mjmp_openshift-console-operator(de241220-5f27-4b0b-b7c6-6d4b350e24e1)\"" pod="openshift-console-operator/console-operator-584dcb6b54-4mjmp" podUID="de241220-5f27-4b0b-b7c6-6d4b350e24e1"
Jul 30 14:49:13.900589 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:13.900551    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/623272af-807b-4e8c-b308-3c2737b1fe31-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9lk5c\" (UID: \"623272af-807b-4e8c-b308-3c2737b1fe31\") " pod="openshift-insights/insights-runtime-extractor-9lk5c"
Jul 30 14:49:13.901048 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:13.900617    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/623272af-807b-4e8c-b308-3c2737b1fe31-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9lk5c\" (UID: \"623272af-807b-4e8c-b308-3c2737b1fe31\") " pod="openshift-insights/insights-runtime-extractor-9lk5c"
Jul 30 14:49:13.901048 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:49:13.900684    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 14:49:13.901048 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:49:13.900713    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 14:49:13.901048 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:49:13.900769    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/623272af-807b-4e8c-b308-3c2737b1fe31-kube-rbac-proxy-cm podName:623272af-807b-4e8c-b308-3c2737b1fe31 nodeName:}" failed. No retries permitted until 2026-07-30 14:49:21.900747136 +0000 UTC m=+73.429685607 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/623272af-807b-4e8c-b308-3c2737b1fe31-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9lk5c" (UID: "623272af-807b-4e8c-b308-3c2737b1fe31") : configmap "kube-rbac-proxy" not found
Jul 30 14:49:13.901048 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:49:13.900789    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/623272af-807b-4e8c-b308-3c2737b1fe31-insights-runtime-extractor-tls podName:623272af-807b-4e8c-b308-3c2737b1fe31 nodeName:}" failed. No retries permitted until 2026-07-30 14:49:21.900780036 +0000 UTC m=+73.429718487 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/623272af-807b-4e8c-b308-3c2737b1fe31-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9lk5c" (UID: "623272af-807b-4e8c-b308-3c2737b1fe31") : secret "insights-runtime-extractor-tls" not found
Jul 30 14:49:14.229469 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:14.229372    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-jrblr" event={"ID":"d41b3168-0581-4969-99db-4c2649053aaf","Type":"ContainerStarted","Data":"b42097bc50c9ae5314d9a1f9ced7ac27660b25f78a67de2b7ddc85952b112236"}
Jul 30 14:49:14.247917 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:14.247855    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-658d7b7c78-jrblr" podStartSLOduration=1.872523153 podStartE2EDuration="4.247834062s" totalImagesPullingTime="2.375310909s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 14:49:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:49:14.246635224 +0000 UTC m=+65.775573699" watchObservedRunningTime="2026-07-30 14:49:14.247834062 +0000 UTC m=+65.776772537"
Jul 30 14:49:14.707762 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:14.707725    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be2b73fa-10f0-4ea3-a68e-fa452e30914b-metrics-certs\") pod \"network-metrics-daemon-rj8gr\" (UID: \"be2b73fa-10f0-4ea3-a68e-fa452e30914b\") " pod="openshift-multus/network-metrics-daemon-rj8gr"
Jul 30 14:49:14.707972 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:14.707815    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rbmt\" (UniqueName: \"kubernetes.io/projected/78565b9e-9521-4e9f-811e-6cc4cd76a4d3-kube-api-access-9rbmt\") pod \"network-check-target-j6hld\" (UID: \"78565b9e-9521-4e9f-811e-6cc4cd76a4d3\") " pod="openshift-network-diagnostics/network-check-target-j6hld"
Jul 30 14:49:14.710021 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:14.709838    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 30 14:49:14.710465 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:14.710443    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rbmt\" (UniqueName: \"kubernetes.io/projected/78565b9e-9521-4e9f-811e-6cc4cd76a4d3-kube-api-access-9rbmt\") pod \"network-check-target-j6hld\" (UID: \"78565b9e-9521-4e9f-811e-6cc4cd76a4d3\") " pod="openshift-network-diagnostics/network-check-target-j6hld"
Jul 30 14:49:14.718393 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:49:14.718371    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 30 14:49:14.718502 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:49:14.718437    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be2b73fa-10f0-4ea3-a68e-fa452e30914b-metrics-certs podName:be2b73fa-10f0-4ea3-a68e-fa452e30914b nodeName:}" failed. No retries permitted until 2026-07-30 14:50:18.718417762 +0000 UTC m=+130.247356228 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/be2b73fa-10f0-4ea3-a68e-fa452e30914b-metrics-certs") pod "network-metrics-daemon-rj8gr" (UID: "be2b73fa-10f0-4ea3-a68e-fa452e30914b") : secret "metrics-daemon-secret" not found
Jul 30 14:49:14.785279 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:14.785251    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dd6sm\""
Jul 30 14:49:14.794194 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:14.794167    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j6hld"
Jul 30 14:49:14.809013 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:14.808575    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/963eac2b-b8af-497c-a206-0a1c976a0e2c-original-pull-secret\") pod \"global-pull-secret-syncer-9lgq9\" (UID: \"963eac2b-b8af-497c-a206-0a1c976a0e2c\") " pod="kube-system/global-pull-secret-syncer-9lgq9"
Jul 30 14:49:14.811058 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:14.811032    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 30 14:49:14.822254 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:14.822228    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/963eac2b-b8af-497c-a206-0a1c976a0e2c-original-pull-secret\") pod \"global-pull-secret-syncer-9lgq9\" (UID: \"963eac2b-b8af-497c-a206-0a1c976a0e2c\") " pod="kube-system/global-pull-secret-syncer-9lgq9"
Jul 30 14:49:14.929964 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:14.929934    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-j6hld"]
Jul 30 14:49:14.933197 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:49:14.933161    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78565b9e_9521_4e9f_811e_6cc4cd76a4d3.slice/crio-1f827ed21d4e83a611feae82fb1cea74914e3d0d20e4c892c5e7fd1751ef10d8 WatchSource:0}: Error finding container 1f827ed21d4e83a611feae82fb1cea74914e3d0d20e4c892c5e7fd1751ef10d8: Status 404 returned error can't find the container with id 1f827ed21d4e83a611feae82fb1cea74914e3d0d20e4c892c5e7fd1751ef10d8
Jul 30 14:49:14.990076 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:14.990057    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lgq9"
Jul 30 14:49:15.109877 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:15.109848    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9lgq9"]
Jul 30 14:49:15.112827 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:49:15.112805    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod963eac2b_b8af_497c_a206_0a1c976a0e2c.slice/crio-663be3b51b07bec92bb3f26596a8509cf7a91c72e256558f6f1731a01f192368 WatchSource:0}: Error finding container 663be3b51b07bec92bb3f26596a8509cf7a91c72e256558f6f1731a01f192368: Status 404 returned error can't find the container with id 663be3b51b07bec92bb3f26596a8509cf7a91c72e256558f6f1731a01f192368
Jul 30 14:49:15.234487 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:15.234406    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-j6hld" event={"ID":"78565b9e-9521-4e9f-811e-6cc4cd76a4d3","Type":"ContainerStarted","Data":"e2cf7bbdcd79eff30b575bdd6037b922af97cc37e5a522a3b04811a581702a08"}
Jul 30 14:49:15.234487 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:15.234455    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-j6hld" event={"ID":"78565b9e-9521-4e9f-811e-6cc4cd76a4d3","Type":"ContainerStarted","Data":"1f827ed21d4e83a611feae82fb1cea74914e3d0d20e4c892c5e7fd1751ef10d8"}
Jul 30 14:49:15.234685 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:15.234522    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-j6hld"
Jul 30 14:49:15.235698 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:15.235680    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9lgq9" event={"ID":"963eac2b-b8af-497c-a206-0a1c976a0e2c","Type":"ContainerStarted","Data":"663be3b51b07bec92bb3f26596a8509cf7a91c72e256558f6f1731a01f192368"}
Jul 30 14:49:15.251095 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:15.251054    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-j6hld" podStartSLOduration=66.251041594 podStartE2EDuration="1m6.251041594s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 14:48:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:49:15.249548751 +0000 UTC m=+66.778487225" watchObservedRunningTime="2026-07-30 14:49:15.251041594 +0000 UTC m=+66.779980067"
Jul 30 14:49:16.223257 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:16.223223    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1-metrics-tls\") pod \"dns-default-nzzqf\" (UID: \"64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1\") " pod="openshift-dns/dns-default-nzzqf"
Jul 30 14:49:16.223643 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:49:16.223355    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 14:49:16.223643 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:49:16.223416    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1-metrics-tls podName:64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1 nodeName:}" failed. No retries permitted until 2026-07-30 14:49:48.223402951 +0000 UTC m=+99.752341403 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1-metrics-tls") pod "dns-default-nzzqf" (UID: "64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1") : secret "dns-default-metrics-tls" not found
Jul 30 14:49:16.928915 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:16.928873    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-registry-tls\") pod \"image-registry-684cf4bf4d-2j296\" (UID: \"6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7\") " pod="openshift-image-registry/image-registry-684cf4bf4d-2j296"
Jul 30 14:49:16.929106 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:49:16.929050    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 30 14:49:16.929106 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:49:16.929071    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-684cf4bf4d-2j296: secret "image-registry-tls" not found
Jul 30 14:49:16.929254 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:49:16.929141    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-registry-tls podName:6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7 nodeName:}" failed. No retries permitted until 2026-07-30 14:49:32.929119817 +0000 UTC m=+84.458058269 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-registry-tls") pod "image-registry-684cf4bf4d-2j296" (UID: "6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7") : secret "image-registry-tls" not found
Jul 30 14:49:19.252100 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:19.252058    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9lgq9" event={"ID":"963eac2b-b8af-497c-a206-0a1c976a0e2c","Type":"ContainerStarted","Data":"f36165e3b9cf721f23111497a2995ac9b375ff4d5d41de67db7e366d230a92a6"}
Jul 30 14:49:19.266926 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:19.266878    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-9lgq9" podStartSLOduration=66.312646205 podStartE2EDuration="1m10.266864824s" totalImagesPullingTime="3.954218619s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 14:48:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:49:19.266282159 +0000 UTC m=+70.795220634" watchObservedRunningTime="2026-07-30 14:49:19.266864824 +0000 UTC m=+70.795803298"
Jul 30 14:49:21.967718 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:21.967681    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/623272af-807b-4e8c-b308-3c2737b1fe31-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9lk5c\" (UID: \"623272af-807b-4e8c-b308-3c2737b1fe31\") " pod="openshift-insights/insights-runtime-extractor-9lk5c"
Jul 30 14:49:21.968130 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:21.967732    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/623272af-807b-4e8c-b308-3c2737b1fe31-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9lk5c\" (UID: \"623272af-807b-4e8c-b308-3c2737b1fe31\") " pod="openshift-insights/insights-runtime-extractor-9lk5c"
Jul 30 14:49:21.968130 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:49:21.967818    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 14:49:21.968130 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:49:21.967868    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/623272af-807b-4e8c-b308-3c2737b1fe31-insights-runtime-extractor-tls podName:623272af-807b-4e8c-b308-3c2737b1fe31 nodeName:}" failed. No retries permitted until 2026-07-30 14:49:37.967855238 +0000 UTC m=+89.496793691 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/623272af-807b-4e8c-b308-3c2737b1fe31-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9lk5c" (UID: "623272af-807b-4e8c-b308-3c2737b1fe31") : secret "insights-runtime-extractor-tls" not found
Jul 30 14:49:21.968352 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:21.968330    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/623272af-807b-4e8c-b308-3c2737b1fe31-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9lk5c\" (UID: \"623272af-807b-4e8c-b308-3c2737b1fe31\") " pod="openshift-insights/insights-runtime-extractor-9lk5c"
Jul 30 14:49:22.973335 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:22.973305    2578 scope.go:117] "RemoveContainer" containerID="940b95bb60645ce7ec92cf7e0f7946851746b4f0822ba7d4814145ce446a3186"
Jul 30 14:49:23.266295 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:23.266218    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4mjmp_de241220-5f27-4b0b-b7c6-6d4b350e24e1/console-operator/1.log"
Jul 30 14:49:23.266295 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:23.266278    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-4mjmp" event={"ID":"de241220-5f27-4b0b-b7c6-6d4b350e24e1","Type":"ContainerStarted","Data":"efdfa5f6eb06db931f8bdb4484b78c48edb5740818e4e11ad945d08926abcc68"}
Jul 30 14:49:23.266581 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:23.266554    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-4mjmp"
Jul 30 14:49:23.285389 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:23.285346    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-4mjmp" podStartSLOduration=18.361500677 podStartE2EDuration="22.285331437s" totalImagesPullingTime="3.92383076s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 14:49:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:49:23.285111636 +0000 UTC m=+74.814050112" watchObservedRunningTime="2026-07-30 14:49:23.285331437 +0000 UTC m=+74.814269974"
Jul 30 14:49:23.723605 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:23.723574    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-4mjmp"
Jul 30 14:49:29.809291 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:29.809257    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cf5ff97c-g7lbb"]
Jul 30 14:49:29.812156 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:29.812129    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cf5ff97c-g7lbb"
Jul 30 14:49:29.814054 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:29.814032    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 30 14:49:29.814474 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:29.814449    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 30 14:49:29.814474 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:29.814459    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 30 14:49:29.814474 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:29.814465    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 30 14:49:29.823175 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:29.823138    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cf5ff97c-g7lbb"]
Jul 30 14:49:29.873526 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:29.873492    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-684cf4bf4d-2j296"]
Jul 30 14:49:29.873774 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:49:29.873753    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-684cf4bf4d-2j296" podUID="6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7"
Jul 30 14:49:29.910491 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:29.910462    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-cw96h"]
Jul 30 14:49:29.913609 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:29.913589    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-cw96h"
Jul 30 14:49:29.915328 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:29.915310    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 30 14:49:29.915449 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:29.915388    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-6ws6d\""
Jul 30 14:49:29.915449 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:29.915408    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 30 14:49:29.919775 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:29.919749    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5b5697b754-9q6rd"]
Jul 30 14:49:29.922854 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:29.922838    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b5697b754-9q6rd"
Jul 30 14:49:29.926769 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:29.926741    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-cw96h"]
Jul 30 14:49:29.933331 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:29.933313    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/b04f9212-57e9-453a-99ad-45191e6458c5-klusterlet-config\") pod \"klusterlet-addon-workmgr-5cf5ff97c-g7lbb\" (UID: \"b04f9212-57e9-453a-99ad-45191e6458c5\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cf5ff97c-g7lbb"
Jul 30 14:49:29.933422 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:29.933395    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b04f9212-57e9-453a-99ad-45191e6458c5-tmp\") pod \"klusterlet-addon-workmgr-5cf5ff97c-g7lbb\" (UID: \"b04f9212-57e9-453a-99ad-45191e6458c5\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cf5ff97c-g7lbb"
Jul 30 14:49:29.933463 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:29.933429    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cb874\" (UniqueName: \"kubernetes.io/projected/b04f9212-57e9-453a-99ad-45191e6458c5-kube-api-access-cb874\") pod \"klusterlet-addon-workmgr-5cf5ff97c-g7lbb\" (UID: \"b04f9212-57e9-453a-99ad-45191e6458c5\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cf5ff97c-g7lbb"
Jul 30 14:49:29.938503 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:29.938482    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5b5697b754-9q6rd"]
Jul 30 14:49:30.034723 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.034691    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/b04f9212-57e9-453a-99ad-45191e6458c5-klusterlet-config\") pod \"klusterlet-addon-workmgr-5cf5ff97c-g7lbb\" (UID: \"b04f9212-57e9-453a-99ad-45191e6458c5\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cf5ff97c-g7lbb"
Jul 30 14:49:30.034881 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.034731    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxk4b\" (UniqueName: \"kubernetes.io/projected/44c1d3a9-8aa4-4515-9a6c-2732f5eca386-kube-api-access-nxk4b\") pod \"downloads-7c97dc4555-cw96h\" (UID: \"44c1d3a9-8aa4-4515-9a6c-2732f5eca386\") " pod="openshift-console/downloads-7c97dc4555-cw96h"
Jul 30 14:49:30.034881 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.034769    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a34002ea-a683-4b17-9e6a-419802c3e1c4-bound-sa-token\") pod \"image-registry-5b5697b754-9q6rd\" (UID: \"a34002ea-a683-4b17-9e6a-419802c3e1c4\") " pod="openshift-image-registry/image-registry-5b5697b754-9q6rd"
Jul 30 14:49:30.034881 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.034852    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a34002ea-a683-4b17-9e6a-419802c3e1c4-registry-certificates\") pod \"image-registry-5b5697b754-9q6rd\" (UID: \"a34002ea-a683-4b17-9e6a-419802c3e1c4\") " pod="openshift-image-registry/image-registry-5b5697b754-9q6rd"
Jul 30 14:49:30.035014 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.034923    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a34002ea-a683-4b17-9e6a-419802c3e1c4-ca-trust-extracted\") pod \"image-registry-5b5697b754-9q6rd\" (UID: \"a34002ea-a683-4b17-9e6a-419802c3e1c4\") " pod="openshift-image-registry/image-registry-5b5697b754-9q6rd"
Jul 30 14:49:30.035014 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.034958    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a34002ea-a683-4b17-9e6a-419802c3e1c4-installation-pull-secrets\") pod \"image-registry-5b5697b754-9q6rd\" (UID: \"a34002ea-a683-4b17-9e6a-419802c3e1c4\") " pod="openshift-image-registry/image-registry-5b5697b754-9q6rd"
Jul 30 14:49:30.035110 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.035029    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b04f9212-57e9-453a-99ad-45191e6458c5-tmp\") pod \"klusterlet-addon-workmgr-5cf5ff97c-g7lbb\" (UID: \"b04f9212-57e9-453a-99ad-45191e6458c5\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cf5ff97c-g7lbb"
Jul 30 14:49:30.035110 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.035064    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbt64\" (UniqueName: \"kubernetes.io/projected/a34002ea-a683-4b17-9e6a-419802c3e1c4-kube-api-access-zbt64\") pod \"image-registry-5b5697b754-9q6rd\" (UID: \"a34002ea-a683-4b17-9e6a-419802c3e1c4\") " pod="openshift-image-registry/image-registry-5b5697b754-9q6rd"
Jul 30 14:49:30.035248 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.035118    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cb874\" (UniqueName: \"kubernetes.io/projected/b04f9212-57e9-453a-99ad-45191e6458c5-kube-api-access-cb874\") pod \"klusterlet-addon-workmgr-5cf5ff97c-g7lbb\" (UID: \"b04f9212-57e9-453a-99ad-45191e6458c5\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cf5ff97c-g7lbb"
Jul 30 14:49:30.035248 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.035169    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a34002ea-a683-4b17-9e6a-419802c3e1c4-image-registry-private-configuration\") pod \"image-registry-5b5697b754-9q6rd\" (UID: \"a34002ea-a683-4b17-9e6a-419802c3e1c4\") " pod="openshift-image-registry/image-registry-5b5697b754-9q6rd"
Jul 30 14:49:30.035248 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.035188    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a34002ea-a683-4b17-9e6a-419802c3e1c4-registry-tls\") pod \"image-registry-5b5697b754-9q6rd\" (UID: \"a34002ea-a683-4b17-9e6a-419802c3e1c4\") " pod="openshift-image-registry/image-registry-5b5697b754-9q6rd"
Jul 30 14:49:30.035248 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.035203    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a34002ea-a683-4b17-9e6a-419802c3e1c4-trusted-ca\") pod \"image-registry-5b5697b754-9q6rd\" (UID: \"a34002ea-a683-4b17-9e6a-419802c3e1c4\") " pod="openshift-image-registry/image-registry-5b5697b754-9q6rd"
Jul 30 14:49:30.035437 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.035384    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b04f9212-57e9-453a-99ad-45191e6458c5-tmp\") pod \"klusterlet-addon-workmgr-5cf5ff97c-g7lbb\" (UID: \"b04f9212-57e9-453a-99ad-45191e6458c5\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cf5ff97c-g7lbb"
Jul 30 14:49:30.037268 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.037253    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/b04f9212-57e9-453a-99ad-45191e6458c5-klusterlet-config\") pod \"klusterlet-addon-workmgr-5cf5ff97c-g7lbb\" (UID: \"b04f9212-57e9-453a-99ad-45191e6458c5\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cf5ff97c-g7lbb"
Jul 30 14:49:30.042248 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.042227    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cb874\" (UniqueName: \"kubernetes.io/projected/b04f9212-57e9-453a-99ad-45191e6458c5-kube-api-access-cb874\") pod \"klusterlet-addon-workmgr-5cf5ff97c-g7lbb\" (UID: \"b04f9212-57e9-453a-99ad-45191e6458c5\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cf5ff97c-g7lbb"
Jul 30 14:49:30.121386 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.121324    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cf5ff97c-g7lbb"
Jul 30 14:49:30.136526 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.136499    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a34002ea-a683-4b17-9e6a-419802c3e1c4-ca-trust-extracted\") pod \"image-registry-5b5697b754-9q6rd\" (UID: \"a34002ea-a683-4b17-9e6a-419802c3e1c4\") " pod="openshift-image-registry/image-registry-5b5697b754-9q6rd"
Jul 30 14:49:30.136639 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.136542    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a34002ea-a683-4b17-9e6a-419802c3e1c4-installation-pull-secrets\") pod \"image-registry-5b5697b754-9q6rd\" (UID: \"a34002ea-a683-4b17-9e6a-419802c3e1c4\") " pod="openshift-image-registry/image-registry-5b5697b754-9q6rd"
Jul 30 14:49:30.136639 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.136586    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbt64\" (UniqueName: \"kubernetes.io/projected/a34002ea-a683-4b17-9e6a-419802c3e1c4-kube-api-access-zbt64\") pod \"image-registry-5b5697b754-9q6rd\" (UID: \"a34002ea-a683-4b17-9e6a-419802c3e1c4\") " pod="openshift-image-registry/image-registry-5b5697b754-9q6rd"
Jul 30 14:49:30.136639 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.136634    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a34002ea-a683-4b17-9e6a-419802c3e1c4-image-registry-private-configuration\") pod \"image-registry-5b5697b754-9q6rd\" (UID: \"a34002ea-a683-4b17-9e6a-419802c3e1c4\") " pod="openshift-image-registry/image-registry-5b5697b754-9q6rd"
Jul 30 14:49:30.136793 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.136657    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a34002ea-a683-4b17-9e6a-419802c3e1c4-registry-tls\") pod \"image-registry-5b5697b754-9q6rd\" (UID: \"a34002ea-a683-4b17-9e6a-419802c3e1c4\") " pod="openshift-image-registry/image-registry-5b5697b754-9q6rd"
Jul 30 14:49:30.136793 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.136683    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a34002ea-a683-4b17-9e6a-419802c3e1c4-trusted-ca\") pod \"image-registry-5b5697b754-9q6rd\" (UID: \"a34002ea-a683-4b17-9e6a-419802c3e1c4\") " pod="openshift-image-registry/image-registry-5b5697b754-9q6rd"
Jul 30 14:49:30.136793 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.136732    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxk4b\" (UniqueName: \"kubernetes.io/projected/44c1d3a9-8aa4-4515-9a6c-2732f5eca386-kube-api-access-nxk4b\") pod \"downloads-7c97dc4555-cw96h\" (UID: \"44c1d3a9-8aa4-4515-9a6c-2732f5eca386\") " pod="openshift-console/downloads-7c97dc4555-cw96h"
Jul 30 14:49:30.136793 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.136775    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a34002ea-a683-4b17-9e6a-419802c3e1c4-bound-sa-token\") pod \"image-registry-5b5697b754-9q6rd\" (UID: \"a34002ea-a683-4b17-9e6a-419802c3e1c4\") " pod="openshift-image-registry/image-registry-5b5697b754-9q6rd"
Jul 30 14:49:30.136980 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.136800    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a34002ea-a683-4b17-9e6a-419802c3e1c4-registry-certificates\") pod \"image-registry-5b5697b754-9q6rd\" (UID: \"a34002ea-a683-4b17-9e6a-419802c3e1c4\") " pod="openshift-image-registry/image-registry-5b5697b754-9q6rd"
Jul 30 14:49:30.136980 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.136944    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a34002ea-a683-4b17-9e6a-419802c3e1c4-ca-trust-extracted\") pod \"image-registry-5b5697b754-9q6rd\" (UID: \"a34002ea-a683-4b17-9e6a-419802c3e1c4\") " pod="openshift-image-registry/image-registry-5b5697b754-9q6rd"
Jul 30 14:49:30.137672 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.137638    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a34002ea-a683-4b17-9e6a-419802c3e1c4-registry-certificates\") pod \"image-registry-5b5697b754-9q6rd\" (UID: \"a34002ea-a683-4b17-9e6a-419802c3e1c4\") " pod="openshift-image-registry/image-registry-5b5697b754-9q6rd"
Jul 30 14:49:30.137847 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.137807    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a34002ea-a683-4b17-9e6a-419802c3e1c4-trusted-ca\") pod \"image-registry-5b5697b754-9q6rd\" (UID: \"a34002ea-a683-4b17-9e6a-419802c3e1c4\") " pod="openshift-image-registry/image-registry-5b5697b754-9q6rd"
Jul 30 14:49:30.139450 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.139428    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a34002ea-a683-4b17-9e6a-419802c3e1c4-image-registry-private-configuration\") pod \"image-registry-5b5697b754-9q6rd\" (UID: \"a34002ea-a683-4b17-9e6a-419802c3e1c4\") " pod="openshift-image-registry/image-registry-5b5697b754-9q6rd"
Jul 30 14:49:30.139725 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.139705    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a34002ea-a683-4b17-9e6a-419802c3e1c4-installation-pull-secrets\") pod \"image-registry-5b5697b754-9q6rd\" (UID: \"a34002ea-a683-4b17-9e6a-419802c3e1c4\") " pod="openshift-image-registry/image-registry-5b5697b754-9q6rd"
Jul 30 14:49:30.139799 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.139779    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a34002ea-a683-4b17-9e6a-419802c3e1c4-registry-tls\") pod \"image-registry-5b5697b754-9q6rd\" (UID: \"a34002ea-a683-4b17-9e6a-419802c3e1c4\") " pod="openshift-image-registry/image-registry-5b5697b754-9q6rd"
Jul 30 14:49:30.147056 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.147035    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbt64\" (UniqueName: \"kubernetes.io/projected/a34002ea-a683-4b17-9e6a-419802c3e1c4-kube-api-access-zbt64\") pod \"image-registry-5b5697b754-9q6rd\" (UID: \"a34002ea-a683-4b17-9e6a-419802c3e1c4\") " pod="openshift-image-registry/image-registry-5b5697b754-9q6rd"
Jul 30 14:49:30.147245 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.147225    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a34002ea-a683-4b17-9e6a-419802c3e1c4-bound-sa-token\") pod \"image-registry-5b5697b754-9q6rd\" (UID: \"a34002ea-a683-4b17-9e6a-419802c3e1c4\") " pod="openshift-image-registry/image-registry-5b5697b754-9q6rd"
Jul 30 14:49:30.147617 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.147599    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxk4b\" (UniqueName: \"kubernetes.io/projected/44c1d3a9-8aa4-4515-9a6c-2732f5eca386-kube-api-access-nxk4b\") pod \"downloads-7c97dc4555-cw96h\" (UID: \"44c1d3a9-8aa4-4515-9a6c-2732f5eca386\") " pod="openshift-console/downloads-7c97dc4555-cw96h"
Jul 30 14:49:30.223131 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.223101    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-cw96h"
Jul 30 14:49:30.232713 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.232691    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-fxhsd\""
Jul 30 14:49:30.241691 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.241670    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b5697b754-9q6rd"
Jul 30 14:49:30.251083 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.251057    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cf5ff97c-g7lbb"]
Jul 30 14:49:30.254163 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:49:30.254117    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb04f9212_57e9_453a_99ad_45191e6458c5.slice/crio-f714bce59087ecb439ac2787c259f10ed267d9aca91403e29c113db8b3b1c535 WatchSource:0}: Error finding container f714bce59087ecb439ac2787c259f10ed267d9aca91403e29c113db8b3b1c535: Status 404 returned error can't find the container with id f714bce59087ecb439ac2787c259f10ed267d9aca91403e29c113db8b3b1c535
Jul 30 14:49:30.291012 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.290977    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-684cf4bf4d-2j296"
Jul 30 14:49:30.291203 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.290976    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cf5ff97c-g7lbb" event={"ID":"b04f9212-57e9-453a-99ad-45191e6458c5","Type":"ContainerStarted","Data":"f714bce59087ecb439ac2787c259f10ed267d9aca91403e29c113db8b3b1c535"}
Jul 30 14:49:30.301264 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.301241    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-684cf4bf4d-2j296"
Jul 30 14:49:30.386086 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.386012    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-cw96h"]
Jul 30 14:49:30.389447 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:49:30.389418    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44c1d3a9_8aa4_4515_9a6c_2732f5eca386.slice/crio-6f9149c069db4b9e4f672321e129a64935012e0bb46f1f346c7a889dbd0e0e40 WatchSource:0}: Error finding container 6f9149c069db4b9e4f672321e129a64935012e0bb46f1f346c7a889dbd0e0e40: Status 404 returned error can't find the container with id 6f9149c069db4b9e4f672321e129a64935012e0bb46f1f346c7a889dbd0e0e40
Jul 30 14:49:30.400204 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.400179    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5b5697b754-9q6rd"]
Jul 30 14:49:30.403450 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:49:30.403426    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda34002ea_a683_4b17_9e6a_419802c3e1c4.slice/crio-525e5c81384d30784efb432548471d54d1ba42901bf0570201ef6a99f8cc0a12 WatchSource:0}: Error finding container 525e5c81384d30784efb432548471d54d1ba42901bf0570201ef6a99f8cc0a12: Status 404 returned error can't find the container with id 525e5c81384d30784efb432548471d54d1ba42901bf0570201ef6a99f8cc0a12
Jul 30 14:49:30.442784 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.442761    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-ca-trust-extracted\") pod \"6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7\" (UID: \"6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7\") "
Jul 30 14:49:30.442894 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.442850    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-registry-certificates\") pod \"6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7\" (UID: \"6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7\") "
Jul 30 14:49:30.442894 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.442880    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-trusted-ca\") pod \"6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7\" (UID: \"6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7\") "
Jul 30 14:49:30.442999 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.442918    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-installation-pull-secrets\") pod \"6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7\" (UID: \"6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7\") "
Jul 30 14:49:30.442999 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.442944    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-bound-sa-token\") pod \"6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7\" (UID: \"6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7\") "
Jul 30 14:49:30.442999 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.442973    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-image-registry-private-configuration\") pod \"6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7\" (UID: \"6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7\") "
Jul 30 14:49:30.442999 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.442978    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7" (UID: "6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 14:49:30.443212 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.443004    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wqjs4\" (UniqueName: \"kubernetes.io/projected/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-kube-api-access-wqjs4\") pod \"6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7\" (UID: \"6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7\") "
Jul 30 14:49:30.443295 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.443279    2578 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-ca-trust-extracted\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 14:49:30.443354 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.443271    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7" (UID: "6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 14:49:30.443354 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.443314    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7" (UID: "6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 14:49:30.445076 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.445049    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7" (UID: "6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 14:49:30.445192 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.445079    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7" (UID: "6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 14:49:30.445464 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.445443    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7" (UID: "6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 14:49:30.445524 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.445493    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-kube-api-access-wqjs4" (OuterVolumeSpecName: "kube-api-access-wqjs4") pod "6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7" (UID: "6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7"). InnerVolumeSpecName "kube-api-access-wqjs4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 14:49:30.544455 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.544424    2578 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-registry-certificates\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 14:49:30.544455 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.544451    2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-trusted-ca\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 14:49:30.544455 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.544463    2578 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-installation-pull-secrets\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 14:49:30.544674 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.544472    2578 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-bound-sa-token\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 14:49:30.544674 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.544482    2578 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-image-registry-private-configuration\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 14:49:30.544674 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:30.544490    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqjs4\" (UniqueName: \"kubernetes.io/projected/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-kube-api-access-wqjs4\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 14:49:31.297752 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:31.297495    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-cw96h" event={"ID":"44c1d3a9-8aa4-4515-9a6c-2732f5eca386","Type":"ContainerStarted","Data":"6f9149c069db4b9e4f672321e129a64935012e0bb46f1f346c7a889dbd0e0e40"}
Jul 30 14:49:31.301792 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:31.300513    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-684cf4bf4d-2j296"
Jul 30 14:49:31.301792 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:31.301501    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b5697b754-9q6rd" event={"ID":"a34002ea-a683-4b17-9e6a-419802c3e1c4","Type":"ContainerStarted","Data":"7130ce798ffaeb62998388897f1a62d3acf9e854156c18d100f7b6bc4d784952"}
Jul 30 14:49:31.301792 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:31.301542    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5b5697b754-9q6rd"
Jul 30 14:49:31.301792 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:31.301562    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b5697b754-9q6rd" event={"ID":"a34002ea-a683-4b17-9e6a-419802c3e1c4","Type":"ContainerStarted","Data":"525e5c81384d30784efb432548471d54d1ba42901bf0570201ef6a99f8cc0a12"}
Jul 30 14:49:31.322333 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:31.321301    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5b5697b754-9q6rd" podStartSLOduration=2.321279633 podStartE2EDuration="2.321279633s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 14:49:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:49:31.320633929 +0000 UTC m=+82.849572406" watchObservedRunningTime="2026-07-30 14:49:31.321279633 +0000 UTC m=+82.850218109"
Jul 30 14:49:31.348903 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:31.348857    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-684cf4bf4d-2j296"]
Jul 30 14:49:31.350787 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:31.350737    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-684cf4bf4d-2j296"]
Jul 30 14:49:31.453466 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:31.452262    2578 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7-registry-tls\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 14:49:32.977731 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:32.977692    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7" path="/var/lib/kubelet/pods/6d51d0c8-2dcd-46ba-bb47-742eb9aff8b7/volumes"
Jul 30 14:49:35.636041 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:35.636007    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6b75c955fd-b7cgs"]
Jul 30 14:49:35.646013 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:35.645989    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b75c955fd-b7cgs"
Jul 30 14:49:35.648449 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:35.648423    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 30 14:49:35.648615 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:35.648436    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 30 14:49:35.649094 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:35.648822    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 30 14:49:35.649094 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:35.648850    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 30 14:49:35.649094 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:35.649055    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 30 14:49:35.649351 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:35.649273    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-m7lc8\""
Jul 30 14:49:35.649487 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:35.649433    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b75c955fd-b7cgs"]
Jul 30 14:49:35.791369 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:35.791339    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/31e691be-921a-4cf8-b046-245fa3cac6f4-service-ca\") pod \"console-6b75c955fd-b7cgs\" (UID: \"31e691be-921a-4cf8-b046-245fa3cac6f4\") " pod="openshift-console/console-6b75c955fd-b7cgs"
Jul 30 14:49:35.791481 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:35.791378    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/31e691be-921a-4cf8-b046-245fa3cac6f4-console-oauth-config\") pod \"console-6b75c955fd-b7cgs\" (UID: \"31e691be-921a-4cf8-b046-245fa3cac6f4\") " pod="openshift-console/console-6b75c955fd-b7cgs"
Jul 30 14:49:35.791536 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:35.791478    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/31e691be-921a-4cf8-b046-245fa3cac6f4-oauth-serving-cert\") pod \"console-6b75c955fd-b7cgs\" (UID: \"31e691be-921a-4cf8-b046-245fa3cac6f4\") " pod="openshift-console/console-6b75c955fd-b7cgs"
Jul 30 14:49:35.791536 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:35.791504    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/31e691be-921a-4cf8-b046-245fa3cac6f4-console-config\") pod \"console-6b75c955fd-b7cgs\" (UID: \"31e691be-921a-4cf8-b046-245fa3cac6f4\") " pod="openshift-console/console-6b75c955fd-b7cgs"
Jul 30 14:49:35.791638 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:35.791560    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9qfl\" (UniqueName: \"kubernetes.io/projected/31e691be-921a-4cf8-b046-245fa3cac6f4-kube-api-access-r9qfl\") pod \"console-6b75c955fd-b7cgs\" (UID: \"31e691be-921a-4cf8-b046-245fa3cac6f4\") " pod="openshift-console/console-6b75c955fd-b7cgs"
Jul 30 14:49:35.791638 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:35.791613    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/31e691be-921a-4cf8-b046-245fa3cac6f4-console-serving-cert\") pod \"console-6b75c955fd-b7cgs\" (UID: \"31e691be-921a-4cf8-b046-245fa3cac6f4\") " pod="openshift-console/console-6b75c955fd-b7cgs"
Jul 30 14:49:35.893040 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:35.892960    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/31e691be-921a-4cf8-b046-245fa3cac6f4-service-ca\") pod \"console-6b75c955fd-b7cgs\" (UID: \"31e691be-921a-4cf8-b046-245fa3cac6f4\") " pod="openshift-console/console-6b75c955fd-b7cgs"
Jul 30 14:49:35.893040 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:35.893008    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/31e691be-921a-4cf8-b046-245fa3cac6f4-console-oauth-config\") pod \"console-6b75c955fd-b7cgs\" (UID: \"31e691be-921a-4cf8-b046-245fa3cac6f4\") " pod="openshift-console/console-6b75c955fd-b7cgs"
Jul 30 14:49:35.893313 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:35.893118    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/31e691be-921a-4cf8-b046-245fa3cac6f4-oauth-serving-cert\") pod \"console-6b75c955fd-b7cgs\" (UID: \"31e691be-921a-4cf8-b046-245fa3cac6f4\") " pod="openshift-console/console-6b75c955fd-b7cgs"
Jul 30 14:49:35.893313 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:35.893163    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/31e691be-921a-4cf8-b046-245fa3cac6f4-console-config\") pod \"console-6b75c955fd-b7cgs\" (UID: \"31e691be-921a-4cf8-b046-245fa3cac6f4\") " pod="openshift-console/console-6b75c955fd-b7cgs"
Jul 30 14:49:35.893313 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:35.893226    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9qfl\" (UniqueName: \"kubernetes.io/projected/31e691be-921a-4cf8-b046-245fa3cac6f4-kube-api-access-r9qfl\") pod \"console-6b75c955fd-b7cgs\" (UID: \"31e691be-921a-4cf8-b046-245fa3cac6f4\") " pod="openshift-console/console-6b75c955fd-b7cgs"
Jul 30 14:49:35.893313 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:35.893255    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/31e691be-921a-4cf8-b046-245fa3cac6f4-console-serving-cert\") pod \"console-6b75c955fd-b7cgs\" (UID: \"31e691be-921a-4cf8-b046-245fa3cac6f4\") " pod="openshift-console/console-6b75c955fd-b7cgs"
Jul 30 14:49:35.893760 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:35.893733    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/31e691be-921a-4cf8-b046-245fa3cac6f4-service-ca\") pod \"console-6b75c955fd-b7cgs\" (UID: \"31e691be-921a-4cf8-b046-245fa3cac6f4\") " pod="openshift-console/console-6b75c955fd-b7cgs"
Jul 30 14:49:35.893878 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:35.893793    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/31e691be-921a-4cf8-b046-245fa3cac6f4-oauth-serving-cert\") pod \"console-6b75c955fd-b7cgs\" (UID: \"31e691be-921a-4cf8-b046-245fa3cac6f4\") " pod="openshift-console/console-6b75c955fd-b7cgs"
Jul 30 14:49:35.893878 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:35.893796    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/31e691be-921a-4cf8-b046-245fa3cac6f4-console-config\") pod \"console-6b75c955fd-b7cgs\" (UID: \"31e691be-921a-4cf8-b046-245fa3cac6f4\") " pod="openshift-console/console-6b75c955fd-b7cgs"
Jul 30 14:49:35.896562 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:35.896536    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/31e691be-921a-4cf8-b046-245fa3cac6f4-console-serving-cert\") pod \"console-6b75c955fd-b7cgs\" (UID: \"31e691be-921a-4cf8-b046-245fa3cac6f4\") " pod="openshift-console/console-6b75c955fd-b7cgs"
Jul 30 14:49:35.896686 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:35.896568    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/31e691be-921a-4cf8-b046-245fa3cac6f4-console-oauth-config\") pod \"console-6b75c955fd-b7cgs\" (UID: \"31e691be-921a-4cf8-b046-245fa3cac6f4\") " pod="openshift-console/console-6b75c955fd-b7cgs"
Jul 30 14:49:35.901324 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:35.901299    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9qfl\" (UniqueName: \"kubernetes.io/projected/31e691be-921a-4cf8-b046-245fa3cac6f4-kube-api-access-r9qfl\") pod \"console-6b75c955fd-b7cgs\" (UID: \"31e691be-921a-4cf8-b046-245fa3cac6f4\") " pod="openshift-console/console-6b75c955fd-b7cgs"
Jul 30 14:49:35.957645 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:35.957585    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b75c955fd-b7cgs"
Jul 30 14:49:36.116198 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:36.116164    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b75c955fd-b7cgs"]
Jul 30 14:49:36.119490 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:49:36.119460    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31e691be_921a_4cf8_b046_245fa3cac6f4.slice/crio-53ebfb9dae2b41ec42653957ba13cd875b6e79b63eaed651d580af9a02d229a1 WatchSource:0}: Error finding container 53ebfb9dae2b41ec42653957ba13cd875b6e79b63eaed651d580af9a02d229a1: Status 404 returned error can't find the container with id 53ebfb9dae2b41ec42653957ba13cd875b6e79b63eaed651d580af9a02d229a1
Jul 30 14:49:36.320633 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:36.320591    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b75c955fd-b7cgs" event={"ID":"31e691be-921a-4cf8-b046-245fa3cac6f4","Type":"ContainerStarted","Data":"53ebfb9dae2b41ec42653957ba13cd875b6e79b63eaed651d580af9a02d229a1"}
Jul 30 14:49:36.322280 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:36.322251    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cf5ff97c-g7lbb" event={"ID":"b04f9212-57e9-453a-99ad-45191e6458c5","Type":"ContainerStarted","Data":"42826de7b717e2a0facd72d5e6b1ed96b2aebaa0b9713891d24b525a6c628085"}
Jul 30 14:49:36.322475 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:36.322457    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cf5ff97c-g7lbb"
Jul 30 14:49:36.324267 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:36.324243    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cf5ff97c-g7lbb"
Jul 30 14:49:36.337924 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:36.337866    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cf5ff97c-g7lbb" podStartSLOduration=1.863313518 podStartE2EDuration="7.337849255s" totalImagesPullingTime="5.474535737s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 14:49:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:49:36.336612539 +0000 UTC m=+87.865551013" watchObservedRunningTime="2026-07-30 14:49:36.337849255 +0000 UTC m=+87.866787756"
Jul 30 14:49:38.012592 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:38.012547    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/623272af-807b-4e8c-b308-3c2737b1fe31-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9lk5c\" (UID: \"623272af-807b-4e8c-b308-3c2737b1fe31\") " pod="openshift-insights/insights-runtime-extractor-9lk5c"
Jul 30 14:49:38.015217 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:38.015188    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/623272af-807b-4e8c-b308-3c2737b1fe31-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9lk5c\" (UID: \"623272af-807b-4e8c-b308-3c2737b1fe31\") " pod="openshift-insights/insights-runtime-extractor-9lk5c"
Jul 30 14:49:38.315455 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:38.315421    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9mfqr\""
Jul 30 14:49:38.323581 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:38.323554    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9lk5c"
Jul 30 14:49:39.573746 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:39.573708    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9lk5c"]
Jul 30 14:49:39.576701 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:49:39.576662    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod623272af_807b_4e8c_b308_3c2737b1fe31.slice/crio-d13e7607a4b6b19885085397e213d866fdd6af9ae7c25bb420ea080e75259274 WatchSource:0}: Error finding container d13e7607a4b6b19885085397e213d866fdd6af9ae7c25bb420ea080e75259274: Status 404 returned error can't find the container with id d13e7607a4b6b19885085397e213d866fdd6af9ae7c25bb420ea080e75259274
Jul 30 14:49:40.342052 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:40.341976    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9lk5c" event={"ID":"623272af-807b-4e8c-b308-3c2737b1fe31","Type":"ContainerStarted","Data":"d13e7607a4b6b19885085397e213d866fdd6af9ae7c25bb420ea080e75259274"}
Jul 30 14:49:40.344245 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:40.344095    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b75c955fd-b7cgs" event={"ID":"31e691be-921a-4cf8-b046-245fa3cac6f4","Type":"ContainerStarted","Data":"e9c12a3a760fdac5159bde79de6805b81946f85e6b0b9fc7819dff3e26cfee2b"}
Jul 30 14:49:40.366179 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:40.366086    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6b75c955fd-b7cgs" podStartSLOduration=2.035791072 podStartE2EDuration="5.366067161s" totalImagesPullingTime="3.330276089s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 14:49:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:49:40.365074353 +0000 UTC m=+91.894012840" watchObservedRunningTime="2026-07-30 14:49:40.366067161 +0000 UTC m=+91.895005636"
Jul 30 14:49:45.958422 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:45.958383    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6b75c955fd-b7cgs"
Jul 30 14:49:45.958863 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:45.958458    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6b75c955fd-b7cgs"
Jul 30 14:49:45.963635 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:45.963610    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6b75c955fd-b7cgs"
Jul 30 14:49:46.243989 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:46.243917    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-j6hld"
Jul 30 14:49:46.371141 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:46.371087    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6b75c955fd-b7cgs"
Jul 30 14:49:48.299319 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:48.299280    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1-metrics-tls\") pod \"dns-default-nzzqf\" (UID: \"64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1\") " pod="openshift-dns/dns-default-nzzqf"
Jul 30 14:49:48.301840 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:48.301816    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1-metrics-tls\") pod \"dns-default-nzzqf\" (UID: \"64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1\") " pod="openshift-dns/dns-default-nzzqf"
Jul 30 14:49:48.531466 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:48.531434    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vrbw2\""
Jul 30 14:49:48.538792 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:48.538757    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nzzqf"
Jul 30 14:49:51.116522 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:51.115089    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-4rg5f"]
Jul 30 14:49:51.130081 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:51.129988    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-bf7cb"]
Jul 30 14:49:51.130264 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:51.130094    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-4rg5f"
Jul 30 14:49:51.133388 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:51.133362    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 30 14:49:51.133585 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:51.133569    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 30 14:49:51.133832 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:51.133804    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 30 14:49:51.134040 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:51.134020    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 30 14:49:51.134119 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:51.134048    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 30 14:49:51.134370 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:51.134283    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-kbwkz\""
Jul 30 14:49:51.160168 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:51.155336    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-4rg5f"]
Jul 30 14:49:51.160168 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:51.155474    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-bf7cb"
Jul 30 14:49:51.160168 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:51.158135    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 30 14:49:51.160168 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:51.158338    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 30 14:49:51.160168 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:51.158498    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-m4t2j\""
Jul 30 14:49:51.160168 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:51.158701    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 30 14:49:51.225594 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:51.225562    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1991b76c-7420-4566-9e5e-9f17fd8d35de-node-exporter-tls\") pod \"node-exporter-bf7cb\" (UID: \"1991b76c-7420-4566-9e5e-9f17fd8d35de\") " pod="openshift-monitoring/node-exporter-bf7cb"
Jul 30 14:49:51.225765 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:51.225608    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9630d2b8-95d8-4c7b-a5a0-643b2746daf0-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-4rg5f\" (UID: \"9630d2b8-95d8-4c7b-a5a0-643b2746daf0\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-4rg5f"
Jul 30 14:49:51.225765 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:51.225672    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1991b76c-7420-4566-9e5e-9f17fd8d35de-node-exporter-textfile\") pod \"node-exporter-bf7cb\" (UID: \"1991b76c-7420-4566-9e5e-9f17fd8d35de\") " pod="openshift-monitoring/node-exporter-bf7cb"
Jul 30 14:49:51.225765 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:51.225729    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1991b76c-7420-4566-9e5e-9f17fd8d35de-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bf7cb\" (UID: \"1991b76c-7420-4566-9e5e-9f17fd8d35de\") " pod="openshift-monitoring/node-exporter-bf7cb"
Jul 30 14:49:51.225765 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:51.225756    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1991b76c-7420-4566-9e5e-9f17fd8d35de-sys\") pod \"node-exporter-bf7cb\" (UID: \"1991b76c-7420-4566-9e5e-9f17fd8d35de\") " pod="openshift-monitoring/node-exporter-bf7cb"
Jul 30 14:49:51.225968 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:51.225826    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sf9fh\" (UniqueName: \"kubernetes.io/projected/9630d2b8-95d8-4c7b-a5a0-643b2746daf0-kube-api-access-sf9fh\") pod \"openshift-state-metrics-b8f49f5c6-4rg5f\" (UID: \"9630d2b8-95d8-4c7b-a5a0-643b2746daf0\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-4rg5f"
Jul 30 14:49:51.225968 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:51.225863    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1991b76c-7420-4566-9e5e-9f17fd8d35de-root\") pod \"node-exporter-bf7cb\" (UID: \"1991b76c-7420-4566-9e5e-9f17fd8d35de\") " pod="openshift-monitoring/node-exporter-bf7cb"
Jul 30 14:49:51.225968 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:51.225892    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1991b76c-7420-4566-9e5e-9f17fd8d35de-metrics-client-ca\") pod \"node-exporter-bf7cb\" (UID: \"1991b76c-7420-4566-9e5e-9f17fd8d35de\") " pod="openshift-monitoring/node-exporter-bf7cb"
Jul 30 14:49:51.225968 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:51.225941    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qvf2\" (UniqueName: \"kubernetes.io/projected/1991b76c-7420-4566-9e5e-9f17fd8d35de-kube-api-access-6qvf2\") pod \"node-exporter-bf7cb\" (UID: \"1991b76c-7420-4566-9e5e-9f17fd8d35de\") " pod="openshift-monitoring/node-exporter-bf7cb"
Jul 30 14:49:51.226098 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:51.225969    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1991b76c-7420-4566-9e5e-9f17fd8d35de-node-exporter-wtmp\") pod \"node-exporter-bf7cb\" (UID: \"1991b76c-7420-4566-9e5e-9f17fd8d35de\") " pod="openshift-monitoring/node-exporter-bf7cb"
Jul 30 14:49:51.226098 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:51.226020    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1991b76c-7420-4566-9e5e-9f17fd8d35de-node-exporter-accelerators-collector-config\") pod \"node-exporter-bf7cb\" (UID: \"1991b76c-7420-4566-9e5e-9f17fd8d35de\") " pod="openshift-monitoring/node-exporter-bf7cb"
Jul 30 14:49:51.226098 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:51.226063    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9630d2b8-95d8-4c7b-a5a0-643b2746daf0-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-4rg5f\" (UID: \"9630d2b8-95d8-4c7b-a5a0-643b2746daf0\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-4rg5f"
Jul 30 14:49:51.226098 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:51.226091    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9630d2b8-95d8-4c7b-a5a0-643b2746daf0-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-4rg5f\" (UID: \"9630d2b8-95d8-4c7b-a5a0-643b2746daf0\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-4rg5f"
Jul 30 14:49:51.327100 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:51.327073    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1991b76c-7420-4566-9e5e-9f17fd8d35de-node-exporter-accelerators-collector-config\") pod \"node-exporter-bf7cb\" (UID: \"1991b76c-7420-4566-9e5e-9f17fd8d35de\") " pod="openshift-monitoring/node-exporter-bf7cb"
Jul 30 14:49:51.327248 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:51.327115    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9630d2b8-95d8-4c7b-a5a0-643b2746daf0-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-4rg5f\" (UID: \"9630d2b8-95d8-4c7b-a5a0-643b2746daf0\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-4rg5f"
Jul 30 14:49:51.327248 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:51.327137    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9630d2b8-95d8-4c7b-a5a0-643b2746daf0-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-4rg5f\" (UID: \"9630d2b8-95d8-4c7b-a5a0-643b2746daf0\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-4rg5f"
Jul 30 14:49:51.327248 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:51.327175    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1991b76c-7420-4566-9e5e-9f17fd8d35de-node-exporter-tls\") pod \"node-exporter-bf7cb\" (UID: \"1991b76c-7420-4566-9e5e-9f17fd8d35de\") " pod="openshift-monitoring/node-exporter-bf7cb"
Jul 30 14:49:51.327248 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:51.327205    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9630d2b8-95d8-4c7b-a5a0-643b2746daf0-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-4rg5f\" (UID: \"9630d2b8-95d8-4c7b-a5a0-643b2746daf0\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-4rg5f"
Jul 30 14:49:51.327248 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:51.327235    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1991b76c-7420-4566-9e5e-9f17fd8d35de-node-exporter-textfile\") pod \"node-exporter-bf7cb\" (UID: \"1991b76c-7420-4566-9e5e-9f17fd8d35de\") " pod="openshift-monitoring/node-exporter-bf7cb"
Jul 30 14:49:51.327509 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:51.327265    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1991b76c-7420-4566-9e5e-9f17fd8d35de-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bf7cb\" (UID: \"1991b76c-7420-4566-9e5e-9f17fd8d35de\") " pod="openshift-monitoring/node-exporter-bf7cb"
Jul 30 14:49:51.327509 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:51.327291    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1991b76c-7420-4566-9e5e-9f17fd8d35de-sys\") pod \"node-exporter-bf7cb\" (UID: \"1991b76c-7420-4566-9e5e-9f17fd8d35de\") " pod="openshift-monitoring/node-exporter-bf7cb"
Jul 30 14:49:51.327509 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:51.327322    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sf9fh\" (UniqueName: \"kubernetes.io/projected/9630d2b8-95d8-4c7b-a5a0-643b2746daf0-kube-api-access-sf9fh\") pod \"openshift-state-metrics-b8f49f5c6-4rg5f\" (UID: \"9630d2b8-95d8-4c7b-a5a0-643b2746daf0\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-4rg5f"
Jul 30 14:49:51.327509 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:51.327344    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1991b76c-7420-4566-9e5e-9f17fd8d35de-root\") pod \"node-exporter-bf7cb\" (UID: \"1991b76c-7420-4566-9e5e-9f17fd8d35de\") " pod="openshift-monitoring/node-exporter-bf7cb"
Jul 30 14:49:51.327509 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:51.327369    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1991b76c-7420-4566-9e5e-9f17fd8d35de-metrics-client-ca\") pod \"node-exporter-bf7cb\" (UID: \"1991b76c-7420-4566-9e5e-9f17fd8d35de\") " pod="openshift-monitoring/node-exporter-bf7cb"
Jul 30 14:49:51.327509 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:51.327403    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qvf2\" (UniqueName: \"kubernetes.io/projected/1991b76c-7420-4566-9e5e-9f17fd8d35de-kube-api-access-6qvf2\") pod \"node-exporter-bf7cb\" (UID: \"1991b76c-7420-4566-9e5e-9f17fd8d35de\") " pod="openshift-monitoring/node-exporter-bf7cb"
Jul 30 14:49:51.327509 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:51.327430    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1991b76c-7420-4566-9e5e-9f17fd8d35de-node-exporter-wtmp\") pod \"node-exporter-bf7cb\" (UID: \"1991b76c-7420-4566-9e5e-9f17fd8d35de\") " pod="openshift-monitoring/node-exporter-bf7cb"
Jul 30 14:49:51.327861 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:51.327604    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1991b76c-7420-4566-9e5e-9f17fd8d35de-node-exporter-wtmp\") pod \"node-exporter-bf7cb\" (UID: \"1991b76c-7420-4566-9e5e-9f17fd8d35de\") " pod="openshift-monitoring/node-exporter-bf7cb"
Jul 30 14:49:51.327861 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:49:51.327711    2578 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 30 14:49:51.327861 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:51.327760    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1991b76c-7420-4566-9e5e-9f17fd8d35de-node-exporter-accelerators-collector-config\") pod \"node-exporter-bf7cb\" (UID: \"1991b76c-7420-4566-9e5e-9f17fd8d35de\") " pod="openshift-monitoring/node-exporter-bf7cb"
Jul 30 14:49:51.327861 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:49:51.327772    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9630d2b8-95d8-4c7b-a5a0-643b2746daf0-openshift-state-metrics-tls podName:9630d2b8-95d8-4c7b-a5a0-643b2746daf0 nodeName:}" failed. No retries permitted until 2026-07-30 14:49:51.827754137 +0000 UTC m=+103.356692590 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/9630d2b8-95d8-4c7b-a5a0-643b2746daf0-openshift-state-metrics-tls") pod "openshift-state-metrics-b8f49f5c6-4rg5f" (UID: "9630d2b8-95d8-4c7b-a5a0-643b2746daf0") : secret "openshift-state-metrics-tls" not found
Jul 30 14:49:51.327861 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:51.327856    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9630d2b8-95d8-4c7b-a5a0-643b2746daf0-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-4rg5f\" (UID: \"9630d2b8-95d8-4c7b-a5a0-643b2746daf0\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-4rg5f"
Jul 30 14:49:51.328116 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:51.327856    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1991b76c-7420-4566-9e5e-9f17fd8d35de-root\") pod \"node-exporter-bf7cb\" (UID: \"1991b76c-7420-4566-9e5e-9f17fd8d35de\") " pod="openshift-monitoring/node-exporter-bf7cb"
Jul 30 14:49:51.328116 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:51.328095    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1991b76c-7420-4566-9e5e-9f17fd8d35de-node-exporter-textfile\") pod \"node-exporter-bf7cb\" (UID: \"1991b76c-7420-4566-9e5e-9f17fd8d35de\") " pod="openshift-monitoring/node-exporter-bf7cb"
Jul 30 14:49:51.328572 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:51.328308    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1991b76c-7420-4566-9e5e-9f17fd8d35de-sys\") pod \"node-exporter-bf7cb\" (UID: \"1991b76c-7420-4566-9e5e-9f17fd8d35de\") " pod="openshift-monitoring/node-exporter-bf7cb"
Jul 30 14:49:51.328572 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:51.328339    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1991b76c-7420-4566-9e5e-9f17fd8d35de-metrics-client-ca\") pod \"node-exporter-bf7cb\" (UID: \"1991b76c-7420-4566-9e5e-9f17fd8d35de\") " pod="openshift-monitoring/node-exporter-bf7cb"
Jul 30 14:49:51.330431 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:51.330412    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1991b76c-7420-4566-9e5e-9f17fd8d35de-node-exporter-tls\") pod \"node-exporter-bf7cb\" (UID: \"1991b76c-7420-4566-9e5e-9f17fd8d35de\") " pod="openshift-monitoring/node-exporter-bf7cb"
Jul 30 14:49:51.330614 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:51.330596    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1991b76c-7420-4566-9e5e-9f17fd8d35de-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bf7cb\" (UID: \"1991b76c-7420-4566-9e5e-9f17fd8d35de\") " pod="openshift-monitoring/node-exporter-bf7cb"
Jul 30 14:49:51.330777 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:51.330754    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9630d2b8-95d8-4c7b-a5a0-643b2746daf0-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-4rg5f\" (UID: \"9630d2b8-95d8-4c7b-a5a0-643b2746daf0\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-4rg5f"
Jul 30 14:49:51.340218 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:51.340141    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sf9fh\" (UniqueName: \"kubernetes.io/projected/9630d2b8-95d8-4c7b-a5a0-643b2746daf0-kube-api-access-sf9fh\") pod \"openshift-state-metrics-b8f49f5c6-4rg5f\" (UID: \"9630d2b8-95d8-4c7b-a5a0-643b2746daf0\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-4rg5f"
Jul 30 14:49:51.342391 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:51.342307    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qvf2\" (UniqueName: \"kubernetes.io/projected/1991b76c-7420-4566-9e5e-9f17fd8d35de-kube-api-access-6qvf2\") pod \"node-exporter-bf7cb\" (UID: \"1991b76c-7420-4566-9e5e-9f17fd8d35de\") " pod="openshift-monitoring/node-exporter-bf7cb"
Jul 30 14:49:51.397113 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:51.397079    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nzzqf"]
Jul 30 14:49:51.398993 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:49:51.398963    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64ecf7a0_7c1e_4b43_b360_2a49baf8fdb1.slice/crio-ed6e142d6dadc200a65dc185f0ee5cd451733309f72dcbb93a874e586103b5d6 WatchSource:0}: Error finding container ed6e142d6dadc200a65dc185f0ee5cd451733309f72dcbb93a874e586103b5d6: Status 404 returned error can't find the container with id ed6e142d6dadc200a65dc185f0ee5cd451733309f72dcbb93a874e586103b5d6
Jul 30 14:49:51.468300 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:51.468271    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-bf7cb"
Jul 30 14:49:51.614848 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:49:51.614817    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1991b76c_7420_4566_9e5e_9f17fd8d35de.slice/crio-c16dda828b089f3b8fe25c37a7576a0b2968c863df12a3a36683cdbca082f158 WatchSource:0}: Error finding container c16dda828b089f3b8fe25c37a7576a0b2968c863df12a3a36683cdbca082f158: Status 404 returned error can't find the container with id c16dda828b089f3b8fe25c37a7576a0b2968c863df12a3a36683cdbca082f158
Jul 30 14:49:51.832658 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:51.832620    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9630d2b8-95d8-4c7b-a5a0-643b2746daf0-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-4rg5f\" (UID: \"9630d2b8-95d8-4c7b-a5a0-643b2746daf0\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-4rg5f"
Jul 30 14:49:51.835310 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:51.835281    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9630d2b8-95d8-4c7b-a5a0-643b2746daf0-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-4rg5f\" (UID: \"9630d2b8-95d8-4c7b-a5a0-643b2746daf0\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-4rg5f"
Jul 30 14:49:52.050375 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:52.050338    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-4rg5f"
Jul 30 14:49:52.242823 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:52.241994    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-4rg5f"]
Jul 30 14:49:52.250678 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:49:52.250646    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9630d2b8_95d8_4c7b_a5a0_643b2746daf0.slice/crio-b80e58fa777890b9b7451be7a3bbde4ca52166a1f8e27dd374989e876f9b3903 WatchSource:0}: Error finding container b80e58fa777890b9b7451be7a3bbde4ca52166a1f8e27dd374989e876f9b3903: Status 404 returned error can't find the container with id b80e58fa777890b9b7451be7a3bbde4ca52166a1f8e27dd374989e876f9b3903
Jul 30 14:49:52.315730 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:52.315210    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5b5697b754-9q6rd"
Jul 30 14:49:52.390850 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:52.390761    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9lk5c" event={"ID":"623272af-807b-4e8c-b308-3c2737b1fe31","Type":"ContainerStarted","Data":"614babe0794bd62e36b0be79ee446a0c34de5c5cff928aa4e65cd87abcecf375"}
Jul 30 14:49:52.401719 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:52.401360    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bf7cb" event={"ID":"1991b76c-7420-4566-9e5e-9f17fd8d35de","Type":"ContainerStarted","Data":"c16dda828b089f3b8fe25c37a7576a0b2968c863df12a3a36683cdbca082f158"}
Jul 30 14:49:52.415038 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:52.415000    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nzzqf" event={"ID":"64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1","Type":"ContainerStarted","Data":"ed6e142d6dadc200a65dc185f0ee5cd451733309f72dcbb93a874e586103b5d6"}
Jul 30 14:49:52.429248 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:52.429131    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-cw96h" event={"ID":"44c1d3a9-8aa4-4515-9a6c-2732f5eca386","Type":"ContainerStarted","Data":"d83d837a7c52a534ecca4832c345f12b05a2169d19c95e2a881089f2a4133ba0"}
Jul 30 14:49:52.432497 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:52.432380    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-4rg5f" event={"ID":"9630d2b8-95d8-4c7b-a5a0-643b2746daf0","Type":"ContainerStarted","Data":"9ab2433150c366298ed2b8ec0c577a11668b211f2fa07f6965870b49281a6a3d"}
Jul 30 14:49:52.432497 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:52.432444    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-4rg5f" event={"ID":"9630d2b8-95d8-4c7b-a5a0-643b2746daf0","Type":"ContainerStarted","Data":"b80e58fa777890b9b7451be7a3bbde4ca52166a1f8e27dd374989e876f9b3903"}
Jul 30 14:49:52.450970 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:52.449260    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-cw96h"
Jul 30 14:49:52.454692 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:52.454607    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-cw96h"
Jul 30 14:49:52.500460 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:52.499973    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-cw96h" podStartSLOduration=2.628352353 podStartE2EDuration="23.499948657s" totalImagesPullingTime="20.871596304s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 14:49:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:49:52.470387697 +0000 UTC m=+103.999326172" watchObservedRunningTime="2026-07-30 14:49:52.499948657 +0000 UTC m=+104.028887131"
Jul 30 14:49:53.444291 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:53.444198    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-4rg5f" event={"ID":"9630d2b8-95d8-4c7b-a5a0-643b2746daf0","Type":"ContainerStarted","Data":"cc24afc4b2fa49a132e751b7a63f2f1f2a4d142f4ac9a54fef6e2db9048d53e1"}
Jul 30 14:49:55.113863 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:55.113831    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b75c955fd-b7cgs"]
Jul 30 14:49:55.520503 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:55.520427    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5c5df58559-ds468"]
Jul 30 14:49:55.585943 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:55.585894    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5c5df58559-ds468"]
Jul 30 14:49:55.586204 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:55.586043    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5c5df58559-ds468"
Jul 30 14:49:55.588798 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:55.588755    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 30 14:49:55.588932 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:55.588790    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 30 14:49:55.589663 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:55.589109    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 30 14:49:55.589663 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:55.589432    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 30 14:49:55.589663 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:55.589603    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-2wc74\""
Jul 30 14:49:55.590104 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:55.590049    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-d7n5n8vhjd0uo\""
Jul 30 14:49:55.677953 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:55.677918    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/00d9e19d-ff30-4139-984d-1c00ffbc85cd-secret-metrics-server-tls\") pod \"metrics-server-5c5df58559-ds468\" (UID: \"00d9e19d-ff30-4139-984d-1c00ffbc85cd\") " pod="openshift-monitoring/metrics-server-5c5df58559-ds468"
Jul 30 14:49:55.678181 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:55.677993    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/00d9e19d-ff30-4139-984d-1c00ffbc85cd-secret-metrics-server-client-certs\") pod \"metrics-server-5c5df58559-ds468\" (UID: \"00d9e19d-ff30-4139-984d-1c00ffbc85cd\") " pod="openshift-monitoring/metrics-server-5c5df58559-ds468"
Jul 30 14:49:55.678181 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:55.678108    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/00d9e19d-ff30-4139-984d-1c00ffbc85cd-metrics-server-audit-profiles\") pod \"metrics-server-5c5df58559-ds468\" (UID: \"00d9e19d-ff30-4139-984d-1c00ffbc85cd\") " pod="openshift-monitoring/metrics-server-5c5df58559-ds468"
Jul 30 14:49:55.678181 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:55.678172    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00d9e19d-ff30-4139-984d-1c00ffbc85cd-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5c5df58559-ds468\" (UID: \"00d9e19d-ff30-4139-984d-1c00ffbc85cd\") " pod="openshift-monitoring/metrics-server-5c5df58559-ds468"
Jul 30 14:49:55.678359 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:55.678212    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/00d9e19d-ff30-4139-984d-1c00ffbc85cd-client-ca-bundle\") pod \"metrics-server-5c5df58559-ds468\" (UID: \"00d9e19d-ff30-4139-984d-1c00ffbc85cd\") " pod="openshift-monitoring/metrics-server-5c5df58559-ds468"
Jul 30 14:49:55.678359 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:55.678238    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdhtv\" (UniqueName: \"kubernetes.io/projected/00d9e19d-ff30-4139-984d-1c00ffbc85cd-kube-api-access-wdhtv\") pod \"metrics-server-5c5df58559-ds468\" (UID: \"00d9e19d-ff30-4139-984d-1c00ffbc85cd\") " pod="openshift-monitoring/metrics-server-5c5df58559-ds468"
Jul 30 14:49:55.678359 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:55.678318    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/00d9e19d-ff30-4139-984d-1c00ffbc85cd-audit-log\") pod \"metrics-server-5c5df58559-ds468\" (UID: \"00d9e19d-ff30-4139-984d-1c00ffbc85cd\") " pod="openshift-monitoring/metrics-server-5c5df58559-ds468"
Jul 30 14:49:55.778974 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:55.778894    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/00d9e19d-ff30-4139-984d-1c00ffbc85cd-metrics-server-audit-profiles\") pod \"metrics-server-5c5df58559-ds468\" (UID: \"00d9e19d-ff30-4139-984d-1c00ffbc85cd\") " pod="openshift-monitoring/metrics-server-5c5df58559-ds468"
Jul 30 14:49:55.778974 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:55.778946    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00d9e19d-ff30-4139-984d-1c00ffbc85cd-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5c5df58559-ds468\" (UID: \"00d9e19d-ff30-4139-984d-1c00ffbc85cd\") " pod="openshift-monitoring/metrics-server-5c5df58559-ds468"
Jul 30 14:49:55.778974 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:55.778971    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/00d9e19d-ff30-4139-984d-1c00ffbc85cd-client-ca-bundle\") pod \"metrics-server-5c5df58559-ds468\" (UID: \"00d9e19d-ff30-4139-984d-1c00ffbc85cd\") " pod="openshift-monitoring/metrics-server-5c5df58559-ds468"
Jul 30 14:49:55.779621 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:55.779006    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdhtv\" (UniqueName: \"kubernetes.io/projected/00d9e19d-ff30-4139-984d-1c00ffbc85cd-kube-api-access-wdhtv\") pod \"metrics-server-5c5df58559-ds468\" (UID: \"00d9e19d-ff30-4139-984d-1c00ffbc85cd\") " pod="openshift-monitoring/metrics-server-5c5df58559-ds468"
Jul 30 14:49:55.779621 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:55.779032    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/00d9e19d-ff30-4139-984d-1c00ffbc85cd-audit-log\") pod \"metrics-server-5c5df58559-ds468\" (UID: \"00d9e19d-ff30-4139-984d-1c00ffbc85cd\") " pod="openshift-monitoring/metrics-server-5c5df58559-ds468"
Jul 30 14:49:55.779621 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:55.779082    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/00d9e19d-ff30-4139-984d-1c00ffbc85cd-secret-metrics-server-tls\") pod \"metrics-server-5c5df58559-ds468\" (UID: \"00d9e19d-ff30-4139-984d-1c00ffbc85cd\") " pod="openshift-monitoring/metrics-server-5c5df58559-ds468"
Jul 30 14:49:55.779621 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:55.779174    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/00d9e19d-ff30-4139-984d-1c00ffbc85cd-secret-metrics-server-client-certs\") pod \"metrics-server-5c5df58559-ds468\" (UID: \"00d9e19d-ff30-4139-984d-1c00ffbc85cd\") " pod="openshift-monitoring/metrics-server-5c5df58559-ds468"
Jul 30 14:49:55.779621 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:55.779515    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/00d9e19d-ff30-4139-984d-1c00ffbc85cd-audit-log\") pod \"metrics-server-5c5df58559-ds468\" (UID: \"00d9e19d-ff30-4139-984d-1c00ffbc85cd\") " pod="openshift-monitoring/metrics-server-5c5df58559-ds468"
Jul 30 14:49:55.780403 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:55.780356    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/00d9e19d-ff30-4139-984d-1c00ffbc85cd-metrics-server-audit-profiles\") pod \"metrics-server-5c5df58559-ds468\" (UID: \"00d9e19d-ff30-4139-984d-1c00ffbc85cd\") " pod="openshift-monitoring/metrics-server-5c5df58559-ds468"
Jul 30 14:49:55.780974 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:55.780927    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00d9e19d-ff30-4139-984d-1c00ffbc85cd-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5c5df58559-ds468\" (UID: \"00d9e19d-ff30-4139-984d-1c00ffbc85cd\") " pod="openshift-monitoring/metrics-server-5c5df58559-ds468"
Jul 30 14:49:55.782920 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:55.782875    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/00d9e19d-ff30-4139-984d-1c00ffbc85cd-secret-metrics-server-client-certs\") pod \"metrics-server-5c5df58559-ds468\" (UID: \"00d9e19d-ff30-4139-984d-1c00ffbc85cd\") " pod="openshift-monitoring/metrics-server-5c5df58559-ds468"
Jul 30 14:49:55.783024 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:55.782950    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/00d9e19d-ff30-4139-984d-1c00ffbc85cd-client-ca-bundle\") pod \"metrics-server-5c5df58559-ds468\" (UID: \"00d9e19d-ff30-4139-984d-1c00ffbc85cd\") " pod="openshift-monitoring/metrics-server-5c5df58559-ds468"
Jul 30 14:49:55.784206 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:55.784182    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/00d9e19d-ff30-4139-984d-1c00ffbc85cd-secret-metrics-server-tls\") pod \"metrics-server-5c5df58559-ds468\" (UID: \"00d9e19d-ff30-4139-984d-1c00ffbc85cd\") " pod="openshift-monitoring/metrics-server-5c5df58559-ds468"
Jul 30 14:49:55.789483 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:55.789433    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdhtv\" (UniqueName: \"kubernetes.io/projected/00d9e19d-ff30-4139-984d-1c00ffbc85cd-kube-api-access-wdhtv\") pod \"metrics-server-5c5df58559-ds468\" (UID: \"00d9e19d-ff30-4139-984d-1c00ffbc85cd\") " pod="openshift-monitoring/metrics-server-5c5df58559-ds468"
Jul 30 14:49:55.888305 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:55.887982    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-z4q4n"]
Jul 30 14:49:55.898445 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:55.898415    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5c5df58559-ds468"
Jul 30 14:49:55.920083 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:55.920039    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-z4q4n"]
Jul 30 14:49:55.920258 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:55.920200    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-z4q4n"
Jul 30 14:49:55.923372 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:55.922948    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-4zjpw\""
Jul 30 14:49:55.923372 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:55.923211    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 30 14:49:55.982124 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:55.981931    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smdgx\" (UniqueName: \"kubernetes.io/projected/52b76eb8-59c8-4644-8dbf-aba3ace91eb2-kube-api-access-smdgx\") pod \"monitoring-plugin-68b7fc9574-z4q4n\" (UID: \"52b76eb8-59c8-4644-8dbf-aba3ace91eb2\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-z4q4n"
Jul 30 14:49:55.982124 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:55.982044    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/52b76eb8-59c8-4644-8dbf-aba3ace91eb2-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-z4q4n\" (UID: \"52b76eb8-59c8-4644-8dbf-aba3ace91eb2\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-z4q4n"
Jul 30 14:49:56.082938 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:56.082902    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/52b76eb8-59c8-4644-8dbf-aba3ace91eb2-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-z4q4n\" (UID: \"52b76eb8-59c8-4644-8dbf-aba3ace91eb2\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-z4q4n"
Jul 30 14:49:56.083314 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:56.083000    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smdgx\" (UniqueName: \"kubernetes.io/projected/52b76eb8-59c8-4644-8dbf-aba3ace91eb2-kube-api-access-smdgx\") pod \"monitoring-plugin-68b7fc9574-z4q4n\" (UID: \"52b76eb8-59c8-4644-8dbf-aba3ace91eb2\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-z4q4n"
Jul 30 14:49:56.083314 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:49:56.083086    2578 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 30 14:49:56.083314 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:49:56.083188    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52b76eb8-59c8-4644-8dbf-aba3ace91eb2-monitoring-plugin-cert podName:52b76eb8-59c8-4644-8dbf-aba3ace91eb2 nodeName:}" failed. No retries permitted until 2026-07-30 14:49:56.583168489 +0000 UTC m=+108.112106954 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/52b76eb8-59c8-4644-8dbf-aba3ace91eb2-monitoring-plugin-cert") pod "monitoring-plugin-68b7fc9574-z4q4n" (UID: "52b76eb8-59c8-4644-8dbf-aba3ace91eb2") : secret "monitoring-plugin-cert" not found
Jul 30 14:49:56.092613 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:56.092588    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smdgx\" (UniqueName: \"kubernetes.io/projected/52b76eb8-59c8-4644-8dbf-aba3ace91eb2-kube-api-access-smdgx\") pod \"monitoring-plugin-68b7fc9574-z4q4n\" (UID: \"52b76eb8-59c8-4644-8dbf-aba3ace91eb2\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-z4q4n"
Jul 30 14:49:56.589333 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:56.589294    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/52b76eb8-59c8-4644-8dbf-aba3ace91eb2-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-z4q4n\" (UID: \"52b76eb8-59c8-4644-8dbf-aba3ace91eb2\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-z4q4n"
Jul 30 14:49:56.592391 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:56.592331    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/52b76eb8-59c8-4644-8dbf-aba3ace91eb2-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-z4q4n\" (UID: \"52b76eb8-59c8-4644-8dbf-aba3ace91eb2\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-z4q4n"
Jul 30 14:49:56.834093 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:56.834050    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-z4q4n"
Jul 30 14:49:57.475057 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.474975    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 30 14:49:57.533439 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.533361    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 30 14:49:57.533796 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.533525    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.536523 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.536497    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 30 14:49:57.536653 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.536637    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 30 14:49:57.537646 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.537074    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 30 14:49:57.537646 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.537291    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 30 14:49:57.537646 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.537312    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 30 14:49:57.537646 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.537631    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 30 14:49:57.538359 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.537969    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 30 14:49:57.538359 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.538179    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 30 14:49:57.538359 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.538296    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 30 14:49:57.538359 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.538333    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 30 14:49:57.539034 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.538333    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 30 14:49:57.539105 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.539045    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 30 14:49:57.539758 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.539684    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-n252m\""
Jul 30 14:49:57.539852 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.539766    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-aavba3djirvbn\""
Jul 30 14:49:57.540440 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.540127    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 30 14:49:57.544447 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.544289    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 30 14:49:57.600380 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.600351    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/13fc7a64-04f6-492b-bffd-c81059c7af28-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.600845 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.600529    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/13fc7a64-04f6-492b-bffd-c81059c7af28-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.600845 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.600608    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/13fc7a64-04f6-492b-bffd-c81059c7af28-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.600845 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.600636    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13fc7a64-04f6-492b-bffd-c81059c7af28-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.600845 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.600744    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/13fc7a64-04f6-492b-bffd-c81059c7af28-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.600845 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.600801    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/13fc7a64-04f6-492b-bffd-c81059c7af28-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.600845 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.600831    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/13fc7a64-04f6-492b-bffd-c81059c7af28-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.601175 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.600856    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/13fc7a64-04f6-492b-bffd-c81059c7af28-config\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.601175 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.600939    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/13fc7a64-04f6-492b-bffd-c81059c7af28-config-out\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.601175 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.600970    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/13fc7a64-04f6-492b-bffd-c81059c7af28-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.601324 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.601176    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/13fc7a64-04f6-492b-bffd-c81059c7af28-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.601324 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.601218    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/13fc7a64-04f6-492b-bffd-c81059c7af28-web-config\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.601324 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.601252    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13fc7a64-04f6-492b-bffd-c81059c7af28-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.601473 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.601369    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13fc7a64-04f6-492b-bffd-c81059c7af28-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.601473 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.601404    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/13fc7a64-04f6-492b-bffd-c81059c7af28-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.601473 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.601444    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/13fc7a64-04f6-492b-bffd-c81059c7af28-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.601610 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.601467    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/13fc7a64-04f6-492b-bffd-c81059c7af28-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.601610 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.601504    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/13fc7a64-04f6-492b-bffd-c81059c7af28-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.601610 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.601533    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skln5\" (UniqueName: \"kubernetes.io/projected/13fc7a64-04f6-492b-bffd-c81059c7af28-kube-api-access-skln5\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.601610 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.601561    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/13fc7a64-04f6-492b-bffd-c81059c7af28-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.702342 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.702293    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/13fc7a64-04f6-492b-bffd-c81059c7af28-config-out\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.702342 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.702343    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/13fc7a64-04f6-492b-bffd-c81059c7af28-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.702651 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.702378    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/13fc7a64-04f6-492b-bffd-c81059c7af28-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.702651 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.702404    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/13fc7a64-04f6-492b-bffd-c81059c7af28-web-config\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.702651 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.702432    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13fc7a64-04f6-492b-bffd-c81059c7af28-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.702651 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.702463    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13fc7a64-04f6-492b-bffd-c81059c7af28-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.702651 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.702489    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/13fc7a64-04f6-492b-bffd-c81059c7af28-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.702651 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.702524    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/13fc7a64-04f6-492b-bffd-c81059c7af28-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.702651 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.702545    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/13fc7a64-04f6-492b-bffd-c81059c7af28-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.702651 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.702571    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/13fc7a64-04f6-492b-bffd-c81059c7af28-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.702651 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.702599    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skln5\" (UniqueName: \"kubernetes.io/projected/13fc7a64-04f6-492b-bffd-c81059c7af28-kube-api-access-skln5\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.702651 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.702628    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/13fc7a64-04f6-492b-bffd-c81059c7af28-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.702651 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.702651    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/13fc7a64-04f6-492b-bffd-c81059c7af28-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.703475 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.702693    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/13fc7a64-04f6-492b-bffd-c81059c7af28-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.703475 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.702762    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/13fc7a64-04f6-492b-bffd-c81059c7af28-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.703475 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.702791    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13fc7a64-04f6-492b-bffd-c81059c7af28-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.703475 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.702816    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/13fc7a64-04f6-492b-bffd-c81059c7af28-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.703475 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.702862    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/13fc7a64-04f6-492b-bffd-c81059c7af28-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.703475 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.702885    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/13fc7a64-04f6-492b-bffd-c81059c7af28-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.703475 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.702912    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/13fc7a64-04f6-492b-bffd-c81059c7af28-config\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.703877 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.703787    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/13fc7a64-04f6-492b-bffd-c81059c7af28-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.703877 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.703818    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13fc7a64-04f6-492b-bffd-c81059c7af28-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.705003 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.704938    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/13fc7a64-04f6-492b-bffd-c81059c7af28-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.706463 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.705833    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/13fc7a64-04f6-492b-bffd-c81059c7af28-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.706463 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.706119    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/13fc7a64-04f6-492b-bffd-c81059c7af28-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.707461 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.707076    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/13fc7a64-04f6-492b-bffd-c81059c7af28-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.707461 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.707297    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/13fc7a64-04f6-492b-bffd-c81059c7af28-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.707461 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.707421    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13fc7a64-04f6-492b-bffd-c81059c7af28-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.707939 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.707757    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/13fc7a64-04f6-492b-bffd-c81059c7af28-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.708370 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.708325    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/13fc7a64-04f6-492b-bffd-c81059c7af28-web-config\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.708642 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.708611    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13fc7a64-04f6-492b-bffd-c81059c7af28-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.709577 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.709465    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/13fc7a64-04f6-492b-bffd-c81059c7af28-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.709577 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.709532    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/13fc7a64-04f6-492b-bffd-c81059c7af28-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.711386 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.711358    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/13fc7a64-04f6-492b-bffd-c81059c7af28-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.711929 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.711905    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/13fc7a64-04f6-492b-bffd-c81059c7af28-config-out\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.712655 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.712590    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/13fc7a64-04f6-492b-bffd-c81059c7af28-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.714259 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.714088    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/13fc7a64-04f6-492b-bffd-c81059c7af28-config\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.714605 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.714521    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/13fc7a64-04f6-492b-bffd-c81059c7af28-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.714789 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.714658    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/13fc7a64-04f6-492b-bffd-c81059c7af28-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.714862 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.714794    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-skln5\" (UniqueName: \"kubernetes.io/projected/13fc7a64-04f6-492b-bffd-c81059c7af28-kube-api-access-skln5\") pod \"prometheus-k8s-0\" (UID: \"13fc7a64-04f6-492b-bffd-c81059c7af28\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:57.849292 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:57.849252    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:49:58.266180 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:58.266080    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-z4q4n"]
Jul 30 14:49:58.609206 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:49:58.609173    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52b76eb8_59c8_4644_8dbf_aba3ace91eb2.slice/crio-dc150a6f77a3c66c29ed887dd9e01ad080b98698bc739e7963b8b96e062fa7aa WatchSource:0}: Error finding container dc150a6f77a3c66c29ed887dd9e01ad080b98698bc739e7963b8b96e062fa7aa: Status 404 returned error can't find the container with id dc150a6f77a3c66c29ed887dd9e01ad080b98698bc739e7963b8b96e062fa7aa
Jul 30 14:49:59.004964 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:59.004902    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5c5df58559-ds468"]
Jul 30 14:49:59.034843 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:59.034735    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 30 14:49:59.040752 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:49:59.040575    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13fc7a64_04f6_492b_bffd_c81059c7af28.slice/crio-afa3e45062051e9ed5b57e6a00e8657c79ed3fe638a37358cdd18a3f60719a8e WatchSource:0}: Error finding container afa3e45062051e9ed5b57e6a00e8657c79ed3fe638a37358cdd18a3f60719a8e: Status 404 returned error can't find the container with id afa3e45062051e9ed5b57e6a00e8657c79ed3fe638a37358cdd18a3f60719a8e
Jul 30 14:49:59.482532 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:59.482442    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"13fc7a64-04f6-492b-bffd-c81059c7af28","Type":"ContainerStarted","Data":"afa3e45062051e9ed5b57e6a00e8657c79ed3fe638a37358cdd18a3f60719a8e"}
Jul 30 14:49:59.484396 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:59.484362    2578 generic.go:356] "Generic (PLEG): container finished" podID="1991b76c-7420-4566-9e5e-9f17fd8d35de" containerID="c3092de7e1fcda9fbf73161a8cf2f91c1e2872c348980409782ebfd743461a12" exitCode=0
Jul 30 14:49:59.484516 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:59.484454    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bf7cb" event={"ID":"1991b76c-7420-4566-9e5e-9f17fd8d35de","Type":"ContainerDied","Data":"c3092de7e1fcda9fbf73161a8cf2f91c1e2872c348980409782ebfd743461a12"}
Jul 30 14:49:59.486709 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:59.486681    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nzzqf" event={"ID":"64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1","Type":"ContainerStarted","Data":"aef2203a3bec4719e1dc0e147b831d4c92a91da774cda00faae5e635cfa7463a"}
Jul 30 14:49:59.486833 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:59.486733    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nzzqf" event={"ID":"64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1","Type":"ContainerStarted","Data":"5499397474df73a4bf935a4e85eeff5b10f9e3cfd45a06278f31b779e67364f3"}
Jul 30 14:49:59.486833 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:59.486796    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-nzzqf"
Jul 30 14:49:59.488201 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:59.488175    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5c5df58559-ds468" event={"ID":"00d9e19d-ff30-4139-984d-1c00ffbc85cd","Type":"ContainerStarted","Data":"17afaf353ea8393c0ae22da96232bd2d07dafc0344251b418d61f054527e35ca"}
Jul 30 14:49:59.490531 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:59.490510    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-4rg5f" event={"ID":"9630d2b8-95d8-4c7b-a5a0-643b2746daf0","Type":"ContainerStarted","Data":"c3f982b9f682b1560994ef802fd0fa61bdeb86c8dc7df2a74b6238b32334aab7"}
Jul 30 14:49:59.492496 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:59.492475    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9lk5c" event={"ID":"623272af-807b-4e8c-b308-3c2737b1fe31","Type":"ContainerStarted","Data":"4e4ce719ab232c702fb9c23453177708ad471741f187407281894bd67498dd90"}
Jul 30 14:49:59.493776 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:59.493750    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-z4q4n" event={"ID":"52b76eb8-59c8-4644-8dbf-aba3ace91eb2","Type":"ContainerStarted","Data":"dc150a6f77a3c66c29ed887dd9e01ad080b98698bc739e7963b8b96e062fa7aa"}
Jul 30 14:49:59.521041 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:59.520989    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-4rg5f" podStartSLOduration=2.272241149 podStartE2EDuration="8.520973118s" totalImagesPullingTime="6.248731969s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 14:49:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:49:59.519829209 +0000 UTC m=+111.048767684" watchObservedRunningTime="2026-07-30 14:49:59.520973118 +0000 UTC m=+111.049911592"
Jul 30 14:49:59.538486 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:49:59.538439    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-nzzqf" podStartSLOduration=69.004438832 podStartE2EDuration="1m15.538424494s" totalImagesPullingTime="6.533985662s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 14:48:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:49:59.537713155 +0000 UTC m=+111.066651670" watchObservedRunningTime="2026-07-30 14:49:59.538424494 +0000 UTC m=+111.067362995"
Jul 30 14:50:00.504982 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:00.504943    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bf7cb" event={"ID":"1991b76c-7420-4566-9e5e-9f17fd8d35de","Type":"ContainerStarted","Data":"a77b8296ab0c9e1e1fd8fd79e9fda9be09930a88cdd3b92ff191e15ae9af7bfb"}
Jul 30 14:50:00.505595 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:00.505013    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bf7cb" event={"ID":"1991b76c-7420-4566-9e5e-9f17fd8d35de","Type":"ContainerStarted","Data":"bf61556e1be8c3b123302f7cac8a51f8fa46dd2e80b28861732a8c4a674f5ab1"}
Jul 30 14:50:00.530031 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:00.529966    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-bf7cb" podStartSLOduration=3.0871053489999998 podStartE2EDuration="9.529943334s" totalImagesPullingTime="6.318398053s" totalInitContainerRuntime="124.439932ms" isStatefulPod=true podCreationTimestamp="2026-07-30 14:49:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:50:00.527775875 +0000 UTC m=+112.056714351" watchObservedRunningTime="2026-07-30 14:50:00.529943334 +0000 UTC m=+112.058881810"
Jul 30 14:50:04.521866 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:04.521782    2578 generic.go:356] "Generic (PLEG): container finished" podID="13fc7a64-04f6-492b-bffd-c81059c7af28" containerID="0ea70b74729853a0ae6087610781812386743507a974a88e87934d5ba7e48636" exitCode=0
Jul 30 14:50:04.522270 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:04.521856    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"13fc7a64-04f6-492b-bffd-c81059c7af28","Type":"ContainerDied","Data":"0ea70b74729853a0ae6087610781812386743507a974a88e87934d5ba7e48636"}
Jul 30 14:50:04.523445 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:04.523388    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5c5df58559-ds468" event={"ID":"00d9e19d-ff30-4139-984d-1c00ffbc85cd","Type":"ContainerStarted","Data":"115051b8adb719c554b11d615f8820677169cb02198026d52e4c027b55c12f64"}
Jul 30 14:50:04.525593 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:04.525574    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9lk5c" event={"ID":"623272af-807b-4e8c-b308-3c2737b1fe31","Type":"ContainerStarted","Data":"8fd9a5d68b145c2c4c0809fcba05023ab7fa9bfab51391e2137c684b348a3fb0"}
Jul 30 14:50:04.526977 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:04.526959    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-z4q4n" event={"ID":"52b76eb8-59c8-4644-8dbf-aba3ace91eb2","Type":"ContainerStarted","Data":"ffdbfd6bcf82e695ee6190720357fdc63d4caaa0bcb2e426516b861d6da5e440"}
Jul 30 14:50:04.527117 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:04.527102    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-z4q4n"
Jul 30 14:50:04.531972 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:04.531953    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-z4q4n"
Jul 30 14:50:04.586524 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:04.586470    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5c5df58559-ds468" podStartSLOduration=4.89956712 podStartE2EDuration="9.58645318s" totalImagesPullingTime="4.68688606s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 14:49:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:50:04.585014417 +0000 UTC m=+116.113953135" watchObservedRunningTime="2026-07-30 14:50:04.58645318 +0000 UTC m=+116.115391653"
Jul 30 14:50:04.602595 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:04.602550    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-z4q4n" podStartSLOduration=4.531354454 podStartE2EDuration="9.602536385s" totalImagesPullingTime="5.071181931s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 14:49:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:50:04.600856716 +0000 UTC m=+116.129795191" watchObservedRunningTime="2026-07-30 14:50:04.602536385 +0000 UTC m=+116.131474900"
Jul 30 14:50:04.620432 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:04.620387    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-9lk5c" podStartSLOduration=34.394361877 podStartE2EDuration="58.620371427s" totalImagesPullingTime="24.22600955s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 14:49:06 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:50:04.619942752 +0000 UTC m=+116.148881227" watchObservedRunningTime="2026-07-30 14:50:04.620371427 +0000 UTC m=+116.149309901"
Jul 30 14:50:08.561613 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:08.561546    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"13fc7a64-04f6-492b-bffd-c81059c7af28","Type":"ContainerStarted","Data":"08d46e6fd7994105178606cc8c57b5d11cc4cc9d0f27b0ebea64d6531aae3eb5"}
Jul 30 14:50:08.562031 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:08.561636    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"13fc7a64-04f6-492b-bffd-c81059c7af28","Type":"ContainerStarted","Data":"b74e247c9dc1c16327fb684ed885e3102e1a297d82386877e1feed575d63e871"}
Jul 30 14:50:09.507907 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:09.507875    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-nzzqf"
Jul 30 14:50:11.577521 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:11.577488    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"13fc7a64-04f6-492b-bffd-c81059c7af28","Type":"ContainerStarted","Data":"94802d69acbc81990e8e8b581812eae4c722360b9ae17e2323f581b845a191e0"}
Jul 30 14:50:11.577969 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:11.577548    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"13fc7a64-04f6-492b-bffd-c81059c7af28","Type":"ContainerStarted","Data":"47e8c095ee401f1d42e85305cbd2b487ea490777b2103c60c537356390031290"}
Jul 30 14:50:11.577969 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:11.577574    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"13fc7a64-04f6-492b-bffd-c81059c7af28","Type":"ContainerStarted","Data":"5daba48223d53cfcb0ad1b06d52d39581caf73af12b7a4b31322d52d62a7db68"}
Jul 30 14:50:11.577969 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:11.577598    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"13fc7a64-04f6-492b-bffd-c81059c7af28","Type":"ContainerStarted","Data":"1de3193c39d519dc4bc0b6bcf3ca088279202df0be9795c2a16c6fd5d2a5e741"}
Jul 30 14:50:11.607676 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:11.607623    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.711939462 podStartE2EDuration="14.607606303s" totalImagesPullingTime="10.881353793s" totalInitContainerRuntime="14.313048ms" isStatefulPod=false podCreationTimestamp="2026-07-30 14:49:57 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:50:11.605218163 +0000 UTC m=+123.134156638" watchObservedRunningTime="2026-07-30 14:50:11.607606303 +0000 UTC m=+123.136544935"
Jul 30 14:50:12.850233 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:12.850194    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:50:15.898783 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:15.898749    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5c5df58559-ds468"
Jul 30 14:50:15.899283 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:15.898805    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5c5df58559-ds468"
Jul 30 14:50:16.596215 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:16.596184    2578 generic.go:356] "Generic (PLEG): container finished" podID="1f07334b-dd72-4771-b26f-bb23fc9a0ada" containerID="deefbaa84c623159f2ddb1ad752b5e45b3c480ad09a676cd0f7afbefcaf44d75" exitCode=0
Jul 30 14:50:16.596505 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:16.596238    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-zqcwj" event={"ID":"1f07334b-dd72-4771-b26f-bb23fc9a0ada","Type":"ContainerDied","Data":"deefbaa84c623159f2ddb1ad752b5e45b3c480ad09a676cd0f7afbefcaf44d75"}
Jul 30 14:50:16.596647 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:16.596630    2578 scope.go:117] "RemoveContainer" containerID="deefbaa84c623159f2ddb1ad752b5e45b3c480ad09a676cd0f7afbefcaf44d75"
Jul 30 14:50:17.602008 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:17.601973    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-zqcwj" event={"ID":"1f07334b-dd72-4771-b26f-bb23fc9a0ada","Type":"ContainerStarted","Data":"01b15550fbecb1d083c1bc52ac7899a68be75b0de41c045459e65bd92533ecb7"}
Jul 30 14:50:18.805249 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:18.805213    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be2b73fa-10f0-4ea3-a68e-fa452e30914b-metrics-certs\") pod \"network-metrics-daemon-rj8gr\" (UID: \"be2b73fa-10f0-4ea3-a68e-fa452e30914b\") " pod="openshift-multus/network-metrics-daemon-rj8gr"
Jul 30 14:50:18.807599 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:18.807575    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be2b73fa-10f0-4ea3-a68e-fa452e30914b-metrics-certs\") pod \"network-metrics-daemon-rj8gr\" (UID: \"be2b73fa-10f0-4ea3-a68e-fa452e30914b\") " pod="openshift-multus/network-metrics-daemon-rj8gr"
Jul 30 14:50:18.886603 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:18.886579    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-82hs8\""
Jul 30 14:50:18.895157 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:18.895131    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rj8gr"
Jul 30 14:50:19.041402 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:19.041368    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rj8gr"]
Jul 30 14:50:19.043406 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:50:19.043376    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe2b73fa_10f0_4ea3_a68e_fa452e30914b.slice/crio-05811ba2fd93da08173450179233ad227c98265398dec180df6fee15658946ac WatchSource:0}: Error finding container 05811ba2fd93da08173450179233ad227c98265398dec180df6fee15658946ac: Status 404 returned error can't find the container with id 05811ba2fd93da08173450179233ad227c98265398dec180df6fee15658946ac
Jul 30 14:50:19.612226 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:19.612187    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rj8gr" event={"ID":"be2b73fa-10f0-4ea3-a68e-fa452e30914b","Type":"ContainerStarted","Data":"05811ba2fd93da08173450179233ad227c98265398dec180df6fee15658946ac"}
Jul 30 14:50:20.140619 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:20.140579    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6b75c955fd-b7cgs" podUID="31e691be-921a-4cf8-b046-245fa3cac6f4" containerName="console" containerID="cri-o://e9c12a3a760fdac5159bde79de6805b81946f85e6b0b9fc7819dff3e26cfee2b" gracePeriod=15
Jul 30 14:50:20.493103 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:20.493079    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b75c955fd-b7cgs"
Jul 30 14:50:20.618141 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:20.618111    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rj8gr" event={"ID":"be2b73fa-10f0-4ea3-a68e-fa452e30914b","Type":"ContainerStarted","Data":"3d811af39a933d80a6f80ccec9a2406a403807ca60cbcfe8e19381b1320e5d57"}
Jul 30 14:50:20.619416 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:20.619390    2578 generic.go:356] "Generic (PLEG): container finished" podID="31e691be-921a-4cf8-b046-245fa3cac6f4" containerID="e9c12a3a760fdac5159bde79de6805b81946f85e6b0b9fc7819dff3e26cfee2b" exitCode=0
Jul 30 14:50:20.619508 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:20.619438    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b75c955fd-b7cgs"
Jul 30 14:50:20.619508 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:20.619474    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b75c955fd-b7cgs" event={"ID":"31e691be-921a-4cf8-b046-245fa3cac6f4","Type":"ContainerDied","Data":"e9c12a3a760fdac5159bde79de6805b81946f85e6b0b9fc7819dff3e26cfee2b"}
Jul 30 14:50:20.619578 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:20.619526    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b75c955fd-b7cgs" event={"ID":"31e691be-921a-4cf8-b046-245fa3cac6f4","Type":"ContainerDied","Data":"53ebfb9dae2b41ec42653957ba13cd875b6e79b63eaed651d580af9a02d229a1"}
Jul 30 14:50:20.619578 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:20.619555    2578 scope.go:117] "RemoveContainer" containerID="e9c12a3a760fdac5159bde79de6805b81946f85e6b0b9fc7819dff3e26cfee2b"
Jul 30 14:50:20.620080 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:20.620053    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/31e691be-921a-4cf8-b046-245fa3cac6f4-console-config\") pod \"31e691be-921a-4cf8-b046-245fa3cac6f4\" (UID: \"31e691be-921a-4cf8-b046-245fa3cac6f4\") "
Jul 30 14:50:20.620164 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:20.620109    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/31e691be-921a-4cf8-b046-245fa3cac6f4-oauth-serving-cert\") pod \"31e691be-921a-4cf8-b046-245fa3cac6f4\" (UID: \"31e691be-921a-4cf8-b046-245fa3cac6f4\") "
Jul 30 14:50:20.620231 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:20.620164    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/31e691be-921a-4cf8-b046-245fa3cac6f4-console-oauth-config\") pod \"31e691be-921a-4cf8-b046-245fa3cac6f4\" (UID: \"31e691be-921a-4cf8-b046-245fa3cac6f4\") "
Jul 30 14:50:20.620280 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:20.620229    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/31e691be-921a-4cf8-b046-245fa3cac6f4-service-ca\") pod \"31e691be-921a-4cf8-b046-245fa3cac6f4\" (UID: \"31e691be-921a-4cf8-b046-245fa3cac6f4\") "
Jul 30 14:50:20.620280 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:20.620251    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/31e691be-921a-4cf8-b046-245fa3cac6f4-console-serving-cert\") pod \"31e691be-921a-4cf8-b046-245fa3cac6f4\" (UID: \"31e691be-921a-4cf8-b046-245fa3cac6f4\") "
Jul 30 14:50:20.620382 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:20.620293    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r9qfl\" (UniqueName: \"kubernetes.io/projected/31e691be-921a-4cf8-b046-245fa3cac6f4-kube-api-access-r9qfl\") pod \"31e691be-921a-4cf8-b046-245fa3cac6f4\" (UID: \"31e691be-921a-4cf8-b046-245fa3cac6f4\") "
Jul 30 14:50:20.620583 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:20.620545    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31e691be-921a-4cf8-b046-245fa3cac6f4-console-config" (OuterVolumeSpecName: "console-config") pod "31e691be-921a-4cf8-b046-245fa3cac6f4" (UID: "31e691be-921a-4cf8-b046-245fa3cac6f4"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 14:50:20.620583 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:20.620555    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31e691be-921a-4cf8-b046-245fa3cac6f4-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "31e691be-921a-4cf8-b046-245fa3cac6f4" (UID: "31e691be-921a-4cf8-b046-245fa3cac6f4"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 14:50:20.620803 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:20.620776    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31e691be-921a-4cf8-b046-245fa3cac6f4-service-ca" (OuterVolumeSpecName: "service-ca") pod "31e691be-921a-4cf8-b046-245fa3cac6f4" (UID: "31e691be-921a-4cf8-b046-245fa3cac6f4"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 14:50:20.622969 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:20.622933    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31e691be-921a-4cf8-b046-245fa3cac6f4-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "31e691be-921a-4cf8-b046-245fa3cac6f4" (UID: "31e691be-921a-4cf8-b046-245fa3cac6f4"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 14:50:20.623082 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:20.623029    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31e691be-921a-4cf8-b046-245fa3cac6f4-kube-api-access-r9qfl" (OuterVolumeSpecName: "kube-api-access-r9qfl") pod "31e691be-921a-4cf8-b046-245fa3cac6f4" (UID: "31e691be-921a-4cf8-b046-245fa3cac6f4"). InnerVolumeSpecName "kube-api-access-r9qfl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 14:50:20.623082 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:20.623041    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31e691be-921a-4cf8-b046-245fa3cac6f4-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "31e691be-921a-4cf8-b046-245fa3cac6f4" (UID: "31e691be-921a-4cf8-b046-245fa3cac6f4"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 14:50:20.633170 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:20.633124    2578 scope.go:117] "RemoveContainer" containerID="e9c12a3a760fdac5159bde79de6805b81946f85e6b0b9fc7819dff3e26cfee2b"
Jul 30 14:50:20.633413 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:50:20.633392    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9c12a3a760fdac5159bde79de6805b81946f85e6b0b9fc7819dff3e26cfee2b\": container with ID starting with e9c12a3a760fdac5159bde79de6805b81946f85e6b0b9fc7819dff3e26cfee2b not found: ID does not exist" containerID="e9c12a3a760fdac5159bde79de6805b81946f85e6b0b9fc7819dff3e26cfee2b"
Jul 30 14:50:20.633490 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:20.633436    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9c12a3a760fdac5159bde79de6805b81946f85e6b0b9fc7819dff3e26cfee2b"} err="failed to get container status \"e9c12a3a760fdac5159bde79de6805b81946f85e6b0b9fc7819dff3e26cfee2b\": rpc error: code = NotFound desc = could not find container \"e9c12a3a760fdac5159bde79de6805b81946f85e6b0b9fc7819dff3e26cfee2b\": container with ID starting with e9c12a3a760fdac5159bde79de6805b81946f85e6b0b9fc7819dff3e26cfee2b not found: ID does not exist"
Jul 30 14:50:20.721131 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:20.721102    2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/31e691be-921a-4cf8-b046-245fa3cac6f4-service-ca\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 14:50:20.721131 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:20.721126    2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/31e691be-921a-4cf8-b046-245fa3cac6f4-console-serving-cert\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 14:50:20.721329 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:20.721138    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r9qfl\" (UniqueName: \"kubernetes.io/projected/31e691be-921a-4cf8-b046-245fa3cac6f4-kube-api-access-r9qfl\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 14:50:20.721329 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:20.721173    2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/31e691be-921a-4cf8-b046-245fa3cac6f4-console-config\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 14:50:20.721329 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:20.721182    2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/31e691be-921a-4cf8-b046-245fa3cac6f4-oauth-serving-cert\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 14:50:20.721329 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:20.721191    2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/31e691be-921a-4cf8-b046-245fa3cac6f4-console-oauth-config\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 14:50:20.939511 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:20.939480    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b75c955fd-b7cgs"]
Jul 30 14:50:20.942494 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:20.942472    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6b75c955fd-b7cgs"]
Jul 30 14:50:20.983237 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:20.983140    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31e691be-921a-4cf8-b046-245fa3cac6f4" path="/var/lib/kubelet/pods/31e691be-921a-4cf8-b046-245fa3cac6f4/volumes"
Jul 30 14:50:21.624677 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:21.624636    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rj8gr" event={"ID":"be2b73fa-10f0-4ea3-a68e-fa452e30914b","Type":"ContainerStarted","Data":"a0effaf554686552aef9e892996520dbdfd0d4579181bedcbde8e31541052293"}
Jul 30 14:50:21.642064 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:21.642013    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-rj8gr" podStartSLOduration=131.210543473 podStartE2EDuration="2m12.641997976s" totalImagesPullingTime="1.431454503s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 14:48:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:50:21.640281937 +0000 UTC m=+133.169220411" watchObservedRunningTime="2026-07-30 14:50:21.641997976 +0000 UTC m=+133.170936500"
Jul 30 14:50:35.904388 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:35.904358    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5c5df58559-ds468"
Jul 30 14:50:35.908167 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:35.908131    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5c5df58559-ds468"
Jul 30 14:50:57.850198 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:57.850165    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:50:57.870749 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:57.870727    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:50:58.784237 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:50:58.784209    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 14:53:01.605184 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:01.605135    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fhmcj"]
Jul 30 14:53:01.605628 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:01.605439    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31e691be-921a-4cf8-b046-245fa3cac6f4" containerName="console"
Jul 30 14:53:01.605628 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:01.605449    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="31e691be-921a-4cf8-b046-245fa3cac6f4" containerName="console"
Jul 30 14:53:01.605628 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:01.605501    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="31e691be-921a-4cf8-b046-245fa3cac6f4" containerName="console"
Jul 30 14:53:01.607213 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:01.607196    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fhmcj"
Jul 30 14:53:01.609220 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:01.609198    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 30 14:53:01.609742 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:01.609723    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-jxbdw\""
Jul 30 14:53:01.609831 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:01.609754    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 30 14:53:01.617182 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:01.617139    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fhmcj"]
Jul 30 14:53:01.719905 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:01.719861    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h68x8\" (UniqueName: \"kubernetes.io/projected/7dc9a4d7-863d-4f08-a805-304c39a22007-kube-api-access-h68x8\") pod \"openshift-lws-operator-fd8ccff4c-fhmcj\" (UID: \"7dc9a4d7-863d-4f08-a805-304c39a22007\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fhmcj"
Jul 30 14:53:01.720078 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:01.719983    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7dc9a4d7-863d-4f08-a805-304c39a22007-tmp\") pod \"openshift-lws-operator-fd8ccff4c-fhmcj\" (UID: \"7dc9a4d7-863d-4f08-a805-304c39a22007\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fhmcj"
Jul 30 14:53:01.820818 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:01.820778    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h68x8\" (UniqueName: \"kubernetes.io/projected/7dc9a4d7-863d-4f08-a805-304c39a22007-kube-api-access-h68x8\") pod \"openshift-lws-operator-fd8ccff4c-fhmcj\" (UID: \"7dc9a4d7-863d-4f08-a805-304c39a22007\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fhmcj"
Jul 30 14:53:01.820975 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:01.820866    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7dc9a4d7-863d-4f08-a805-304c39a22007-tmp\") pod \"openshift-lws-operator-fd8ccff4c-fhmcj\" (UID: \"7dc9a4d7-863d-4f08-a805-304c39a22007\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fhmcj"
Jul 30 14:53:01.821301 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:01.821282    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7dc9a4d7-863d-4f08-a805-304c39a22007-tmp\") pod \"openshift-lws-operator-fd8ccff4c-fhmcj\" (UID: \"7dc9a4d7-863d-4f08-a805-304c39a22007\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fhmcj"
Jul 30 14:53:01.828610 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:01.828588    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h68x8\" (UniqueName: \"kubernetes.io/projected/7dc9a4d7-863d-4f08-a805-304c39a22007-kube-api-access-h68x8\") pod \"openshift-lws-operator-fd8ccff4c-fhmcj\" (UID: \"7dc9a4d7-863d-4f08-a805-304c39a22007\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fhmcj"
Jul 30 14:53:01.928915 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:01.928848    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fhmcj"
Jul 30 14:53:02.049666 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:02.049598    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fhmcj"]
Jul 30 14:53:02.052028 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:53:02.051992    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7dc9a4d7_863d_4f08_a805_304c39a22007.slice/crio-d96a2351c477ef7f04a36a91d98c31f153947841eab1fb83af4043d3ed79a947 WatchSource:0}: Error finding container d96a2351c477ef7f04a36a91d98c31f153947841eab1fb83af4043d3ed79a947: Status 404 returned error can't find the container with id d96a2351c477ef7f04a36a91d98c31f153947841eab1fb83af4043d3ed79a947
Jul 30 14:53:02.204261 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:02.204180    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fhmcj" event={"ID":"7dc9a4d7-863d-4f08-a805-304c39a22007","Type":"ContainerStarted","Data":"d96a2351c477ef7f04a36a91d98c31f153947841eab1fb83af4043d3ed79a947"}
Jul 30 14:53:06.221413 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:06.221379    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fhmcj" event={"ID":"7dc9a4d7-863d-4f08-a805-304c39a22007","Type":"ContainerStarted","Data":"58c7dbc372856c43be3801b324d6afdb7e805f1de65a87a6134a25f2289d3de9"}
Jul 30 14:53:06.239490 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:06.239438    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fhmcj" podStartSLOduration=2.091989947 podStartE2EDuration="5.239421639s" totalImagesPullingTime="3.147431692s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 14:53:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:53:06.237329272 +0000 UTC m=+297.766267746" watchObservedRunningTime="2026-07-30 14:53:06.239421639 +0000 UTC m=+297.768360112"
Jul 30 14:53:08.884724 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:08.884693    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4mjmp_de241220-5f27-4b0b-b7c6-6d4b350e24e1/console-operator/1.log"
Jul 30 14:53:08.890722 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:08.890699    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4mjmp_de241220-5f27-4b0b-b7c6-6d4b350e24e1/console-operator/1.log"
Jul 30 14:53:08.895353 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:08.895333    2578 kubelet.go:1648] "Image garbage collection succeeded"
Jul 30 14:53:21.203983 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:21.203940    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7b696b7c-wdh8g"]
Jul 30 14:53:21.210197 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:21.210177    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7b696b7c-wdh8g"
Jul 30 14:53:21.214104 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:21.214081    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 30 14:53:21.214260 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:21.214082    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 30 14:53:21.214654 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:21.214633    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-q26xj\""
Jul 30 14:53:21.214837 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:21.214821    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 30 14:53:21.218739 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:21.218715    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7b696b7c-wdh8g"]
Jul 30 14:53:21.285945 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:21.285919    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8179b7c2-1ef1-43a4-965f-7e44e6f58fb0-cert\") pod \"lws-controller-manager-7b696b7c-wdh8g\" (UID: \"8179b7c2-1ef1-43a4-965f-7e44e6f58fb0\") " pod="openshift-lws-operator/lws-controller-manager-7b696b7c-wdh8g"
Jul 30 14:53:21.286086 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:21.285950    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8179b7c2-1ef1-43a4-965f-7e44e6f58fb0-metrics-cert\") pod \"lws-controller-manager-7b696b7c-wdh8g\" (UID: \"8179b7c2-1ef1-43a4-965f-7e44e6f58fb0\") " pod="openshift-lws-operator/lws-controller-manager-7b696b7c-wdh8g"
Jul 30 14:53:21.286086 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:21.285974    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jmdm\" (UniqueName: \"kubernetes.io/projected/8179b7c2-1ef1-43a4-965f-7e44e6f58fb0-kube-api-access-7jmdm\") pod \"lws-controller-manager-7b696b7c-wdh8g\" (UID: \"8179b7c2-1ef1-43a4-965f-7e44e6f58fb0\") " pod="openshift-lws-operator/lws-controller-manager-7b696b7c-wdh8g"
Jul 30 14:53:21.286086 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:21.286025    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8179b7c2-1ef1-43a4-965f-7e44e6f58fb0-manager-config\") pod \"lws-controller-manager-7b696b7c-wdh8g\" (UID: \"8179b7c2-1ef1-43a4-965f-7e44e6f58fb0\") " pod="openshift-lws-operator/lws-controller-manager-7b696b7c-wdh8g"
Jul 30 14:53:21.386400 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:21.386365    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8179b7c2-1ef1-43a4-965f-7e44e6f58fb0-manager-config\") pod \"lws-controller-manager-7b696b7c-wdh8g\" (UID: \"8179b7c2-1ef1-43a4-965f-7e44e6f58fb0\") " pod="openshift-lws-operator/lws-controller-manager-7b696b7c-wdh8g"
Jul 30 14:53:21.386567 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:21.386440    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8179b7c2-1ef1-43a4-965f-7e44e6f58fb0-cert\") pod \"lws-controller-manager-7b696b7c-wdh8g\" (UID: \"8179b7c2-1ef1-43a4-965f-7e44e6f58fb0\") " pod="openshift-lws-operator/lws-controller-manager-7b696b7c-wdh8g"
Jul 30 14:53:21.386567 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:21.386464    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8179b7c2-1ef1-43a4-965f-7e44e6f58fb0-metrics-cert\") pod \"lws-controller-manager-7b696b7c-wdh8g\" (UID: \"8179b7c2-1ef1-43a4-965f-7e44e6f58fb0\") " pod="openshift-lws-operator/lws-controller-manager-7b696b7c-wdh8g"
Jul 30 14:53:21.386567 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:21.386487    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jmdm\" (UniqueName: \"kubernetes.io/projected/8179b7c2-1ef1-43a4-965f-7e44e6f58fb0-kube-api-access-7jmdm\") pod \"lws-controller-manager-7b696b7c-wdh8g\" (UID: \"8179b7c2-1ef1-43a4-965f-7e44e6f58fb0\") " pod="openshift-lws-operator/lws-controller-manager-7b696b7c-wdh8g"
Jul 30 14:53:21.386981 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:21.386961    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8179b7c2-1ef1-43a4-965f-7e44e6f58fb0-manager-config\") pod \"lws-controller-manager-7b696b7c-wdh8g\" (UID: \"8179b7c2-1ef1-43a4-965f-7e44e6f58fb0\") " pod="openshift-lws-operator/lws-controller-manager-7b696b7c-wdh8g"
Jul 30 14:53:21.388831 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:21.388812    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8179b7c2-1ef1-43a4-965f-7e44e6f58fb0-cert\") pod \"lws-controller-manager-7b696b7c-wdh8g\" (UID: \"8179b7c2-1ef1-43a4-965f-7e44e6f58fb0\") " pod="openshift-lws-operator/lws-controller-manager-7b696b7c-wdh8g"
Jul 30 14:53:21.388903 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:21.388850    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8179b7c2-1ef1-43a4-965f-7e44e6f58fb0-metrics-cert\") pod \"lws-controller-manager-7b696b7c-wdh8g\" (UID: \"8179b7c2-1ef1-43a4-965f-7e44e6f58fb0\") " pod="openshift-lws-operator/lws-controller-manager-7b696b7c-wdh8g"
Jul 30 14:53:21.402784 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:21.402752    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jmdm\" (UniqueName: \"kubernetes.io/projected/8179b7c2-1ef1-43a4-965f-7e44e6f58fb0-kube-api-access-7jmdm\") pod \"lws-controller-manager-7b696b7c-wdh8g\" (UID: \"8179b7c2-1ef1-43a4-965f-7e44e6f58fb0\") " pod="openshift-lws-operator/lws-controller-manager-7b696b7c-wdh8g"
Jul 30 14:53:21.520182 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:21.520102    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7b696b7c-wdh8g"
Jul 30 14:53:21.651132 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:21.651098    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7b696b7c-wdh8g"]
Jul 30 14:53:21.654545 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:53:21.654523    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8179b7c2_1ef1_43a4_965f_7e44e6f58fb0.slice/crio-4b49dff8afb0642a734cc45dba22d4a31ee23a9e9d0640d417e408a36c2c62de WatchSource:0}: Error finding container 4b49dff8afb0642a734cc45dba22d4a31ee23a9e9d0640d417e408a36c2c62de: Status 404 returned error can't find the container with id 4b49dff8afb0642a734cc45dba22d4a31ee23a9e9d0640d417e408a36c2c62de
Jul 30 14:53:21.656377 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:21.656361    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 14:53:22.118594 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:22.118552    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6bdff7d7c7-xqjhm"]
Jul 30 14:53:22.123416 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:22.123393    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6bdff7d7c7-xqjhm"
Jul 30 14:53:22.126752 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:22.126732    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 30 14:53:22.126832 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:22.126781    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 30 14:53:22.126979 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:22.126961    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 30 14:53:22.127246 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:22.127233    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 30 14:53:22.127290 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:22.127233    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-zsh4t\""
Jul 30 14:53:22.159266 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:22.159233    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6bdff7d7c7-xqjhm"]
Jul 30 14:53:22.191399 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:22.191361    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4086f535-988b-442d-b9cd-e2e7858f65e4-tmp\") pod \"opendatahub-operator-controller-manager-6bdff7d7c7-xqjhm\" (UID: \"4086f535-988b-442d-b9cd-e2e7858f65e4\") " pod="opendatahub/opendatahub-operator-controller-manager-6bdff7d7c7-xqjhm"
Jul 30 14:53:22.191530 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:22.191403    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4086f535-988b-442d-b9cd-e2e7858f65e4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6bdff7d7c7-xqjhm\" (UID: \"4086f535-988b-442d-b9cd-e2e7858f65e4\") " pod="opendatahub/opendatahub-operator-controller-manager-6bdff7d7c7-xqjhm"
Jul 30 14:53:22.191530 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:22.191439    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mq5q8\" (UniqueName: \"kubernetes.io/projected/4086f535-988b-442d-b9cd-e2e7858f65e4-kube-api-access-mq5q8\") pod \"opendatahub-operator-controller-manager-6bdff7d7c7-xqjhm\" (UID: \"4086f535-988b-442d-b9cd-e2e7858f65e4\") " pod="opendatahub/opendatahub-operator-controller-manager-6bdff7d7c7-xqjhm"
Jul 30 14:53:22.191530 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:22.191490    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4086f535-988b-442d-b9cd-e2e7858f65e4-webhook-cert\") pod \"opendatahub-operator-controller-manager-6bdff7d7c7-xqjhm\" (UID: \"4086f535-988b-442d-b9cd-e2e7858f65e4\") " pod="opendatahub/opendatahub-operator-controller-manager-6bdff7d7c7-xqjhm"
Jul 30 14:53:22.191680 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:22.191533    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4086f535-988b-442d-b9cd-e2e7858f65e4-manifests\") pod \"opendatahub-operator-controller-manager-6bdff7d7c7-xqjhm\" (UID: \"4086f535-988b-442d-b9cd-e2e7858f65e4\") " pod="opendatahub/opendatahub-operator-controller-manager-6bdff7d7c7-xqjhm"
Jul 30 14:53:22.285538 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:22.285501    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7b696b7c-wdh8g" event={"ID":"8179b7c2-1ef1-43a4-965f-7e44e6f58fb0","Type":"ContainerStarted","Data":"4b49dff8afb0642a734cc45dba22d4a31ee23a9e9d0640d417e408a36c2c62de"}
Jul 30 14:53:22.291927 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:22.291890    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mq5q8\" (UniqueName: \"kubernetes.io/projected/4086f535-988b-442d-b9cd-e2e7858f65e4-kube-api-access-mq5q8\") pod \"opendatahub-operator-controller-manager-6bdff7d7c7-xqjhm\" (UID: \"4086f535-988b-442d-b9cd-e2e7858f65e4\") " pod="opendatahub/opendatahub-operator-controller-manager-6bdff7d7c7-xqjhm"
Jul 30 14:53:22.291927 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:22.291935    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4086f535-988b-442d-b9cd-e2e7858f65e4-webhook-cert\") pod \"opendatahub-operator-controller-manager-6bdff7d7c7-xqjhm\" (UID: \"4086f535-988b-442d-b9cd-e2e7858f65e4\") " pod="opendatahub/opendatahub-operator-controller-manager-6bdff7d7c7-xqjhm"
Jul 30 14:53:22.292093 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:22.292035    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4086f535-988b-442d-b9cd-e2e7858f65e4-manifests\") pod \"opendatahub-operator-controller-manager-6bdff7d7c7-xqjhm\" (UID: \"4086f535-988b-442d-b9cd-e2e7858f65e4\") " pod="opendatahub/opendatahub-operator-controller-manager-6bdff7d7c7-xqjhm"
Jul 30 14:53:22.292093 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:22.292071    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4086f535-988b-442d-b9cd-e2e7858f65e4-tmp\") pod \"opendatahub-operator-controller-manager-6bdff7d7c7-xqjhm\" (UID: \"4086f535-988b-442d-b9cd-e2e7858f65e4\") " pod="opendatahub/opendatahub-operator-controller-manager-6bdff7d7c7-xqjhm"
Jul 30 14:53:22.292376 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:22.292097    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4086f535-988b-442d-b9cd-e2e7858f65e4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6bdff7d7c7-xqjhm\" (UID: \"4086f535-988b-442d-b9cd-e2e7858f65e4\") " pod="opendatahub/opendatahub-operator-controller-manager-6bdff7d7c7-xqjhm"
Jul 30 14:53:22.292520 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:22.292482    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4086f535-988b-442d-b9cd-e2e7858f65e4-manifests\") pod \"opendatahub-operator-controller-manager-6bdff7d7c7-xqjhm\" (UID: \"4086f535-988b-442d-b9cd-e2e7858f65e4\") " pod="opendatahub/opendatahub-operator-controller-manager-6bdff7d7c7-xqjhm"
Jul 30 14:53:22.294336 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:22.294316    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4086f535-988b-442d-b9cd-e2e7858f65e4-tmp\") pod \"opendatahub-operator-controller-manager-6bdff7d7c7-xqjhm\" (UID: \"4086f535-988b-442d-b9cd-e2e7858f65e4\") " pod="opendatahub/opendatahub-operator-controller-manager-6bdff7d7c7-xqjhm"
Jul 30 14:53:22.294559 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:22.294542    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4086f535-988b-442d-b9cd-e2e7858f65e4-webhook-cert\") pod \"opendatahub-operator-controller-manager-6bdff7d7c7-xqjhm\" (UID: \"4086f535-988b-442d-b9cd-e2e7858f65e4\") " pod="opendatahub/opendatahub-operator-controller-manager-6bdff7d7c7-xqjhm"
Jul 30 14:53:22.294614 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:22.294596    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4086f535-988b-442d-b9cd-e2e7858f65e4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6bdff7d7c7-xqjhm\" (UID: \"4086f535-988b-442d-b9cd-e2e7858f65e4\") " pod="opendatahub/opendatahub-operator-controller-manager-6bdff7d7c7-xqjhm"
Jul 30 14:53:22.306356 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:22.306336    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mq5q8\" (UniqueName: \"kubernetes.io/projected/4086f535-988b-442d-b9cd-e2e7858f65e4-kube-api-access-mq5q8\") pod \"opendatahub-operator-controller-manager-6bdff7d7c7-xqjhm\" (UID: \"4086f535-988b-442d-b9cd-e2e7858f65e4\") " pod="opendatahub/opendatahub-operator-controller-manager-6bdff7d7c7-xqjhm"
Jul 30 14:53:22.433739 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:22.433667    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6bdff7d7c7-xqjhm"
Jul 30 14:53:22.589922 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:22.589896    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6bdff7d7c7-xqjhm"]
Jul 30 14:53:22.592871 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:53:22.592837    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4086f535_988b_442d_b9cd_e2e7858f65e4.slice/crio-6019fbadb16c80a3f5a390543bf5b83ba891932cb098ab7646b8bf6c0d957a82 WatchSource:0}: Error finding container 6019fbadb16c80a3f5a390543bf5b83ba891932cb098ab7646b8bf6c0d957a82: Status 404 returned error can't find the container with id 6019fbadb16c80a3f5a390543bf5b83ba891932cb098ab7646b8bf6c0d957a82
Jul 30 14:53:23.290482 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:23.290447    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6bdff7d7c7-xqjhm" event={"ID":"4086f535-988b-442d-b9cd-e2e7858f65e4","Type":"ContainerStarted","Data":"6019fbadb16c80a3f5a390543bf5b83ba891932cb098ab7646b8bf6c0d957a82"}
Jul 30 14:53:24.297711 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:24.297667    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7b696b7c-wdh8g" event={"ID":"8179b7c2-1ef1-43a4-965f-7e44e6f58fb0","Type":"ContainerStarted","Data":"fbeffe171353ce898abc02b48208c8d20c96dd7bf92cf019737aa16556e3e5fd"}
Jul 30 14:53:24.298187 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:24.297795    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7b696b7c-wdh8g"
Jul 30 14:53:24.323111 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:24.323057    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7b696b7c-wdh8g" podStartSLOduration=1.677357878 podStartE2EDuration="3.323037345s" totalImagesPullingTime="1.645679467s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 14:53:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:53:24.321510098 +0000 UTC m=+315.850448573" watchObservedRunningTime="2026-07-30 14:53:24.323037345 +0000 UTC m=+315.851975821"
Jul 30 14:53:26.308202 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:26.308164    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6bdff7d7c7-xqjhm" event={"ID":"4086f535-988b-442d-b9cd-e2e7858f65e4","Type":"ContainerStarted","Data":"4afd11ff87afcf6ecaef57e4c7cd982e80255f2269ccd70dc8e2e077954717b6"}
Jul 30 14:53:27.313976 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:27.313941    2578 generic.go:356] "Generic (PLEG): container finished" podID="4086f535-988b-442d-b9cd-e2e7858f65e4" containerID="4afd11ff87afcf6ecaef57e4c7cd982e80255f2269ccd70dc8e2e077954717b6" exitCode=0
Jul 30 14:53:27.314376 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:27.313982    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6bdff7d7c7-xqjhm" event={"ID":"4086f535-988b-442d-b9cd-e2e7858f65e4","Type":"ContainerDied","Data":"4afd11ff87afcf6ecaef57e4c7cd982e80255f2269ccd70dc8e2e077954717b6"}
Jul 30 14:53:28.320473 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:28.320437    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6bdff7d7c7-xqjhm" event={"ID":"4086f535-988b-442d-b9cd-e2e7858f65e4","Type":"ContainerStarted","Data":"07fbad4945b841efb234c86b4d29576505c670e83ad3af83a3fbe38052a318cf"}
Jul 30 14:53:28.320866 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:28.320653    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6bdff7d7c7-xqjhm"
Jul 30 14:53:28.339503 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:28.339453    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6bdff7d7c7-xqjhm" podStartSLOduration=1.822652069 podStartE2EDuration="6.339433545s" totalImagesPullingTime="3.476373014s" totalInitContainerRuntime="1.040408462s" isStatefulPod=false podCreationTimestamp="2026-07-30 14:53:22 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:53:28.338581469 +0000 UTC m=+319.867519977" watchObservedRunningTime="2026-07-30 14:53:28.339433545 +0000 UTC m=+319.868372021"
Jul 30 14:53:35.305289 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:35.305254    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7b696b7c-wdh8g"
Jul 30 14:53:39.331931 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:39.331902    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6bdff7d7c7-xqjhm"
Jul 30 14:53:42.203853 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:42.203813    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-59c4bbddff-cr7cb"]
Jul 30 14:53:42.207267 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:42.207241    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-59c4bbddff-cr7cb"
Jul 30 14:53:42.209747 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:42.209726    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-h8nw8\""
Jul 30 14:53:42.210017 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:42.210000    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 30 14:53:42.210480 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:42.210441    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 30 14:53:42.210773 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:42.210749    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 30 14:53:42.213385 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:42.213361    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 30 14:53:42.227585 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:42.227562    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-59c4bbddff-cr7cb"]
Jul 30 14:53:42.261755 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:42.261718    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bd797b96-255a-47e8-abb1-b2aa5620dd89-tmp\") pod \"kube-auth-proxy-59c4bbddff-cr7cb\" (UID: \"bd797b96-255a-47e8-abb1-b2aa5620dd89\") " pod="openshift-ingress/kube-auth-proxy-59c4bbddff-cr7cb"
Jul 30 14:53:42.261755 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:42.261764    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd797b96-255a-47e8-abb1-b2aa5620dd89-tls-certs\") pod \"kube-auth-proxy-59c4bbddff-cr7cb\" (UID: \"bd797b96-255a-47e8-abb1-b2aa5620dd89\") " pod="openshift-ingress/kube-auth-proxy-59c4bbddff-cr7cb"
Jul 30 14:53:42.261949 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:42.261814    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkvkj\" (UniqueName: \"kubernetes.io/projected/bd797b96-255a-47e8-abb1-b2aa5620dd89-kube-api-access-kkvkj\") pod \"kube-auth-proxy-59c4bbddff-cr7cb\" (UID: \"bd797b96-255a-47e8-abb1-b2aa5620dd89\") " pod="openshift-ingress/kube-auth-proxy-59c4bbddff-cr7cb"
Jul 30 14:53:42.362608 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:42.362580    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bd797b96-255a-47e8-abb1-b2aa5620dd89-tmp\") pod \"kube-auth-proxy-59c4bbddff-cr7cb\" (UID: \"bd797b96-255a-47e8-abb1-b2aa5620dd89\") " pod="openshift-ingress/kube-auth-proxy-59c4bbddff-cr7cb"
Jul 30 14:53:42.362777 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:42.362616    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd797b96-255a-47e8-abb1-b2aa5620dd89-tls-certs\") pod \"kube-auth-proxy-59c4bbddff-cr7cb\" (UID: \"bd797b96-255a-47e8-abb1-b2aa5620dd89\") " pod="openshift-ingress/kube-auth-proxy-59c4bbddff-cr7cb"
Jul 30 14:53:42.362777 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:42.362749    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkvkj\" (UniqueName: \"kubernetes.io/projected/bd797b96-255a-47e8-abb1-b2aa5620dd89-kube-api-access-kkvkj\") pod \"kube-auth-proxy-59c4bbddff-cr7cb\" (UID: \"bd797b96-255a-47e8-abb1-b2aa5620dd89\") " pod="openshift-ingress/kube-auth-proxy-59c4bbddff-cr7cb"
Jul 30 14:53:42.364873 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:42.364851    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bd797b96-255a-47e8-abb1-b2aa5620dd89-tmp\") pod \"kube-auth-proxy-59c4bbddff-cr7cb\" (UID: \"bd797b96-255a-47e8-abb1-b2aa5620dd89\") " pod="openshift-ingress/kube-auth-proxy-59c4bbddff-cr7cb"
Jul 30 14:53:42.364982 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:42.364965    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd797b96-255a-47e8-abb1-b2aa5620dd89-tls-certs\") pod \"kube-auth-proxy-59c4bbddff-cr7cb\" (UID: \"bd797b96-255a-47e8-abb1-b2aa5620dd89\") " pod="openshift-ingress/kube-auth-proxy-59c4bbddff-cr7cb"
Jul 30 14:53:42.370337 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:42.370310    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkvkj\" (UniqueName: \"kubernetes.io/projected/bd797b96-255a-47e8-abb1-b2aa5620dd89-kube-api-access-kkvkj\") pod \"kube-auth-proxy-59c4bbddff-cr7cb\" (UID: \"bd797b96-255a-47e8-abb1-b2aa5620dd89\") " pod="openshift-ingress/kube-auth-proxy-59c4bbddff-cr7cb"
Jul 30 14:53:42.516934 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:42.516860    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-59c4bbddff-cr7cb"
Jul 30 14:53:42.658841 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:42.658806    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-59c4bbddff-cr7cb"]
Jul 30 14:53:42.662298 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:53:42.662266    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd797b96_255a_47e8_abb1_b2aa5620dd89.slice/crio-a2c6ea8aa612ac34f263bdee3ca43ea1502aed282f9c73d1a01469b46cea8dd9 WatchSource:0}: Error finding container a2c6ea8aa612ac34f263bdee3ca43ea1502aed282f9c73d1a01469b46cea8dd9: Status 404 returned error can't find the container with id a2c6ea8aa612ac34f263bdee3ca43ea1502aed282f9c73d1a01469b46cea8dd9
Jul 30 14:53:43.388981 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:43.388947    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-59c4bbddff-cr7cb" event={"ID":"bd797b96-255a-47e8-abb1-b2aa5620dd89","Type":"ContainerStarted","Data":"a2c6ea8aa612ac34f263bdee3ca43ea1502aed282f9c73d1a01469b46cea8dd9"}
Jul 30 14:53:46.404119 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:46.404080    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-59c4bbddff-cr7cb" event={"ID":"bd797b96-255a-47e8-abb1-b2aa5620dd89","Type":"ContainerStarted","Data":"72bf10b594112462f32e1e2824746037ec2953603495ba84f9f0156abadf1f54"}
Jul 30 14:53:46.420889 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:53:46.420843    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-59c4bbddff-cr7cb" podStartSLOduration=1.673684636 podStartE2EDuration="4.420824711s" totalImagesPullingTime="2.747140075s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 14:53:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:53:46.419250303 +0000 UTC m=+337.948188777" watchObservedRunningTime="2026-07-30 14:53:46.420824711 +0000 UTC m=+337.949763185"
Jul 30 14:56:24.279955 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:24.279922    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-hnr6r"]
Jul 30 14:56:24.283204 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:24.283186    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-hnr6r"
Jul 30 14:56:24.286833 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:24.286809    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 30 14:56:24.287558 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:24.287539    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 30 14:56:24.287786 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:24.287769    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-k6jbg\""
Jul 30 14:56:24.288345 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:24.288330    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 30 14:56:24.316778 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:24.316749    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-hnr6r"]
Jul 30 14:56:24.384486 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:24.384457    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-hnr6r"]
Jul 30 14:56:24.449714 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:24.449683    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c8842d8a-98c5-4695-b33b-c3db37ee81f5-config-file\") pod \"limitador-limitador-64c94fd64f-hnr6r\" (UID: \"c8842d8a-98c5-4695-b33b-c3db37ee81f5\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-hnr6r"
Jul 30 14:56:24.449878 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:24.449737    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5lv9\" (UniqueName: \"kubernetes.io/projected/c8842d8a-98c5-4695-b33b-c3db37ee81f5-kube-api-access-c5lv9\") pod \"limitador-limitador-64c94fd64f-hnr6r\" (UID: \"c8842d8a-98c5-4695-b33b-c3db37ee81f5\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-hnr6r"
Jul 30 14:56:24.550895 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:24.550864    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5lv9\" (UniqueName: \"kubernetes.io/projected/c8842d8a-98c5-4695-b33b-c3db37ee81f5-kube-api-access-c5lv9\") pod \"limitador-limitador-64c94fd64f-hnr6r\" (UID: \"c8842d8a-98c5-4695-b33b-c3db37ee81f5\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-hnr6r"
Jul 30 14:56:24.551057 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:24.550947    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c8842d8a-98c5-4695-b33b-c3db37ee81f5-config-file\") pod \"limitador-limitador-64c94fd64f-hnr6r\" (UID: \"c8842d8a-98c5-4695-b33b-c3db37ee81f5\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-hnr6r"
Jul 30 14:56:24.551538 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:24.551518    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c8842d8a-98c5-4695-b33b-c3db37ee81f5-config-file\") pod \"limitador-limitador-64c94fd64f-hnr6r\" (UID: \"c8842d8a-98c5-4695-b33b-c3db37ee81f5\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-hnr6r"
Jul 30 14:56:24.559523 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:24.559497    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5lv9\" (UniqueName: \"kubernetes.io/projected/c8842d8a-98c5-4695-b33b-c3db37ee81f5-kube-api-access-c5lv9\") pod \"limitador-limitador-64c94fd64f-hnr6r\" (UID: \"c8842d8a-98c5-4695-b33b-c3db37ee81f5\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-hnr6r"
Jul 30 14:56:24.593430 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:24.593401    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-hnr6r"
Jul 30 14:56:24.722402 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:24.722378    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-hnr6r"]
Jul 30 14:56:24.724426 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:56:24.724402    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8842d8a_98c5_4695_b33b_c3db37ee81f5.slice/crio-51f8d847c2d421ff5ca3efc45b86804d4ef2a86d24a439287b7194922163c82f WatchSource:0}: Error finding container 51f8d847c2d421ff5ca3efc45b86804d4ef2a86d24a439287b7194922163c82f: Status 404 returned error can't find the container with id 51f8d847c2d421ff5ca3efc45b86804d4ef2a86d24a439287b7194922163c82f
Jul 30 14:56:25.058544 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:25.058510    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-hnr6r" event={"ID":"c8842d8a-98c5-4695-b33b-c3db37ee81f5","Type":"ContainerStarted","Data":"51f8d847c2d421ff5ca3efc45b86804d4ef2a86d24a439287b7194922163c82f"}
Jul 30 14:56:28.074354 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:28.074318    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-hnr6r" event={"ID":"c8842d8a-98c5-4695-b33b-c3db37ee81f5","Type":"ContainerStarted","Data":"a3da858449ce0e0de53e2d7d87c4683f7e711775803933e42f369f0cdf2e4259"}
Jul 30 14:56:28.074753 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:28.074462    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-hnr6r"
Jul 30 14:56:28.092495 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:28.092444    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-hnr6r" podStartSLOduration=1.358352416 podStartE2EDuration="4.092429267s" totalImagesPullingTime="2.734076851s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 14:56:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:56:28.091013624 +0000 UTC m=+499.619952098" watchObservedRunningTime="2026-07-30 14:56:28.092429267 +0000 UTC m=+499.621367780"
Jul 30 14:56:39.079037 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:39.079003    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-hnr6r"
Jul 30 14:56:39.659757 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:39.659725    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-hnr6r"]
Jul 30 14:56:39.659929 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:39.659888    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-hnr6r" podUID="c8842d8a-98c5-4695-b33b-c3db37ee81f5" containerName="limitador" containerID="cri-o://a3da858449ce0e0de53e2d7d87c4683f7e711775803933e42f369f0cdf2e4259" gracePeriod=30
Jul 30 14:56:40.125486 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:40.124756    2578 generic.go:356] "Generic (PLEG): container finished" podID="c8842d8a-98c5-4695-b33b-c3db37ee81f5" containerID="a3da858449ce0e0de53e2d7d87c4683f7e711775803933e42f369f0cdf2e4259" exitCode=0
Jul 30 14:56:40.125486 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:40.124797    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-hnr6r" event={"ID":"c8842d8a-98c5-4695-b33b-c3db37ee81f5","Type":"ContainerDied","Data":"a3da858449ce0e0de53e2d7d87c4683f7e711775803933e42f369f0cdf2e4259"}
Jul 30 14:56:40.200676 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:40.200649    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-hnr6r"
Jul 30 14:56:40.281949 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:40.281878    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c8842d8a-98c5-4695-b33b-c3db37ee81f5-config-file\") pod \"c8842d8a-98c5-4695-b33b-c3db37ee81f5\" (UID: \"c8842d8a-98c5-4695-b33b-c3db37ee81f5\") "
Jul 30 14:56:40.282091 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:40.281977    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c5lv9\" (UniqueName: \"kubernetes.io/projected/c8842d8a-98c5-4695-b33b-c3db37ee81f5-kube-api-access-c5lv9\") pod \"c8842d8a-98c5-4695-b33b-c3db37ee81f5\" (UID: \"c8842d8a-98c5-4695-b33b-c3db37ee81f5\") "
Jul 30 14:56:40.282233 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:40.282213    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8842d8a-98c5-4695-b33b-c3db37ee81f5-config-file" (OuterVolumeSpecName: "config-file") pod "c8842d8a-98c5-4695-b33b-c3db37ee81f5" (UID: "c8842d8a-98c5-4695-b33b-c3db37ee81f5"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 14:56:40.284344 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:40.284311    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8842d8a-98c5-4695-b33b-c3db37ee81f5-kube-api-access-c5lv9" (OuterVolumeSpecName: "kube-api-access-c5lv9") pod "c8842d8a-98c5-4695-b33b-c3db37ee81f5" (UID: "c8842d8a-98c5-4695-b33b-c3db37ee81f5"). InnerVolumeSpecName "kube-api-access-c5lv9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 14:56:40.382546 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:40.382516    2578 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c8842d8a-98c5-4695-b33b-c3db37ee81f5-config-file\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 14:56:40.382546 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:40.382541    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c5lv9\" (UniqueName: \"kubernetes.io/projected/c8842d8a-98c5-4695-b33b-c3db37ee81f5-kube-api-access-c5lv9\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 14:56:41.130116 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:41.130075    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-hnr6r" event={"ID":"c8842d8a-98c5-4695-b33b-c3db37ee81f5","Type":"ContainerDied","Data":"51f8d847c2d421ff5ca3efc45b86804d4ef2a86d24a439287b7194922163c82f"}
Jul 30 14:56:41.130116 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:41.130099    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-hnr6r"
Jul 30 14:56:41.130635 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:41.130160    2578 scope.go:117] "RemoveContainer" containerID="a3da858449ce0e0de53e2d7d87c4683f7e711775803933e42f369f0cdf2e4259"
Jul 30 14:56:41.144616 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:41.144591    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-hnr6r"]
Jul 30 14:56:41.150036 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:41.150013    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-hnr6r"]
Jul 30 14:56:42.979085 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:42.979053    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8842d8a-98c5-4695-b33b-c3db37ee81f5" path="/var/lib/kubelet/pods/c8842d8a-98c5-4695-b33b-c3db37ee81f5/volumes"
Jul 30 14:56:46.047794 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:46.047764    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-wwfgt"]
Jul 30 14:56:46.048173 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:46.048081    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8842d8a-98c5-4695-b33b-c3db37ee81f5" containerName="limitador"
Jul 30 14:56:46.048173 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:46.048094    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8842d8a-98c5-4695-b33b-c3db37ee81f5" containerName="limitador"
Jul 30 14:56:46.048242 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:46.048178    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8842d8a-98c5-4695-b33b-c3db37ee81f5" containerName="limitador"
Jul 30 14:56:46.052444 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:46.052427    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wwfgt"
Jul 30 14:56:46.054338 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:46.054315    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 30 14:56:46.054338 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:46.054326    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 30 14:56:46.054495 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:46.054326    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-278gf\""
Jul 30 14:56:46.054649 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:46.054617    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 30 14:56:46.057616 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:46.057595    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-wwfgt"]
Jul 30 14:56:46.127326 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:46.127293    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62j2x\" (UniqueName: \"kubernetes.io/projected/3b517781-1975-48de-978c-c2148dc78f22-kube-api-access-62j2x\") pod \"postgres-69fb4594cf-wwfgt\" (UID: \"3b517781-1975-48de-978c-c2148dc78f22\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wwfgt"
Jul 30 14:56:46.127326 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:46.127330    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/3b517781-1975-48de-978c-c2148dc78f22-data\") pod \"postgres-69fb4594cf-wwfgt\" (UID: \"3b517781-1975-48de-978c-c2148dc78f22\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wwfgt"
Jul 30 14:56:46.228614 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:46.228585    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62j2x\" (UniqueName: \"kubernetes.io/projected/3b517781-1975-48de-978c-c2148dc78f22-kube-api-access-62j2x\") pod \"postgres-69fb4594cf-wwfgt\" (UID: \"3b517781-1975-48de-978c-c2148dc78f22\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wwfgt"
Jul 30 14:56:46.228781 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:46.228621    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/3b517781-1975-48de-978c-c2148dc78f22-data\") pod \"postgres-69fb4594cf-wwfgt\" (UID: \"3b517781-1975-48de-978c-c2148dc78f22\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wwfgt"
Jul 30 14:56:46.228989 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:46.228971    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/3b517781-1975-48de-978c-c2148dc78f22-data\") pod \"postgres-69fb4594cf-wwfgt\" (UID: \"3b517781-1975-48de-978c-c2148dc78f22\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wwfgt"
Jul 30 14:56:46.236622 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:46.236597    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-62j2x\" (UniqueName: \"kubernetes.io/projected/3b517781-1975-48de-978c-c2148dc78f22-kube-api-access-62j2x\") pod \"postgres-69fb4594cf-wwfgt\" (UID: \"3b517781-1975-48de-978c-c2148dc78f22\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wwfgt"
Jul 30 14:56:46.363780 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:46.363750    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wwfgt"
Jul 30 14:56:46.490247 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:46.490224    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-wwfgt"]
Jul 30 14:56:46.492085 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:56:46.492061    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b517781_1975_48de_978c_c2148dc78f22.slice/crio-403e3909fcda741d139cb040661aeb0f8b0d62be8f7471099993198a3b789ff0 WatchSource:0}: Error finding container 403e3909fcda741d139cb040661aeb0f8b0d62be8f7471099993198a3b789ff0: Status 404 returned error can't find the container with id 403e3909fcda741d139cb040661aeb0f8b0d62be8f7471099993198a3b789ff0
Jul 30 14:56:47.155729 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:47.155697    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wwfgt" event={"ID":"3b517781-1975-48de-978c-c2148dc78f22","Type":"ContainerStarted","Data":"403e3909fcda741d139cb040661aeb0f8b0d62be8f7471099993198a3b789ff0"}
Jul 30 14:56:52.179366 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:52.179332    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wwfgt" event={"ID":"3b517781-1975-48de-978c-c2148dc78f22","Type":"ContainerStarted","Data":"924c3c6584d1408dfefff6528e82e852dd8e4b5a8c810430300d1223d307e932"}
Jul 30 14:56:52.179722 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:52.179413    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wwfgt"
Jul 30 14:56:52.195686 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:52.195640    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wwfgt" podStartSLOduration=1.328240005 podStartE2EDuration="6.195623054s" totalImagesPullingTime="4.867383049s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 14:56:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:56:52.194859403 +0000 UTC m=+523.723797890" watchObservedRunningTime="2026-07-30 14:56:52.195623054 +0000 UTC m=+523.724561528"
Jul 30 14:56:58.211317 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:56:58.211288    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wwfgt"
Jul 30 14:57:05.263659 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:05.263582    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6fd596b994-nr2j6"]
Jul 30 14:57:05.271304 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:05.271284    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6fd596b994-nr2j6"
Jul 30 14:57:05.273676 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:05.273654    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-qdb2j\""
Jul 30 14:57:05.274223 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:05.274208    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 30 14:57:05.274299 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:05.274244    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 30 14:57:05.278315 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:05.278293    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6fd596b994-nr2j6"]
Jul 30 14:57:05.394407 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:05.394373    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bd0124de-47e4-480a-a500-d4c4f8b31401-webhook-cert\") pod \"maas-controller-6fd596b994-nr2j6\" (UID: \"bd0124de-47e4-480a-a500-d4c4f8b31401\") " pod="opendatahub/maas-controller-6fd596b994-nr2j6"
Jul 30 14:57:05.394567 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:05.394423    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lk654\" (UniqueName: \"kubernetes.io/projected/bd0124de-47e4-480a-a500-d4c4f8b31401-kube-api-access-lk654\") pod \"maas-controller-6fd596b994-nr2j6\" (UID: \"bd0124de-47e4-480a-a500-d4c4f8b31401\") " pod="opendatahub/maas-controller-6fd596b994-nr2j6"
Jul 30 14:57:05.464066 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:05.464034    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6f8f467c55-xcjp8"]
Jul 30 14:57:05.466284 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:05.466268    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f8f467c55-xcjp8"
Jul 30 14:57:05.477822 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:05.477800    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6f8f467c55-xcjp8"]
Jul 30 14:57:05.495034 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:05.495004    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bd0124de-47e4-480a-a500-d4c4f8b31401-webhook-cert\") pod \"maas-controller-6fd596b994-nr2j6\" (UID: \"bd0124de-47e4-480a-a500-d4c4f8b31401\") " pod="opendatahub/maas-controller-6fd596b994-nr2j6"
Jul 30 14:57:05.495191 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:05.495064    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lk654\" (UniqueName: \"kubernetes.io/projected/bd0124de-47e4-480a-a500-d4c4f8b31401-kube-api-access-lk654\") pod \"maas-controller-6fd596b994-nr2j6\" (UID: \"bd0124de-47e4-480a-a500-d4c4f8b31401\") " pod="opendatahub/maas-controller-6fd596b994-nr2j6"
Jul 30 14:57:05.495267 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:57:05.495182    2578 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 30 14:57:05.495267 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:57:05.495254    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd0124de-47e4-480a-a500-d4c4f8b31401-webhook-cert podName:bd0124de-47e4-480a-a500-d4c4f8b31401 nodeName:}" failed. No retries permitted until 2026-07-30 14:57:05.995233634 +0000 UTC m=+537.524172100 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/bd0124de-47e4-480a-a500-d4c4f8b31401-webhook-cert") pod "maas-controller-6fd596b994-nr2j6" (UID: "bd0124de-47e4-480a-a500-d4c4f8b31401") : secret "maas-controller-webhook-cert" not found
Jul 30 14:57:05.503192 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:05.503168    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lk654\" (UniqueName: \"kubernetes.io/projected/bd0124de-47e4-480a-a500-d4c4f8b31401-kube-api-access-lk654\") pod \"maas-controller-6fd596b994-nr2j6\" (UID: \"bd0124de-47e4-480a-a500-d4c4f8b31401\") " pod="opendatahub/maas-controller-6fd596b994-nr2j6"
Jul 30 14:57:05.596520 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:05.596482    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d3d6f5c8-4667-48b2-983a-08ee8da201f1-webhook-cert\") pod \"maas-controller-6f8f467c55-xcjp8\" (UID: \"d3d6f5c8-4667-48b2-983a-08ee8da201f1\") " pod="opendatahub/maas-controller-6f8f467c55-xcjp8"
Jul 30 14:57:05.596520 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:05.596518    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nqkp\" (UniqueName: \"kubernetes.io/projected/d3d6f5c8-4667-48b2-983a-08ee8da201f1-kube-api-access-2nqkp\") pod \"maas-controller-6f8f467c55-xcjp8\" (UID: \"d3d6f5c8-4667-48b2-983a-08ee8da201f1\") " pod="opendatahub/maas-controller-6f8f467c55-xcjp8"
Jul 30 14:57:05.697495 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:05.697459    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d3d6f5c8-4667-48b2-983a-08ee8da201f1-webhook-cert\") pod \"maas-controller-6f8f467c55-xcjp8\" (UID: \"d3d6f5c8-4667-48b2-983a-08ee8da201f1\") " pod="opendatahub/maas-controller-6f8f467c55-xcjp8"
Jul 30 14:57:05.697679 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:05.697497    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nqkp\" (UniqueName: \"kubernetes.io/projected/d3d6f5c8-4667-48b2-983a-08ee8da201f1-kube-api-access-2nqkp\") pod \"maas-controller-6f8f467c55-xcjp8\" (UID: \"d3d6f5c8-4667-48b2-983a-08ee8da201f1\") " pod="opendatahub/maas-controller-6f8f467c55-xcjp8"
Jul 30 14:57:05.699874 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:05.699853    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d3d6f5c8-4667-48b2-983a-08ee8da201f1-webhook-cert\") pod \"maas-controller-6f8f467c55-xcjp8\" (UID: \"d3d6f5c8-4667-48b2-983a-08ee8da201f1\") " pod="opendatahub/maas-controller-6f8f467c55-xcjp8"
Jul 30 14:57:05.705138 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:05.705114    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nqkp\" (UniqueName: \"kubernetes.io/projected/d3d6f5c8-4667-48b2-983a-08ee8da201f1-kube-api-access-2nqkp\") pod \"maas-controller-6f8f467c55-xcjp8\" (UID: \"d3d6f5c8-4667-48b2-983a-08ee8da201f1\") " pod="opendatahub/maas-controller-6f8f467c55-xcjp8"
Jul 30 14:57:05.779098 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:05.779064    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f8f467c55-xcjp8"
Jul 30 14:57:05.904207 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:05.904185    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6f8f467c55-xcjp8"]
Jul 30 14:57:05.906020 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:57:05.905998    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3d6f5c8_4667_48b2_983a_08ee8da201f1.slice/crio-82442fdfa320ecb290e557891e0cd19177e95e9e20f1c3237d8281790444c45b WatchSource:0}: Error finding container 82442fdfa320ecb290e557891e0cd19177e95e9e20f1c3237d8281790444c45b: Status 404 returned error can't find the container with id 82442fdfa320ecb290e557891e0cd19177e95e9e20f1c3237d8281790444c45b
Jul 30 14:57:06.000835 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:06.000810    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bd0124de-47e4-480a-a500-d4c4f8b31401-webhook-cert\") pod \"maas-controller-6fd596b994-nr2j6\" (UID: \"bd0124de-47e4-480a-a500-d4c4f8b31401\") " pod="opendatahub/maas-controller-6fd596b994-nr2j6"
Jul 30 14:57:06.003117 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:06.003092    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bd0124de-47e4-480a-a500-d4c4f8b31401-webhook-cert\") pod \"maas-controller-6fd596b994-nr2j6\" (UID: \"bd0124de-47e4-480a-a500-d4c4f8b31401\") " pod="opendatahub/maas-controller-6fd596b994-nr2j6"
Jul 30 14:57:06.183520 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:06.183434    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6fd596b994-nr2j6"
Jul 30 14:57:06.242215 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:06.242182    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f8f467c55-xcjp8" event={"ID":"d3d6f5c8-4667-48b2-983a-08ee8da201f1","Type":"ContainerStarted","Data":"82442fdfa320ecb290e557891e0cd19177e95e9e20f1c3237d8281790444c45b"}
Jul 30 14:57:06.315121 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:06.315088    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6fd596b994-nr2j6"]
Jul 30 14:57:06.317868 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:57:06.317844    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd0124de_47e4_480a_a500_d4c4f8b31401.slice/crio-6fafb091b11df4bbcb4600c0f1eb078d8b431926ee7b2d0ca2f1d4da16ab1f34 WatchSource:0}: Error finding container 6fafb091b11df4bbcb4600c0f1eb078d8b431926ee7b2d0ca2f1d4da16ab1f34: Status 404 returned error can't find the container with id 6fafb091b11df4bbcb4600c0f1eb078d8b431926ee7b2d0ca2f1d4da16ab1f34
Jul 30 14:57:07.248908 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:07.248865    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6fd596b994-nr2j6" event={"ID":"bd0124de-47e4-480a-a500-d4c4f8b31401","Type":"ContainerStarted","Data":"6fafb091b11df4bbcb4600c0f1eb078d8b431926ee7b2d0ca2f1d4da16ab1f34"}
Jul 30 14:57:09.260470 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:09.260373    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f8f467c55-xcjp8" event={"ID":"d3d6f5c8-4667-48b2-983a-08ee8da201f1","Type":"ContainerStarted","Data":"7c9d4b582af76b466730ca85847e75d534489c21d0a9356b1d45751e20e00c68"}
Jul 30 14:57:09.260898 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:09.260519    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6f8f467c55-xcjp8"
Jul 30 14:57:09.262087 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:09.262062    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6fd596b994-nr2j6" event={"ID":"bd0124de-47e4-480a-a500-d4c4f8b31401","Type":"ContainerStarted","Data":"5877e715193584544b11350647244dfbbee696a0060f91dea34039ad3286ecac"}
Jul 30 14:57:09.262235 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:09.262163    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6fd596b994-nr2j6"
Jul 30 14:57:09.289935 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:09.289886    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6f8f467c55-xcjp8" podStartSLOduration=1.358579588 podStartE2EDuration="4.289870664s" totalImagesPullingTime="2.931291076s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 14:57:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:57:09.288380204 +0000 UTC m=+540.817318677" watchObservedRunningTime="2026-07-30 14:57:09.289870664 +0000 UTC m=+540.818809138"
Jul 30 14:57:09.329899 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:09.329849    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6fd596b994-nr2j6" podStartSLOduration=1.815440856 podStartE2EDuration="4.329811419s" totalImagesPullingTime="2.514370563s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 14:57:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:57:09.328297086 +0000 UTC m=+540.857235560" watchObservedRunningTime="2026-07-30 14:57:09.329811419 +0000 UTC m=+540.858749892"
Jul 30 14:57:20.268774 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:20.268745    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6fd596b994-nr2j6"
Jul 30 14:57:20.269126 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:20.268811    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6f8f467c55-xcjp8"
Jul 30 14:57:20.331026 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:20.330990    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6fd596b994-nr2j6"]
Jul 30 14:57:20.331243 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:20.331218    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6fd596b994-nr2j6" podUID="bd0124de-47e4-480a-a500-d4c4f8b31401" containerName="manager" containerID="cri-o://5877e715193584544b11350647244dfbbee696a0060f91dea34039ad3286ecac" gracePeriod=30
Jul 30 14:57:20.564985 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:20.564956    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6fd596b994-nr2j6"
Jul 30 14:57:20.734282 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:20.734252    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bd0124de-47e4-480a-a500-d4c4f8b31401-webhook-cert\") pod \"bd0124de-47e4-480a-a500-d4c4f8b31401\" (UID: \"bd0124de-47e4-480a-a500-d4c4f8b31401\") "
Jul 30 14:57:20.734466 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:20.734329    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lk654\" (UniqueName: \"kubernetes.io/projected/bd0124de-47e4-480a-a500-d4c4f8b31401-kube-api-access-lk654\") pod \"bd0124de-47e4-480a-a500-d4c4f8b31401\" (UID: \"bd0124de-47e4-480a-a500-d4c4f8b31401\") "
Jul 30 14:57:20.736394 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:20.736368    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd0124de-47e4-480a-a500-d4c4f8b31401-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "bd0124de-47e4-480a-a500-d4c4f8b31401" (UID: "bd0124de-47e4-480a-a500-d4c4f8b31401"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 14:57:20.736507 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:20.736419    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd0124de-47e4-480a-a500-d4c4f8b31401-kube-api-access-lk654" (OuterVolumeSpecName: "kube-api-access-lk654") pod "bd0124de-47e4-480a-a500-d4c4f8b31401" (UID: "bd0124de-47e4-480a-a500-d4c4f8b31401"). InnerVolumeSpecName "kube-api-access-lk654". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 14:57:20.835379 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:20.835355    2578 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bd0124de-47e4-480a-a500-d4c4f8b31401-webhook-cert\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 14:57:20.835379 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:20.835378    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lk654\" (UniqueName: \"kubernetes.io/projected/bd0124de-47e4-480a-a500-d4c4f8b31401-kube-api-access-lk654\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 14:57:21.314860 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:21.314827    2578 generic.go:356] "Generic (PLEG): container finished" podID="bd0124de-47e4-480a-a500-d4c4f8b31401" containerID="5877e715193584544b11350647244dfbbee696a0060f91dea34039ad3286ecac" exitCode=0
Jul 30 14:57:21.315352 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:21.314876    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6fd596b994-nr2j6"
Jul 30 14:57:21.315352 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:21.314901    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6fd596b994-nr2j6" event={"ID":"bd0124de-47e4-480a-a500-d4c4f8b31401","Type":"ContainerDied","Data":"5877e715193584544b11350647244dfbbee696a0060f91dea34039ad3286ecac"}
Jul 30 14:57:21.315352 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:21.314944    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6fd596b994-nr2j6" event={"ID":"bd0124de-47e4-480a-a500-d4c4f8b31401","Type":"ContainerDied","Data":"6fafb091b11df4bbcb4600c0f1eb078d8b431926ee7b2d0ca2f1d4da16ab1f34"}
Jul 30 14:57:21.315352 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:21.314966    2578 scope.go:117] "RemoveContainer" containerID="5877e715193584544b11350647244dfbbee696a0060f91dea34039ad3286ecac"
Jul 30 14:57:21.323084 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:21.323065    2578 scope.go:117] "RemoveContainer" containerID="5877e715193584544b11350647244dfbbee696a0060f91dea34039ad3286ecac"
Jul 30 14:57:21.323351 ip-10-0-133-117 kubenswrapper[2578]: E0730 14:57:21.323330    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5877e715193584544b11350647244dfbbee696a0060f91dea34039ad3286ecac\": container with ID starting with 5877e715193584544b11350647244dfbbee696a0060f91dea34039ad3286ecac not found: ID does not exist" containerID="5877e715193584544b11350647244dfbbee696a0060f91dea34039ad3286ecac"
Jul 30 14:57:21.323424 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:21.323362    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5877e715193584544b11350647244dfbbee696a0060f91dea34039ad3286ecac"} err="failed to get container status \"5877e715193584544b11350647244dfbbee696a0060f91dea34039ad3286ecac\": rpc error: code = NotFound desc = could not find container \"5877e715193584544b11350647244dfbbee696a0060f91dea34039ad3286ecac\": container with ID starting with 5877e715193584544b11350647244dfbbee696a0060f91dea34039ad3286ecac not found: ID does not exist"
Jul 30 14:57:21.330191 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:21.330170    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6fd596b994-nr2j6"]
Jul 30 14:57:21.334053 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:21.334033    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6fd596b994-nr2j6"]
Jul 30 14:57:22.978078 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:22.978047    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd0124de-47e4-480a-a500-d4c4f8b31401" path="/var/lib/kubelet/pods/bd0124de-47e4-480a-a500-d4c4f8b31401/volumes"
Jul 30 14:57:42.382028 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:42.381992    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-ff4449fc7-cdnxs"]
Jul 30 14:57:42.382503 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:42.382347    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd0124de-47e4-480a-a500-d4c4f8b31401" containerName="manager"
Jul 30 14:57:42.382503 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:42.382359    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd0124de-47e4-480a-a500-d4c4f8b31401" containerName="manager"
Jul 30 14:57:42.382503 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:42.382423    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd0124de-47e4-480a-a500-d4c4f8b31401" containerName="manager"
Jul 30 14:57:42.387642 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:42.387622    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-ff4449fc7-cdnxs"
Jul 30 14:57:42.389867 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:42.389843    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 30 14:57:42.389867 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:42.389859    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-q7854\""
Jul 30 14:57:42.395205 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:42.395183    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-ff4449fc7-cdnxs"]
Jul 30 14:57:42.525311 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:42.525279    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/54eeb596-5029-4848-997b-aeb298cda9b4-openshift-service-ca\") pod \"maas-api-ff4449fc7-cdnxs\" (UID: \"54eeb596-5029-4848-997b-aeb298cda9b4\") " pod="odh-ai-gateway-infra/maas-api-ff4449fc7-cdnxs"
Jul 30 14:57:42.525479 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:42.525316    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bq82\" (UniqueName: \"kubernetes.io/projected/54eeb596-5029-4848-997b-aeb298cda9b4-kube-api-access-7bq82\") pod \"maas-api-ff4449fc7-cdnxs\" (UID: \"54eeb596-5029-4848-997b-aeb298cda9b4\") " pod="odh-ai-gateway-infra/maas-api-ff4449fc7-cdnxs"
Jul 30 14:57:42.525479 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:42.525403    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/54eeb596-5029-4848-997b-aeb298cda9b4-maas-api-tls\") pod \"maas-api-ff4449fc7-cdnxs\" (UID: \"54eeb596-5029-4848-997b-aeb298cda9b4\") " pod="odh-ai-gateway-infra/maas-api-ff4449fc7-cdnxs"
Jul 30 14:57:42.625867 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:42.625827    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/54eeb596-5029-4848-997b-aeb298cda9b4-maas-api-tls\") pod \"maas-api-ff4449fc7-cdnxs\" (UID: \"54eeb596-5029-4848-997b-aeb298cda9b4\") " pod="odh-ai-gateway-infra/maas-api-ff4449fc7-cdnxs"
Jul 30 14:57:42.626051 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:42.625928    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/54eeb596-5029-4848-997b-aeb298cda9b4-openshift-service-ca\") pod \"maas-api-ff4449fc7-cdnxs\" (UID: \"54eeb596-5029-4848-997b-aeb298cda9b4\") " pod="odh-ai-gateway-infra/maas-api-ff4449fc7-cdnxs"
Jul 30 14:57:42.626051 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:42.625952    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bq82\" (UniqueName: \"kubernetes.io/projected/54eeb596-5029-4848-997b-aeb298cda9b4-kube-api-access-7bq82\") pod \"maas-api-ff4449fc7-cdnxs\" (UID: \"54eeb596-5029-4848-997b-aeb298cda9b4\") " pod="odh-ai-gateway-infra/maas-api-ff4449fc7-cdnxs"
Jul 30 14:57:42.626573 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:42.626553    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/54eeb596-5029-4848-997b-aeb298cda9b4-openshift-service-ca\") pod \"maas-api-ff4449fc7-cdnxs\" (UID: \"54eeb596-5029-4848-997b-aeb298cda9b4\") " pod="odh-ai-gateway-infra/maas-api-ff4449fc7-cdnxs"
Jul 30 14:57:42.628312 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:42.628291    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/54eeb596-5029-4848-997b-aeb298cda9b4-maas-api-tls\") pod \"maas-api-ff4449fc7-cdnxs\" (UID: \"54eeb596-5029-4848-997b-aeb298cda9b4\") " pod="odh-ai-gateway-infra/maas-api-ff4449fc7-cdnxs"
Jul 30 14:57:42.635019 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:42.634961    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bq82\" (UniqueName: \"kubernetes.io/projected/54eeb596-5029-4848-997b-aeb298cda9b4-kube-api-access-7bq82\") pod \"maas-api-ff4449fc7-cdnxs\" (UID: \"54eeb596-5029-4848-997b-aeb298cda9b4\") " pod="odh-ai-gateway-infra/maas-api-ff4449fc7-cdnxs"
Jul 30 14:57:42.698600 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:42.698569    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-ff4449fc7-cdnxs"
Jul 30 14:57:42.839108 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:42.839077    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-ff4449fc7-cdnxs"]
Jul 30 14:57:42.842160 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:57:42.842119    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54eeb596_5029_4848_997b_aeb298cda9b4.slice/crio-29d1d22fbeb65f44a48ceaa95a87c72d9f0f5dd6dbfe96664bca93ec1109ed86 WatchSource:0}: Error finding container 29d1d22fbeb65f44a48ceaa95a87c72d9f0f5dd6dbfe96664bca93ec1109ed86: Status 404 returned error can't find the container with id 29d1d22fbeb65f44a48ceaa95a87c72d9f0f5dd6dbfe96664bca93ec1109ed86
Jul 30 14:57:43.151993 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:43.151962    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-bjkzn"]
Jul 30 14:57:43.156006 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:43.155985    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-bjkzn"
Jul 30 14:57:43.157903 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:43.157884    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 30 14:57:43.158027 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:43.157989    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-lt24z\""
Jul 30 14:57:43.165548 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:43.165517    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-bjkzn"]
Jul 30 14:57:43.231083 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:43.231055    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2cb2ce43-84fb-480e-9b51-ea6c36a181b2-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-bjkzn\" (UID: \"2cb2ce43-84fb-480e-9b51-ea6c36a181b2\") " pod="openshift-ingress/payload-processing-5cc6cd444b-bjkzn"
Jul 30 14:57:43.231255 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:43.231089    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdfgc\" (UniqueName: \"kubernetes.io/projected/2cb2ce43-84fb-480e-9b51-ea6c36a181b2-kube-api-access-kdfgc\") pod \"payload-processing-5cc6cd444b-bjkzn\" (UID: \"2cb2ce43-84fb-480e-9b51-ea6c36a181b2\") " pod="openshift-ingress/payload-processing-5cc6cd444b-bjkzn"
Jul 30 14:57:43.332204 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:43.332175    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2cb2ce43-84fb-480e-9b51-ea6c36a181b2-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-bjkzn\" (UID: \"2cb2ce43-84fb-480e-9b51-ea6c36a181b2\") " pod="openshift-ingress/payload-processing-5cc6cd444b-bjkzn"
Jul 30 14:57:43.332204 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:43.332213    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdfgc\" (UniqueName: \"kubernetes.io/projected/2cb2ce43-84fb-480e-9b51-ea6c36a181b2-kube-api-access-kdfgc\") pod \"payload-processing-5cc6cd444b-bjkzn\" (UID: \"2cb2ce43-84fb-480e-9b51-ea6c36a181b2\") " pod="openshift-ingress/payload-processing-5cc6cd444b-bjkzn"
Jul 30 14:57:43.333102 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:43.333073    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2cb2ce43-84fb-480e-9b51-ea6c36a181b2-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-bjkzn\" (UID: \"2cb2ce43-84fb-480e-9b51-ea6c36a181b2\") " pod="openshift-ingress/payload-processing-5cc6cd444b-bjkzn"
Jul 30 14:57:43.340012 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:43.339988    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdfgc\" (UniqueName: \"kubernetes.io/projected/2cb2ce43-84fb-480e-9b51-ea6c36a181b2-kube-api-access-kdfgc\") pod \"payload-processing-5cc6cd444b-bjkzn\" (UID: \"2cb2ce43-84fb-480e-9b51-ea6c36a181b2\") " pod="openshift-ingress/payload-processing-5cc6cd444b-bjkzn"
Jul 30 14:57:43.420947 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:43.420861    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-ff4449fc7-cdnxs" event={"ID":"54eeb596-5029-4848-997b-aeb298cda9b4","Type":"ContainerStarted","Data":"29d1d22fbeb65f44a48ceaa95a87c72d9f0f5dd6dbfe96664bca93ec1109ed86"}
Jul 30 14:57:43.468233 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:43.468202    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-bjkzn"
Jul 30 14:57:43.601970 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:43.601928    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-bjkzn"]
Jul 30 14:57:43.604299 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:57:43.604270    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cb2ce43_84fb_480e_9b51_ea6c36a181b2.slice/crio-701f55e042b0bec465b458c4cff1a77bfee193d94ce2e930840939e630ae3e92 WatchSource:0}: Error finding container 701f55e042b0bec465b458c4cff1a77bfee193d94ce2e930840939e630ae3e92: Status 404 returned error can't find the container with id 701f55e042b0bec465b458c4cff1a77bfee193d94ce2e930840939e630ae3e92
Jul 30 14:57:44.430272 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:44.430231    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-bjkzn" event={"ID":"2cb2ce43-84fb-480e-9b51-ea6c36a181b2","Type":"ContainerStarted","Data":"701f55e042b0bec465b458c4cff1a77bfee193d94ce2e930840939e630ae3e92"}
Jul 30 14:57:45.438414 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:45.438373    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-ff4449fc7-cdnxs" event={"ID":"54eeb596-5029-4848-997b-aeb298cda9b4","Type":"ContainerStarted","Data":"66d94ba347505eada7c1f5428f9a2107c4985ce9aa759e25dddb84e8c00d8998"}
Jul 30 14:57:45.438831 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:45.438636    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-ff4449fc7-cdnxs"
Jul 30 14:57:45.456501 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:45.456435    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-ff4449fc7-cdnxs" podStartSLOduration=1.697775417 podStartE2EDuration="3.456414787s" totalImagesPullingTime="1.75863937s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 14:57:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:57:45.454866363 +0000 UTC m=+576.983804838" watchObservedRunningTime="2026-07-30 14:57:45.456414787 +0000 UTC m=+576.985353264"
Jul 30 14:57:46.444818 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:46.444780    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-bjkzn" event={"ID":"2cb2ce43-84fb-480e-9b51-ea6c36a181b2","Type":"ContainerStarted","Data":"7a34f75d1371371d85931015b398429ea0bbca8f0663048f871a5764aa13c2c9"}
Jul 30 14:57:46.462438 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:46.462383    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5cc6cd444b-bjkzn" podStartSLOduration=0.713414886 podStartE2EDuration="3.462363378s" totalImagesPullingTime="2.748948492s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 14:57:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:57:46.460029161 +0000 UTC m=+577.988967663" watchObservedRunningTime="2026-07-30 14:57:46.462363378 +0000 UTC m=+577.991301852"
Jul 30 14:57:47.450421 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:47.450380    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5cc6cd444b-bjkzn"
Jul 30 14:57:51.450868 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:51.450838    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-ff4449fc7-cdnxs"
Jul 30 14:57:58.458081 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:57:58.458044    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5cc6cd444b-bjkzn"
Jul 30 14:58:08.921069 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:08.921037    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4mjmp_de241220-5f27-4b0b-b7c6-6d4b350e24e1/console-operator/1.log"
Jul 30 14:58:08.922890 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:08.922870    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4mjmp_de241220-5f27-4b0b-b7c6-6d4b350e24e1/console-operator/1.log"
Jul 30 14:58:33.407020 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:33.406935    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-g64tv"]
Jul 30 14:58:33.410278 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:33.410258    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g64tv"
Jul 30 14:58:33.412344 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:33.412323    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 30 14:58:33.412457 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:33.412361    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-756b8\""
Jul 30 14:58:33.412914 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:33.412898    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 30 14:58:33.413140 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:33.413122    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 30 14:58:33.426597 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:33.423885    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-g64tv"]
Jul 30 14:58:33.453505 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:33.453473    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a874a8f6-a84a-493b-949e-6420d2278658-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g64tv\" (UID: \"a874a8f6-a84a-493b-949e-6420d2278658\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g64tv"
Jul 30 14:58:33.453639 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:33.453515    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a874a8f6-a84a-493b-949e-6420d2278658-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g64tv\" (UID: \"a874a8f6-a84a-493b-949e-6420d2278658\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g64tv"
Jul 30 14:58:33.453639 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:33.453533    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9vbj\" (UniqueName: \"kubernetes.io/projected/a874a8f6-a84a-493b-949e-6420d2278658-kube-api-access-d9vbj\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g64tv\" (UID: \"a874a8f6-a84a-493b-949e-6420d2278658\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g64tv"
Jul 30 14:58:33.453716 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:33.453653    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a874a8f6-a84a-493b-949e-6420d2278658-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g64tv\" (UID: \"a874a8f6-a84a-493b-949e-6420d2278658\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g64tv"
Jul 30 14:58:33.453716 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:33.453702    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a874a8f6-a84a-493b-949e-6420d2278658-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g64tv\" (UID: \"a874a8f6-a84a-493b-949e-6420d2278658\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g64tv"
Jul 30 14:58:33.453777 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:33.453732    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a874a8f6-a84a-493b-949e-6420d2278658-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g64tv\" (UID: \"a874a8f6-a84a-493b-949e-6420d2278658\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g64tv"
Jul 30 14:58:33.554333 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:33.554301    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a874a8f6-a84a-493b-949e-6420d2278658-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g64tv\" (UID: \"a874a8f6-a84a-493b-949e-6420d2278658\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g64tv"
Jul 30 14:58:33.554480 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:33.554347    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a874a8f6-a84a-493b-949e-6420d2278658-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g64tv\" (UID: \"a874a8f6-a84a-493b-949e-6420d2278658\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g64tv"
Jul 30 14:58:33.554480 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:33.554377    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a874a8f6-a84a-493b-949e-6420d2278658-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g64tv\" (UID: \"a874a8f6-a84a-493b-949e-6420d2278658\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g64tv"
Jul 30 14:58:33.554480 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:33.554402    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a874a8f6-a84a-493b-949e-6420d2278658-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g64tv\" (UID: \"a874a8f6-a84a-493b-949e-6420d2278658\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g64tv"
Jul 30 14:58:33.554480 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:33.554419    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9vbj\" (UniqueName: \"kubernetes.io/projected/a874a8f6-a84a-493b-949e-6420d2278658-kube-api-access-d9vbj\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g64tv\" (UID: \"a874a8f6-a84a-493b-949e-6420d2278658\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g64tv"
Jul 30 14:58:33.554480 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:33.554465    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a874a8f6-a84a-493b-949e-6420d2278658-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g64tv\" (UID: \"a874a8f6-a84a-493b-949e-6420d2278658\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g64tv"
Jul 30 14:58:33.554815 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:33.554788    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a874a8f6-a84a-493b-949e-6420d2278658-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g64tv\" (UID: \"a874a8f6-a84a-493b-949e-6420d2278658\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g64tv"
Jul 30 14:58:33.554886 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:33.554857    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a874a8f6-a84a-493b-949e-6420d2278658-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g64tv\" (UID: \"a874a8f6-a84a-493b-949e-6420d2278658\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g64tv"
Jul 30 14:58:33.554994 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:33.554887    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a874a8f6-a84a-493b-949e-6420d2278658-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g64tv\" (UID: \"a874a8f6-a84a-493b-949e-6420d2278658\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g64tv"
Jul 30 14:58:33.556686 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:33.556655    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a874a8f6-a84a-493b-949e-6420d2278658-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g64tv\" (UID: \"a874a8f6-a84a-493b-949e-6420d2278658\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g64tv"
Jul 30 14:58:33.556949 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:33.556926    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a874a8f6-a84a-493b-949e-6420d2278658-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g64tv\" (UID: \"a874a8f6-a84a-493b-949e-6420d2278658\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g64tv"
Jul 30 14:58:33.562646 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:33.562623    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9vbj\" (UniqueName: \"kubernetes.io/projected/a874a8f6-a84a-493b-949e-6420d2278658-kube-api-access-d9vbj\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g64tv\" (UID: \"a874a8f6-a84a-493b-949e-6420d2278658\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g64tv"
Jul 30 14:58:33.723208 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:33.723108    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g64tv"
Jul 30 14:58:33.855250 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:33.855224    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-g64tv"]
Jul 30 14:58:33.857171 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:58:33.857129    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda874a8f6_a84a_493b_949e_6420d2278658.slice/crio-15b3e833de5c5cda4d563dc9306f6914acbfcff48cc5474fcc889e73f37184dd WatchSource:0}: Error finding container 15b3e833de5c5cda4d563dc9306f6914acbfcff48cc5474fcc889e73f37184dd: Status 404 returned error can't find the container with id 15b3e833de5c5cda4d563dc9306f6914acbfcff48cc5474fcc889e73f37184dd
Jul 30 14:58:33.858410 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:33.858392    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 14:58:34.665634 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:34.665589    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g64tv" event={"ID":"a874a8f6-a84a-493b-949e-6420d2278658","Type":"ContainerStarted","Data":"15b3e833de5c5cda4d563dc9306f6914acbfcff48cc5474fcc889e73f37184dd"}
Jul 30 14:58:36.676948 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:36.676904    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g64tv" event={"ID":"a874a8f6-a84a-493b-949e-6420d2278658","Type":"ContainerStarted","Data":"6948d3eab8f6b410220497cea2358ee3c38f21505ab5e6f3b718b7fffe0bf2f5"}
Jul 30 14:58:36.695788 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:36.695740    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g64tv" podStartSLOduration=1.544456461 podStartE2EDuration="3.695722977s" totalImagesPullingTime="2.151266516s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 14:58:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:58:36.694379825 +0000 UTC m=+628.223318299" watchObservedRunningTime="2026-07-30 14:58:36.695722977 +0000 UTC m=+628.224661451"
Jul 30 14:58:43.723408 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:43.723373    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g64tv"
Jul 30 14:58:43.723976 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:43.723446    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g64tv"
Jul 30 14:58:43.735921 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:43.735896    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g64tv"
Jul 30 14:58:44.723614 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:44.723585    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g64tv"
Jul 30 14:58:57.840061 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:57.840025    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8vc55"]
Jul 30 14:58:57.844907 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:57.844882    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8vc55"
Jul 30 14:58:57.847130 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:57.847104    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 30 14:58:57.854639 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:57.854611    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8vc55"]
Jul 30 14:58:57.882369 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:57.882339    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3664ed3-abea-4fa6-9437-1d6c941f85db-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8vc55\" (UID: \"d3664ed3-abea-4fa6-9437-1d6c941f85db\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8vc55"
Jul 30 14:58:57.882514 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:57.882406    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3664ed3-abea-4fa6-9437-1d6c941f85db-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8vc55\" (UID: \"d3664ed3-abea-4fa6-9437-1d6c941f85db\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8vc55"
Jul 30 14:58:57.882514 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:57.882430    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3664ed3-abea-4fa6-9437-1d6c941f85db-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8vc55\" (UID: \"d3664ed3-abea-4fa6-9437-1d6c941f85db\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8vc55"
Jul 30 14:58:57.882514 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:57.882451    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3664ed3-abea-4fa6-9437-1d6c941f85db-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8vc55\" (UID: \"d3664ed3-abea-4fa6-9437-1d6c941f85db\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8vc55"
Jul 30 14:58:57.882653 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:57.882515    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3664ed3-abea-4fa6-9437-1d6c941f85db-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8vc55\" (UID: \"d3664ed3-abea-4fa6-9437-1d6c941f85db\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8vc55"
Jul 30 14:58:57.882653 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:57.882539    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgtl4\" (UniqueName: \"kubernetes.io/projected/d3664ed3-abea-4fa6-9437-1d6c941f85db-kube-api-access-bgtl4\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8vc55\" (UID: \"d3664ed3-abea-4fa6-9437-1d6c941f85db\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8vc55"
Jul 30 14:58:57.983252 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:57.983220    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3664ed3-abea-4fa6-9437-1d6c941f85db-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8vc55\" (UID: \"d3664ed3-abea-4fa6-9437-1d6c941f85db\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8vc55"
Jul 30 14:58:57.983393 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:57.983292    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3664ed3-abea-4fa6-9437-1d6c941f85db-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8vc55\" (UID: \"d3664ed3-abea-4fa6-9437-1d6c941f85db\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8vc55"
Jul 30 14:58:57.983393 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:57.983313    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3664ed3-abea-4fa6-9437-1d6c941f85db-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8vc55\" (UID: \"d3664ed3-abea-4fa6-9437-1d6c941f85db\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8vc55"
Jul 30 14:58:57.983393 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:57.983333    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3664ed3-abea-4fa6-9437-1d6c941f85db-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8vc55\" (UID: \"d3664ed3-abea-4fa6-9437-1d6c941f85db\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8vc55"
Jul 30 14:58:57.983393 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:57.983356    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3664ed3-abea-4fa6-9437-1d6c941f85db-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8vc55\" (UID: \"d3664ed3-abea-4fa6-9437-1d6c941f85db\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8vc55"
Jul 30 14:58:57.983393 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:57.983371    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgtl4\" (UniqueName: \"kubernetes.io/projected/d3664ed3-abea-4fa6-9437-1d6c941f85db-kube-api-access-bgtl4\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8vc55\" (UID: \"d3664ed3-abea-4fa6-9437-1d6c941f85db\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8vc55"
Jul 30 14:58:57.983694 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:57.983670    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3664ed3-abea-4fa6-9437-1d6c941f85db-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8vc55\" (UID: \"d3664ed3-abea-4fa6-9437-1d6c941f85db\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8vc55"
Jul 30 14:58:57.983767 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:57.983715    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3664ed3-abea-4fa6-9437-1d6c941f85db-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8vc55\" (UID: \"d3664ed3-abea-4fa6-9437-1d6c941f85db\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8vc55"
Jul 30 14:58:57.983821 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:57.983771    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3664ed3-abea-4fa6-9437-1d6c941f85db-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8vc55\" (UID: \"d3664ed3-abea-4fa6-9437-1d6c941f85db\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8vc55"
Jul 30 14:58:57.985484 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:57.985460    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3664ed3-abea-4fa6-9437-1d6c941f85db-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8vc55\" (UID: \"d3664ed3-abea-4fa6-9437-1d6c941f85db\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8vc55"
Jul 30 14:58:57.985894 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:57.985872    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3664ed3-abea-4fa6-9437-1d6c941f85db-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8vc55\" (UID: \"d3664ed3-abea-4fa6-9437-1d6c941f85db\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8vc55"
Jul 30 14:58:57.991557 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:57.991533    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgtl4\" (UniqueName: \"kubernetes.io/projected/d3664ed3-abea-4fa6-9437-1d6c941f85db-kube-api-access-bgtl4\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8vc55\" (UID: \"d3664ed3-abea-4fa6-9437-1d6c941f85db\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8vc55"
Jul 30 14:58:58.156057 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:58.155981    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8vc55"
Jul 30 14:58:58.287307 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:58.287281    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8vc55"]
Jul 30 14:58:58.290692 ip-10-0-133-117 kubenswrapper[2578]: W0730 14:58:58.290666    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3664ed3_abea_4fa6_9437_1d6c941f85db.slice/crio-75c598524e2245eeb803f2f8420bbb1035b6ad79cb0572d81ffb8959e77dfe0f WatchSource:0}: Error finding container 75c598524e2245eeb803f2f8420bbb1035b6ad79cb0572d81ffb8959e77dfe0f: Status 404 returned error can't find the container with id 75c598524e2245eeb803f2f8420bbb1035b6ad79cb0572d81ffb8959e77dfe0f
Jul 30 14:58:58.774894 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:58.774815    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8vc55" event={"ID":"d3664ed3-abea-4fa6-9437-1d6c941f85db","Type":"ContainerStarted","Data":"8dd6f49601e817cb987f4e8a79a4a65679601abbaebb05cb837ee816667228c1"}
Jul 30 14:58:58.774894 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:58.774861    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8vc55" event={"ID":"d3664ed3-abea-4fa6-9437-1d6c941f85db","Type":"ContainerStarted","Data":"75c598524e2245eeb803f2f8420bbb1035b6ad79cb0572d81ffb8959e77dfe0f"}
Jul 30 14:58:58.798274 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:58:58.798224    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8vc55" podStartSLOduration=1.605518641 podStartE2EDuration="1.798207469s" totalImagesPullingTime="192.688828ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 14:58:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:58:58.795842587 +0000 UTC m=+650.324781061" watchObservedRunningTime="2026-07-30 14:58:58.798207469 +0000 UTC m=+650.327145943"
Jul 30 14:59:08.156209 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:59:08.156179    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8vc55"
Jul 30 14:59:08.156727 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:59:08.156238    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8vc55"
Jul 30 14:59:08.168962 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:59:08.168937    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8vc55"
Jul 30 14:59:08.830651 ip-10-0-133-117 kubenswrapper[2578]: I0730 14:59:08.830611    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8vc55"
Jul 30 15:00:00.145195 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:00:00.145155    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29757060-g2z7x"]
Jul 30 15:00:00.148719 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:00:00.148703    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757060-g2z7x"
Jul 30 15:00:00.150494 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:00:00.150474    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rmv59\""
Jul 30 15:00:00.155095 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:00:00.155066    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29757060-g2z7x"]
Jul 30 15:00:00.159940 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:00:00.159920    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757060-g2z7x"
Jul 30 15:00:00.290094 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:00:00.290061    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29757060-g2z7x"]
Jul 30 15:00:00.293480 ip-10-0-133-117 kubenswrapper[2578]: W0730 15:00:00.293448    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8d2ee95_3d18_4385_9dcf_e67c75850a73.slice/crio-7847cd7088ccb7e8064512b06ffe4d58a1766352ecf1b52d23ada86854fd48b9 WatchSource:0}: Error finding container 7847cd7088ccb7e8064512b06ffe4d58a1766352ecf1b52d23ada86854fd48b9: Status 404 returned error can't find the container with id 7847cd7088ccb7e8064512b06ffe4d58a1766352ecf1b52d23ada86854fd48b9
Jul 30 15:00:01.048466 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:00:01.048437    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757060-g2z7x" event={"ID":"f8d2ee95-3d18-4385-9dcf-e67c75850a73","Type":"ContainerStarted","Data":"7847cd7088ccb7e8064512b06ffe4d58a1766352ecf1b52d23ada86854fd48b9"}
Jul 30 15:00:04.063920 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:00:04.063887    2578 generic.go:356] "Generic (PLEG): container finished" podID="f8d2ee95-3d18-4385-9dcf-e67c75850a73" containerID="5a9b19883d174b14465418ee526d59d6ad18435eab8243c0f56e796db9f46207" exitCode=0
Jul 30 15:00:04.064263 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:00:04.063947    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757060-g2z7x" event={"ID":"f8d2ee95-3d18-4385-9dcf-e67c75850a73","Type":"ContainerDied","Data":"5a9b19883d174b14465418ee526d59d6ad18435eab8243c0f56e796db9f46207"}
Jul 30 15:00:05.200406 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:00:05.200380    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757060-g2z7x"
Jul 30 15:00:06.073896 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:00:06.073870    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757060-g2z7x"
Jul 30 15:00:06.074078 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:00:06.073869    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757060-g2z7x" event={"ID":"f8d2ee95-3d18-4385-9dcf-e67c75850a73","Type":"ContainerDied","Data":"7847cd7088ccb7e8064512b06ffe4d58a1766352ecf1b52d23ada86854fd48b9"}
Jul 30 15:00:06.074078 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:00:06.073989    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7847cd7088ccb7e8064512b06ffe4d58a1766352ecf1b52d23ada86854fd48b9"
Jul 30 15:02:01.367534 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:02:01.367500    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6f8f467c55-xcjp8"]
Jul 30 15:02:01.368025 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:02:01.367742    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6f8f467c55-xcjp8" podUID="d3d6f5c8-4667-48b2-983a-08ee8da201f1" containerName="manager" containerID="cri-o://7c9d4b582af76b466730ca85847e75d534489c21d0a9356b1d45751e20e00c68" gracePeriod=30
Jul 30 15:02:01.597436 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:02:01.597407    2578 generic.go:356] "Generic (PLEG): container finished" podID="d3d6f5c8-4667-48b2-983a-08ee8da201f1" containerID="7c9d4b582af76b466730ca85847e75d534489c21d0a9356b1d45751e20e00c68" exitCode=0
Jul 30 15:02:01.597556 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:02:01.597482    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f8f467c55-xcjp8" event={"ID":"d3d6f5c8-4667-48b2-983a-08ee8da201f1","Type":"ContainerDied","Data":"7c9d4b582af76b466730ca85847e75d534489c21d0a9356b1d45751e20e00c68"}
Jul 30 15:02:01.615023 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:02:01.614999    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f8f467c55-xcjp8"
Jul 30 15:02:01.745697 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:02:01.745610    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2nqkp\" (UniqueName: \"kubernetes.io/projected/d3d6f5c8-4667-48b2-983a-08ee8da201f1-kube-api-access-2nqkp\") pod \"d3d6f5c8-4667-48b2-983a-08ee8da201f1\" (UID: \"d3d6f5c8-4667-48b2-983a-08ee8da201f1\") "
Jul 30 15:02:01.745697 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:02:01.745682    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d3d6f5c8-4667-48b2-983a-08ee8da201f1-webhook-cert\") pod \"d3d6f5c8-4667-48b2-983a-08ee8da201f1\" (UID: \"d3d6f5c8-4667-48b2-983a-08ee8da201f1\") "
Jul 30 15:02:01.747680 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:02:01.747645    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3d6f5c8-4667-48b2-983a-08ee8da201f1-kube-api-access-2nqkp" (OuterVolumeSpecName: "kube-api-access-2nqkp") pod "d3d6f5c8-4667-48b2-983a-08ee8da201f1" (UID: "d3d6f5c8-4667-48b2-983a-08ee8da201f1"). InnerVolumeSpecName "kube-api-access-2nqkp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:02:01.747680 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:02:01.747669    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3d6f5c8-4667-48b2-983a-08ee8da201f1-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "d3d6f5c8-4667-48b2-983a-08ee8da201f1" (UID: "d3d6f5c8-4667-48b2-983a-08ee8da201f1"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 15:02:01.846508 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:02:01.846475    2578 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d3d6f5c8-4667-48b2-983a-08ee8da201f1-webhook-cert\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:02:01.846508 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:02:01.846500    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2nqkp\" (UniqueName: \"kubernetes.io/projected/d3d6f5c8-4667-48b2-983a-08ee8da201f1-kube-api-access-2nqkp\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:02:02.603466 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:02:02.603436    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f8f467c55-xcjp8"
Jul 30 15:02:02.603466 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:02:02.603452    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f8f467c55-xcjp8" event={"ID":"d3d6f5c8-4667-48b2-983a-08ee8da201f1","Type":"ContainerDied","Data":"82442fdfa320ecb290e557891e0cd19177e95e9e20f1c3237d8281790444c45b"}
Jul 30 15:02:02.603970 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:02:02.603518    2578 scope.go:117] "RemoveContainer" containerID="7c9d4b582af76b466730ca85847e75d534489c21d0a9356b1d45751e20e00c68"
Jul 30 15:02:02.625063 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:02:02.625036    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6f8f467c55-xcjp8"]
Jul 30 15:02:02.634289 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:02:02.630078    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6f8f467c55-xcjp8"]
Jul 30 15:02:02.979394 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:02:02.979315    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3d6f5c8-4667-48b2-983a-08ee8da201f1" path="/var/lib/kubelet/pods/d3d6f5c8-4667-48b2-983a-08ee8da201f1/volumes"
Jul 30 15:03:03.524251 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:03.524170    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6f8f467c55-qdkrz"]
Jul 30 15:03:03.524665 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:03.524607    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8d2ee95-3d18-4385-9dcf-e67c75850a73" containerName="cleanup"
Jul 30 15:03:03.524665 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:03.524624    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8d2ee95-3d18-4385-9dcf-e67c75850a73" containerName="cleanup"
Jul 30 15:03:03.524665 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:03.524641    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3d6f5c8-4667-48b2-983a-08ee8da201f1" containerName="manager"
Jul 30 15:03:03.524665 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:03.524651    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3d6f5c8-4667-48b2-983a-08ee8da201f1" containerName="manager"
Jul 30 15:03:03.524841 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:03.524743    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3d6f5c8-4667-48b2-983a-08ee8da201f1" containerName="manager"
Jul 30 15:03:03.524841 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:03.524756    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8d2ee95-3d18-4385-9dcf-e67c75850a73" containerName="cleanup"
Jul 30 15:03:03.526914 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:03.526892    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f8f467c55-qdkrz"
Jul 30 15:03:03.529360 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:03.529343    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 30 15:03:03.529415 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:03.529375    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 30 15:03:03.529462 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:03.529343    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-qdb2j\""
Jul 30 15:03:03.538253 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:03.538232    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6f8f467c55-qdkrz"]
Jul 30 15:03:03.688525 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:03.688494    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/966f4eab-60a1-4cb8-852d-ddc0799450b5-webhook-cert\") pod \"maas-controller-6f8f467c55-qdkrz\" (UID: \"966f4eab-60a1-4cb8-852d-ddc0799450b5\") " pod="opendatahub/maas-controller-6f8f467c55-qdkrz"
Jul 30 15:03:03.688525 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:03.688529    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d462l\" (UniqueName: \"kubernetes.io/projected/966f4eab-60a1-4cb8-852d-ddc0799450b5-kube-api-access-d462l\") pod \"maas-controller-6f8f467c55-qdkrz\" (UID: \"966f4eab-60a1-4cb8-852d-ddc0799450b5\") " pod="opendatahub/maas-controller-6f8f467c55-qdkrz"
Jul 30 15:03:03.789282 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:03.789218    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/966f4eab-60a1-4cb8-852d-ddc0799450b5-webhook-cert\") pod \"maas-controller-6f8f467c55-qdkrz\" (UID: \"966f4eab-60a1-4cb8-852d-ddc0799450b5\") " pod="opendatahub/maas-controller-6f8f467c55-qdkrz"
Jul 30 15:03:03.789282 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:03.789248    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d462l\" (UniqueName: \"kubernetes.io/projected/966f4eab-60a1-4cb8-852d-ddc0799450b5-kube-api-access-d462l\") pod \"maas-controller-6f8f467c55-qdkrz\" (UID: \"966f4eab-60a1-4cb8-852d-ddc0799450b5\") " pod="opendatahub/maas-controller-6f8f467c55-qdkrz"
Jul 30 15:03:03.791355 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:03.791335    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/966f4eab-60a1-4cb8-852d-ddc0799450b5-webhook-cert\") pod \"maas-controller-6f8f467c55-qdkrz\" (UID: \"966f4eab-60a1-4cb8-852d-ddc0799450b5\") " pod="opendatahub/maas-controller-6f8f467c55-qdkrz"
Jul 30 15:03:03.798006 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:03.797986    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d462l\" (UniqueName: \"kubernetes.io/projected/966f4eab-60a1-4cb8-852d-ddc0799450b5-kube-api-access-d462l\") pod \"maas-controller-6f8f467c55-qdkrz\" (UID: \"966f4eab-60a1-4cb8-852d-ddc0799450b5\") " pod="opendatahub/maas-controller-6f8f467c55-qdkrz"
Jul 30 15:03:03.856200 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:03.856175    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f8f467c55-qdkrz"
Jul 30 15:03:03.997991 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:03.997965    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6f8f467c55-qdkrz"]
Jul 30 15:03:04.000101 ip-10-0-133-117 kubenswrapper[2578]: W0730 15:03:04.000075    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod966f4eab_60a1_4cb8_852d_ddc0799450b5.slice/crio-031f8d69b5ca657d32405dceeaff72452f0da2071c757865371d2f2e0674d0da WatchSource:0}: Error finding container 031f8d69b5ca657d32405dceeaff72452f0da2071c757865371d2f2e0674d0da: Status 404 returned error can't find the container with id 031f8d69b5ca657d32405dceeaff72452f0da2071c757865371d2f2e0674d0da
Jul 30 15:03:04.880636 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:04.880592    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f8f467c55-qdkrz" event={"ID":"966f4eab-60a1-4cb8-852d-ddc0799450b5","Type":"ContainerStarted","Data":"e926c0001e5992902074edeb5817a261d5c713ec8b014ce89b7cca136f2d6d8b"}
Jul 30 15:03:04.881007 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:04.880656    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f8f467c55-qdkrz" event={"ID":"966f4eab-60a1-4cb8-852d-ddc0799450b5","Type":"ContainerStarted","Data":"031f8d69b5ca657d32405dceeaff72452f0da2071c757865371d2f2e0674d0da"}
Jul 30 15:03:04.881007 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:04.880687    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6f8f467c55-qdkrz"
Jul 30 15:03:04.899799 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:04.899747    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6f8f467c55-qdkrz" podStartSLOduration=1.231640711 podStartE2EDuration="1.899730385s" totalImagesPullingTime="668.089674ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:03:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:03:04.899004548 +0000 UTC m=+896.427943023" watchObservedRunningTime="2026-07-30 15:03:04.899730385 +0000 UTC m=+896.428668859"
Jul 30 15:03:08.950300 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:08.950272    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4mjmp_de241220-5f27-4b0b-b7c6-6d4b350e24e1/console-operator/1.log"
Jul 30 15:03:08.950674 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:08.950325    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4mjmp_de241220-5f27-4b0b-b7c6-6d4b350e24e1/console-operator/1.log"
Jul 30 15:03:15.887301 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:15.887266    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6f8f467c55-qdkrz"
Jul 30 15:03:20.575424 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:20.575381    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6f8f467c55-qdkrz"]
Jul 30 15:03:20.575997 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:20.575659    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6f8f467c55-qdkrz" podUID="966f4eab-60a1-4cb8-852d-ddc0799450b5" containerName="manager" containerID="cri-o://e926c0001e5992902074edeb5817a261d5c713ec8b014ce89b7cca136f2d6d8b" gracePeriod=30
Jul 30 15:03:20.823959 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:20.823933    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f8f467c55-qdkrz"
Jul 30 15:03:20.846019 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:20.845939    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d462l\" (UniqueName: \"kubernetes.io/projected/966f4eab-60a1-4cb8-852d-ddc0799450b5-kube-api-access-d462l\") pod \"966f4eab-60a1-4cb8-852d-ddc0799450b5\" (UID: \"966f4eab-60a1-4cb8-852d-ddc0799450b5\") "
Jul 30 15:03:20.846190 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:20.846095    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/966f4eab-60a1-4cb8-852d-ddc0799450b5-webhook-cert\") pod \"966f4eab-60a1-4cb8-852d-ddc0799450b5\" (UID: \"966f4eab-60a1-4cb8-852d-ddc0799450b5\") "
Jul 30 15:03:20.848409 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:20.848378    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/966f4eab-60a1-4cb8-852d-ddc0799450b5-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "966f4eab-60a1-4cb8-852d-ddc0799450b5" (UID: "966f4eab-60a1-4cb8-852d-ddc0799450b5"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 15:03:20.848409 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:20.848390    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/966f4eab-60a1-4cb8-852d-ddc0799450b5-kube-api-access-d462l" (OuterVolumeSpecName: "kube-api-access-d462l") pod "966f4eab-60a1-4cb8-852d-ddc0799450b5" (UID: "966f4eab-60a1-4cb8-852d-ddc0799450b5"). InnerVolumeSpecName "kube-api-access-d462l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:03:20.947051 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:20.947008    2578 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/966f4eab-60a1-4cb8-852d-ddc0799450b5-webhook-cert\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:03:20.947051 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:20.947046    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d462l\" (UniqueName: \"kubernetes.io/projected/966f4eab-60a1-4cb8-852d-ddc0799450b5-kube-api-access-d462l\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:03:20.955052 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:20.955024    2578 generic.go:356] "Generic (PLEG): container finished" podID="966f4eab-60a1-4cb8-852d-ddc0799450b5" containerID="e926c0001e5992902074edeb5817a261d5c713ec8b014ce89b7cca136f2d6d8b" exitCode=0
Jul 30 15:03:20.955218 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:20.955067    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f8f467c55-qdkrz"
Jul 30 15:03:20.955218 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:20.955075    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f8f467c55-qdkrz" event={"ID":"966f4eab-60a1-4cb8-852d-ddc0799450b5","Type":"ContainerDied","Data":"e926c0001e5992902074edeb5817a261d5c713ec8b014ce89b7cca136f2d6d8b"}
Jul 30 15:03:20.955218 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:20.955116    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f8f467c55-qdkrz" event={"ID":"966f4eab-60a1-4cb8-852d-ddc0799450b5","Type":"ContainerDied","Data":"031f8d69b5ca657d32405dceeaff72452f0da2071c757865371d2f2e0674d0da"}
Jul 30 15:03:20.955218 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:20.955139    2578 scope.go:117] "RemoveContainer" containerID="e926c0001e5992902074edeb5817a261d5c713ec8b014ce89b7cca136f2d6d8b"
Jul 30 15:03:20.963832 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:20.963817    2578 scope.go:117] "RemoveContainer" containerID="e926c0001e5992902074edeb5817a261d5c713ec8b014ce89b7cca136f2d6d8b"
Jul 30 15:03:20.964068 ip-10-0-133-117 kubenswrapper[2578]: E0730 15:03:20.964050    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e926c0001e5992902074edeb5817a261d5c713ec8b014ce89b7cca136f2d6d8b\": container with ID starting with e926c0001e5992902074edeb5817a261d5c713ec8b014ce89b7cca136f2d6d8b not found: ID does not exist" containerID="e926c0001e5992902074edeb5817a261d5c713ec8b014ce89b7cca136f2d6d8b"
Jul 30 15:03:20.964113 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:20.964076    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e926c0001e5992902074edeb5817a261d5c713ec8b014ce89b7cca136f2d6d8b"} err="failed to get container status \"e926c0001e5992902074edeb5817a261d5c713ec8b014ce89b7cca136f2d6d8b\": rpc error: code = NotFound desc = could not find container \"e926c0001e5992902074edeb5817a261d5c713ec8b014ce89b7cca136f2d6d8b\": container with ID starting with e926c0001e5992902074edeb5817a261d5c713ec8b014ce89b7cca136f2d6d8b not found: ID does not exist"
Jul 30 15:03:20.979708 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:20.979680    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6f8f467c55-qdkrz"]
Jul 30 15:03:20.979708 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:20.979714    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6f8f467c55-qdkrz"]
Jul 30 15:03:21.799872 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:21.799833    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6f8f467c55-jln6p"]
Jul 30 15:03:21.800275 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:21.800197    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="966f4eab-60a1-4cb8-852d-ddc0799450b5" containerName="manager"
Jul 30 15:03:21.800275 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:21.800208    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="966f4eab-60a1-4cb8-852d-ddc0799450b5" containerName="manager"
Jul 30 15:03:21.800275 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:21.800267    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="966f4eab-60a1-4cb8-852d-ddc0799450b5" containerName="manager"
Jul 30 15:03:21.802930 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:21.802913    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f8f467c55-jln6p"
Jul 30 15:03:21.804825 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:21.804800    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 30 15:03:21.805213 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:21.805198    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-qdb2j\""
Jul 30 15:03:21.805299 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:21.805240    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 30 15:03:21.811902 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:21.811874    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6f8f467c55-jln6p"]
Jul 30 15:03:21.854976 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:21.854928    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5f74\" (UniqueName: \"kubernetes.io/projected/b17ea779-45c5-44d7-a8f7-f21d609224e0-kube-api-access-z5f74\") pod \"maas-controller-6f8f467c55-jln6p\" (UID: \"b17ea779-45c5-44d7-a8f7-f21d609224e0\") " pod="opendatahub/maas-controller-6f8f467c55-jln6p"
Jul 30 15:03:21.855175 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:21.855012    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b17ea779-45c5-44d7-a8f7-f21d609224e0-webhook-cert\") pod \"maas-controller-6f8f467c55-jln6p\" (UID: \"b17ea779-45c5-44d7-a8f7-f21d609224e0\") " pod="opendatahub/maas-controller-6f8f467c55-jln6p"
Jul 30 15:03:21.956111 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:21.956080    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b17ea779-45c5-44d7-a8f7-f21d609224e0-webhook-cert\") pod \"maas-controller-6f8f467c55-jln6p\" (UID: \"b17ea779-45c5-44d7-a8f7-f21d609224e0\") " pod="opendatahub/maas-controller-6f8f467c55-jln6p"
Jul 30 15:03:21.956312 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:21.956230    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5f74\" (UniqueName: \"kubernetes.io/projected/b17ea779-45c5-44d7-a8f7-f21d609224e0-kube-api-access-z5f74\") pod \"maas-controller-6f8f467c55-jln6p\" (UID: \"b17ea779-45c5-44d7-a8f7-f21d609224e0\") " pod="opendatahub/maas-controller-6f8f467c55-jln6p"
Jul 30 15:03:21.959300 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:21.959273    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b17ea779-45c5-44d7-a8f7-f21d609224e0-webhook-cert\") pod \"maas-controller-6f8f467c55-jln6p\" (UID: \"b17ea779-45c5-44d7-a8f7-f21d609224e0\") " pod="opendatahub/maas-controller-6f8f467c55-jln6p"
Jul 30 15:03:21.964356 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:21.964328    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5f74\" (UniqueName: \"kubernetes.io/projected/b17ea779-45c5-44d7-a8f7-f21d609224e0-kube-api-access-z5f74\") pod \"maas-controller-6f8f467c55-jln6p\" (UID: \"b17ea779-45c5-44d7-a8f7-f21d609224e0\") " pod="opendatahub/maas-controller-6f8f467c55-jln6p"
Jul 30 15:03:22.114081 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:22.113995    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f8f467c55-jln6p"
Jul 30 15:03:22.250639 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:22.250603    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6f8f467c55-jln6p"]
Jul 30 15:03:22.256586 ip-10-0-133-117 kubenswrapper[2578]: W0730 15:03:22.256561    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb17ea779_45c5_44d7_a8f7_f21d609224e0.slice/crio-4c5ecb8bd2fc0a4d5d232b18f74c5cf10bc56d2227468331d53ef42f41978820 WatchSource:0}: Error finding container 4c5ecb8bd2fc0a4d5d232b18f74c5cf10bc56d2227468331d53ef42f41978820: Status 404 returned error can't find the container with id 4c5ecb8bd2fc0a4d5d232b18f74c5cf10bc56d2227468331d53ef42f41978820
Jul 30 15:03:22.969270 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:22.969182    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f8f467c55-jln6p" event={"ID":"b17ea779-45c5-44d7-a8f7-f21d609224e0","Type":"ContainerStarted","Data":"adc11b4eb7501762bb69cfcb65ca11c61d920ce14c888ecbb2d8d7648e323901"}
Jul 30 15:03:22.969270 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:22.969246    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f8f467c55-jln6p" event={"ID":"b17ea779-45c5-44d7-a8f7-f21d609224e0","Type":"ContainerStarted","Data":"4c5ecb8bd2fc0a4d5d232b18f74c5cf10bc56d2227468331d53ef42f41978820"}
Jul 30 15:03:22.969672 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:22.969277    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6f8f467c55-jln6p"
Jul 30 15:03:22.982779 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:22.982740    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="966f4eab-60a1-4cb8-852d-ddc0799450b5" path="/var/lib/kubelet/pods/966f4eab-60a1-4cb8-852d-ddc0799450b5/volumes"
Jul 30 15:03:22.987003 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:22.986952    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6f8f467c55-jln6p" podStartSLOduration=1.688491639 podStartE2EDuration="1.986937476s" totalImagesPullingTime="298.445837ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:03:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:03:22.986410173 +0000 UTC m=+914.515348661" watchObservedRunningTime="2026-07-30 15:03:22.986937476 +0000 UTC m=+914.515875950"
Jul 30 15:03:33.976557 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:03:33.976525    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6f8f467c55-jln6p"
Jul 30 15:08:08.978459 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:08:08.978375    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4mjmp_de241220-5f27-4b0b-b7c6-6d4b350e24e1/console-operator/1.log"
Jul 30 15:08:08.978958 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:08:08.978904    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4mjmp_de241220-5f27-4b0b-b7c6-6d4b350e24e1/console-operator/1.log"
Jul 30 15:13:09.004721 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:13:09.004690    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4mjmp_de241220-5f27-4b0b-b7c6-6d4b350e24e1/console-operator/1.log"
Jul 30 15:13:09.006865 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:13:09.006845    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4mjmp_de241220-5f27-4b0b-b7c6-6d4b350e24e1/console-operator/1.log"
Jul 30 15:15:00.139572 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:15:00.139536    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29757075-pfxlq"]
Jul 30 15:15:00.142975 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:15:00.142959    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757075-pfxlq"
Jul 30 15:15:00.145015 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:15:00.144990    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rmv59\""
Jul 30 15:15:00.152284 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:15:00.152265    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757075-pfxlq"
Jul 30 15:15:00.159279 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:15:00.159259    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29757075-pfxlq"]
Jul 30 15:15:00.286303 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:15:00.286273    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29757075-pfxlq"]
Jul 30 15:15:00.288068 ip-10-0-133-117 kubenswrapper[2578]: W0730 15:15:00.288042    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod093ed22f_2895_4cc9_ac0a_cfdcfbd240b9.slice/crio-f62b5d0f7b1e9ca293886daf9f37cea32fda2857f1bd09c769cbf8774945cef9 WatchSource:0}: Error finding container f62b5d0f7b1e9ca293886daf9f37cea32fda2857f1bd09c769cbf8774945cef9: Status 404 returned error can't find the container with id f62b5d0f7b1e9ca293886daf9f37cea32fda2857f1bd09c769cbf8774945cef9
Jul 30 15:15:01.112554 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:15:01.112520    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757075-pfxlq" event={"ID":"093ed22f-2895-4cc9-ac0a-cfdcfbd240b9","Type":"ContainerStarted","Data":"02b1483756f158cd8990c79f87dfb888285d4c54a754392a8b7f1c702f9c1abd"}
Jul 30 15:15:01.112737 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:15:01.112571    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757075-pfxlq" event={"ID":"093ed22f-2895-4cc9-ac0a-cfdcfbd240b9","Type":"ContainerStarted","Data":"f62b5d0f7b1e9ca293886daf9f37cea32fda2857f1bd09c769cbf8774945cef9"}
Jul 30 15:15:01.128229 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:15:01.128187    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757075-pfxlq" podStartSLOduration=1.128172268 podStartE2EDuration="1.128172268s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:15:01.126319222 +0000 UTC m=+1612.655257707" watchObservedRunningTime="2026-07-30 15:15:01.128172268 +0000 UTC m=+1612.657110742"
Jul 30 15:15:02.118122 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:15:02.118090    2578 generic.go:356] "Generic (PLEG): container finished" podID="093ed22f-2895-4cc9-ac0a-cfdcfbd240b9" containerID="02b1483756f158cd8990c79f87dfb888285d4c54a754392a8b7f1c702f9c1abd" exitCode=0
Jul 30 15:15:02.118504 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:15:02.118166    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757075-pfxlq" event={"ID":"093ed22f-2895-4cc9-ac0a-cfdcfbd240b9","Type":"ContainerDied","Data":"02b1483756f158cd8990c79f87dfb888285d4c54a754392a8b7f1c702f9c1abd"}
Jul 30 15:15:03.249001 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:15:03.248973    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757075-pfxlq"
Jul 30 15:15:04.127857 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:15:04.127834    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757075-pfxlq"
Jul 30 15:15:04.128021 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:15:04.127851    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757075-pfxlq" event={"ID":"093ed22f-2895-4cc9-ac0a-cfdcfbd240b9","Type":"ContainerDied","Data":"f62b5d0f7b1e9ca293886daf9f37cea32fda2857f1bd09c769cbf8774945cef9"}
Jul 30 15:15:04.128021 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:15:04.127897    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f62b5d0f7b1e9ca293886daf9f37cea32fda2857f1bd09c769cbf8774945cef9"
Jul 30 15:15:27.030695 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:15:27.030665    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-6405f46e-kserve-c9dcff99-6d72q"]
Jul 30 15:15:27.031100 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:15:27.031018    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="093ed22f-2895-4cc9-ac0a-cfdcfbd240b9" containerName="cleanup"
Jul 30 15:15:27.031100 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:15:27.031029    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="093ed22f-2895-4cc9-ac0a-cfdcfbd240b9" containerName="cleanup"
Jul 30 15:15:27.031100 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:15:27.031091    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="093ed22f-2895-4cc9-ac0a-cfdcfbd240b9" containerName="cleanup"
Jul 30 15:15:27.034264 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:15:27.034244    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-6405f46e-kserve-c9dcff99-6d72q"
Jul 30 15:15:27.036040 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:15:27.036019    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-6405f46e-kserve-self-signed-certs\""
Jul 30 15:15:27.044820 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:15:27.044794    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-6405f46e-kserve-c9dcff99-6d72q"]
Jul 30 15:15:27.171352 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:15:27.171319    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45m86\" (UniqueName: \"kubernetes.io/projected/84bb3bd4-6b79-4e28-be01-ebcba2551453-kube-api-access-45m86\") pod \"e2e-same-id-a-6405f46e-kserve-c9dcff99-6d72q\" (UID: \"84bb3bd4-6b79-4e28-be01-ebcba2551453\") " pod="llm/e2e-same-id-a-6405f46e-kserve-c9dcff99-6d72q"
Jul 30 15:15:27.171352 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:15:27.171357    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84bb3bd4-6b79-4e28-be01-ebcba2551453-home\") pod \"e2e-same-id-a-6405f46e-kserve-c9dcff99-6d72q\" (UID: \"84bb3bd4-6b79-4e28-be01-ebcba2551453\") " pod="llm/e2e-same-id-a-6405f46e-kserve-c9dcff99-6d72q"
Jul 30 15:15:27.171580 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:15:27.171373    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84bb3bd4-6b79-4e28-be01-ebcba2551453-tmp-dir\") pod \"e2e-same-id-a-6405f46e-kserve-c9dcff99-6d72q\" (UID: \"84bb3bd4-6b79-4e28-be01-ebcba2551453\") " pod="llm/e2e-same-id-a-6405f46e-kserve-c9dcff99-6d72q"
Jul 30 15:15:27.171580 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:15:27.171403    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84bb3bd4-6b79-4e28-be01-ebcba2551453-dshm\") pod \"e2e-same-id-a-6405f46e-kserve-c9dcff99-6d72q\" (UID: \"84bb3bd4-6b79-4e28-be01-ebcba2551453\") " pod="llm/e2e-same-id-a-6405f46e-kserve-c9dcff99-6d72q"
Jul 30 15:15:27.171580 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:15:27.171450    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84bb3bd4-6b79-4e28-be01-ebcba2551453-model-cache\") pod \"e2e-same-id-a-6405f46e-kserve-c9dcff99-6d72q\" (UID: \"84bb3bd4-6b79-4e28-be01-ebcba2551453\") " pod="llm/e2e-same-id-a-6405f46e-kserve-c9dcff99-6d72q"
Jul 30 15:15:27.171580 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:15:27.171482    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84bb3bd4-6b79-4e28-be01-ebcba2551453-tls-certs\") pod \"e2e-same-id-a-6405f46e-kserve-c9dcff99-6d72q\" (UID: \"84bb3bd4-6b79-4e28-be01-ebcba2551453\") " pod="llm/e2e-same-id-a-6405f46e-kserve-c9dcff99-6d72q"
Jul 30 15:15:27.272648 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:15:27.272616    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45m86\" (UniqueName: \"kubernetes.io/projected/84bb3bd4-6b79-4e28-be01-ebcba2551453-kube-api-access-45m86\") pod \"e2e-same-id-a-6405f46e-kserve-c9dcff99-6d72q\" (UID: \"84bb3bd4-6b79-4e28-be01-ebcba2551453\") " pod="llm/e2e-same-id-a-6405f46e-kserve-c9dcff99-6d72q"
Jul 30 15:15:27.272648 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:15:27.272653    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84bb3bd4-6b79-4e28-be01-ebcba2551453-home\") pod \"e2e-same-id-a-6405f46e-kserve-c9dcff99-6d72q\" (UID: \"84bb3bd4-6b79-4e28-be01-ebcba2551453\") " pod="llm/e2e-same-id-a-6405f46e-kserve-c9dcff99-6d72q"
Jul 30 15:15:27.272878 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:15:27.272671    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84bb3bd4-6b79-4e28-be01-ebcba2551453-tmp-dir\") pod \"e2e-same-id-a-6405f46e-kserve-c9dcff99-6d72q\" (UID: \"84bb3bd4-6b79-4e28-be01-ebcba2551453\") " pod="llm/e2e-same-id-a-6405f46e-kserve-c9dcff99-6d72q"
Jul 30 15:15:27.272878 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:15:27.272703    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84bb3bd4-6b79-4e28-be01-ebcba2551453-dshm\") pod \"e2e-same-id-a-6405f46e-kserve-c9dcff99-6d72q\" (UID: \"84bb3bd4-6b79-4e28-be01-ebcba2551453\") " pod="llm/e2e-same-id-a-6405f46e-kserve-c9dcff99-6d72q"
Jul 30 15:15:27.272878 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:15:27.272726    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84bb3bd4-6b79-4e28-be01-ebcba2551453-model-cache\") pod \"e2e-same-id-a-6405f46e-kserve-c9dcff99-6d72q\" (UID: \"84bb3bd4-6b79-4e28-be01-ebcba2551453\") " pod="llm/e2e-same-id-a-6405f46e-kserve-c9dcff99-6d72q"
Jul 30 15:15:27.272878 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:15:27.272752    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84bb3bd4-6b79-4e28-be01-ebcba2551453-tls-certs\") pod \"e2e-same-id-a-6405f46e-kserve-c9dcff99-6d72q\" (UID: \"84bb3bd4-6b79-4e28-be01-ebcba2551453\") " pod="llm/e2e-same-id-a-6405f46e-kserve-c9dcff99-6d72q"
Jul 30 15:15:27.273128 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:15:27.273101    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84bb3bd4-6b79-4e28-be01-ebcba2551453-home\") pod \"e2e-same-id-a-6405f46e-kserve-c9dcff99-6d72q\" (UID: \"84bb3bd4-6b79-4e28-be01-ebcba2551453\") " pod="llm/e2e-same-id-a-6405f46e-kserve-c9dcff99-6d72q"
Jul 30 15:15:27.273226 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:15:27.273192    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84bb3bd4-6b79-4e28-be01-ebcba2551453-tmp-dir\") pod \"e2e-same-id-a-6405f46e-kserve-c9dcff99-6d72q\" (UID: \"84bb3bd4-6b79-4e28-be01-ebcba2551453\") " pod="llm/e2e-same-id-a-6405f46e-kserve-c9dcff99-6d72q"
Jul 30 15:15:27.273226 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:15:27.273208    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84bb3bd4-6b79-4e28-be01-ebcba2551453-model-cache\") pod \"e2e-same-id-a-6405f46e-kserve-c9dcff99-6d72q\" (UID: \"84bb3bd4-6b79-4e28-be01-ebcba2551453\") " pod="llm/e2e-same-id-a-6405f46e-kserve-c9dcff99-6d72q"
Jul 30 15:15:27.274962 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:15:27.274940    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84bb3bd4-6b79-4e28-be01-ebcba2551453-dshm\") pod \"e2e-same-id-a-6405f46e-kserve-c9dcff99-6d72q\" (UID: \"84bb3bd4-6b79-4e28-be01-ebcba2551453\") " pod="llm/e2e-same-id-a-6405f46e-kserve-c9dcff99-6d72q"
Jul 30 15:15:27.275236 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:15:27.275219    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84bb3bd4-6b79-4e28-be01-ebcba2551453-tls-certs\") pod \"e2e-same-id-a-6405f46e-kserve-c9dcff99-6d72q\" (UID: \"84bb3bd4-6b79-4e28-be01-ebcba2551453\") " pod="llm/e2e-same-id-a-6405f46e-kserve-c9dcff99-6d72q"
Jul 30 15:15:27.279647 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:15:27.279625    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45m86\" (UniqueName: \"kubernetes.io/projected/84bb3bd4-6b79-4e28-be01-ebcba2551453-kube-api-access-45m86\") pod \"e2e-same-id-a-6405f46e-kserve-c9dcff99-6d72q\" (UID: \"84bb3bd4-6b79-4e28-be01-ebcba2551453\") " pod="llm/e2e-same-id-a-6405f46e-kserve-c9dcff99-6d72q"
Jul 30 15:15:27.346228 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:15:27.346202    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-6405f46e-kserve-c9dcff99-6d72q"
Jul 30 15:15:27.474279 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:15:27.474253    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-6405f46e-kserve-c9dcff99-6d72q"]
Jul 30 15:15:27.476443 ip-10-0-133-117 kubenswrapper[2578]: W0730 15:15:27.476412    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod84bb3bd4_6b79_4e28_be01_ebcba2551453.slice/crio-73809f0a8d9ff2cfd3c85731c5dcbc7296d8aa47f8a552eb69506c98cf34d5ad WatchSource:0}: Error finding container 73809f0a8d9ff2cfd3c85731c5dcbc7296d8aa47f8a552eb69506c98cf34d5ad: Status 404 returned error can't find the container with id 73809f0a8d9ff2cfd3c85731c5dcbc7296d8aa47f8a552eb69506c98cf34d5ad
Jul 30 15:15:28.242194 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:15:28.242160    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-6405f46e-kserve-c9dcff99-6d72q" event={"ID":"84bb3bd4-6b79-4e28-be01-ebcba2551453","Type":"ContainerStarted","Data":"1d6f861accef0d285442dedd6737db57b4ee844239e9e7d17ee082033797a1aa"}
Jul 30 15:15:28.242556 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:15:28.242203    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-6405f46e-kserve-c9dcff99-6d72q" event={"ID":"84bb3bd4-6b79-4e28-be01-ebcba2551453","Type":"ContainerStarted","Data":"73809f0a8d9ff2cfd3c85731c5dcbc7296d8aa47f8a552eb69506c98cf34d5ad"}
Jul 30 15:15:28.258451 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:15:28.258402    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-6405f46e-kserve-c9dcff99-6d72q" podStartSLOduration=1.258384707 podStartE2EDuration="1.258384707s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:15:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:15:28.256785286 +0000 UTC m=+1639.785723760" watchObservedRunningTime="2026-07-30 15:15:28.258384707 +0000 UTC m=+1639.787323178"
Jul 30 15:15:37.346908 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:15:37.346875    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-6405f46e-kserve-c9dcff99-6d72q"
Jul 30 15:15:37.347395 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:15:37.347175    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-6405f46e-kserve-c9dcff99-6d72q"
Jul 30 15:15:37.359636 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:15:37.359601    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-6405f46e-kserve-c9dcff99-6d72q"
Jul 30 15:15:38.301886 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:15:38.301860    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-6405f46e-kserve-c9dcff99-6d72q"
Jul 30 15:16:14.579564 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:16:14.579533    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-6405f46e-kserve-c9dcff99-6d72q"]
Jul 30 15:16:29.598963 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:16:29.598909    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-6405f46e-kserve-c9dcff99-6d72q" podUID="84bb3bd4-6b79-4e28-be01-ebcba2551453" containerName="main" containerID="cri-o://1d6f861accef0d285442dedd6737db57b4ee844239e9e7d17ee082033797a1aa" gracePeriod=45
Jul 30 15:16:29.836234 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:16:29.836207    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-6405f46e-kserve-c9dcff99-6d72q"
Jul 30 15:16:29.953928 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:16:29.953853    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84bb3bd4-6b79-4e28-be01-ebcba2551453-tmp-dir\") pod \"84bb3bd4-6b79-4e28-be01-ebcba2551453\" (UID: \"84bb3bd4-6b79-4e28-be01-ebcba2551453\") "
Jul 30 15:16:29.953928 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:16:29.953915    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84bb3bd4-6b79-4e28-be01-ebcba2551453-home\") pod \"84bb3bd4-6b79-4e28-be01-ebcba2551453\" (UID: \"84bb3bd4-6b79-4e28-be01-ebcba2551453\") "
Jul 30 15:16:29.954176 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:16:29.953936    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84bb3bd4-6b79-4e28-be01-ebcba2551453-tls-certs\") pod \"84bb3bd4-6b79-4e28-be01-ebcba2551453\" (UID: \"84bb3bd4-6b79-4e28-be01-ebcba2551453\") "
Jul 30 15:16:29.954176 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:16:29.953972    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84bb3bd4-6b79-4e28-be01-ebcba2551453-dshm\") pod \"84bb3bd4-6b79-4e28-be01-ebcba2551453\" (UID: \"84bb3bd4-6b79-4e28-be01-ebcba2551453\") "
Jul 30 15:16:29.954176 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:16:29.953999    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-45m86\" (UniqueName: \"kubernetes.io/projected/84bb3bd4-6b79-4e28-be01-ebcba2551453-kube-api-access-45m86\") pod \"84bb3bd4-6b79-4e28-be01-ebcba2551453\" (UID: \"84bb3bd4-6b79-4e28-be01-ebcba2551453\") "
Jul 30 15:16:29.954176 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:16:29.954023    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84bb3bd4-6b79-4e28-be01-ebcba2551453-model-cache\") pod \"84bb3bd4-6b79-4e28-be01-ebcba2551453\" (UID: \"84bb3bd4-6b79-4e28-be01-ebcba2551453\") "
Jul 30 15:16:29.954176 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:16:29.954103    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84bb3bd4-6b79-4e28-be01-ebcba2551453-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "84bb3bd4-6b79-4e28-be01-ebcba2551453" (UID: "84bb3bd4-6b79-4e28-be01-ebcba2551453"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:16:29.954433 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:16:29.954245    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84bb3bd4-6b79-4e28-be01-ebcba2551453-home" (OuterVolumeSpecName: "home") pod "84bb3bd4-6b79-4e28-be01-ebcba2551453" (UID: "84bb3bd4-6b79-4e28-be01-ebcba2551453"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:16:29.954433 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:16:29.954332    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84bb3bd4-6b79-4e28-be01-ebcba2551453-model-cache" (OuterVolumeSpecName: "model-cache") pod "84bb3bd4-6b79-4e28-be01-ebcba2551453" (UID: "84bb3bd4-6b79-4e28-be01-ebcba2551453"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:16:29.954433 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:16:29.954396    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84bb3bd4-6b79-4e28-be01-ebcba2551453-tmp-dir\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:16:29.954433 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:16:29.954417    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84bb3bd4-6b79-4e28-be01-ebcba2551453-home\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:16:29.956232 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:16:29.956209    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84bb3bd4-6b79-4e28-be01-ebcba2551453-dshm" (OuterVolumeSpecName: "dshm") pod "84bb3bd4-6b79-4e28-be01-ebcba2551453" (UID: "84bb3bd4-6b79-4e28-be01-ebcba2551453"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:16:29.956303 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:16:29.956263    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84bb3bd4-6b79-4e28-be01-ebcba2551453-kube-api-access-45m86" (OuterVolumeSpecName: "kube-api-access-45m86") pod "84bb3bd4-6b79-4e28-be01-ebcba2551453" (UID: "84bb3bd4-6b79-4e28-be01-ebcba2551453"). InnerVolumeSpecName "kube-api-access-45m86". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:16:29.956346 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:16:29.956330    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84bb3bd4-6b79-4e28-be01-ebcba2551453-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "84bb3bd4-6b79-4e28-be01-ebcba2551453" (UID: "84bb3bd4-6b79-4e28-be01-ebcba2551453"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 15:16:30.055391 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:16:30.055362    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84bb3bd4-6b79-4e28-be01-ebcba2551453-tls-certs\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:16:30.055391 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:16:30.055389    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84bb3bd4-6b79-4e28-be01-ebcba2551453-dshm\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:16:30.055567 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:16:30.055399    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-45m86\" (UniqueName: \"kubernetes.io/projected/84bb3bd4-6b79-4e28-be01-ebcba2551453-kube-api-access-45m86\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:16:30.055567 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:16:30.055408    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84bb3bd4-6b79-4e28-be01-ebcba2551453-model-cache\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:16:30.543189 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:16:30.543141    2578 generic.go:356] "Generic (PLEG): container finished" podID="84bb3bd4-6b79-4e28-be01-ebcba2551453" containerID="1d6f861accef0d285442dedd6737db57b4ee844239e9e7d17ee082033797a1aa" exitCode=0
Jul 30 15:16:30.543358 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:16:30.543200    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-6405f46e-kserve-c9dcff99-6d72q"
Jul 30 15:16:30.543358 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:16:30.543245    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-6405f46e-kserve-c9dcff99-6d72q" event={"ID":"84bb3bd4-6b79-4e28-be01-ebcba2551453","Type":"ContainerDied","Data":"1d6f861accef0d285442dedd6737db57b4ee844239e9e7d17ee082033797a1aa"}
Jul 30 15:16:30.543358 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:16:30.543281    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-6405f46e-kserve-c9dcff99-6d72q" event={"ID":"84bb3bd4-6b79-4e28-be01-ebcba2551453","Type":"ContainerDied","Data":"73809f0a8d9ff2cfd3c85731c5dcbc7296d8aa47f8a552eb69506c98cf34d5ad"}
Jul 30 15:16:30.543358 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:16:30.543299    2578 scope.go:117] "RemoveContainer" containerID="1d6f861accef0d285442dedd6737db57b4ee844239e9e7d17ee082033797a1aa"
Jul 30 15:16:30.552946 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:16:30.552930    2578 scope.go:117] "RemoveContainer" containerID="1d6f861accef0d285442dedd6737db57b4ee844239e9e7d17ee082033797a1aa"
Jul 30 15:16:30.553211 ip-10-0-133-117 kubenswrapper[2578]: E0730 15:16:30.553193    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d6f861accef0d285442dedd6737db57b4ee844239e9e7d17ee082033797a1aa\": container with ID starting with 1d6f861accef0d285442dedd6737db57b4ee844239e9e7d17ee082033797a1aa not found: ID does not exist" containerID="1d6f861accef0d285442dedd6737db57b4ee844239e9e7d17ee082033797a1aa"
Jul 30 15:16:30.553302 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:16:30.553217    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d6f861accef0d285442dedd6737db57b4ee844239e9e7d17ee082033797a1aa"} err="failed to get container status \"1d6f861accef0d285442dedd6737db57b4ee844239e9e7d17ee082033797a1aa\": rpc error: code = NotFound desc = could not find container \"1d6f861accef0d285442dedd6737db57b4ee844239e9e7d17ee082033797a1aa\": container with ID starting with 1d6f861accef0d285442dedd6737db57b4ee844239e9e7d17ee082033797a1aa not found: ID does not exist"
Jul 30 15:16:30.565234 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:16:30.565212    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-6405f46e-kserve-c9dcff99-6d72q"]
Jul 30 15:16:30.569288 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:16:30.569264    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-6405f46e-kserve-c9dcff99-6d72q"]
Jul 30 15:16:30.979197 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:16:30.979137    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84bb3bd4-6b79-4e28-be01-ebcba2551453" path="/var/lib/kubelet/pods/84bb3bd4-6b79-4e28-be01-ebcba2551453/volumes"
Jul 30 15:18:09.030940 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:18:09.030912    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4mjmp_de241220-5f27-4b0b-b7c6-6d4b350e24e1/console-operator/1.log"
Jul 30 15:18:09.035374 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:18:09.035346    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4mjmp_de241220-5f27-4b0b-b7c6-6d4b350e24e1/console-operator/1.log"
Jul 30 15:21:02.142414 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:02.142320    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj"]
Jul 30 15:21:02.142984 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:02.142882    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84bb3bd4-6b79-4e28-be01-ebcba2551453" containerName="main"
Jul 30 15:21:02.142984 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:02.142902    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="84bb3bd4-6b79-4e28-be01-ebcba2551453" containerName="main"
Jul 30 15:21:02.143095 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:02.143015    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="84bb3bd4-6b79-4e28-be01-ebcba2551453" containerName="main"
Jul 30 15:21:02.156312 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:02.156279    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj"
Jul 30 15:21:02.158235 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:02.158208    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-624a770e-openshift-default-dockercfg-55xhc\""
Jul 30 15:21:02.158386 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:02.158340    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 30 15:21:02.171617 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:02.171591    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj"]
Jul 30 15:21:02.228984 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:02.228955    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ebe26db-241d-404b-bae1-aa7ef5c7bc74-istio-podinfo\") pod \"e2e-ait-624a770e-openshift-default-6cf65b696-zlghj\" (UID: \"8ebe26db-241d-404b-bae1-aa7ef5c7bc74\") " pod="openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj"
Jul 30 15:21:02.229134 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:02.228988    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ebe26db-241d-404b-bae1-aa7ef5c7bc74-workload-certs\") pod \"e2e-ait-624a770e-openshift-default-6cf65b696-zlghj\" (UID: \"8ebe26db-241d-404b-bae1-aa7ef5c7bc74\") " pod="openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj"
Jul 30 15:21:02.229134 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:02.229023    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wzxh\" (UniqueName: \"kubernetes.io/projected/8ebe26db-241d-404b-bae1-aa7ef5c7bc74-kube-api-access-5wzxh\") pod \"e2e-ait-624a770e-openshift-default-6cf65b696-zlghj\" (UID: \"8ebe26db-241d-404b-bae1-aa7ef5c7bc74\") " pod="openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj"
Jul 30 15:21:02.229134 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:02.229041    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ebe26db-241d-404b-bae1-aa7ef5c7bc74-workload-socket\") pod \"e2e-ait-624a770e-openshift-default-6cf65b696-zlghj\" (UID: \"8ebe26db-241d-404b-bae1-aa7ef5c7bc74\") " pod="openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj"
Jul 30 15:21:02.229134 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:02.229092    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ebe26db-241d-404b-bae1-aa7ef5c7bc74-istio-token\") pod \"e2e-ait-624a770e-openshift-default-6cf65b696-zlghj\" (UID: \"8ebe26db-241d-404b-bae1-aa7ef5c7bc74\") " pod="openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj"
Jul 30 15:21:02.229134 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:02.229110    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ebe26db-241d-404b-bae1-aa7ef5c7bc74-istio-envoy\") pod \"e2e-ait-624a770e-openshift-default-6cf65b696-zlghj\" (UID: \"8ebe26db-241d-404b-bae1-aa7ef5c7bc74\") " pod="openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj"
Jul 30 15:21:02.229313 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:02.229183    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ebe26db-241d-404b-bae1-aa7ef5c7bc74-istiod-ca-cert\") pod \"e2e-ait-624a770e-openshift-default-6cf65b696-zlghj\" (UID: \"8ebe26db-241d-404b-bae1-aa7ef5c7bc74\") " pod="openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj"
Jul 30 15:21:02.229313 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:02.229205    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ebe26db-241d-404b-bae1-aa7ef5c7bc74-credential-socket\") pod \"e2e-ait-624a770e-openshift-default-6cf65b696-zlghj\" (UID: \"8ebe26db-241d-404b-bae1-aa7ef5c7bc74\") " pod="openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj"
Jul 30 15:21:02.229313 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:02.229221    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ebe26db-241d-404b-bae1-aa7ef5c7bc74-istio-data\") pod \"e2e-ait-624a770e-openshift-default-6cf65b696-zlghj\" (UID: \"8ebe26db-241d-404b-bae1-aa7ef5c7bc74\") " pod="openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj"
Jul 30 15:21:02.330096 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:02.330067    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ebe26db-241d-404b-bae1-aa7ef5c7bc74-istio-podinfo\") pod \"e2e-ait-624a770e-openshift-default-6cf65b696-zlghj\" (UID: \"8ebe26db-241d-404b-bae1-aa7ef5c7bc74\") " pod="openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj"
Jul 30 15:21:02.330096 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:02.330100    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ebe26db-241d-404b-bae1-aa7ef5c7bc74-workload-certs\") pod \"e2e-ait-624a770e-openshift-default-6cf65b696-zlghj\" (UID: \"8ebe26db-241d-404b-bae1-aa7ef5c7bc74\") " pod="openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj"
Jul 30 15:21:02.330657 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:02.330127    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wzxh\" (UniqueName: \"kubernetes.io/projected/8ebe26db-241d-404b-bae1-aa7ef5c7bc74-kube-api-access-5wzxh\") pod \"e2e-ait-624a770e-openshift-default-6cf65b696-zlghj\" (UID: \"8ebe26db-241d-404b-bae1-aa7ef5c7bc74\") " pod="openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj"
Jul 30 15:21:02.330657 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:02.330177    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ebe26db-241d-404b-bae1-aa7ef5c7bc74-workload-socket\") pod \"e2e-ait-624a770e-openshift-default-6cf65b696-zlghj\" (UID: \"8ebe26db-241d-404b-bae1-aa7ef5c7bc74\") " pod="openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj"
Jul 30 15:21:02.330657 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:02.330224    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ebe26db-241d-404b-bae1-aa7ef5c7bc74-istio-token\") pod \"e2e-ait-624a770e-openshift-default-6cf65b696-zlghj\" (UID: \"8ebe26db-241d-404b-bae1-aa7ef5c7bc74\") " pod="openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj"
Jul 30 15:21:02.330657 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:02.330253    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ebe26db-241d-404b-bae1-aa7ef5c7bc74-istio-envoy\") pod \"e2e-ait-624a770e-openshift-default-6cf65b696-zlghj\" (UID: \"8ebe26db-241d-404b-bae1-aa7ef5c7bc74\") " pod="openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj"
Jul 30 15:21:02.330657 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:02.330304    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ebe26db-241d-404b-bae1-aa7ef5c7bc74-istiod-ca-cert\") pod \"e2e-ait-624a770e-openshift-default-6cf65b696-zlghj\" (UID: \"8ebe26db-241d-404b-bae1-aa7ef5c7bc74\") " pod="openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj"
Jul 30 15:21:02.330657 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:02.330340    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ebe26db-241d-404b-bae1-aa7ef5c7bc74-credential-socket\") pod \"e2e-ait-624a770e-openshift-default-6cf65b696-zlghj\" (UID: \"8ebe26db-241d-404b-bae1-aa7ef5c7bc74\") " pod="openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj"
Jul 30 15:21:02.330657 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:02.330369    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ebe26db-241d-404b-bae1-aa7ef5c7bc74-istio-data\") pod \"e2e-ait-624a770e-openshift-default-6cf65b696-zlghj\" (UID: \"8ebe26db-241d-404b-bae1-aa7ef5c7bc74\") " pod="openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj"
Jul 30 15:21:02.331073 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:02.330700    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ebe26db-241d-404b-bae1-aa7ef5c7bc74-workload-certs\") pod \"e2e-ait-624a770e-openshift-default-6cf65b696-zlghj\" (UID: \"8ebe26db-241d-404b-bae1-aa7ef5c7bc74\") " pod="openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj"
Jul 30 15:21:02.331073 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:02.330752    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ebe26db-241d-404b-bae1-aa7ef5c7bc74-workload-socket\") pod \"e2e-ait-624a770e-openshift-default-6cf65b696-zlghj\" (UID: \"8ebe26db-241d-404b-bae1-aa7ef5c7bc74\") " pod="openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj"
Jul 30 15:21:02.331073 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:02.330834    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ebe26db-241d-404b-bae1-aa7ef5c7bc74-istio-data\") pod \"e2e-ait-624a770e-openshift-default-6cf65b696-zlghj\" (UID: \"8ebe26db-241d-404b-bae1-aa7ef5c7bc74\") " pod="openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj"
Jul 30 15:21:02.331073 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:02.330872    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ebe26db-241d-404b-bae1-aa7ef5c7bc74-credential-socket\") pod \"e2e-ait-624a770e-openshift-default-6cf65b696-zlghj\" (UID: \"8ebe26db-241d-404b-bae1-aa7ef5c7bc74\") " pod="openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj"
Jul 30 15:21:02.331275 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:02.331142    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ebe26db-241d-404b-bae1-aa7ef5c7bc74-istiod-ca-cert\") pod \"e2e-ait-624a770e-openshift-default-6cf65b696-zlghj\" (UID: \"8ebe26db-241d-404b-bae1-aa7ef5c7bc74\") " pod="openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj"
Jul 30 15:21:02.332534 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:02.332515    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ebe26db-241d-404b-bae1-aa7ef5c7bc74-istio-envoy\") pod \"e2e-ait-624a770e-openshift-default-6cf65b696-zlghj\" (UID: \"8ebe26db-241d-404b-bae1-aa7ef5c7bc74\") " pod="openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj"
Jul 30 15:21:02.332663 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:02.332645    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ebe26db-241d-404b-bae1-aa7ef5c7bc74-istio-podinfo\") pod \"e2e-ait-624a770e-openshift-default-6cf65b696-zlghj\" (UID: \"8ebe26db-241d-404b-bae1-aa7ef5c7bc74\") " pod="openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj"
Jul 30 15:21:02.338552 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:02.338528    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wzxh\" (UniqueName: \"kubernetes.io/projected/8ebe26db-241d-404b-bae1-aa7ef5c7bc74-kube-api-access-5wzxh\") pod \"e2e-ait-624a770e-openshift-default-6cf65b696-zlghj\" (UID: \"8ebe26db-241d-404b-bae1-aa7ef5c7bc74\") " pod="openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj"
Jul 30 15:21:02.338743 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:02.338725    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ebe26db-241d-404b-bae1-aa7ef5c7bc74-istio-token\") pod \"e2e-ait-624a770e-openshift-default-6cf65b696-zlghj\" (UID: \"8ebe26db-241d-404b-bae1-aa7ef5c7bc74\") " pod="openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj"
Jul 30 15:21:02.467990 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:02.467923    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj"
Jul 30 15:21:02.602964 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:02.602941    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj"]
Jul 30 15:21:02.604709 ip-10-0-133-117 kubenswrapper[2578]: W0730 15:21:02.604681    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ebe26db_241d_404b_bae1_aa7ef5c7bc74.slice/crio-f2236d4bd9b4c0e619f364e2fcbe641c162c884995b1641065ed0ebbefe1935c WatchSource:0}: Error finding container f2236d4bd9b4c0e619f364e2fcbe641c162c884995b1641065ed0ebbefe1935c: Status 404 returned error can't find the container with id f2236d4bd9b4c0e619f364e2fcbe641c162c884995b1641065ed0ebbefe1935c
Jul 30 15:21:02.607142 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:02.607125    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 15:21:02.785240 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:02.785165    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj" event={"ID":"8ebe26db-241d-404b-bae1-aa7ef5c7bc74","Type":"ContainerStarted","Data":"f2236d4bd9b4c0e619f364e2fcbe641c162c884995b1641065ed0ebbefe1935c"}
Jul 30 15:21:05.348179 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:05.348125    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:21:05.348538 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:05.348234    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:21:05.348538 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:05.348283    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:21:05.803138 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:05.803101    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj" event={"ID":"8ebe26db-241d-404b-bae1-aa7ef5c7bc74","Type":"ContainerStarted","Data":"0b6d1efc41a9c5cbafb78c040f8476d2c3cd10483f14b077f80af360fe377aef"}
Jul 30 15:21:05.823387 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:05.823335    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj" podStartSLOduration=1.082748636 podStartE2EDuration="3.823320416s" totalImagesPullingTime="2.74057178s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:21:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:21:05.821570003 +0000 UTC m=+1977.350508474" watchObservedRunningTime="2026-07-30 15:21:05.823320416 +0000 UTC m=+1977.352258889"
Jul 30 15:21:06.468652 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:06.468616    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj"
Jul 30 15:21:06.473446 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:06.473423    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj"
Jul 30 15:21:06.809255 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:06.809226    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj"
Jul 30 15:21:06.813476 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:06.813451    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj"
Jul 30 15:21:08.392413 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:08.392378    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-624a770e-6598f56556-4dd6v"]
Jul 30 15:21:08.396498 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:08.396474    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-624a770e-6598f56556-4dd6v"
Jul 30 15:21:08.398317 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:08.398294    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-624a770e-serving-cert\""
Jul 30 15:21:08.404097 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:08.404073    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-624a770e-6598f56556-4dd6v"]
Jul 30 15:21:08.489345 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:08.489307    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98dd53d8-2dbe-4b73-b442-165dd7c0c30f-openshift-service-ca\") pod \"maas-api-e2e-ait-624a770e-6598f56556-4dd6v\" (UID: \"98dd53d8-2dbe-4b73-b442-165dd7c0c30f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-624a770e-6598f56556-4dd6v"
Jul 30 15:21:08.489529 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:08.489467    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bm7q\" (UniqueName: \"kubernetes.io/projected/98dd53d8-2dbe-4b73-b442-165dd7c0c30f-kube-api-access-8bm7q\") pod \"maas-api-e2e-ait-624a770e-6598f56556-4dd6v\" (UID: \"98dd53d8-2dbe-4b73-b442-165dd7c0c30f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-624a770e-6598f56556-4dd6v"
Jul 30 15:21:08.489529 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:08.489497    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98dd53d8-2dbe-4b73-b442-165dd7c0c30f-maas-api-tls\") pod \"maas-api-e2e-ait-624a770e-6598f56556-4dd6v\" (UID: \"98dd53d8-2dbe-4b73-b442-165dd7c0c30f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-624a770e-6598f56556-4dd6v"
Jul 30 15:21:08.590355 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:08.590305    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bm7q\" (UniqueName: \"kubernetes.io/projected/98dd53d8-2dbe-4b73-b442-165dd7c0c30f-kube-api-access-8bm7q\") pod \"maas-api-e2e-ait-624a770e-6598f56556-4dd6v\" (UID: \"98dd53d8-2dbe-4b73-b442-165dd7c0c30f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-624a770e-6598f56556-4dd6v"
Jul 30 15:21:08.590355 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:08.590357    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98dd53d8-2dbe-4b73-b442-165dd7c0c30f-maas-api-tls\") pod \"maas-api-e2e-ait-624a770e-6598f56556-4dd6v\" (UID: \"98dd53d8-2dbe-4b73-b442-165dd7c0c30f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-624a770e-6598f56556-4dd6v"
Jul 30 15:21:08.590612 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:08.590400    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98dd53d8-2dbe-4b73-b442-165dd7c0c30f-openshift-service-ca\") pod \"maas-api-e2e-ait-624a770e-6598f56556-4dd6v\" (UID: \"98dd53d8-2dbe-4b73-b442-165dd7c0c30f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-624a770e-6598f56556-4dd6v"
Jul 30 15:21:08.590612 ip-10-0-133-117 kubenswrapper[2578]: E0730 15:21:08.590540    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-624a770e-serving-cert: secret "maas-api-e2e-ait-624a770e-serving-cert" not found
Jul 30 15:21:08.590720 ip-10-0-133-117 kubenswrapper[2578]: E0730 15:21:08.590614    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98dd53d8-2dbe-4b73-b442-165dd7c0c30f-maas-api-tls podName:98dd53d8-2dbe-4b73-b442-165dd7c0c30f nodeName:}" failed. No retries permitted until 2026-07-30 15:21:09.090593072 +0000 UTC m=+1980.619531538 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/98dd53d8-2dbe-4b73-b442-165dd7c0c30f-maas-api-tls") pod "maas-api-e2e-ait-624a770e-6598f56556-4dd6v" (UID: "98dd53d8-2dbe-4b73-b442-165dd7c0c30f") : secret "maas-api-e2e-ait-624a770e-serving-cert" not found
Jul 30 15:21:08.591098 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:08.591081    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98dd53d8-2dbe-4b73-b442-165dd7c0c30f-openshift-service-ca\") pod \"maas-api-e2e-ait-624a770e-6598f56556-4dd6v\" (UID: \"98dd53d8-2dbe-4b73-b442-165dd7c0c30f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-624a770e-6598f56556-4dd6v"
Jul 30 15:21:08.599829 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:08.599803    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bm7q\" (UniqueName: \"kubernetes.io/projected/98dd53d8-2dbe-4b73-b442-165dd7c0c30f-kube-api-access-8bm7q\") pod \"maas-api-e2e-ait-624a770e-6598f56556-4dd6v\" (UID: \"98dd53d8-2dbe-4b73-b442-165dd7c0c30f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-624a770e-6598f56556-4dd6v"
Jul 30 15:21:09.005292 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:09.005258    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-624a770e-8447b8dcb8-m9lhz"]
Jul 30 15:21:09.008576 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:09.008555    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-624a770e-8447b8dcb8-m9lhz"
Jul 30 15:21:09.010753 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:09.010731    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-624a770e-dockercfg-zvkhx\""
Jul 30 15:21:09.011093 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:09.011071    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-624a770e\""
Jul 30 15:21:09.018527 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:09.018500    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-624a770e-8447b8dcb8-m9lhz"]
Jul 30 15:21:09.094815 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:09.094784    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vwqg\" (UniqueName: \"kubernetes.io/projected/ef7df91b-23f7-4406-abcb-f8d90823c285-kube-api-access-7vwqg\") pod \"payload-processing-e2e-ait-624a770e-8447b8dcb8-m9lhz\" (UID: \"ef7df91b-23f7-4406-abcb-f8d90823c285\") " pod="openshift-ingress/payload-processing-e2e-ait-624a770e-8447b8dcb8-m9lhz"
Jul 30 15:21:09.095106 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:09.095081    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef7df91b-23f7-4406-abcb-f8d90823c285-plugins-config-volume\") pod \"payload-processing-e2e-ait-624a770e-8447b8dcb8-m9lhz\" (UID: \"ef7df91b-23f7-4406-abcb-f8d90823c285\") " pod="openshift-ingress/payload-processing-e2e-ait-624a770e-8447b8dcb8-m9lhz"
Jul 30 15:21:09.095232 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:09.095222    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98dd53d8-2dbe-4b73-b442-165dd7c0c30f-maas-api-tls\") pod \"maas-api-e2e-ait-624a770e-6598f56556-4dd6v\" (UID: \"98dd53d8-2dbe-4b73-b442-165dd7c0c30f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-624a770e-6598f56556-4dd6v"
Jul 30 15:21:09.097977 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:09.097958    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98dd53d8-2dbe-4b73-b442-165dd7c0c30f-maas-api-tls\") pod \"maas-api-e2e-ait-624a770e-6598f56556-4dd6v\" (UID: \"98dd53d8-2dbe-4b73-b442-165dd7c0c30f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-624a770e-6598f56556-4dd6v"
Jul 30 15:21:09.195834 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:09.195799    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef7df91b-23f7-4406-abcb-f8d90823c285-plugins-config-volume\") pod \"payload-processing-e2e-ait-624a770e-8447b8dcb8-m9lhz\" (UID: \"ef7df91b-23f7-4406-abcb-f8d90823c285\") " pod="openshift-ingress/payload-processing-e2e-ait-624a770e-8447b8dcb8-m9lhz"
Jul 30 15:21:09.195990 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:09.195908    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vwqg\" (UniqueName: \"kubernetes.io/projected/ef7df91b-23f7-4406-abcb-f8d90823c285-kube-api-access-7vwqg\") pod \"payload-processing-e2e-ait-624a770e-8447b8dcb8-m9lhz\" (UID: \"ef7df91b-23f7-4406-abcb-f8d90823c285\") " pod="openshift-ingress/payload-processing-e2e-ait-624a770e-8447b8dcb8-m9lhz"
Jul 30 15:21:09.196594 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:09.196574    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef7df91b-23f7-4406-abcb-f8d90823c285-plugins-config-volume\") pod \"payload-processing-e2e-ait-624a770e-8447b8dcb8-m9lhz\" (UID: \"ef7df91b-23f7-4406-abcb-f8d90823c285\") " pod="openshift-ingress/payload-processing-e2e-ait-624a770e-8447b8dcb8-m9lhz"
Jul 30 15:21:09.203132 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:09.203108    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vwqg\" (UniqueName: \"kubernetes.io/projected/ef7df91b-23f7-4406-abcb-f8d90823c285-kube-api-access-7vwqg\") pod \"payload-processing-e2e-ait-624a770e-8447b8dcb8-m9lhz\" (UID: \"ef7df91b-23f7-4406-abcb-f8d90823c285\") " pod="openshift-ingress/payload-processing-e2e-ait-624a770e-8447b8dcb8-m9lhz"
Jul 30 15:21:09.309313 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:09.309274    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-624a770e-6598f56556-4dd6v"
Jul 30 15:21:09.320794 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:09.320774    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-624a770e-8447b8dcb8-m9lhz"
Jul 30 15:21:09.461979 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:09.461944    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-624a770e-6598f56556-4dd6v"]
Jul 30 15:21:09.465100 ip-10-0-133-117 kubenswrapper[2578]: W0730 15:21:09.465069    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98dd53d8_2dbe_4b73_b442_165dd7c0c30f.slice/crio-d1015ad3be744ac373e46a0e4ca945830712229adaacc6939e218ad86df893aa WatchSource:0}: Error finding container d1015ad3be744ac373e46a0e4ca945830712229adaacc6939e218ad86df893aa: Status 404 returned error can't find the container with id d1015ad3be744ac373e46a0e4ca945830712229adaacc6939e218ad86df893aa
Jul 30 15:21:09.484139 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:09.484117    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-624a770e-8447b8dcb8-m9lhz"]
Jul 30 15:21:09.486212 ip-10-0-133-117 kubenswrapper[2578]: W0730 15:21:09.486193    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef7df91b_23f7_4406_abcb_f8d90823c285.slice/crio-ea3158f9ae3a7814d7744aab6ceff687abe2973239a7245b7306535adaf671a3 WatchSource:0}: Error finding container ea3158f9ae3a7814d7744aab6ceff687abe2973239a7245b7306535adaf671a3: Status 404 returned error can't find the container with id ea3158f9ae3a7814d7744aab6ceff687abe2973239a7245b7306535adaf671a3
Jul 30 15:21:09.823683 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:09.823597    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-624a770e-8447b8dcb8-m9lhz" event={"ID":"ef7df91b-23f7-4406-abcb-f8d90823c285","Type":"ContainerStarted","Data":"8300c5c7ec26aee3637a8fb27625865cc2d2769bc9918e061705d3c8cf8d66e9"}
Jul 30 15:21:09.823683 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:09.823659    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-624a770e-8447b8dcb8-m9lhz" event={"ID":"ef7df91b-23f7-4406-abcb-f8d90823c285","Type":"ContainerStarted","Data":"ea3158f9ae3a7814d7744aab6ceff687abe2973239a7245b7306535adaf671a3"}
Jul 30 15:21:09.823890 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:09.823690    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-624a770e-8447b8dcb8-m9lhz"
Jul 30 15:21:09.824788 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:09.824763    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-624a770e-6598f56556-4dd6v" event={"ID":"98dd53d8-2dbe-4b73-b442-165dd7c0c30f","Type":"ContainerStarted","Data":"d1015ad3be744ac373e46a0e4ca945830712229adaacc6939e218ad86df893aa"}
Jul 30 15:21:09.839515 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:09.839471    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-624a770e-8447b8dcb8-m9lhz" podStartSLOduration=1.839455902 podStartE2EDuration="1.839455902s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:21:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:21:09.838398126 +0000 UTC m=+1981.367336601" watchObservedRunningTime="2026-07-30 15:21:09.839455902 +0000 UTC m=+1981.368394374"
Jul 30 15:21:10.831291 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:10.831247    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-624a770e-6598f56556-4dd6v" event={"ID":"98dd53d8-2dbe-4b73-b442-165dd7c0c30f","Type":"ContainerStarted","Data":"3018b7948676c42464d479d001331f897c345e29810cb4f7df559cf08395c1e5"}
Jul 30 15:21:10.833708 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:10.831341    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-624a770e-6598f56556-4dd6v"
Jul 30 15:21:10.852167 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:10.852109    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-624a770e-6598f56556-4dd6v" podStartSLOduration=2.351238409 podStartE2EDuration="2.852091821s" totalImagesPullingTime="500.853412ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:21:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:21:10.850342551 +0000 UTC m=+1982.379281028" watchObservedRunningTime="2026-07-30 15:21:10.852091821 +0000 UTC m=+1982.381030295"
Jul 30 15:21:16.850454 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:16.850412    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-624a770e-6598f56556-4dd6v"
Jul 30 15:21:20.832943 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:20.832912    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-624a770e-8447b8dcb8-m9lhz"
Jul 30 15:21:20.886337 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:20.886264    2578 generic.go:356] "Generic (PLEG): container finished" podID="8ebe26db-241d-404b-bae1-aa7ef5c7bc74" containerID="0b6d1efc41a9c5cbafb78c040f8476d2c3cd10483f14b077f80af360fe377aef" exitCode=0
Jul 30 15:21:20.886469 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:20.886348    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj" event={"ID":"8ebe26db-241d-404b-bae1-aa7ef5c7bc74","Type":"ContainerDied","Data":"0b6d1efc41a9c5cbafb78c040f8476d2c3cd10483f14b077f80af360fe377aef"}
Jul 30 15:21:20.886873 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:20.886845    2578 scope.go:117] "RemoveContainer" containerID="0b6d1efc41a9c5cbafb78c040f8476d2c3cd10483f14b077f80af360fe377aef"
Jul 30 15:21:20.887981 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:20.887959    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:21:20.888047 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:20.888025    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:21:20.888081 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:20.888056    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:21:21.809486 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:21.809438    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj"
Jul 30 15:21:21.893346 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:21.893309    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj" event={"ID":"8ebe26db-241d-404b-bae1-aa7ef5c7bc74","Type":"ContainerStarted","Data":"467ac8ace581aebd03376f1f71b62c222abfddb85765064b68dc0c62e7140872"}
Jul 30 15:21:22.468295 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:22.468257    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj"
Jul 30 15:21:22.469562 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:22.469531    2578 patch_prober.go:28] interesting pod/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Jul 30 15:21:22.469699 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:22.469583    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj" podUID="8ebe26db-241d-404b-bae1-aa7ef5c7bc74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Jul 30 15:21:23.468441 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:23.468402    2578 patch_prober.go:28] interesting pod/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Jul 30 15:21:23.468831 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:23.468472    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj" podUID="8ebe26db-241d-404b-bae1-aa7ef5c7bc74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Jul 30 15:21:24.468749 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:24.468711    2578 patch_prober.go:28] interesting pod/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Jul 30 15:21:24.469172 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:24.468780    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj" podUID="8ebe26db-241d-404b-bae1-aa7ef5c7bc74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Jul 30 15:21:25.468853 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:25.468813    2578 patch_prober.go:28] interesting pod/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Jul 30 15:21:25.469262 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:25.468878    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj" podUID="8ebe26db-241d-404b-bae1-aa7ef5c7bc74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Jul 30 15:21:26.472265 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:26.472234    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj"
Jul 30 15:21:26.472627 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:26.472561    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj"
Jul 30 15:21:26.473311 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:26.473293    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj"
Jul 30 15:21:48.812942 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:48.812906    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-624a770e-6598f56556-4dd6v"]
Jul 30 15:21:48.813463 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:48.813117    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-624a770e-6598f56556-4dd6v" podUID="98dd53d8-2dbe-4b73-b442-165dd7c0c30f" containerName="maas-api" containerID="cri-o://3018b7948676c42464d479d001331f897c345e29810cb4f7df559cf08395c1e5" gracePeriod=30
Jul 30 15:21:48.861366 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:48.861330    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-624a770e-8447b8dcb8-m9lhz"]
Jul 30 15:21:48.861654 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:48.861593    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-624a770e-8447b8dcb8-m9lhz" podUID="ef7df91b-23f7-4406-abcb-f8d90823c285" containerName="payload-processing" containerID="cri-o://8300c5c7ec26aee3637a8fb27625865cc2d2769bc9918e061705d3c8cf8d66e9" gracePeriod=30
Jul 30 15:21:49.037188 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:49.037132    2578 generic.go:356] "Generic (PLEG): container finished" podID="ef7df91b-23f7-4406-abcb-f8d90823c285" containerID="8300c5c7ec26aee3637a8fb27625865cc2d2769bc9918e061705d3c8cf8d66e9" exitCode=0
Jul 30 15:21:49.037317 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:49.037247    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-624a770e-8447b8dcb8-m9lhz" event={"ID":"ef7df91b-23f7-4406-abcb-f8d90823c285","Type":"ContainerDied","Data":"8300c5c7ec26aee3637a8fb27625865cc2d2769bc9918e061705d3c8cf8d66e9"}
Jul 30 15:21:49.039350 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:49.039317    2578 generic.go:356] "Generic (PLEG): container finished" podID="98dd53d8-2dbe-4b73-b442-165dd7c0c30f" containerID="3018b7948676c42464d479d001331f897c345e29810cb4f7df559cf08395c1e5" exitCode=0
Jul 30 15:21:49.039463 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:49.039433    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-624a770e-6598f56556-4dd6v" event={"ID":"98dd53d8-2dbe-4b73-b442-165dd7c0c30f","Type":"ContainerDied","Data":"3018b7948676c42464d479d001331f897c345e29810cb4f7df559cf08395c1e5"}
Jul 30 15:21:49.110303 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:49.110278    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-624a770e-6598f56556-4dd6v"
Jul 30 15:21:49.142386 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:49.142362    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-624a770e-8447b8dcb8-m9lhz"
Jul 30 15:21:49.159357 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:49.158606    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj"]
Jul 30 15:21:49.159357 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:49.158840    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj" podUID="8ebe26db-241d-404b-bae1-aa7ef5c7bc74" containerName="istio-proxy" containerID="cri-o://467ac8ace581aebd03376f1f71b62c222abfddb85765064b68dc0c62e7140872" gracePeriod=30
Jul 30 15:21:49.189214 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:49.189187    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7vwqg\" (UniqueName: \"kubernetes.io/projected/ef7df91b-23f7-4406-abcb-f8d90823c285-kube-api-access-7vwqg\") pod \"ef7df91b-23f7-4406-abcb-f8d90823c285\" (UID: \"ef7df91b-23f7-4406-abcb-f8d90823c285\") "
Jul 30 15:21:49.189387 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:49.189249    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98dd53d8-2dbe-4b73-b442-165dd7c0c30f-openshift-service-ca\") pod \"98dd53d8-2dbe-4b73-b442-165dd7c0c30f\" (UID: \"98dd53d8-2dbe-4b73-b442-165dd7c0c30f\") "
Jul 30 15:21:49.189387 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:49.189316    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8bm7q\" (UniqueName: \"kubernetes.io/projected/98dd53d8-2dbe-4b73-b442-165dd7c0c30f-kube-api-access-8bm7q\") pod \"98dd53d8-2dbe-4b73-b442-165dd7c0c30f\" (UID: \"98dd53d8-2dbe-4b73-b442-165dd7c0c30f\") "
Jul 30 15:21:49.189387 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:49.189382    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef7df91b-23f7-4406-abcb-f8d90823c285-plugins-config-volume\") pod \"ef7df91b-23f7-4406-abcb-f8d90823c285\" (UID: \"ef7df91b-23f7-4406-abcb-f8d90823c285\") "
Jul 30 15:21:49.189550 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:49.189432    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98dd53d8-2dbe-4b73-b442-165dd7c0c30f-maas-api-tls\") pod \"98dd53d8-2dbe-4b73-b442-165dd7c0c30f\" (UID: \"98dd53d8-2dbe-4b73-b442-165dd7c0c30f\") "
Jul 30 15:21:49.189959 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:49.189817    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef7df91b-23f7-4406-abcb-f8d90823c285-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ef7df91b-23f7-4406-abcb-f8d90823c285" (UID: "ef7df91b-23f7-4406-abcb-f8d90823c285"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:21:49.190066 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:49.189977    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98dd53d8-2dbe-4b73-b442-165dd7c0c30f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "98dd53d8-2dbe-4b73-b442-165dd7c0c30f" (UID: "98dd53d8-2dbe-4b73-b442-165dd7c0c30f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:21:49.191611 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:49.191587    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef7df91b-23f7-4406-abcb-f8d90823c285-kube-api-access-7vwqg" (OuterVolumeSpecName: "kube-api-access-7vwqg") pod "ef7df91b-23f7-4406-abcb-f8d90823c285" (UID: "ef7df91b-23f7-4406-abcb-f8d90823c285"). InnerVolumeSpecName "kube-api-access-7vwqg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:21:49.191721 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:49.191651    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98dd53d8-2dbe-4b73-b442-165dd7c0c30f-kube-api-access-8bm7q" (OuterVolumeSpecName: "kube-api-access-8bm7q") pod "98dd53d8-2dbe-4b73-b442-165dd7c0c30f" (UID: "98dd53d8-2dbe-4b73-b442-165dd7c0c30f"). InnerVolumeSpecName "kube-api-access-8bm7q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:21:49.191998 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:49.191977    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98dd53d8-2dbe-4b73-b442-165dd7c0c30f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "98dd53d8-2dbe-4b73-b442-165dd7c0c30f" (UID: "98dd53d8-2dbe-4b73-b442-165dd7c0c30f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 15:21:49.290789 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:49.290756    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98dd53d8-2dbe-4b73-b442-165dd7c0c30f-openshift-service-ca\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:21:49.290789 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:49.290788    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8bm7q\" (UniqueName: \"kubernetes.io/projected/98dd53d8-2dbe-4b73-b442-165dd7c0c30f-kube-api-access-8bm7q\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:21:49.290981 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:49.290801    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef7df91b-23f7-4406-abcb-f8d90823c285-plugins-config-volume\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:21:49.290981 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:49.290814    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98dd53d8-2dbe-4b73-b442-165dd7c0c30f-maas-api-tls\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:21:49.290981 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:49.290826    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7vwqg\" (UniqueName: \"kubernetes.io/projected/ef7df91b-23f7-4406-abcb-f8d90823c285-kube-api-access-7vwqg\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:21:50.045569 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:50.045538    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-624a770e-8447b8dcb8-m9lhz"
Jul 30 15:21:50.046024 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:50.045537    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-624a770e-8447b8dcb8-m9lhz" event={"ID":"ef7df91b-23f7-4406-abcb-f8d90823c285","Type":"ContainerDied","Data":"ea3158f9ae3a7814d7744aab6ceff687abe2973239a7245b7306535adaf671a3"}
Jul 30 15:21:50.046024 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:50.045706    2578 scope.go:117] "RemoveContainer" containerID="8300c5c7ec26aee3637a8fb27625865cc2d2769bc9918e061705d3c8cf8d66e9"
Jul 30 15:21:50.047097 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:50.046858    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-624a770e-6598f56556-4dd6v"
Jul 30 15:21:50.047097 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:50.046906    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-624a770e-6598f56556-4dd6v" event={"ID":"98dd53d8-2dbe-4b73-b442-165dd7c0c30f","Type":"ContainerDied","Data":"d1015ad3be744ac373e46a0e4ca945830712229adaacc6939e218ad86df893aa"}
Jul 30 15:21:50.054883 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:50.054657    2578 scope.go:117] "RemoveContainer" containerID="3018b7948676c42464d479d001331f897c345e29810cb4f7df559cf08395c1e5"
Jul 30 15:21:50.068154 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:50.068121    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-624a770e-8447b8dcb8-m9lhz"]
Jul 30 15:21:50.074388 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:50.074361    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-624a770e-8447b8dcb8-m9lhz"]
Jul 30 15:21:50.083207 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:50.083134    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-624a770e-6598f56556-4dd6v"]
Jul 30 15:21:50.089094 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:50.089073    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-624a770e-6598f56556-4dd6v"]
Jul 30 15:21:50.273862 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:50.273830    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv"]
Jul 30 15:21:50.274392 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:50.274372    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef7df91b-23f7-4406-abcb-f8d90823c285" containerName="payload-processing"
Jul 30 15:21:50.274451 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:50.274395    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef7df91b-23f7-4406-abcb-f8d90823c285" containerName="payload-processing"
Jul 30 15:21:50.274451 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:50.274407    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98dd53d8-2dbe-4b73-b442-165dd7c0c30f" containerName="maas-api"
Jul 30 15:21:50.274451 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:50.274415    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="98dd53d8-2dbe-4b73-b442-165dd7c0c30f" containerName="maas-api"
Jul 30 15:21:50.274550 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:50.274528    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="98dd53d8-2dbe-4b73-b442-165dd7c0c30f" containerName="maas-api"
Jul 30 15:21:50.274550 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:50.274540    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef7df91b-23f7-4406-abcb-f8d90823c285" containerName="payload-processing"
Jul 30 15:21:50.278544 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:50.278523    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv"
Jul 30 15:21:50.280569 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:50.280544    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-5d54eb26-gw-openshift-default-dockercfg-tkxn6\""
Jul 30 15:21:50.289810 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:50.289782    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv"]
Jul 30 15:21:50.402732 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:50.402640    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/122ffe00-0f17-4101-ad04-f0a3ad41b35c-workload-certs\") pod \"e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv\" (UID: \"122ffe00-0f17-4101-ad04-f0a3ad41b35c\") " pod="openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv"
Jul 30 15:21:50.402732 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:50.402712    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/122ffe00-0f17-4101-ad04-f0a3ad41b35c-istio-envoy\") pod \"e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv\" (UID: \"122ffe00-0f17-4101-ad04-f0a3ad41b35c\") " pod="openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv"
Jul 30 15:21:50.402954 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:50.402748    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/122ffe00-0f17-4101-ad04-f0a3ad41b35c-istio-data\") pod \"e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv\" (UID: \"122ffe00-0f17-4101-ad04-f0a3ad41b35c\") " pod="openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv"
Jul 30 15:21:50.402954 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:50.402779    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/122ffe00-0f17-4101-ad04-f0a3ad41b35c-istio-podinfo\") pod \"e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv\" (UID: \"122ffe00-0f17-4101-ad04-f0a3ad41b35c\") " pod="openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv"
Jul 30 15:21:50.402954 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:50.402811    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/122ffe00-0f17-4101-ad04-f0a3ad41b35c-istio-token\") pod \"e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv\" (UID: \"122ffe00-0f17-4101-ad04-f0a3ad41b35c\") " pod="openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv"
Jul 30 15:21:50.402954 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:50.402837    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/122ffe00-0f17-4101-ad04-f0a3ad41b35c-istiod-ca-cert\") pod \"e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv\" (UID: \"122ffe00-0f17-4101-ad04-f0a3ad41b35c\") " pod="openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv"
Jul 30 15:21:50.402954 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:50.402880    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/122ffe00-0f17-4101-ad04-f0a3ad41b35c-workload-socket\") pod \"e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv\" (UID: \"122ffe00-0f17-4101-ad04-f0a3ad41b35c\") " pod="openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv"
Jul 30 15:21:50.402954 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:50.402909    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/122ffe00-0f17-4101-ad04-f0a3ad41b35c-credential-socket\") pod \"e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv\" (UID: \"122ffe00-0f17-4101-ad04-f0a3ad41b35c\") " pod="openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv"
Jul 30 15:21:50.403265 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:50.403011    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95n49\" (UniqueName: \"kubernetes.io/projected/122ffe00-0f17-4101-ad04-f0a3ad41b35c-kube-api-access-95n49\") pod \"e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv\" (UID: \"122ffe00-0f17-4101-ad04-f0a3ad41b35c\") " pod="openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv"
Jul 30 15:21:50.504164 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:50.504114    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95n49\" (UniqueName: \"kubernetes.io/projected/122ffe00-0f17-4101-ad04-f0a3ad41b35c-kube-api-access-95n49\") pod \"e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv\" (UID: \"122ffe00-0f17-4101-ad04-f0a3ad41b35c\") " pod="openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv"
Jul 30 15:21:50.504351 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:50.504232    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/122ffe00-0f17-4101-ad04-f0a3ad41b35c-workload-certs\") pod \"e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv\" (UID: \"122ffe00-0f17-4101-ad04-f0a3ad41b35c\") " pod="openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv"
Jul 30 15:21:50.504351 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:50.504271    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/122ffe00-0f17-4101-ad04-f0a3ad41b35c-istio-envoy\") pod \"e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv\" (UID: \"122ffe00-0f17-4101-ad04-f0a3ad41b35c\") " pod="openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv"
Jul 30 15:21:50.504351 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:50.504293    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/122ffe00-0f17-4101-ad04-f0a3ad41b35c-istio-data\") pod \"e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv\" (UID: \"122ffe00-0f17-4101-ad04-f0a3ad41b35c\") " pod="openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv"
Jul 30 15:21:50.504351 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:50.504316    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/122ffe00-0f17-4101-ad04-f0a3ad41b35c-istio-podinfo\") pod \"e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv\" (UID: \"122ffe00-0f17-4101-ad04-f0a3ad41b35c\") " pod="openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv"
Jul 30 15:21:50.504351 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:50.504339    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/122ffe00-0f17-4101-ad04-f0a3ad41b35c-istio-token\") pod \"e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv\" (UID: \"122ffe00-0f17-4101-ad04-f0a3ad41b35c\") " pod="openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv"
Jul 30 15:21:50.504609 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:50.504444    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/122ffe00-0f17-4101-ad04-f0a3ad41b35c-istiod-ca-cert\") pod \"e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv\" (UID: \"122ffe00-0f17-4101-ad04-f0a3ad41b35c\") " pod="openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv"
Jul 30 15:21:50.504609 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:50.504485    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/122ffe00-0f17-4101-ad04-f0a3ad41b35c-workload-socket\") pod \"e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv\" (UID: \"122ffe00-0f17-4101-ad04-f0a3ad41b35c\") " pod="openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv"
Jul 30 15:21:50.504609 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:50.504540    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/122ffe00-0f17-4101-ad04-f0a3ad41b35c-credential-socket\") pod \"e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv\" (UID: \"122ffe00-0f17-4101-ad04-f0a3ad41b35c\") " pod="openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv"
Jul 30 15:21:50.504812 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:50.504705    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/122ffe00-0f17-4101-ad04-f0a3ad41b35c-workload-certs\") pod \"e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv\" (UID: \"122ffe00-0f17-4101-ad04-f0a3ad41b35c\") " pod="openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv"
Jul 30 15:21:50.504812 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:50.504747    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/122ffe00-0f17-4101-ad04-f0a3ad41b35c-istio-data\") pod \"e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv\" (UID: \"122ffe00-0f17-4101-ad04-f0a3ad41b35c\") " pod="openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv"
Jul 30 15:21:50.504929 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:50.504908    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/122ffe00-0f17-4101-ad04-f0a3ad41b35c-credential-socket\") pod \"e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv\" (UID: \"122ffe00-0f17-4101-ad04-f0a3ad41b35c\") " pod="openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv"
Jul 30 15:21:50.505003 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:50.504984    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/122ffe00-0f17-4101-ad04-f0a3ad41b35c-workload-socket\") pod \"e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv\" (UID: \"122ffe00-0f17-4101-ad04-f0a3ad41b35c\") " pod="openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv"
Jul 30 15:21:50.505294 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:50.505276    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/122ffe00-0f17-4101-ad04-f0a3ad41b35c-istiod-ca-cert\") pod \"e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv\" (UID: \"122ffe00-0f17-4101-ad04-f0a3ad41b35c\") " pod="openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv"
Jul 30 15:21:50.507135 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:50.507111    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/122ffe00-0f17-4101-ad04-f0a3ad41b35c-istio-envoy\") pod \"e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv\" (UID: \"122ffe00-0f17-4101-ad04-f0a3ad41b35c\") " pod="openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv"
Jul 30 15:21:50.507454 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:50.507432    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/122ffe00-0f17-4101-ad04-f0a3ad41b35c-istio-podinfo\") pod \"e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv\" (UID: \"122ffe00-0f17-4101-ad04-f0a3ad41b35c\") " pod="openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv"
Jul 30 15:21:50.511884 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:50.511862    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/122ffe00-0f17-4101-ad04-f0a3ad41b35c-istio-token\") pod \"e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv\" (UID: \"122ffe00-0f17-4101-ad04-f0a3ad41b35c\") " pod="openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv"
Jul 30 15:21:50.512066 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:50.512045    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-95n49\" (UniqueName: \"kubernetes.io/projected/122ffe00-0f17-4101-ad04-f0a3ad41b35c-kube-api-access-95n49\") pod \"e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv\" (UID: \"122ffe00-0f17-4101-ad04-f0a3ad41b35c\") " pod="openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv"
Jul 30 15:21:50.590123 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:50.590092    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv"
Jul 30 15:21:50.731343 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:50.731315    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv"]
Jul 30 15:21:50.732750 ip-10-0-133-117 kubenswrapper[2578]: W0730 15:21:50.732728    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod122ffe00_0f17_4101_ad04_f0a3ad41b35c.slice/crio-2ac8e611298f1f5bf0044db4c5f0731b04aaf2b65661c0fbe03e08a8ee8ef168 WatchSource:0}: Error finding container 2ac8e611298f1f5bf0044db4c5f0731b04aaf2b65661c0fbe03e08a8ee8ef168: Status 404 returned error can't find the container with id 2ac8e611298f1f5bf0044db4c5f0731b04aaf2b65661c0fbe03e08a8ee8ef168
Jul 30 15:21:50.734925 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:50.734894    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:21:50.734998 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:50.734955    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:21:50.735034 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:50.735007    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:21:50.980978 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:50.980904    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98dd53d8-2dbe-4b73-b442-165dd7c0c30f" path="/var/lib/kubelet/pods/98dd53d8-2dbe-4b73-b442-165dd7c0c30f/volumes"
Jul 30 15:21:50.981291 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:50.981277    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef7df91b-23f7-4406-abcb-f8d90823c285" path="/var/lib/kubelet/pods/ef7df91b-23f7-4406-abcb-f8d90823c285/volumes"
Jul 30 15:21:51.054543 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:51.054510    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv" event={"ID":"122ffe00-0f17-4101-ad04-f0a3ad41b35c","Type":"ContainerStarted","Data":"2fccf5acb2167ccbe23ff47b824e510128f4be0824bc3124173180733a38b6fd"}
Jul 30 15:21:51.055002 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:51.054573    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv" event={"ID":"122ffe00-0f17-4101-ad04-f0a3ad41b35c","Type":"ContainerStarted","Data":"2ac8e611298f1f5bf0044db4c5f0731b04aaf2b65661c0fbe03e08a8ee8ef168"}
Jul 30 15:21:51.073941 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:51.073884    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv" podStartSLOduration=1.073861677 podStartE2EDuration="1.073861677s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:21:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:21:51.07240125 +0000 UTC m=+2022.601339737" watchObservedRunningTime="2026-07-30 15:21:51.073861677 +0000 UTC m=+2022.602800152"
Jul 30 15:21:51.590645 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:51.590611    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv"
Jul 30 15:21:51.595619 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:51.595593    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv"
Jul 30 15:21:52.062409 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:52.062382    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv"
Jul 30 15:21:52.063500 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:52.063480    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv"
Jul 30 15:21:52.703750 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:52.703717    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-5d54eb26-8ddff55c4-7klrg"]
Jul 30 15:21:52.708115 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:52.708096    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5d54eb26-8ddff55c4-7klrg"
Jul 30 15:21:52.709837 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:52.709816    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-5d54eb26-serving-cert\""
Jul 30 15:21:52.715643 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:52.715619    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-5d54eb26-8ddff55c4-7klrg"]
Jul 30 15:21:52.826703 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:52.826663    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b5b9fa2-eb94-4c47-b783-43f975683454-maas-api-tls\") pod \"maas-api-e2e-migrate-5d54eb26-8ddff55c4-7klrg\" (UID: \"3b5b9fa2-eb94-4c47-b783-43f975683454\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5d54eb26-8ddff55c4-7klrg"
Jul 30 15:21:52.826910 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:52.826743    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwlcq\" (UniqueName: \"kubernetes.io/projected/3b5b9fa2-eb94-4c47-b783-43f975683454-kube-api-access-mwlcq\") pod \"maas-api-e2e-migrate-5d54eb26-8ddff55c4-7klrg\" (UID: \"3b5b9fa2-eb94-4c47-b783-43f975683454\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5d54eb26-8ddff55c4-7klrg"
Jul 30 15:21:52.826910 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:52.826788    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b5b9fa2-eb94-4c47-b783-43f975683454-openshift-service-ca\") pod \"maas-api-e2e-migrate-5d54eb26-8ddff55c4-7klrg\" (UID: \"3b5b9fa2-eb94-4c47-b783-43f975683454\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5d54eb26-8ddff55c4-7klrg"
Jul 30 15:21:52.927525 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:52.927492    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b5b9fa2-eb94-4c47-b783-43f975683454-maas-api-tls\") pod \"maas-api-e2e-migrate-5d54eb26-8ddff55c4-7klrg\" (UID: \"3b5b9fa2-eb94-4c47-b783-43f975683454\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5d54eb26-8ddff55c4-7klrg"
Jul 30 15:21:52.927720 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:52.927550    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwlcq\" (UniqueName: \"kubernetes.io/projected/3b5b9fa2-eb94-4c47-b783-43f975683454-kube-api-access-mwlcq\") pod \"maas-api-e2e-migrate-5d54eb26-8ddff55c4-7klrg\" (UID: \"3b5b9fa2-eb94-4c47-b783-43f975683454\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5d54eb26-8ddff55c4-7klrg"
Jul 30 15:21:52.927720 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:52.927590    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b5b9fa2-eb94-4c47-b783-43f975683454-openshift-service-ca\") pod \"maas-api-e2e-migrate-5d54eb26-8ddff55c4-7klrg\" (UID: \"3b5b9fa2-eb94-4c47-b783-43f975683454\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5d54eb26-8ddff55c4-7klrg"
Jul 30 15:21:52.927720 ip-10-0-133-117 kubenswrapper[2578]: E0730 15:21:52.927656    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-5d54eb26-serving-cert: secret "maas-api-e2e-migrate-5d54eb26-serving-cert" not found
Jul 30 15:21:52.927876 ip-10-0-133-117 kubenswrapper[2578]: E0730 15:21:52.927735    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b5b9fa2-eb94-4c47-b783-43f975683454-maas-api-tls podName:3b5b9fa2-eb94-4c47-b783-43f975683454 nodeName:}" failed. No retries permitted until 2026-07-30 15:21:53.42771283 +0000 UTC m=+2024.956651284 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3b5b9fa2-eb94-4c47-b783-43f975683454-maas-api-tls") pod "maas-api-e2e-migrate-5d54eb26-8ddff55c4-7klrg" (UID: "3b5b9fa2-eb94-4c47-b783-43f975683454") : secret "maas-api-e2e-migrate-5d54eb26-serving-cert" not found
Jul 30 15:21:52.928250 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:52.928233    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b5b9fa2-eb94-4c47-b783-43f975683454-openshift-service-ca\") pod \"maas-api-e2e-migrate-5d54eb26-8ddff55c4-7klrg\" (UID: \"3b5b9fa2-eb94-4c47-b783-43f975683454\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5d54eb26-8ddff55c4-7klrg"
Jul 30 15:21:52.935598 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:52.935574    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwlcq\" (UniqueName: \"kubernetes.io/projected/3b5b9fa2-eb94-4c47-b783-43f975683454-kube-api-access-mwlcq\") pod \"maas-api-e2e-migrate-5d54eb26-8ddff55c4-7klrg\" (UID: \"3b5b9fa2-eb94-4c47-b783-43f975683454\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5d54eb26-8ddff55c4-7klrg"
Jul 30 15:21:53.393541 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:53.393501    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-5d54eb26-77479bf77-hnmb4"]
Jul 30 15:21:53.396921 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:53.396897    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-5d54eb26-77479bf77-hnmb4"
Jul 30 15:21:53.399066 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:53.399024    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-5d54eb26\""
Jul 30 15:21:53.399239 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:53.399222    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-5d54eb26-dockercfg-dshs4\""
Jul 30 15:21:53.407071 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:53.407044    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-5d54eb26-77479bf77-hnmb4"]
Jul 30 15:21:53.433156 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:53.433111    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b5b9fa2-eb94-4c47-b783-43f975683454-maas-api-tls\") pod \"maas-api-e2e-migrate-5d54eb26-8ddff55c4-7klrg\" (UID: \"3b5b9fa2-eb94-4c47-b783-43f975683454\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5d54eb26-8ddff55c4-7klrg"
Jul 30 15:21:53.435697 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:53.435670    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b5b9fa2-eb94-4c47-b783-43f975683454-maas-api-tls\") pod \"maas-api-e2e-migrate-5d54eb26-8ddff55c4-7klrg\" (UID: \"3b5b9fa2-eb94-4c47-b783-43f975683454\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5d54eb26-8ddff55c4-7klrg"
Jul 30 15:21:53.534594 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:53.534554    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fq786\" (UniqueName: \"kubernetes.io/projected/49092391-f648-41f5-b0f4-0ed0cee3a34d-kube-api-access-fq786\") pod \"payload-processing-e2e-migrate-5d54eb26-77479bf77-hnmb4\" (UID: \"49092391-f648-41f5-b0f4-0ed0cee3a34d\") " pod="openshift-ingress/payload-processing-e2e-migrate-5d54eb26-77479bf77-hnmb4"
Jul 30 15:21:53.534795 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:53.534759    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49092391-f648-41f5-b0f4-0ed0cee3a34d-plugins-config-volume\") pod \"payload-processing-e2e-migrate-5d54eb26-77479bf77-hnmb4\" (UID: \"49092391-f648-41f5-b0f4-0ed0cee3a34d\") " pod="openshift-ingress/payload-processing-e2e-migrate-5d54eb26-77479bf77-hnmb4"
Jul 30 15:21:53.621104 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:53.621067    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5d54eb26-8ddff55c4-7klrg"
Jul 30 15:21:53.635659 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:53.635625    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49092391-f648-41f5-b0f4-0ed0cee3a34d-plugins-config-volume\") pod \"payload-processing-e2e-migrate-5d54eb26-77479bf77-hnmb4\" (UID: \"49092391-f648-41f5-b0f4-0ed0cee3a34d\") " pod="openshift-ingress/payload-processing-e2e-migrate-5d54eb26-77479bf77-hnmb4"
Jul 30 15:21:53.635780 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:53.635729    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fq786\" (UniqueName: \"kubernetes.io/projected/49092391-f648-41f5-b0f4-0ed0cee3a34d-kube-api-access-fq786\") pod \"payload-processing-e2e-migrate-5d54eb26-77479bf77-hnmb4\" (UID: \"49092391-f648-41f5-b0f4-0ed0cee3a34d\") " pod="openshift-ingress/payload-processing-e2e-migrate-5d54eb26-77479bf77-hnmb4"
Jul 30 15:21:53.636387 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:53.636367    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49092391-f648-41f5-b0f4-0ed0cee3a34d-plugins-config-volume\") pod \"payload-processing-e2e-migrate-5d54eb26-77479bf77-hnmb4\" (UID: \"49092391-f648-41f5-b0f4-0ed0cee3a34d\") " pod="openshift-ingress/payload-processing-e2e-migrate-5d54eb26-77479bf77-hnmb4"
Jul 30 15:21:53.642833 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:53.642809    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fq786\" (UniqueName: \"kubernetes.io/projected/49092391-f648-41f5-b0f4-0ed0cee3a34d-kube-api-access-fq786\") pod \"payload-processing-e2e-migrate-5d54eb26-77479bf77-hnmb4\" (UID: \"49092391-f648-41f5-b0f4-0ed0cee3a34d\") " pod="openshift-ingress/payload-processing-e2e-migrate-5d54eb26-77479bf77-hnmb4"
Jul 30 15:21:53.709206 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:53.709163    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-5d54eb26-77479bf77-hnmb4"
Jul 30 15:21:53.781381 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:53.780402    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-5d54eb26-8ddff55c4-7klrg"]
Jul 30 15:21:53.787785 ip-10-0-133-117 kubenswrapper[2578]: W0730 15:21:53.787711    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b5b9fa2_eb94_4c47_b783_43f975683454.slice/crio-c5c3ceb3d87b3ea94506e0651c6f19c0ecad429187ee223c31b407d3c6c20900 WatchSource:0}: Error finding container c5c3ceb3d87b3ea94506e0651c6f19c0ecad429187ee223c31b407d3c6c20900: Status 404 returned error can't find the container with id c5c3ceb3d87b3ea94506e0651c6f19c0ecad429187ee223c31b407d3c6c20900
Jul 30 15:21:53.860398 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:53.860351    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-5d54eb26-77479bf77-hnmb4"]
Jul 30 15:21:53.861623 ip-10-0-133-117 kubenswrapper[2578]: W0730 15:21:53.861594    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49092391_f648_41f5_b0f4_0ed0cee3a34d.slice/crio-e759237431c90692eaec8810b98ea9ff6c44aebbf1d9d518ea8c39ff608e51e4 WatchSource:0}: Error finding container e759237431c90692eaec8810b98ea9ff6c44aebbf1d9d518ea8c39ff608e51e4: Status 404 returned error can't find the container with id e759237431c90692eaec8810b98ea9ff6c44aebbf1d9d518ea8c39ff608e51e4
Jul 30 15:21:54.073524 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:54.073489    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-5d54eb26-77479bf77-hnmb4" event={"ID":"49092391-f648-41f5-b0f4-0ed0cee3a34d","Type":"ContainerStarted","Data":"e18f05922c5cc9360f3673fb74b8fe411c0363375fccdc89818a580bb2f6ba4e"}
Jul 30 15:21:54.073713 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:54.073551    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-5d54eb26-77479bf77-hnmb4" event={"ID":"49092391-f648-41f5-b0f4-0ed0cee3a34d","Type":"ContainerStarted","Data":"e759237431c90692eaec8810b98ea9ff6c44aebbf1d9d518ea8c39ff608e51e4"}
Jul 30 15:21:54.073713 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:54.073584    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-5d54eb26-77479bf77-hnmb4"
Jul 30 15:21:54.074657 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:54.074632    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5d54eb26-8ddff55c4-7klrg" event={"ID":"3b5b9fa2-eb94-4c47-b783-43f975683454","Type":"ContainerStarted","Data":"c5c3ceb3d87b3ea94506e0651c6f19c0ecad429187ee223c31b407d3c6c20900"}
Jul 30 15:21:54.421525 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:54.421500    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj"
Jul 30 15:21:54.441975 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:54.441913    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-5d54eb26-77479bf77-hnmb4" podStartSLOduration=1.441890629 podStartE2EDuration="1.441890629s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:21:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:21:54.089351349 +0000 UTC m=+2025.618289849" watchObservedRunningTime="2026-07-30 15:21:54.441890629 +0000 UTC m=+2025.970829103"
Jul 30 15:21:54.544496 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:54.544467    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5wzxh\" (UniqueName: \"kubernetes.io/projected/8ebe26db-241d-404b-bae1-aa7ef5c7bc74-kube-api-access-5wzxh\") pod \"8ebe26db-241d-404b-bae1-aa7ef5c7bc74\" (UID: \"8ebe26db-241d-404b-bae1-aa7ef5c7bc74\") "
Jul 30 15:21:54.544689 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:54.544548    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ebe26db-241d-404b-bae1-aa7ef5c7bc74-istio-data\") pod \"8ebe26db-241d-404b-bae1-aa7ef5c7bc74\" (UID: \"8ebe26db-241d-404b-bae1-aa7ef5c7bc74\") "
Jul 30 15:21:54.544689 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:54.544571    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ebe26db-241d-404b-bae1-aa7ef5c7bc74-credential-socket\") pod \"8ebe26db-241d-404b-bae1-aa7ef5c7bc74\" (UID: \"8ebe26db-241d-404b-bae1-aa7ef5c7bc74\") "
Jul 30 15:21:54.544689 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:54.544593    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ebe26db-241d-404b-bae1-aa7ef5c7bc74-istio-podinfo\") pod \"8ebe26db-241d-404b-bae1-aa7ef5c7bc74\" (UID: \"8ebe26db-241d-404b-bae1-aa7ef5c7bc74\") "
Jul 30 15:21:54.544689 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:54.544614    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ebe26db-241d-404b-bae1-aa7ef5c7bc74-workload-socket\") pod \"8ebe26db-241d-404b-bae1-aa7ef5c7bc74\" (UID: \"8ebe26db-241d-404b-bae1-aa7ef5c7bc74\") "
Jul 30 15:21:54.544689 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:54.544637    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ebe26db-241d-404b-bae1-aa7ef5c7bc74-istiod-ca-cert\") pod \"8ebe26db-241d-404b-bae1-aa7ef5c7bc74\" (UID: \"8ebe26db-241d-404b-bae1-aa7ef5c7bc74\") "
Jul 30 15:21:54.544689 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:54.544673    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ebe26db-241d-404b-bae1-aa7ef5c7bc74-istio-envoy\") pod \"8ebe26db-241d-404b-bae1-aa7ef5c7bc74\" (UID: \"8ebe26db-241d-404b-bae1-aa7ef5c7bc74\") "
Jul 30 15:21:54.545009 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:54.544751    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ebe26db-241d-404b-bae1-aa7ef5c7bc74-istio-token\") pod \"8ebe26db-241d-404b-bae1-aa7ef5c7bc74\" (UID: \"8ebe26db-241d-404b-bae1-aa7ef5c7bc74\") "
Jul 30 15:21:54.545009 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:54.544799    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ebe26db-241d-404b-bae1-aa7ef5c7bc74-workload-certs\") pod \"8ebe26db-241d-404b-bae1-aa7ef5c7bc74\" (UID: \"8ebe26db-241d-404b-bae1-aa7ef5c7bc74\") "
Jul 30 15:21:54.545307 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:54.545134    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ebe26db-241d-404b-bae1-aa7ef5c7bc74-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8ebe26db-241d-404b-bae1-aa7ef5c7bc74" (UID: "8ebe26db-241d-404b-bae1-aa7ef5c7bc74"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:21:54.545421 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:54.545307    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ebe26db-241d-404b-bae1-aa7ef5c7bc74-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8ebe26db-241d-404b-bae1-aa7ef5c7bc74" (UID: "8ebe26db-241d-404b-bae1-aa7ef5c7bc74"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:21:54.545478 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:54.545414    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ebe26db-241d-404b-bae1-aa7ef5c7bc74-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8ebe26db-241d-404b-bae1-aa7ef5c7bc74" (UID: "8ebe26db-241d-404b-bae1-aa7ef5c7bc74"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:21:54.545478 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:54.545430    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ebe26db-241d-404b-bae1-aa7ef5c7bc74-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8ebe26db-241d-404b-bae1-aa7ef5c7bc74" (UID: "8ebe26db-241d-404b-bae1-aa7ef5c7bc74"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:21:54.545714 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:54.545678    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ebe26db-241d-404b-bae1-aa7ef5c7bc74-istio-data" (OuterVolumeSpecName: "istio-data") pod "8ebe26db-241d-404b-bae1-aa7ef5c7bc74" (UID: "8ebe26db-241d-404b-bae1-aa7ef5c7bc74"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:21:54.546915 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:54.546885    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ebe26db-241d-404b-bae1-aa7ef5c7bc74-kube-api-access-5wzxh" (OuterVolumeSpecName: "kube-api-access-5wzxh") pod "8ebe26db-241d-404b-bae1-aa7ef5c7bc74" (UID: "8ebe26db-241d-404b-bae1-aa7ef5c7bc74"). InnerVolumeSpecName "kube-api-access-5wzxh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:21:54.547255 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:54.547109    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ebe26db-241d-404b-bae1-aa7ef5c7bc74-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8ebe26db-241d-404b-bae1-aa7ef5c7bc74" (UID: "8ebe26db-241d-404b-bae1-aa7ef5c7bc74"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:21:54.547255 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:54.547211    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8ebe26db-241d-404b-bae1-aa7ef5c7bc74-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8ebe26db-241d-404b-bae1-aa7ef5c7bc74" (UID: "8ebe26db-241d-404b-bae1-aa7ef5c7bc74"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 15:21:54.547406 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:54.547380    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ebe26db-241d-404b-bae1-aa7ef5c7bc74-istio-token" (OuterVolumeSpecName: "istio-token") pod "8ebe26db-241d-404b-bae1-aa7ef5c7bc74" (UID: "8ebe26db-241d-404b-bae1-aa7ef5c7bc74"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:21:54.646290 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:54.646208    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5wzxh\" (UniqueName: \"kubernetes.io/projected/8ebe26db-241d-404b-bae1-aa7ef5c7bc74-kube-api-access-5wzxh\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:21:54.646290 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:54.646239    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ebe26db-241d-404b-bae1-aa7ef5c7bc74-istio-data\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:21:54.646290 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:54.646249    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ebe26db-241d-404b-bae1-aa7ef5c7bc74-credential-socket\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:21:54.646290 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:54.646262    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ebe26db-241d-404b-bae1-aa7ef5c7bc74-istio-podinfo\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:21:54.646290 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:54.646278    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ebe26db-241d-404b-bae1-aa7ef5c7bc74-workload-socket\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:21:54.646290 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:54.646289    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ebe26db-241d-404b-bae1-aa7ef5c7bc74-istiod-ca-cert\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:21:54.646290 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:54.646297    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ebe26db-241d-404b-bae1-aa7ef5c7bc74-istio-envoy\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:21:54.646654 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:54.646304    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ebe26db-241d-404b-bae1-aa7ef5c7bc74-istio-token\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:21:54.646654 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:54.646311    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ebe26db-241d-404b-bae1-aa7ef5c7bc74-workload-certs\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:21:55.081856 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:55.081824    2578 generic.go:356] "Generic (PLEG): container finished" podID="8ebe26db-241d-404b-bae1-aa7ef5c7bc74" containerID="467ac8ace581aebd03376f1f71b62c222abfddb85765064b68dc0c62e7140872" exitCode=0
Jul 30 15:21:55.082050 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:55.081853    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj" event={"ID":"8ebe26db-241d-404b-bae1-aa7ef5c7bc74","Type":"ContainerDied","Data":"467ac8ace581aebd03376f1f71b62c222abfddb85765064b68dc0c62e7140872"}
Jul 30 15:21:55.082050 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:55.081884    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj"
Jul 30 15:21:55.082050 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:55.081909    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj" event={"ID":"8ebe26db-241d-404b-bae1-aa7ef5c7bc74","Type":"ContainerDied","Data":"f2236d4bd9b4c0e619f364e2fcbe641c162c884995b1641065ed0ebbefe1935c"}
Jul 30 15:21:55.082050 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:55.081932    2578 scope.go:117] "RemoveContainer" containerID="467ac8ace581aebd03376f1f71b62c222abfddb85765064b68dc0c62e7140872"
Jul 30 15:21:55.090677 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:55.090649    2578 scope.go:117] "RemoveContainer" containerID="0b6d1efc41a9c5cbafb78c040f8476d2c3cd10483f14b077f80af360fe377aef"
Jul 30 15:21:55.097798 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:55.097771    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj"]
Jul 30 15:21:55.099274 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:55.099250    2578 scope.go:117] "RemoveContainer" containerID="467ac8ace581aebd03376f1f71b62c222abfddb85765064b68dc0c62e7140872"
Jul 30 15:21:55.099538 ip-10-0-133-117 kubenswrapper[2578]: E0730 15:21:55.099515    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"467ac8ace581aebd03376f1f71b62c222abfddb85765064b68dc0c62e7140872\": container with ID starting with 467ac8ace581aebd03376f1f71b62c222abfddb85765064b68dc0c62e7140872 not found: ID does not exist" containerID="467ac8ace581aebd03376f1f71b62c222abfddb85765064b68dc0c62e7140872"
Jul 30 15:21:55.099624 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:55.099550    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"467ac8ace581aebd03376f1f71b62c222abfddb85765064b68dc0c62e7140872"} err="failed to get container status \"467ac8ace581aebd03376f1f71b62c222abfddb85765064b68dc0c62e7140872\": rpc error: code = NotFound desc = could not find container \"467ac8ace581aebd03376f1f71b62c222abfddb85765064b68dc0c62e7140872\": container with ID starting with 467ac8ace581aebd03376f1f71b62c222abfddb85765064b68dc0c62e7140872 not found: ID does not exist"
Jul 30 15:21:55.099624 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:55.099574    2578 scope.go:117] "RemoveContainer" containerID="0b6d1efc41a9c5cbafb78c040f8476d2c3cd10483f14b077f80af360fe377aef"
Jul 30 15:21:55.099822 ip-10-0-133-117 kubenswrapper[2578]: E0730 15:21:55.099806    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b6d1efc41a9c5cbafb78c040f8476d2c3cd10483f14b077f80af360fe377aef\": container with ID starting with 0b6d1efc41a9c5cbafb78c040f8476d2c3cd10483f14b077f80af360fe377aef not found: ID does not exist" containerID="0b6d1efc41a9c5cbafb78c040f8476d2c3cd10483f14b077f80af360fe377aef"
Jul 30 15:21:55.099877 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:55.099830    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b6d1efc41a9c5cbafb78c040f8476d2c3cd10483f14b077f80af360fe377aef"} err="failed to get container status \"0b6d1efc41a9c5cbafb78c040f8476d2c3cd10483f14b077f80af360fe377aef\": rpc error: code = NotFound desc = could not find container \"0b6d1efc41a9c5cbafb78c040f8476d2c3cd10483f14b077f80af360fe377aef\": container with ID starting with 0b6d1efc41a9c5cbafb78c040f8476d2c3cd10483f14b077f80af360fe377aef not found: ID does not exist"
Jul 30 15:21:55.100850 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:55.100830    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-624a770e-openshift-default-6cf65b696-zlghj"]
Jul 30 15:21:56.088498 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:56.088462    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5d54eb26-8ddff55c4-7klrg" event={"ID":"3b5b9fa2-eb94-4c47-b783-43f975683454","Type":"ContainerStarted","Data":"5fe8490c3e060987e010f70ee146ef7bf212f510c234d1a3454ed55120acae91"}
Jul 30 15:21:56.088971 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:56.088565    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5d54eb26-8ddff55c4-7klrg"
Jul 30 15:21:56.104631 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:56.104581    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5d54eb26-8ddff55c4-7klrg" podStartSLOduration=2.513197508 podStartE2EDuration="4.104564848s" totalImagesPullingTime="1.59136734s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:21:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:21:56.103403842 +0000 UTC m=+2027.632342315" watchObservedRunningTime="2026-07-30 15:21:56.104564848 +0000 UTC m=+2027.633503322"
Jul 30 15:21:56.979736 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:21:56.979705    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ebe26db-241d-404b-bae1-aa7ef5c7bc74" path="/var/lib/kubelet/pods/8ebe26db-241d-404b-bae1-aa7ef5c7bc74/volumes"
Jul 30 15:22:02.100995 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:02.100956    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5d54eb26-8ddff55c4-7klrg"
Jul 30 15:22:05.084326 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:05.084297    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-5d54eb26-77479bf77-hnmb4"
Jul 30 15:22:08.105313 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:08.105272    2578 patch_prober.go:28] interesting pod/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 30 15:22:08.105686 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:08.105329    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv" podUID="122ffe00-0f17-4101-ad04-f0a3ad41b35c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 30 15:22:09.161418 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:09.161389    2578 generic.go:356] "Generic (PLEG): container finished" podID="122ffe00-0f17-4101-ad04-f0a3ad41b35c" containerID="2fccf5acb2167ccbe23ff47b824e510128f4be0824bc3124173180733a38b6fd" exitCode=0
Jul 30 15:22:09.161753 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:09.161461    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv" event={"ID":"122ffe00-0f17-4101-ad04-f0a3ad41b35c","Type":"ContainerDied","Data":"2fccf5acb2167ccbe23ff47b824e510128f4be0824bc3124173180733a38b6fd"}
Jul 30 15:22:09.161892 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:09.161878    2578 scope.go:117] "RemoveContainer" containerID="2fccf5acb2167ccbe23ff47b824e510128f4be0824bc3124173180733a38b6fd"
Jul 30 15:22:09.162889 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:09.162864    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:22:09.162966 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:09.162929    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:22:09.162966 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:09.162959    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:22:10.168218 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:10.168185    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv" event={"ID":"122ffe00-0f17-4101-ad04-f0a3ad41b35c","Type":"ContainerStarted","Data":"d9a694438d7c1eb2e8494e71566835335bd42646c66c131370865c72b00c45f3"}
Jul 30 15:22:10.591159 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:10.591121    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv"
Jul 30 15:22:10.591483 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:10.591460    2578 patch_prober.go:28] interesting pod/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Jul 30 15:22:10.591545 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:10.591513    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv" podUID="122ffe00-0f17-4101-ad04-f0a3ad41b35c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Jul 30 15:22:11.590994 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:11.590962    2578 patch_prober.go:28] interesting pod/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Jul 30 15:22:11.591449 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:11.591018    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv" podUID="122ffe00-0f17-4101-ad04-f0a3ad41b35c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Jul 30 15:22:12.590479 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:12.590444    2578 patch_prober.go:28] interesting pod/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Jul 30 15:22:12.590650 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:12.590531    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv" podUID="122ffe00-0f17-4101-ad04-f0a3ad41b35c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Jul 30 15:22:13.591532 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:13.591486    2578 patch_prober.go:28] interesting pod/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Jul 30 15:22:13.591928 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:13.591556    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv" podUID="122ffe00-0f17-4101-ad04-f0a3ad41b35c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Jul 30 15:22:14.594298 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:14.594266    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv"
Jul 30 15:22:14.594688 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:14.594651    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv"
Jul 30 15:22:14.595277 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:14.595258    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv"
Jul 30 15:22:23.963671 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:23.963635    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-5d54eb26-8ddff55c4-7klrg"]
Jul 30 15:22:23.964330 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:23.963871    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5d54eb26-8ddff55c4-7klrg" podUID="3b5b9fa2-eb94-4c47-b783-43f975683454" containerName="maas-api" containerID="cri-o://5fe8490c3e060987e010f70ee146ef7bf212f510c234d1a3454ed55120acae91" gracePeriod=30
Jul 30 15:22:24.023541 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:24.023504    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-5d54eb26-77479bf77-hnmb4"]
Jul 30 15:22:24.023997 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:24.023778    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-5d54eb26-77479bf77-hnmb4" podUID="49092391-f648-41f5-b0f4-0ed0cee3a34d" containerName="payload-processing" containerID="cri-o://e18f05922c5cc9360f3673fb74b8fe411c0363375fccdc89818a580bb2f6ba4e" gracePeriod=30
Jul 30 15:22:24.249787 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:24.249745    2578 generic.go:356] "Generic (PLEG): container finished" podID="3b5b9fa2-eb94-4c47-b783-43f975683454" containerID="5fe8490c3e060987e010f70ee146ef7bf212f510c234d1a3454ed55120acae91" exitCode=0
Jul 30 15:22:24.249937 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:24.249794    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5d54eb26-8ddff55c4-7klrg" event={"ID":"3b5b9fa2-eb94-4c47-b783-43f975683454","Type":"ContainerDied","Data":"5fe8490c3e060987e010f70ee146ef7bf212f510c234d1a3454ed55120acae91"}
Jul 30 15:22:24.251655 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:24.251630    2578 generic.go:356] "Generic (PLEG): container finished" podID="49092391-f648-41f5-b0f4-0ed0cee3a34d" containerID="e18f05922c5cc9360f3673fb74b8fe411c0363375fccdc89818a580bb2f6ba4e" exitCode=0
Jul 30 15:22:24.251766 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:24.251699    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-5d54eb26-77479bf77-hnmb4" event={"ID":"49092391-f648-41f5-b0f4-0ed0cee3a34d","Type":"ContainerDied","Data":"e18f05922c5cc9360f3673fb74b8fe411c0363375fccdc89818a580bb2f6ba4e"}
Jul 30 15:22:24.280502 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:24.280475    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5d54eb26-8ddff55c4-7klrg"
Jul 30 15:22:24.310104 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:24.310070    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-5d54eb26-77479bf77-hnmb4"
Jul 30 15:22:24.351311 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:24.351278    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv"]
Jul 30 15:22:24.351493 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:24.351470    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv" podUID="122ffe00-0f17-4101-ad04-f0a3ad41b35c" containerName="istio-proxy" containerID="cri-o://d9a694438d7c1eb2e8494e71566835335bd42646c66c131370865c72b00c45f3" gracePeriod=30
Jul 30 15:22:24.364142 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:24.364110    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b5b9fa2-eb94-4c47-b783-43f975683454-maas-api-tls\") pod \"3b5b9fa2-eb94-4c47-b783-43f975683454\" (UID: \"3b5b9fa2-eb94-4c47-b783-43f975683454\") "
Jul 30 15:22:24.364142 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:24.364206    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mwlcq\" (UniqueName: \"kubernetes.io/projected/3b5b9fa2-eb94-4c47-b783-43f975683454-kube-api-access-mwlcq\") pod \"3b5b9fa2-eb94-4c47-b783-43f975683454\" (UID: \"3b5b9fa2-eb94-4c47-b783-43f975683454\") "
Jul 30 15:22:24.364447 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:24.364237    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49092391-f648-41f5-b0f4-0ed0cee3a34d-plugins-config-volume\") pod \"49092391-f648-41f5-b0f4-0ed0cee3a34d\" (UID: \"49092391-f648-41f5-b0f4-0ed0cee3a34d\") "
Jul 30 15:22:24.364447 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:24.364312    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fq786\" (UniqueName: \"kubernetes.io/projected/49092391-f648-41f5-b0f4-0ed0cee3a34d-kube-api-access-fq786\") pod \"49092391-f648-41f5-b0f4-0ed0cee3a34d\" (UID: \"49092391-f648-41f5-b0f4-0ed0cee3a34d\") "
Jul 30 15:22:24.364447 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:24.364400    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b5b9fa2-eb94-4c47-b783-43f975683454-openshift-service-ca\") pod \"3b5b9fa2-eb94-4c47-b783-43f975683454\" (UID: \"3b5b9fa2-eb94-4c47-b783-43f975683454\") "
Jul 30 15:22:24.367480 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:24.365499    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49092391-f648-41f5-b0f4-0ed0cee3a34d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "49092391-f648-41f5-b0f4-0ed0cee3a34d" (UID: "49092391-f648-41f5-b0f4-0ed0cee3a34d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:22:24.367480 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:24.365730    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b5b9fa2-eb94-4c47-b783-43f975683454-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3b5b9fa2-eb94-4c47-b783-43f975683454" (UID: "3b5b9fa2-eb94-4c47-b783-43f975683454"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:22:24.367480 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:24.367308    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b5b9fa2-eb94-4c47-b783-43f975683454-kube-api-access-mwlcq" (OuterVolumeSpecName: "kube-api-access-mwlcq") pod "3b5b9fa2-eb94-4c47-b783-43f975683454" (UID: "3b5b9fa2-eb94-4c47-b783-43f975683454"). InnerVolumeSpecName "kube-api-access-mwlcq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:22:24.372731 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:24.372687    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b5b9fa2-eb94-4c47-b783-43f975683454-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3b5b9fa2-eb94-4c47-b783-43f975683454" (UID: "3b5b9fa2-eb94-4c47-b783-43f975683454"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 15:22:24.375446 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:24.375326    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49092391-f648-41f5-b0f4-0ed0cee3a34d-kube-api-access-fq786" (OuterVolumeSpecName: "kube-api-access-fq786") pod "49092391-f648-41f5-b0f4-0ed0cee3a34d" (UID: "49092391-f648-41f5-b0f4-0ed0cee3a34d"). InnerVolumeSpecName "kube-api-access-fq786". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:22:24.466640 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:24.466604    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b5b9fa2-eb94-4c47-b783-43f975683454-openshift-service-ca\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:22:24.466640 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:24.466635    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b5b9fa2-eb94-4c47-b783-43f975683454-maas-api-tls\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:22:24.466640 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:24.466645    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mwlcq\" (UniqueName: \"kubernetes.io/projected/3b5b9fa2-eb94-4c47-b783-43f975683454-kube-api-access-mwlcq\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:22:24.466876 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:24.466654    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49092391-f648-41f5-b0f4-0ed0cee3a34d-plugins-config-volume\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:22:24.466876 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:24.466663    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fq786\" (UniqueName: \"kubernetes.io/projected/49092391-f648-41f5-b0f4-0ed0cee3a34d-kube-api-access-fq786\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:22:25.121083 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.121043    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr"]
Jul 30 15:22:25.121628 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.121601    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ebe26db-241d-404b-bae1-aa7ef5c7bc74" containerName="istio-proxy"
Jul 30 15:22:25.121628 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.121626    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ebe26db-241d-404b-bae1-aa7ef5c7bc74" containerName="istio-proxy"
Jul 30 15:22:25.121714 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.121639    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b5b9fa2-eb94-4c47-b783-43f975683454" containerName="maas-api"
Jul 30 15:22:25.121714 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.121647    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b5b9fa2-eb94-4c47-b783-43f975683454" containerName="maas-api"
Jul 30 15:22:25.121714 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.121658    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ebe26db-241d-404b-bae1-aa7ef5c7bc74" containerName="istio-proxy"
Jul 30 15:22:25.121714 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.121667    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ebe26db-241d-404b-bae1-aa7ef5c7bc74" containerName="istio-proxy"
Jul 30 15:22:25.121714 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.121686    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49092391-f648-41f5-b0f4-0ed0cee3a34d" containerName="payload-processing"
Jul 30 15:22:25.121714 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.121694    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="49092391-f648-41f5-b0f4-0ed0cee3a34d" containerName="payload-processing"
Jul 30 15:22:25.121911 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.121774    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b5b9fa2-eb94-4c47-b783-43f975683454" containerName="maas-api"
Jul 30 15:22:25.121911 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.121785    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="49092391-f648-41f5-b0f4-0ed0cee3a34d" containerName="payload-processing"
Jul 30 15:22:25.121911 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.121796    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ebe26db-241d-404b-bae1-aa7ef5c7bc74" containerName="istio-proxy"
Jul 30 15:22:25.121911 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.121805    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ebe26db-241d-404b-bae1-aa7ef5c7bc74" containerName="istio-proxy"
Jul 30 15:22:25.126420 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.126398    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr"
Jul 30 15:22:25.128440 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.128408    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-73521211-openshift-default-dockercfg-lmgmg\""
Jul 30 15:22:25.139628 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.139603    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr"]
Jul 30 15:22:25.258219 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.258115    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-5d54eb26-77479bf77-hnmb4"
Jul 30 15:22:25.258219 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.258131    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-5d54eb26-77479bf77-hnmb4" event={"ID":"49092391-f648-41f5-b0f4-0ed0cee3a34d","Type":"ContainerDied","Data":"e759237431c90692eaec8810b98ea9ff6c44aebbf1d9d518ea8c39ff608e51e4"}
Jul 30 15:22:25.258457 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.258219    2578 scope.go:117] "RemoveContainer" containerID="e18f05922c5cc9360f3673fb74b8fe411c0363375fccdc89818a580bb2f6ba4e"
Jul 30 15:22:25.259614 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.259556    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5d54eb26-8ddff55c4-7klrg" event={"ID":"3b5b9fa2-eb94-4c47-b783-43f975683454","Type":"ContainerDied","Data":"c5c3ceb3d87b3ea94506e0651c6f19c0ecad429187ee223c31b407d3c6c20900"}
Jul 30 15:22:25.259614 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.259570    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5d54eb26-8ddff55c4-7klrg"
Jul 30 15:22:25.268642 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.268623    2578 scope.go:117] "RemoveContainer" containerID="5fe8490c3e060987e010f70ee146ef7bf212f510c234d1a3454ed55120acae91"
Jul 30 15:22:25.273310 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.273286    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/190c779f-555e-4c6b-9d2b-2261442c56e1-workload-socket\") pod \"e2e-ait-73521211-openshift-default-549976f7f6-xj8nr\" (UID: \"190c779f-555e-4c6b-9d2b-2261442c56e1\") " pod="openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr"
Jul 30 15:22:25.273439 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.273322    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cv9qg\" (UniqueName: \"kubernetes.io/projected/190c779f-555e-4c6b-9d2b-2261442c56e1-kube-api-access-cv9qg\") pod \"e2e-ait-73521211-openshift-default-549976f7f6-xj8nr\" (UID: \"190c779f-555e-4c6b-9d2b-2261442c56e1\") " pod="openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr"
Jul 30 15:22:25.273503 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.273441    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/190c779f-555e-4c6b-9d2b-2261442c56e1-workload-certs\") pod \"e2e-ait-73521211-openshift-default-549976f7f6-xj8nr\" (UID: \"190c779f-555e-4c6b-9d2b-2261442c56e1\") " pod="openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr"
Jul 30 15:22:25.273503 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.273474    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/190c779f-555e-4c6b-9d2b-2261442c56e1-istio-token\") pod \"e2e-ait-73521211-openshift-default-549976f7f6-xj8nr\" (UID: \"190c779f-555e-4c6b-9d2b-2261442c56e1\") " pod="openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr"
Jul 30 15:22:25.273631 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.273532    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/190c779f-555e-4c6b-9d2b-2261442c56e1-credential-socket\") pod \"e2e-ait-73521211-openshift-default-549976f7f6-xj8nr\" (UID: \"190c779f-555e-4c6b-9d2b-2261442c56e1\") " pod="openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr"
Jul 30 15:22:25.273631 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.273557    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/190c779f-555e-4c6b-9d2b-2261442c56e1-istio-podinfo\") pod \"e2e-ait-73521211-openshift-default-549976f7f6-xj8nr\" (UID: \"190c779f-555e-4c6b-9d2b-2261442c56e1\") " pod="openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr"
Jul 30 15:22:25.273743 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.273636    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/190c779f-555e-4c6b-9d2b-2261442c56e1-istiod-ca-cert\") pod \"e2e-ait-73521211-openshift-default-549976f7f6-xj8nr\" (UID: \"190c779f-555e-4c6b-9d2b-2261442c56e1\") " pod="openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr"
Jul 30 15:22:25.273743 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.273670    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/190c779f-555e-4c6b-9d2b-2261442c56e1-istio-envoy\") pod \"e2e-ait-73521211-openshift-default-549976f7f6-xj8nr\" (UID: \"190c779f-555e-4c6b-9d2b-2261442c56e1\") " pod="openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr"
Jul 30 15:22:25.273743 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.273693    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/190c779f-555e-4c6b-9d2b-2261442c56e1-istio-data\") pod \"e2e-ait-73521211-openshift-default-549976f7f6-xj8nr\" (UID: \"190c779f-555e-4c6b-9d2b-2261442c56e1\") " pod="openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr"
Jul 30 15:22:25.275848 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.275824    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-5d54eb26-8ddff55c4-7klrg"]
Jul 30 15:22:25.280258 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.280233    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-5d54eb26-8ddff55c4-7klrg"]
Jul 30 15:22:25.288522 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.288500    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-5d54eb26-77479bf77-hnmb4"]
Jul 30 15:22:25.292433 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.292410    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-5d54eb26-77479bf77-hnmb4"]
Jul 30 15:22:25.375095 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.375052    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/190c779f-555e-4c6b-9d2b-2261442c56e1-istio-envoy\") pod \"e2e-ait-73521211-openshift-default-549976f7f6-xj8nr\" (UID: \"190c779f-555e-4c6b-9d2b-2261442c56e1\") " pod="openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr"
Jul 30 15:22:25.375263 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.375114    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/190c779f-555e-4c6b-9d2b-2261442c56e1-istio-data\") pod \"e2e-ait-73521211-openshift-default-549976f7f6-xj8nr\" (UID: \"190c779f-555e-4c6b-9d2b-2261442c56e1\") " pod="openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr"
Jul 30 15:22:25.375263 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.375223    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/190c779f-555e-4c6b-9d2b-2261442c56e1-workload-socket\") pod \"e2e-ait-73521211-openshift-default-549976f7f6-xj8nr\" (UID: \"190c779f-555e-4c6b-9d2b-2261442c56e1\") " pod="openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr"
Jul 30 15:22:25.375384 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.375259    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cv9qg\" (UniqueName: \"kubernetes.io/projected/190c779f-555e-4c6b-9d2b-2261442c56e1-kube-api-access-cv9qg\") pod \"e2e-ait-73521211-openshift-default-549976f7f6-xj8nr\" (UID: \"190c779f-555e-4c6b-9d2b-2261442c56e1\") " pod="openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr"
Jul 30 15:22:25.375384 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.375332    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/190c779f-555e-4c6b-9d2b-2261442c56e1-workload-certs\") pod \"e2e-ait-73521211-openshift-default-549976f7f6-xj8nr\" (UID: \"190c779f-555e-4c6b-9d2b-2261442c56e1\") " pod="openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr"
Jul 30 15:22:25.375384 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.375358    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/190c779f-555e-4c6b-9d2b-2261442c56e1-istio-token\") pod \"e2e-ait-73521211-openshift-default-549976f7f6-xj8nr\" (UID: \"190c779f-555e-4c6b-9d2b-2261442c56e1\") " pod="openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr"
Jul 30 15:22:25.375542 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.375402    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/190c779f-555e-4c6b-9d2b-2261442c56e1-credential-socket\") pod \"e2e-ait-73521211-openshift-default-549976f7f6-xj8nr\" (UID: \"190c779f-555e-4c6b-9d2b-2261442c56e1\") " pod="openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr"
Jul 30 15:22:25.375542 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.375428    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/190c779f-555e-4c6b-9d2b-2261442c56e1-istio-podinfo\") pod \"e2e-ait-73521211-openshift-default-549976f7f6-xj8nr\" (UID: \"190c779f-555e-4c6b-9d2b-2261442c56e1\") " pod="openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr"
Jul 30 15:22:25.375542 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.375460    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/190c779f-555e-4c6b-9d2b-2261442c56e1-istiod-ca-cert\") pod \"e2e-ait-73521211-openshift-default-549976f7f6-xj8nr\" (UID: \"190c779f-555e-4c6b-9d2b-2261442c56e1\") " pod="openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr"
Jul 30 15:22:25.375702 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.375643    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/190c779f-555e-4c6b-9d2b-2261442c56e1-istio-data\") pod \"e2e-ait-73521211-openshift-default-549976f7f6-xj8nr\" (UID: \"190c779f-555e-4c6b-9d2b-2261442c56e1\") " pod="openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr"
Jul 30 15:22:25.375761 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.375742    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/190c779f-555e-4c6b-9d2b-2261442c56e1-workload-certs\") pod \"e2e-ait-73521211-openshift-default-549976f7f6-xj8nr\" (UID: \"190c779f-555e-4c6b-9d2b-2261442c56e1\") " pod="openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr"
Jul 30 15:22:25.375844 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.375820    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/190c779f-555e-4c6b-9d2b-2261442c56e1-credential-socket\") pod \"e2e-ait-73521211-openshift-default-549976f7f6-xj8nr\" (UID: \"190c779f-555e-4c6b-9d2b-2261442c56e1\") " pod="openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr"
Jul 30 15:22:25.376033 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.376014    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/190c779f-555e-4c6b-9d2b-2261442c56e1-workload-socket\") pod \"e2e-ait-73521211-openshift-default-549976f7f6-xj8nr\" (UID: \"190c779f-555e-4c6b-9d2b-2261442c56e1\") " pod="openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr"
Jul 30 15:22:25.376178 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.376106    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/190c779f-555e-4c6b-9d2b-2261442c56e1-istiod-ca-cert\") pod \"e2e-ait-73521211-openshift-default-549976f7f6-xj8nr\" (UID: \"190c779f-555e-4c6b-9d2b-2261442c56e1\") " pod="openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr"
Jul 30 15:22:25.377664 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.377627    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/190c779f-555e-4c6b-9d2b-2261442c56e1-istio-envoy\") pod \"e2e-ait-73521211-openshift-default-549976f7f6-xj8nr\" (UID: \"190c779f-555e-4c6b-9d2b-2261442c56e1\") " pod="openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr"
Jul 30 15:22:25.377902 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.377878    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/190c779f-555e-4c6b-9d2b-2261442c56e1-istio-podinfo\") pod \"e2e-ait-73521211-openshift-default-549976f7f6-xj8nr\" (UID: \"190c779f-555e-4c6b-9d2b-2261442c56e1\") " pod="openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr"
Jul 30 15:22:25.395951 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.395924    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/190c779f-555e-4c6b-9d2b-2261442c56e1-istio-token\") pod \"e2e-ait-73521211-openshift-default-549976f7f6-xj8nr\" (UID: \"190c779f-555e-4c6b-9d2b-2261442c56e1\") " pod="openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr"
Jul 30 15:22:25.399114 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.399094    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cv9qg\" (UniqueName: \"kubernetes.io/projected/190c779f-555e-4c6b-9d2b-2261442c56e1-kube-api-access-cv9qg\") pod \"e2e-ait-73521211-openshift-default-549976f7f6-xj8nr\" (UID: \"190c779f-555e-4c6b-9d2b-2261442c56e1\") " pod="openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr"
Jul 30 15:22:25.436185 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.436139    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr"
Jul 30 15:22:25.571353 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.571322    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr"]
Jul 30 15:22:25.573500 ip-10-0-133-117 kubenswrapper[2578]: W0730 15:22:25.573474    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod190c779f_555e_4c6b_9d2b_2261442c56e1.slice/crio-9c538584a229b68bcbb1c35ca0378fc2dad0c14fffa96bc2fc8359370b2ecbfd WatchSource:0}: Error finding container 9c538584a229b68bcbb1c35ca0378fc2dad0c14fffa96bc2fc8359370b2ecbfd: Status 404 returned error can't find the container with id 9c538584a229b68bcbb1c35ca0378fc2dad0c14fffa96bc2fc8359370b2ecbfd
Jul 30 15:22:25.575631 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.575604    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:22:25.575693 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.575666    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:22:25.575730 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:25.575694    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:22:26.267556 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:26.267523    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr" event={"ID":"190c779f-555e-4c6b-9d2b-2261442c56e1","Type":"ContainerStarted","Data":"e53e7b3647645b68e6a8c3bd4653c8f8c864f4dc45934851167110442c0f29f0"}
Jul 30 15:22:26.267953 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:26.267577    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr" event={"ID":"190c779f-555e-4c6b-9d2b-2261442c56e1","Type":"ContainerStarted","Data":"9c538584a229b68bcbb1c35ca0378fc2dad0c14fffa96bc2fc8359370b2ecbfd"}
Jul 30 15:22:26.286168 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:26.286087    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr" podStartSLOduration=1.28606938 podStartE2EDuration="1.28606938s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:22:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:22:26.28435661 +0000 UTC m=+2057.813295085" watchObservedRunningTime="2026-07-30 15:22:26.28606938 +0000 UTC m=+2057.815007854"
Jul 30 15:22:26.437034 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:26.436992    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr"
Jul 30 15:22:26.442012 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:26.441984    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr"
Jul 30 15:22:26.979288 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:26.979254    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b5b9fa2-eb94-4c47-b783-43f975683454" path="/var/lib/kubelet/pods/3b5b9fa2-eb94-4c47-b783-43f975683454/volumes"
Jul 30 15:22:26.979654 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:26.979641    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49092391-f648-41f5-b0f4-0ed0cee3a34d" path="/var/lib/kubelet/pods/49092391-f648-41f5-b0f4-0ed0cee3a34d/volumes"
Jul 30 15:22:27.272447 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:27.272354    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr"
Jul 30 15:22:27.273354 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:27.273331    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr"
Jul 30 15:22:28.722210 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:28.722175    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-73521211-698f77f4c8-d6rw8"]
Jul 30 15:22:28.727098 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:28.727079    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-73521211-698f77f4c8-d6rw8"
Jul 30 15:22:28.728994 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:28.728975    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-73521211-serving-cert\""
Jul 30 15:22:28.735490 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:28.735466    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-73521211-698f77f4c8-d6rw8"]
Jul 30 15:22:28.810206 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:28.810158    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/489f4895-fece-404e-bc68-a7f0035488b7-maas-api-tls\") pod \"maas-api-e2e-ait-73521211-698f77f4c8-d6rw8\" (UID: \"489f4895-fece-404e-bc68-a7f0035488b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-73521211-698f77f4c8-d6rw8"
Jul 30 15:22:28.810375 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:28.810251    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/489f4895-fece-404e-bc68-a7f0035488b7-openshift-service-ca\") pod \"maas-api-e2e-ait-73521211-698f77f4c8-d6rw8\" (UID: \"489f4895-fece-404e-bc68-a7f0035488b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-73521211-698f77f4c8-d6rw8"
Jul 30 15:22:28.810375 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:28.810282    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njblq\" (UniqueName: \"kubernetes.io/projected/489f4895-fece-404e-bc68-a7f0035488b7-kube-api-access-njblq\") pod \"maas-api-e2e-ait-73521211-698f77f4c8-d6rw8\" (UID: \"489f4895-fece-404e-bc68-a7f0035488b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-73521211-698f77f4c8-d6rw8"
Jul 30 15:22:28.911787 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:28.911745    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/489f4895-fece-404e-bc68-a7f0035488b7-openshift-service-ca\") pod \"maas-api-e2e-ait-73521211-698f77f4c8-d6rw8\" (UID: \"489f4895-fece-404e-bc68-a7f0035488b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-73521211-698f77f4c8-d6rw8"
Jul 30 15:22:28.911972 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:28.911805    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njblq\" (UniqueName: \"kubernetes.io/projected/489f4895-fece-404e-bc68-a7f0035488b7-kube-api-access-njblq\") pod \"maas-api-e2e-ait-73521211-698f77f4c8-d6rw8\" (UID: \"489f4895-fece-404e-bc68-a7f0035488b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-73521211-698f77f4c8-d6rw8"
Jul 30 15:22:28.911972 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:28.911886    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/489f4895-fece-404e-bc68-a7f0035488b7-maas-api-tls\") pod \"maas-api-e2e-ait-73521211-698f77f4c8-d6rw8\" (UID: \"489f4895-fece-404e-bc68-a7f0035488b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-73521211-698f77f4c8-d6rw8"
Jul 30 15:22:28.912082 ip-10-0-133-117 kubenswrapper[2578]: E0730 15:22:28.911986    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-73521211-serving-cert: secret "maas-api-e2e-ait-73521211-serving-cert" not found
Jul 30 15:22:28.912082 ip-10-0-133-117 kubenswrapper[2578]: E0730 15:22:28.912065    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/489f4895-fece-404e-bc68-a7f0035488b7-maas-api-tls podName:489f4895-fece-404e-bc68-a7f0035488b7 nodeName:}" failed. No retries permitted until 2026-07-30 15:22:29.412047652 +0000 UTC m=+2060.940986104 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/489f4895-fece-404e-bc68-a7f0035488b7-maas-api-tls") pod "maas-api-e2e-ait-73521211-698f77f4c8-d6rw8" (UID: "489f4895-fece-404e-bc68-a7f0035488b7") : secret "maas-api-e2e-ait-73521211-serving-cert" not found
Jul 30 15:22:28.912601 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:28.912578    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/489f4895-fece-404e-bc68-a7f0035488b7-openshift-service-ca\") pod \"maas-api-e2e-ait-73521211-698f77f4c8-d6rw8\" (UID: \"489f4895-fece-404e-bc68-a7f0035488b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-73521211-698f77f4c8-d6rw8"
Jul 30 15:22:28.919763 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:28.919742    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njblq\" (UniqueName: \"kubernetes.io/projected/489f4895-fece-404e-bc68-a7f0035488b7-kube-api-access-njblq\") pod \"maas-api-e2e-ait-73521211-698f77f4c8-d6rw8\" (UID: \"489f4895-fece-404e-bc68-a7f0035488b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-73521211-698f77f4c8-d6rw8"
Jul 30 15:22:29.418017 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:29.417979    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/489f4895-fece-404e-bc68-a7f0035488b7-maas-api-tls\") pod \"maas-api-e2e-ait-73521211-698f77f4c8-d6rw8\" (UID: \"489f4895-fece-404e-bc68-a7f0035488b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-73521211-698f77f4c8-d6rw8"
Jul 30 15:22:29.420423 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:29.420399    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/489f4895-fece-404e-bc68-a7f0035488b7-maas-api-tls\") pod \"maas-api-e2e-ait-73521211-698f77f4c8-d6rw8\" (UID: \"489f4895-fece-404e-bc68-a7f0035488b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-73521211-698f77f4c8-d6rw8"
Jul 30 15:22:29.482263 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:29.482230    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-73521211-5c774fd84-vj58c"]
Jul 30 15:22:29.485769 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:29.485743    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-73521211-5c774fd84-vj58c"
Jul 30 15:22:29.487626 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:29.487604    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-73521211-dockercfg-87bm2\""
Jul 30 15:22:29.487753 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:29.487604    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-73521211\""
Jul 30 15:22:29.497017 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:29.496987    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-73521211-5c774fd84-vj58c"]
Jul 30 15:22:29.605235 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:29.605211    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv"
Jul 30 15:22:29.620529 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:29.620450    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3a01f352-6b34-4845-b5e5-f352b179b861-plugins-config-volume\") pod \"payload-processing-e2e-ait-73521211-5c774fd84-vj58c\" (UID: \"3a01f352-6b34-4845-b5e5-f352b179b861\") " pod="openshift-ingress/payload-processing-e2e-ait-73521211-5c774fd84-vj58c"
Jul 30 15:22:29.620529 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:29.620498    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kt5s\" (UniqueName: \"kubernetes.io/projected/3a01f352-6b34-4845-b5e5-f352b179b861-kube-api-access-8kt5s\") pod \"payload-processing-e2e-ait-73521211-5c774fd84-vj58c\" (UID: \"3a01f352-6b34-4845-b5e5-f352b179b861\") " pod="openshift-ingress/payload-processing-e2e-ait-73521211-5c774fd84-vj58c"
Jul 30 15:22:29.640128 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:29.640091    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-73521211-698f77f4c8-d6rw8"
Jul 30 15:22:29.721224 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:29.721190    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/122ffe00-0f17-4101-ad04-f0a3ad41b35c-credential-socket\") pod \"122ffe00-0f17-4101-ad04-f0a3ad41b35c\" (UID: \"122ffe00-0f17-4101-ad04-f0a3ad41b35c\") "
Jul 30 15:22:29.721405 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:29.721248    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/122ffe00-0f17-4101-ad04-f0a3ad41b35c-istiod-ca-cert\") pod \"122ffe00-0f17-4101-ad04-f0a3ad41b35c\" (UID: \"122ffe00-0f17-4101-ad04-f0a3ad41b35c\") "
Jul 30 15:22:29.721405 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:29.721275    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/122ffe00-0f17-4101-ad04-f0a3ad41b35c-workload-certs\") pod \"122ffe00-0f17-4101-ad04-f0a3ad41b35c\" (UID: \"122ffe00-0f17-4101-ad04-f0a3ad41b35c\") "
Jul 30 15:22:29.721405 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:29.721293    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/122ffe00-0f17-4101-ad04-f0a3ad41b35c-workload-socket\") pod \"122ffe00-0f17-4101-ad04-f0a3ad41b35c\" (UID: \"122ffe00-0f17-4101-ad04-f0a3ad41b35c\") "
Jul 30 15:22:29.721405 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:29.721341    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-95n49\" (UniqueName: \"kubernetes.io/projected/122ffe00-0f17-4101-ad04-f0a3ad41b35c-kube-api-access-95n49\") pod \"122ffe00-0f17-4101-ad04-f0a3ad41b35c\" (UID: \"122ffe00-0f17-4101-ad04-f0a3ad41b35c\") "
Jul 30 15:22:29.721405 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:29.721380    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/122ffe00-0f17-4101-ad04-f0a3ad41b35c-istio-envoy\") pod \"122ffe00-0f17-4101-ad04-f0a3ad41b35c\" (UID: \"122ffe00-0f17-4101-ad04-f0a3ad41b35c\") "
Jul 30 15:22:29.721405 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:29.721408    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/122ffe00-0f17-4101-ad04-f0a3ad41b35c-istio-data\") pod \"122ffe00-0f17-4101-ad04-f0a3ad41b35c\" (UID: \"122ffe00-0f17-4101-ad04-f0a3ad41b35c\") "
Jul 30 15:22:29.721738 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:29.721430    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/122ffe00-0f17-4101-ad04-f0a3ad41b35c-istio-podinfo\") pod \"122ffe00-0f17-4101-ad04-f0a3ad41b35c\" (UID: \"122ffe00-0f17-4101-ad04-f0a3ad41b35c\") "
Jul 30 15:22:29.721738 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:29.721462    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/122ffe00-0f17-4101-ad04-f0a3ad41b35c-istio-token\") pod \"122ffe00-0f17-4101-ad04-f0a3ad41b35c\" (UID: \"122ffe00-0f17-4101-ad04-f0a3ad41b35c\") "
Jul 30 15:22:29.721738 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:29.721507    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/122ffe00-0f17-4101-ad04-f0a3ad41b35c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "122ffe00-0f17-4101-ad04-f0a3ad41b35c" (UID: "122ffe00-0f17-4101-ad04-f0a3ad41b35c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:22:29.721738 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:29.721611    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kt5s\" (UniqueName: \"kubernetes.io/projected/3a01f352-6b34-4845-b5e5-f352b179b861-kube-api-access-8kt5s\") pod \"payload-processing-e2e-ait-73521211-5c774fd84-vj58c\" (UID: \"3a01f352-6b34-4845-b5e5-f352b179b861\") " pod="openshift-ingress/payload-processing-e2e-ait-73521211-5c774fd84-vj58c"
Jul 30 15:22:29.721738 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:29.721681    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/122ffe00-0f17-4101-ad04-f0a3ad41b35c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "122ffe00-0f17-4101-ad04-f0a3ad41b35c" (UID: "122ffe00-0f17-4101-ad04-f0a3ad41b35c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:22:29.721968 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:29.721775    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/122ffe00-0f17-4101-ad04-f0a3ad41b35c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "122ffe00-0f17-4101-ad04-f0a3ad41b35c" (UID: "122ffe00-0f17-4101-ad04-f0a3ad41b35c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:22:29.721968 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:29.721940    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3a01f352-6b34-4845-b5e5-f352b179b861-plugins-config-volume\") pod \"payload-processing-e2e-ait-73521211-5c774fd84-vj58c\" (UID: \"3a01f352-6b34-4845-b5e5-f352b179b861\") " pod="openshift-ingress/payload-processing-e2e-ait-73521211-5c774fd84-vj58c"
Jul 30 15:22:29.721968 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:29.721947    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/122ffe00-0f17-4101-ad04-f0a3ad41b35c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "122ffe00-0f17-4101-ad04-f0a3ad41b35c" (UID: "122ffe00-0f17-4101-ad04-f0a3ad41b35c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:22:29.722850 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:29.722817    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/122ffe00-0f17-4101-ad04-f0a3ad41b35c-istio-data" (OuterVolumeSpecName: "istio-data") pod "122ffe00-0f17-4101-ad04-f0a3ad41b35c" (UID: "122ffe00-0f17-4101-ad04-f0a3ad41b35c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:22:29.723796 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:29.723771    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3a01f352-6b34-4845-b5e5-f352b179b861-plugins-config-volume\") pod \"payload-processing-e2e-ait-73521211-5c774fd84-vj58c\" (UID: \"3a01f352-6b34-4845-b5e5-f352b179b861\") " pod="openshift-ingress/payload-processing-e2e-ait-73521211-5c774fd84-vj58c"
Jul 30 15:22:29.723989 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:29.723834    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/122ffe00-0f17-4101-ad04-f0a3ad41b35c-workload-certs\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:22:29.723989 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:29.723854    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/122ffe00-0f17-4101-ad04-f0a3ad41b35c-credential-socket\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:22:29.723989 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:29.723870    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/122ffe00-0f17-4101-ad04-f0a3ad41b35c-istiod-ca-cert\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:22:29.724188 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:29.723983    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/122ffe00-0f17-4101-ad04-f0a3ad41b35c-istio-token" (OuterVolumeSpecName: "istio-token") pod "122ffe00-0f17-4101-ad04-f0a3ad41b35c" (UID: "122ffe00-0f17-4101-ad04-f0a3ad41b35c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:22:29.724252 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:29.724227    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/122ffe00-0f17-4101-ad04-f0a3ad41b35c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "122ffe00-0f17-4101-ad04-f0a3ad41b35c" (UID: "122ffe00-0f17-4101-ad04-f0a3ad41b35c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 15:22:29.724300 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:29.724241    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/122ffe00-0f17-4101-ad04-f0a3ad41b35c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "122ffe00-0f17-4101-ad04-f0a3ad41b35c" (UID: "122ffe00-0f17-4101-ad04-f0a3ad41b35c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:22:29.724350 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:29.724331    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/122ffe00-0f17-4101-ad04-f0a3ad41b35c-kube-api-access-95n49" (OuterVolumeSpecName: "kube-api-access-95n49") pod "122ffe00-0f17-4101-ad04-f0a3ad41b35c" (UID: "122ffe00-0f17-4101-ad04-f0a3ad41b35c"). InnerVolumeSpecName "kube-api-access-95n49". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:22:29.728941 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:29.728919    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kt5s\" (UniqueName: \"kubernetes.io/projected/3a01f352-6b34-4845-b5e5-f352b179b861-kube-api-access-8kt5s\") pod \"payload-processing-e2e-ait-73521211-5c774fd84-vj58c\" (UID: \"3a01f352-6b34-4845-b5e5-f352b179b861\") " pod="openshift-ingress/payload-processing-e2e-ait-73521211-5c774fd84-vj58c"
Jul 30 15:22:29.782360 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:29.782332    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-73521211-698f77f4c8-d6rw8"]
Jul 30 15:22:29.784352 ip-10-0-133-117 kubenswrapper[2578]: W0730 15:22:29.784320    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod489f4895_fece_404e_bc68_a7f0035488b7.slice/crio-3c017d898a2c36006cb3246d2795b943e5ea5bc6e09e41240ac1f58bb0b4ef3c WatchSource:0}: Error finding container 3c017d898a2c36006cb3246d2795b943e5ea5bc6e09e41240ac1f58bb0b4ef3c: Status 404 returned error can't find the container with id 3c017d898a2c36006cb3246d2795b943e5ea5bc6e09e41240ac1f58bb0b4ef3c
Jul 30 15:22:29.798364 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:29.798337    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-73521211-5c774fd84-vj58c"
Jul 30 15:22:29.824824 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:29.824802    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/122ffe00-0f17-4101-ad04-f0a3ad41b35c-istio-data\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:22:29.824824 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:29.824824    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/122ffe00-0f17-4101-ad04-f0a3ad41b35c-istio-podinfo\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:22:29.824984 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:29.824832    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/122ffe00-0f17-4101-ad04-f0a3ad41b35c-istio-token\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:22:29.824984 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:29.824846    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/122ffe00-0f17-4101-ad04-f0a3ad41b35c-workload-socket\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:22:29.824984 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:29.824856    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-95n49\" (UniqueName: \"kubernetes.io/projected/122ffe00-0f17-4101-ad04-f0a3ad41b35c-kube-api-access-95n49\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:22:29.824984 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:29.824866    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/122ffe00-0f17-4101-ad04-f0a3ad41b35c-istio-envoy\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:22:29.929249 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:29.929109    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-73521211-5c774fd84-vj58c"]
Jul 30 15:22:29.931858 ip-10-0-133-117 kubenswrapper[2578]: W0730 15:22:29.931823    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a01f352_6b34_4845_b5e5_f352b179b861.slice/crio-c9210681e87efb2d2a4304a9f3e686859c813a04a8084ba5ac4d800ea78fe6ed WatchSource:0}: Error finding container c9210681e87efb2d2a4304a9f3e686859c813a04a8084ba5ac4d800ea78fe6ed: Status 404 returned error can't find the container with id c9210681e87efb2d2a4304a9f3e686859c813a04a8084ba5ac4d800ea78fe6ed
Jul 30 15:22:30.289918 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:30.289826    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-73521211-5c774fd84-vj58c" event={"ID":"3a01f352-6b34-4845-b5e5-f352b179b861","Type":"ContainerStarted","Data":"a888158e49ad0cfbebd478f88b6e672549e2f03e39f5e00156a0715713e0c2a6"}
Jul 30 15:22:30.289918 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:30.289899    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-73521211-5c774fd84-vj58c"
Jul 30 15:22:30.290173 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:30.289921    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-73521211-5c774fd84-vj58c" event={"ID":"3a01f352-6b34-4845-b5e5-f352b179b861","Type":"ContainerStarted","Data":"c9210681e87efb2d2a4304a9f3e686859c813a04a8084ba5ac4d800ea78fe6ed"}
Jul 30 15:22:30.291464 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:30.291444    2578 generic.go:356] "Generic (PLEG): container finished" podID="122ffe00-0f17-4101-ad04-f0a3ad41b35c" containerID="d9a694438d7c1eb2e8494e71566835335bd42646c66c131370865c72b00c45f3" exitCode=0
Jul 30 15:22:30.291534 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:30.291487    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv"
Jul 30 15:22:30.291534 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:30.291508    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv" event={"ID":"122ffe00-0f17-4101-ad04-f0a3ad41b35c","Type":"ContainerDied","Data":"d9a694438d7c1eb2e8494e71566835335bd42646c66c131370865c72b00c45f3"}
Jul 30 15:22:30.291615 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:30.291550    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv" event={"ID":"122ffe00-0f17-4101-ad04-f0a3ad41b35c","Type":"ContainerDied","Data":"2ac8e611298f1f5bf0044db4c5f0731b04aaf2b65661c0fbe03e08a8ee8ef168"}
Jul 30 15:22:30.291615 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:30.291577    2578 scope.go:117] "RemoveContainer" containerID="d9a694438d7c1eb2e8494e71566835335bd42646c66c131370865c72b00c45f3"
Jul 30 15:22:30.292770 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:30.292749    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-73521211-698f77f4c8-d6rw8" event={"ID":"489f4895-fece-404e-bc68-a7f0035488b7","Type":"ContainerStarted","Data":"3c017d898a2c36006cb3246d2795b943e5ea5bc6e09e41240ac1f58bb0b4ef3c"}
Jul 30 15:22:30.300296 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:30.300274    2578 scope.go:117] "RemoveContainer" containerID="2fccf5acb2167ccbe23ff47b824e510128f4be0824bc3124173180733a38b6fd"
Jul 30 15:22:30.306546 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:30.306502    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-73521211-5c774fd84-vj58c" podStartSLOduration=1.3064882469999999 podStartE2EDuration="1.306488247s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:22:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:22:30.304596782 +0000 UTC m=+2061.833535256" watchObservedRunningTime="2026-07-30 15:22:30.306488247 +0000 UTC m=+2061.835426721"
Jul 30 15:22:30.308619 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:30.308602    2578 scope.go:117] "RemoveContainer" containerID="d9a694438d7c1eb2e8494e71566835335bd42646c66c131370865c72b00c45f3"
Jul 30 15:22:30.308854 ip-10-0-133-117 kubenswrapper[2578]: E0730 15:22:30.308833    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9a694438d7c1eb2e8494e71566835335bd42646c66c131370865c72b00c45f3\": container with ID starting with d9a694438d7c1eb2e8494e71566835335bd42646c66c131370865c72b00c45f3 not found: ID does not exist" containerID="d9a694438d7c1eb2e8494e71566835335bd42646c66c131370865c72b00c45f3"
Jul 30 15:22:30.308894 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:30.308863    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9a694438d7c1eb2e8494e71566835335bd42646c66c131370865c72b00c45f3"} err="failed to get container status \"d9a694438d7c1eb2e8494e71566835335bd42646c66c131370865c72b00c45f3\": rpc error: code = NotFound desc = could not find container \"d9a694438d7c1eb2e8494e71566835335bd42646c66c131370865c72b00c45f3\": container with ID starting with d9a694438d7c1eb2e8494e71566835335bd42646c66c131370865c72b00c45f3 not found: ID does not exist"
Jul 30 15:22:30.308931 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:30.308894    2578 scope.go:117] "RemoveContainer" containerID="2fccf5acb2167ccbe23ff47b824e510128f4be0824bc3124173180733a38b6fd"
Jul 30 15:22:30.309114 ip-10-0-133-117 kubenswrapper[2578]: E0730 15:22:30.309094    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2fccf5acb2167ccbe23ff47b824e510128f4be0824bc3124173180733a38b6fd\": container with ID starting with 2fccf5acb2167ccbe23ff47b824e510128f4be0824bc3124173180733a38b6fd not found: ID does not exist" containerID="2fccf5acb2167ccbe23ff47b824e510128f4be0824bc3124173180733a38b6fd"
Jul 30 15:22:30.309168 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:30.309121    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2fccf5acb2167ccbe23ff47b824e510128f4be0824bc3124173180733a38b6fd"} err="failed to get container status \"2fccf5acb2167ccbe23ff47b824e510128f4be0824bc3124173180733a38b6fd\": rpc error: code = NotFound desc = could not find container \"2fccf5acb2167ccbe23ff47b824e510128f4be0824bc3124173180733a38b6fd\": container with ID starting with 2fccf5acb2167ccbe23ff47b824e510128f4be0824bc3124173180733a38b6fd not found: ID does not exist"
Jul 30 15:22:30.319258 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:30.319235    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv"]
Jul 30 15:22:30.321023 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:30.321003    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv"]
Jul 30 15:22:30.595754 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:30.595714    2578 patch_prober.go:28] interesting pod/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 30 15:22:30.595918 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:30.595773    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-5d54eb26-gw-openshift-default-7cf944d88d-nvzpv" podUID="122ffe00-0f17-4101-ad04-f0a3ad41b35c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 30 15:22:30.980324 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:30.980249    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="122ffe00-0f17-4101-ad04-f0a3ad41b35c" path="/var/lib/kubelet/pods/122ffe00-0f17-4101-ad04-f0a3ad41b35c/volumes"
Jul 30 15:22:32.305523 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:32.305489    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-73521211-698f77f4c8-d6rw8" event={"ID":"489f4895-fece-404e-bc68-a7f0035488b7","Type":"ContainerStarted","Data":"da1f4bcc308c44188c06db628445f0830cad20e73446df2906dcfab529cbc7a5"}
Jul 30 15:22:32.305916 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:32.305553    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-73521211-698f77f4c8-d6rw8"
Jul 30 15:22:32.323741 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:32.323690    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-73521211-698f77f4c8-d6rw8" podStartSLOduration=2.395472908 podStartE2EDuration="4.323675521s" totalImagesPullingTime="1.928202613s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:22:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:22:32.321376177 +0000 UTC m=+2063.850314651" watchObservedRunningTime="2026-07-30 15:22:32.323675521 +0000 UTC m=+2063.852613994"
Jul 30 15:22:38.314824 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:38.314790    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-73521211-698f77f4c8-d6rw8"
Jul 30 15:22:41.301134 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:41.301100    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-73521211-5c774fd84-vj58c"
Jul 30 15:22:42.273860 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:42.273783    2578 patch_prober.go:28] interesting pod/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Jul 30 15:22:42.273860 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:42.273835    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr" podUID="190c779f-555e-4c6b-9d2b-2261442c56e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Jul 30 15:22:42.359024 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:42.358952    2578 generic.go:356] "Generic (PLEG): container finished" podID="190c779f-555e-4c6b-9d2b-2261442c56e1" containerID="e53e7b3647645b68e6a8c3bd4653c8f8c864f4dc45934851167110442c0f29f0" exitCode=0
Jul 30 15:22:42.359387 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:42.359026    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr" event={"ID":"190c779f-555e-4c6b-9d2b-2261442c56e1","Type":"ContainerDied","Data":"e53e7b3647645b68e6a8c3bd4653c8f8c864f4dc45934851167110442c0f29f0"}
Jul 30 15:22:42.359454 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:42.359437    2578 scope.go:117] "RemoveContainer" containerID="e53e7b3647645b68e6a8c3bd4653c8f8c864f4dc45934851167110442c0f29f0"
Jul 30 15:22:42.360470 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:42.360442    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:22:42.360527 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:42.360510    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:22:42.360562 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:42.360542    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:22:43.365539 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:43.365504    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr" event={"ID":"190c779f-555e-4c6b-9d2b-2261442c56e1","Type":"ContainerStarted","Data":"91ee894b01ec38d50fbd69827310d56427baa605110a1d1eadd992324e66c7b2"}
Jul 30 15:22:43.437505 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:43.437460    2578 patch_prober.go:28] interesting pod/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Jul 30 15:22:43.437724 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:43.437530    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr" podUID="190c779f-555e-4c6b-9d2b-2261442c56e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Jul 30 15:22:43.437832 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:43.437076    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr"
Jul 30 15:22:44.436758 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:44.436714    2578 patch_prober.go:28] interesting pod/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Jul 30 15:22:44.437263 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:44.436779    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr" podUID="190c779f-555e-4c6b-9d2b-2261442c56e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Jul 30 15:22:45.437073 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:45.437024    2578 patch_prober.go:28] interesting pod/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Jul 30 15:22:45.437590 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:45.437098    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr" podUID="190c779f-555e-4c6b-9d2b-2261442c56e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Jul 30 15:22:46.436728 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:46.436686    2578 patch_prober.go:28] interesting pod/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Jul 30 15:22:46.437020 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:46.436990    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr" podUID="190c779f-555e-4c6b-9d2b-2261442c56e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Jul 30 15:22:47.440793 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:47.440763    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr"
Jul 30 15:22:47.441227 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:47.441036    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr"
Jul 30 15:22:47.441817 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:47.441801    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr"
Jul 30 15:22:59.402882 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:59.402845    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-73521211-698f77f4c8-d6rw8"]
Jul 30 15:22:59.403448 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:59.403098    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-73521211-698f77f4c8-d6rw8" podUID="489f4895-fece-404e-bc68-a7f0035488b7" containerName="maas-api" containerID="cri-o://da1f4bcc308c44188c06db628445f0830cad20e73446df2906dcfab529cbc7a5" gracePeriod=30
Jul 30 15:22:59.456543 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:59.456514    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-73521211-5c774fd84-vj58c"]
Jul 30 15:22:59.456763 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:59.456734    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-73521211-5c774fd84-vj58c" podUID="3a01f352-6b34-4845-b5e5-f352b179b861" containerName="payload-processing" containerID="cri-o://a888158e49ad0cfbebd478f88b6e672549e2f03e39f5e00156a0715713e0c2a6" gracePeriod=30
Jul 30 15:22:59.704267 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:59.704243    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-73521211-698f77f4c8-d6rw8"
Jul 30 15:22:59.733742 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:59.733718    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-73521211-5c774fd84-vj58c"
Jul 30 15:22:59.808352 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:59.808323    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/489f4895-fece-404e-bc68-a7f0035488b7-openshift-service-ca\") pod \"489f4895-fece-404e-bc68-a7f0035488b7\" (UID: \"489f4895-fece-404e-bc68-a7f0035488b7\") "
Jul 30 15:22:59.808520 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:59.808404    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/489f4895-fece-404e-bc68-a7f0035488b7-maas-api-tls\") pod \"489f4895-fece-404e-bc68-a7f0035488b7\" (UID: \"489f4895-fece-404e-bc68-a7f0035488b7\") "
Jul 30 15:22:59.808520 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:59.808434    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3a01f352-6b34-4845-b5e5-f352b179b861-plugins-config-volume\") pod \"3a01f352-6b34-4845-b5e5-f352b179b861\" (UID: \"3a01f352-6b34-4845-b5e5-f352b179b861\") "
Jul 30 15:22:59.808520 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:59.808450    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8kt5s\" (UniqueName: \"kubernetes.io/projected/3a01f352-6b34-4845-b5e5-f352b179b861-kube-api-access-8kt5s\") pod \"3a01f352-6b34-4845-b5e5-f352b179b861\" (UID: \"3a01f352-6b34-4845-b5e5-f352b179b861\") "
Jul 30 15:22:59.808643 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:59.808605    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-njblq\" (UniqueName: \"kubernetes.io/projected/489f4895-fece-404e-bc68-a7f0035488b7-kube-api-access-njblq\") pod \"489f4895-fece-404e-bc68-a7f0035488b7\" (UID: \"489f4895-fece-404e-bc68-a7f0035488b7\") "
Jul 30 15:22:59.808759 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:59.808738    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/489f4895-fece-404e-bc68-a7f0035488b7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "489f4895-fece-404e-bc68-a7f0035488b7" (UID: "489f4895-fece-404e-bc68-a7f0035488b7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:22:59.808831 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:59.808816    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a01f352-6b34-4845-b5e5-f352b179b861-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3a01f352-6b34-4845-b5e5-f352b179b861" (UID: "3a01f352-6b34-4845-b5e5-f352b179b861"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:22:59.808993 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:59.808971    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/489f4895-fece-404e-bc68-a7f0035488b7-openshift-service-ca\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:22:59.808993 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:59.808991    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3a01f352-6b34-4845-b5e5-f352b179b861-plugins-config-volume\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:22:59.810632 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:59.810608    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a01f352-6b34-4845-b5e5-f352b179b861-kube-api-access-8kt5s" (OuterVolumeSpecName: "kube-api-access-8kt5s") pod "3a01f352-6b34-4845-b5e5-f352b179b861" (UID: "3a01f352-6b34-4845-b5e5-f352b179b861"). InnerVolumeSpecName "kube-api-access-8kt5s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:22:59.810632 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:59.810632    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/489f4895-fece-404e-bc68-a7f0035488b7-kube-api-access-njblq" (OuterVolumeSpecName: "kube-api-access-njblq") pod "489f4895-fece-404e-bc68-a7f0035488b7" (UID: "489f4895-fece-404e-bc68-a7f0035488b7"). InnerVolumeSpecName "kube-api-access-njblq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:22:59.810756 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:59.810633    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/489f4895-fece-404e-bc68-a7f0035488b7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "489f4895-fece-404e-bc68-a7f0035488b7" (UID: "489f4895-fece-404e-bc68-a7f0035488b7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 15:22:59.910168 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:59.910117    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-njblq\" (UniqueName: \"kubernetes.io/projected/489f4895-fece-404e-bc68-a7f0035488b7-kube-api-access-njblq\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:22:59.910168 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:59.910163    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/489f4895-fece-404e-bc68-a7f0035488b7-maas-api-tls\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:22:59.910168 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:22:59.910174    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8kt5s\" (UniqueName: \"kubernetes.io/projected/3a01f352-6b34-4845-b5e5-f352b179b861-kube-api-access-8kt5s\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:23:00.445094 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:00.445061    2578 generic.go:356] "Generic (PLEG): container finished" podID="3a01f352-6b34-4845-b5e5-f352b179b861" containerID="a888158e49ad0cfbebd478f88b6e672549e2f03e39f5e00156a0715713e0c2a6" exitCode=0
Jul 30 15:23:00.445526 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:00.445104    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-73521211-5c774fd84-vj58c"
Jul 30 15:23:00.445526 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:00.445141    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-73521211-5c774fd84-vj58c" event={"ID":"3a01f352-6b34-4845-b5e5-f352b179b861","Type":"ContainerDied","Data":"a888158e49ad0cfbebd478f88b6e672549e2f03e39f5e00156a0715713e0c2a6"}
Jul 30 15:23:00.445526 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:00.445209    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-73521211-5c774fd84-vj58c" event={"ID":"3a01f352-6b34-4845-b5e5-f352b179b861","Type":"ContainerDied","Data":"c9210681e87efb2d2a4304a9f3e686859c813a04a8084ba5ac4d800ea78fe6ed"}
Jul 30 15:23:00.445526 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:00.445234    2578 scope.go:117] "RemoveContainer" containerID="a888158e49ad0cfbebd478f88b6e672549e2f03e39f5e00156a0715713e0c2a6"
Jul 30 15:23:00.446598 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:00.446531    2578 generic.go:356] "Generic (PLEG): container finished" podID="489f4895-fece-404e-bc68-a7f0035488b7" containerID="da1f4bcc308c44188c06db628445f0830cad20e73446df2906dcfab529cbc7a5" exitCode=0
Jul 30 15:23:00.446598 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:00.446577    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-73521211-698f77f4c8-d6rw8"
Jul 30 15:23:00.446688 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:00.446576    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-73521211-698f77f4c8-d6rw8" event={"ID":"489f4895-fece-404e-bc68-a7f0035488b7","Type":"ContainerDied","Data":"da1f4bcc308c44188c06db628445f0830cad20e73446df2906dcfab529cbc7a5"}
Jul 30 15:23:00.446688 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:00.446680    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-73521211-698f77f4c8-d6rw8" event={"ID":"489f4895-fece-404e-bc68-a7f0035488b7","Type":"ContainerDied","Data":"3c017d898a2c36006cb3246d2795b943e5ea5bc6e09e41240ac1f58bb0b4ef3c"}
Jul 30 15:23:00.454615 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:00.454421    2578 scope.go:117] "RemoveContainer" containerID="a888158e49ad0cfbebd478f88b6e672549e2f03e39f5e00156a0715713e0c2a6"
Jul 30 15:23:00.454727 ip-10-0-133-117 kubenswrapper[2578]: E0730 15:23:00.454706    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a888158e49ad0cfbebd478f88b6e672549e2f03e39f5e00156a0715713e0c2a6\": container with ID starting with a888158e49ad0cfbebd478f88b6e672549e2f03e39f5e00156a0715713e0c2a6 not found: ID does not exist" containerID="a888158e49ad0cfbebd478f88b6e672549e2f03e39f5e00156a0715713e0c2a6"
Jul 30 15:23:00.454790 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:00.454735    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a888158e49ad0cfbebd478f88b6e672549e2f03e39f5e00156a0715713e0c2a6"} err="failed to get container status \"a888158e49ad0cfbebd478f88b6e672549e2f03e39f5e00156a0715713e0c2a6\": rpc error: code = NotFound desc = could not find container \"a888158e49ad0cfbebd478f88b6e672549e2f03e39f5e00156a0715713e0c2a6\": container with ID starting with a888158e49ad0cfbebd478f88b6e672549e2f03e39f5e00156a0715713e0c2a6 not found: ID does not exist"
Jul 30 15:23:00.454790 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:00.454753    2578 scope.go:117] "RemoveContainer" containerID="da1f4bcc308c44188c06db628445f0830cad20e73446df2906dcfab529cbc7a5"
Jul 30 15:23:00.462877 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:00.462860    2578 scope.go:117] "RemoveContainer" containerID="da1f4bcc308c44188c06db628445f0830cad20e73446df2906dcfab529cbc7a5"
Jul 30 15:23:00.463121 ip-10-0-133-117 kubenswrapper[2578]: E0730 15:23:00.463101    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da1f4bcc308c44188c06db628445f0830cad20e73446df2906dcfab529cbc7a5\": container with ID starting with da1f4bcc308c44188c06db628445f0830cad20e73446df2906dcfab529cbc7a5 not found: ID does not exist" containerID="da1f4bcc308c44188c06db628445f0830cad20e73446df2906dcfab529cbc7a5"
Jul 30 15:23:00.463233 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:00.463125    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da1f4bcc308c44188c06db628445f0830cad20e73446df2906dcfab529cbc7a5"} err="failed to get container status \"da1f4bcc308c44188c06db628445f0830cad20e73446df2906dcfab529cbc7a5\": rpc error: code = NotFound desc = could not find container \"da1f4bcc308c44188c06db628445f0830cad20e73446df2906dcfab529cbc7a5\": container with ID starting with da1f4bcc308c44188c06db628445f0830cad20e73446df2906dcfab529cbc7a5 not found: ID does not exist"
Jul 30 15:23:00.468451 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:00.468431    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-73521211-698f77f4c8-d6rw8"]
Jul 30 15:23:00.474039 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:00.474018    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-73521211-698f77f4c8-d6rw8"]
Jul 30 15:23:00.485278 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:00.485252    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-73521211-5c774fd84-vj58c"]
Jul 30 15:23:00.488171 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:00.488131    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-73521211-5c774fd84-vj58c"]
Jul 30 15:23:00.827206 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:00.827170    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr"]
Jul 30 15:23:00.827444 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:00.827418    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr" podUID="190c779f-555e-4c6b-9d2b-2261442c56e1" containerName="istio-proxy" containerID="cri-o://91ee894b01ec38d50fbd69827310d56427baa605110a1d1eadd992324e66c7b2" gracePeriod=30
Jul 30 15:23:00.978564 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:00.978532    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a01f352-6b34-4845-b5e5-f352b179b861" path="/var/lib/kubelet/pods/3a01f352-6b34-4845-b5e5-f352b179b861/volumes"
Jul 30 15:23:00.978933 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:00.978919    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="489f4895-fece-404e-bc68-a7f0035488b7" path="/var/lib/kubelet/pods/489f4895-fece-404e-bc68-a7f0035488b7/volumes"
Jul 30 15:23:01.624439 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:01.624405    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx"]
Jul 30 15:23:01.624916 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:01.624786    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="122ffe00-0f17-4101-ad04-f0a3ad41b35c" containerName="istio-proxy"
Jul 30 15:23:01.624916 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:01.624797    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="122ffe00-0f17-4101-ad04-f0a3ad41b35c" containerName="istio-proxy"
Jul 30 15:23:01.624916 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:01.624804    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="122ffe00-0f17-4101-ad04-f0a3ad41b35c" containerName="istio-proxy"
Jul 30 15:23:01.624916 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:01.624810    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="122ffe00-0f17-4101-ad04-f0a3ad41b35c" containerName="istio-proxy"
Jul 30 15:23:01.624916 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:01.624831    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="489f4895-fece-404e-bc68-a7f0035488b7" containerName="maas-api"
Jul 30 15:23:01.624916 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:01.624839    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="489f4895-fece-404e-bc68-a7f0035488b7" containerName="maas-api"
Jul 30 15:23:01.624916 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:01.624849    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a01f352-6b34-4845-b5e5-f352b179b861" containerName="payload-processing"
Jul 30 15:23:01.624916 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:01.624854    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a01f352-6b34-4845-b5e5-f352b179b861" containerName="payload-processing"
Jul 30 15:23:01.624916 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:01.624913    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="122ffe00-0f17-4101-ad04-f0a3ad41b35c" containerName="istio-proxy"
Jul 30 15:23:01.625398 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:01.624925    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a01f352-6b34-4845-b5e5-f352b179b861" containerName="payload-processing"
Jul 30 15:23:01.625398 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:01.624932    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="489f4895-fece-404e-bc68-a7f0035488b7" containerName="maas-api"
Jul 30 15:23:01.629491 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:01.629470    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx"
Jul 30 15:23:01.631529 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:01.631506    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-7ba3a489-openshift-default-dockercfg-hg4pd\""
Jul 30 15:23:01.650634 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:01.650603    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx"]
Jul 30 15:23:01.727681 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:01.727642    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56e83def-04be-4211-adb9-ca44eb58654c-istio-envoy\") pod \"e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx\" (UID: \"56e83def-04be-4211-adb9-ca44eb58654c\") " pod="openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx"
Jul 30 15:23:01.727681 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:01.727678    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56e83def-04be-4211-adb9-ca44eb58654c-istiod-ca-cert\") pod \"e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx\" (UID: \"56e83def-04be-4211-adb9-ca44eb58654c\") " pod="openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx"
Jul 30 15:23:01.727880 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:01.727700    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56e83def-04be-4211-adb9-ca44eb58654c-credential-socket\") pod \"e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx\" (UID: \"56e83def-04be-4211-adb9-ca44eb58654c\") " pod="openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx"
Jul 30 15:23:01.727880 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:01.727846    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56e83def-04be-4211-adb9-ca44eb58654c-istio-data\") pod \"e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx\" (UID: \"56e83def-04be-4211-adb9-ca44eb58654c\") " pod="openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx"
Jul 30 15:23:01.727946 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:01.727887    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fw8lk\" (UniqueName: \"kubernetes.io/projected/56e83def-04be-4211-adb9-ca44eb58654c-kube-api-access-fw8lk\") pod \"e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx\" (UID: \"56e83def-04be-4211-adb9-ca44eb58654c\") " pod="openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx"
Jul 30 15:23:01.728033 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:01.727972    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56e83def-04be-4211-adb9-ca44eb58654c-workload-socket\") pod \"e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx\" (UID: \"56e83def-04be-4211-adb9-ca44eb58654c\") " pod="openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx"
Jul 30 15:23:01.728033 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:01.727996    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56e83def-04be-4211-adb9-ca44eb58654c-istio-token\") pod \"e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx\" (UID: \"56e83def-04be-4211-adb9-ca44eb58654c\") " pod="openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx"
Jul 30 15:23:01.728124 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:01.728033    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56e83def-04be-4211-adb9-ca44eb58654c-istio-podinfo\") pod \"e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx\" (UID: \"56e83def-04be-4211-adb9-ca44eb58654c\") " pod="openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx"
Jul 30 15:23:01.728124 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:01.728076    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56e83def-04be-4211-adb9-ca44eb58654c-workload-certs\") pod \"e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx\" (UID: \"56e83def-04be-4211-adb9-ca44eb58654c\") " pod="openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx"
Jul 30 15:23:01.829128 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:01.829093    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56e83def-04be-4211-adb9-ca44eb58654c-credential-socket\") pod \"e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx\" (UID: \"56e83def-04be-4211-adb9-ca44eb58654c\") " pod="openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx"
Jul 30 15:23:01.829325 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:01.829200    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56e83def-04be-4211-adb9-ca44eb58654c-istio-data\") pod \"e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx\" (UID: \"56e83def-04be-4211-adb9-ca44eb58654c\") " pod="openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx"
Jul 30 15:23:01.829325 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:01.829222    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fw8lk\" (UniqueName: \"kubernetes.io/projected/56e83def-04be-4211-adb9-ca44eb58654c-kube-api-access-fw8lk\") pod \"e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx\" (UID: \"56e83def-04be-4211-adb9-ca44eb58654c\") " pod="openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx"
Jul 30 15:23:01.829400 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:01.829362    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56e83def-04be-4211-adb9-ca44eb58654c-workload-socket\") pod \"e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx\" (UID: \"56e83def-04be-4211-adb9-ca44eb58654c\") " pod="openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx"
Jul 30 15:23:01.829400 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:01.829392    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56e83def-04be-4211-adb9-ca44eb58654c-istio-token\") pod \"e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx\" (UID: \"56e83def-04be-4211-adb9-ca44eb58654c\") " pod="openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx"
Jul 30 15:23:01.829494 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:01.829430    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56e83def-04be-4211-adb9-ca44eb58654c-istio-podinfo\") pod \"e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx\" (UID: \"56e83def-04be-4211-adb9-ca44eb58654c\") " pod="openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx"
Jul 30 15:23:01.829494 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:01.829464    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56e83def-04be-4211-adb9-ca44eb58654c-workload-certs\") pod \"e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx\" (UID: \"56e83def-04be-4211-adb9-ca44eb58654c\") " pod="openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx"
Jul 30 15:23:01.829494 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:01.829487    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56e83def-04be-4211-adb9-ca44eb58654c-istio-envoy\") pod \"e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx\" (UID: \"56e83def-04be-4211-adb9-ca44eb58654c\") " pod="openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx"
Jul 30 15:23:01.829649 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:01.829509    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56e83def-04be-4211-adb9-ca44eb58654c-istiod-ca-cert\") pod \"e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx\" (UID: \"56e83def-04be-4211-adb9-ca44eb58654c\") " pod="openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx"
Jul 30 15:23:01.829649 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:01.829619    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56e83def-04be-4211-adb9-ca44eb58654c-istio-data\") pod \"e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx\" (UID: \"56e83def-04be-4211-adb9-ca44eb58654c\") " pod="openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx"
Jul 30 15:23:01.829768 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:01.829560    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56e83def-04be-4211-adb9-ca44eb58654c-credential-socket\") pod \"e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx\" (UID: \"56e83def-04be-4211-adb9-ca44eb58654c\") " pod="openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx"
Jul 30 15:23:01.829768 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:01.829721    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56e83def-04be-4211-adb9-ca44eb58654c-workload-socket\") pod \"e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx\" (UID: \"56e83def-04be-4211-adb9-ca44eb58654c\") " pod="openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx"
Jul 30 15:23:01.829876 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:01.829823    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56e83def-04be-4211-adb9-ca44eb58654c-workload-certs\") pod \"e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx\" (UID: \"56e83def-04be-4211-adb9-ca44eb58654c\") " pod="openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx"
Jul 30 15:23:01.830711 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:01.830677    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56e83def-04be-4211-adb9-ca44eb58654c-istiod-ca-cert\") pod \"e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx\" (UID: \"56e83def-04be-4211-adb9-ca44eb58654c\") " pod="openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx"
Jul 30 15:23:01.831892 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:01.831873    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56e83def-04be-4211-adb9-ca44eb58654c-istio-envoy\") pod \"e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx\" (UID: \"56e83def-04be-4211-adb9-ca44eb58654c\") " pod="openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx"
Jul 30 15:23:01.832108 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:01.832091    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56e83def-04be-4211-adb9-ca44eb58654c-istio-podinfo\") pod \"e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx\" (UID: \"56e83def-04be-4211-adb9-ca44eb58654c\") " pod="openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx"
Jul 30 15:23:01.836647 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:01.836626    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56e83def-04be-4211-adb9-ca44eb58654c-istio-token\") pod \"e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx\" (UID: \"56e83def-04be-4211-adb9-ca44eb58654c\") " pod="openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx"
Jul 30 15:23:01.836794 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:01.836777    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fw8lk\" (UniqueName: \"kubernetes.io/projected/56e83def-04be-4211-adb9-ca44eb58654c-kube-api-access-fw8lk\") pod \"e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx\" (UID: \"56e83def-04be-4211-adb9-ca44eb58654c\") " pod="openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx"
Jul 30 15:23:01.939796 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:01.939712    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx"
Jul 30 15:23:02.079476 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:02.079448    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx"]
Jul 30 15:23:02.081172 ip-10-0-133-117 kubenswrapper[2578]: W0730 15:23:02.081123    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56e83def_04be_4211_adb9_ca44eb58654c.slice/crio-9a5ee474fa82daf26d0b4a9621bb862dc241c1fb593eddeeb2bae973b7d521b3 WatchSource:0}: Error finding container 9a5ee474fa82daf26d0b4a9621bb862dc241c1fb593eddeeb2bae973b7d521b3: Status 404 returned error can't find the container with id 9a5ee474fa82daf26d0b4a9621bb862dc241c1fb593eddeeb2bae973b7d521b3
Jul 30 15:23:02.083483 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:02.083450    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:23:02.083542 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:02.083516    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:23:02.083586 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:02.083549    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:23:02.443128 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:02.443091    2578 patch_prober.go:28] interesting pod/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 30 15:23:02.443321 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:02.443185    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr" podUID="190c779f-555e-4c6b-9d2b-2261442c56e1" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 30 15:23:02.460495 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:02.460463    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx" event={"ID":"56e83def-04be-4211-adb9-ca44eb58654c","Type":"ContainerStarted","Data":"754b5ea589d4ed4ac7a4a685055c2d85aebd7b6f38b30d516ed4f4b47a526905"}
Jul 30 15:23:02.460629 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:02.460517    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx" event={"ID":"56e83def-04be-4211-adb9-ca44eb58654c","Type":"ContainerStarted","Data":"9a5ee474fa82daf26d0b4a9621bb862dc241c1fb593eddeeb2bae973b7d521b3"}
Jul 30 15:23:02.481453 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:02.481408    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx" podStartSLOduration=1.481394152 podStartE2EDuration="1.481394152s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:23:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:23:02.480253296 +0000 UTC m=+2094.009191782" watchObservedRunningTime="2026-07-30 15:23:02.481394152 +0000 UTC m=+2094.010332625"
Jul 30 15:23:02.940162 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:02.940111    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx"
Jul 30 15:23:03.944754 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:03.944718    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx"
Jul 30 15:23:04.471293 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:04.471265    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx"
Jul 30 15:23:04.472099 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:04.472080    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx"
Jul 30 15:23:04.910680 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:04.910646    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7ba3a489-658697c88-s7xl2"]
Jul 30 15:23:04.911100 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:04.911087    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="122ffe00-0f17-4101-ad04-f0a3ad41b35c" containerName="istio-proxy"
Jul 30 15:23:04.914376 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:04.914353    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7ba3a489-658697c88-s7xl2"
Jul 30 15:23:04.916178 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:04.916140    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-7ba3a489-serving-cert\""
Jul 30 15:23:04.925936 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:04.925907    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7ba3a489-658697c88-s7xl2"]
Jul 30 15:23:05.062702 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:05.062652    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05f00a28-d937-41d2-b175-7a8cfbfedafc-openshift-service-ca\") pod \"maas-api-e2e-derive-7ba3a489-658697c88-s7xl2\" (UID: \"05f00a28-d937-41d2-b175-7a8cfbfedafc\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7ba3a489-658697c88-s7xl2"
Jul 30 15:23:05.063112 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:05.062718    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fth4\" (UniqueName: \"kubernetes.io/projected/05f00a28-d937-41d2-b175-7a8cfbfedafc-kube-api-access-5fth4\") pod \"maas-api-e2e-derive-7ba3a489-658697c88-s7xl2\" (UID: \"05f00a28-d937-41d2-b175-7a8cfbfedafc\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7ba3a489-658697c88-s7xl2"
Jul 30 15:23:05.063112 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:05.062796    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05f00a28-d937-41d2-b175-7a8cfbfedafc-maas-api-tls\") pod \"maas-api-e2e-derive-7ba3a489-658697c88-s7xl2\" (UID: \"05f00a28-d937-41d2-b175-7a8cfbfedafc\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7ba3a489-658697c88-s7xl2"
Jul 30 15:23:05.165696 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:05.164382    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05f00a28-d937-41d2-b175-7a8cfbfedafc-maas-api-tls\") pod \"maas-api-e2e-derive-7ba3a489-658697c88-s7xl2\" (UID: \"05f00a28-d937-41d2-b175-7a8cfbfedafc\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7ba3a489-658697c88-s7xl2"
Jul 30 15:23:05.165696 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:05.164555    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05f00a28-d937-41d2-b175-7a8cfbfedafc-openshift-service-ca\") pod \"maas-api-e2e-derive-7ba3a489-658697c88-s7xl2\" (UID: \"05f00a28-d937-41d2-b175-7a8cfbfedafc\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7ba3a489-658697c88-s7xl2"
Jul 30 15:23:05.165696 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:05.164603    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fth4\" (UniqueName: \"kubernetes.io/projected/05f00a28-d937-41d2-b175-7a8cfbfedafc-kube-api-access-5fth4\") pod \"maas-api-e2e-derive-7ba3a489-658697c88-s7xl2\" (UID: \"05f00a28-d937-41d2-b175-7a8cfbfedafc\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7ba3a489-658697c88-s7xl2"
Jul 30 15:23:05.165696 ip-10-0-133-117 kubenswrapper[2578]: E0730 15:23:05.165087    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-7ba3a489-serving-cert: secret "maas-api-e2e-derive-7ba3a489-serving-cert" not found
Jul 30 15:23:05.165696 ip-10-0-133-117 kubenswrapper[2578]: E0730 15:23:05.165180    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05f00a28-d937-41d2-b175-7a8cfbfedafc-maas-api-tls podName:05f00a28-d937-41d2-b175-7a8cfbfedafc nodeName:}" failed. No retries permitted until 2026-07-30 15:23:05.665137601 +0000 UTC m=+2097.194076054 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/05f00a28-d937-41d2-b175-7a8cfbfedafc-maas-api-tls") pod "maas-api-e2e-derive-7ba3a489-658697c88-s7xl2" (UID: "05f00a28-d937-41d2-b175-7a8cfbfedafc") : secret "maas-api-e2e-derive-7ba3a489-serving-cert" not found
Jul 30 15:23:05.168739 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:05.166811    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05f00a28-d937-41d2-b175-7a8cfbfedafc-openshift-service-ca\") pod \"maas-api-e2e-derive-7ba3a489-658697c88-s7xl2\" (UID: \"05f00a28-d937-41d2-b175-7a8cfbfedafc\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7ba3a489-658697c88-s7xl2"
Jul 30 15:23:05.174133 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:05.174111    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fth4\" (UniqueName: \"kubernetes.io/projected/05f00a28-d937-41d2-b175-7a8cfbfedafc-kube-api-access-5fth4\") pod \"maas-api-e2e-derive-7ba3a489-658697c88-s7xl2\" (UID: \"05f00a28-d937-41d2-b175-7a8cfbfedafc\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7ba3a489-658697c88-s7xl2"
Jul 30 15:23:05.548360 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:05.548278    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-7ba3a489-c66996d9b-54n8g"]
Jul 30 15:23:05.552003 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:05.551985    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-7ba3a489-c66996d9b-54n8g"
Jul 30 15:23:05.553957 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:05.553932    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-7ba3a489\""
Jul 30 15:23:05.553957 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:05.553953    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-7ba3a489-dockercfg-pqnpq\""
Jul 30 15:23:05.561708 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:05.561680    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-7ba3a489-c66996d9b-54n8g"]
Jul 30 15:23:05.579966 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:05.579934    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-7ba3a489-8554d5d454-ljsln"]
Jul 30 15:23:05.583740 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:05.583724    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-7ba3a489-8554d5d454-ljsln"
Jul 30 15:23:05.592861 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:05.592833    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-7ba3a489-8554d5d454-ljsln"]
Jul 30 15:23:05.668554 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:05.668514    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05f00a28-d937-41d2-b175-7a8cfbfedafc-maas-api-tls\") pod \"maas-api-e2e-derive-7ba3a489-658697c88-s7xl2\" (UID: \"05f00a28-d937-41d2-b175-7a8cfbfedafc\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7ba3a489-658697c88-s7xl2"
Jul 30 15:23:05.668711 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:05.668591    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5513da55-7e86-4521-abcb-fcccd40cd186-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-7ba3a489-c66996d9b-54n8g\" (UID: \"5513da55-7e86-4521-abcb-fcccd40cd186\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-7ba3a489-c66996d9b-54n8g"
Jul 30 15:23:05.668711 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:05.668645    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fb6388bb-353b-4131-9c51-74a47cf8a61a-plugins-config-volume\") pod \"payload-processing-e2e-derive-7ba3a489-8554d5d454-ljsln\" (UID: \"fb6388bb-353b-4131-9c51-74a47cf8a61a\") " pod="openshift-ingress/payload-processing-e2e-derive-7ba3a489-8554d5d454-ljsln"
Jul 30 15:23:05.668807 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:05.668761    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnxv2\" (UniqueName: \"kubernetes.io/projected/5513da55-7e86-4521-abcb-fcccd40cd186-kube-api-access-jnxv2\") pod \"payload-pre-processing-e2e-derive-7ba3a489-c66996d9b-54n8g\" (UID: \"5513da55-7e86-4521-abcb-fcccd40cd186\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-7ba3a489-c66996d9b-54n8g"
Jul 30 15:23:05.668807 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:05.668798    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kg4lt\" (UniqueName: \"kubernetes.io/projected/fb6388bb-353b-4131-9c51-74a47cf8a61a-kube-api-access-kg4lt\") pod \"payload-processing-e2e-derive-7ba3a489-8554d5d454-ljsln\" (UID: \"fb6388bb-353b-4131-9c51-74a47cf8a61a\") " pod="openshift-ingress/payload-processing-e2e-derive-7ba3a489-8554d5d454-ljsln"
Jul 30 15:23:05.671013 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:05.670985    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05f00a28-d937-41d2-b175-7a8cfbfedafc-maas-api-tls\") pod \"maas-api-e2e-derive-7ba3a489-658697c88-s7xl2\" (UID: \"05f00a28-d937-41d2-b175-7a8cfbfedafc\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7ba3a489-658697c88-s7xl2"
Jul 30 15:23:05.770061 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:05.770026    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnxv2\" (UniqueName: \"kubernetes.io/projected/5513da55-7e86-4521-abcb-fcccd40cd186-kube-api-access-jnxv2\") pod \"payload-pre-processing-e2e-derive-7ba3a489-c66996d9b-54n8g\" (UID: \"5513da55-7e86-4521-abcb-fcccd40cd186\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-7ba3a489-c66996d9b-54n8g"
Jul 30 15:23:05.770264 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:05.770071    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kg4lt\" (UniqueName: \"kubernetes.io/projected/fb6388bb-353b-4131-9c51-74a47cf8a61a-kube-api-access-kg4lt\") pod \"payload-processing-e2e-derive-7ba3a489-8554d5d454-ljsln\" (UID: \"fb6388bb-353b-4131-9c51-74a47cf8a61a\") " pod="openshift-ingress/payload-processing-e2e-derive-7ba3a489-8554d5d454-ljsln"
Jul 30 15:23:05.770264 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:05.770177    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5513da55-7e86-4521-abcb-fcccd40cd186-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-7ba3a489-c66996d9b-54n8g\" (UID: \"5513da55-7e86-4521-abcb-fcccd40cd186\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-7ba3a489-c66996d9b-54n8g"
Jul 30 15:23:05.770264 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:05.770219    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fb6388bb-353b-4131-9c51-74a47cf8a61a-plugins-config-volume\") pod \"payload-processing-e2e-derive-7ba3a489-8554d5d454-ljsln\" (UID: \"fb6388bb-353b-4131-9c51-74a47cf8a61a\") " pod="openshift-ingress/payload-processing-e2e-derive-7ba3a489-8554d5d454-ljsln"
Jul 30 15:23:05.770895 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:05.770871    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fb6388bb-353b-4131-9c51-74a47cf8a61a-plugins-config-volume\") pod \"payload-processing-e2e-derive-7ba3a489-8554d5d454-ljsln\" (UID: \"fb6388bb-353b-4131-9c51-74a47cf8a61a\") " pod="openshift-ingress/payload-processing-e2e-derive-7ba3a489-8554d5d454-ljsln"
Jul 30 15:23:05.770982 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:05.770897    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5513da55-7e86-4521-abcb-fcccd40cd186-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-7ba3a489-c66996d9b-54n8g\" (UID: \"5513da55-7e86-4521-abcb-fcccd40cd186\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-7ba3a489-c66996d9b-54n8g"
Jul 30 15:23:05.777568 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:05.777548    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnxv2\" (UniqueName: \"kubernetes.io/projected/5513da55-7e86-4521-abcb-fcccd40cd186-kube-api-access-jnxv2\") pod \"payload-pre-processing-e2e-derive-7ba3a489-c66996d9b-54n8g\" (UID: \"5513da55-7e86-4521-abcb-fcccd40cd186\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-7ba3a489-c66996d9b-54n8g"
Jul 30 15:23:05.778003 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:05.777988    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kg4lt\" (UniqueName: \"kubernetes.io/projected/fb6388bb-353b-4131-9c51-74a47cf8a61a-kube-api-access-kg4lt\") pod \"payload-processing-e2e-derive-7ba3a489-8554d5d454-ljsln\" (UID: \"fb6388bb-353b-4131-9c51-74a47cf8a61a\") " pod="openshift-ingress/payload-processing-e2e-derive-7ba3a489-8554d5d454-ljsln"
Jul 30 15:23:05.828860 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:05.828835    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7ba3a489-658697c88-s7xl2"
Jul 30 15:23:05.863357 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:05.863329    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-7ba3a489-c66996d9b-54n8g"
Jul 30 15:23:05.895582 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:05.895550    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-7ba3a489-8554d5d454-ljsln"
Jul 30 15:23:05.986069 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:05.985976    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7ba3a489-658697c88-s7xl2"]
Jul 30 15:23:05.988030 ip-10-0-133-117 kubenswrapper[2578]: W0730 15:23:05.987941    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05f00a28_d937_41d2_b175_7a8cfbfedafc.slice/crio-12efc661db1e1523c53c0aeb1bc810575e03ed832c17627b31642c73fb22739c WatchSource:0}: Error finding container 12efc661db1e1523c53c0aeb1bc810575e03ed832c17627b31642c73fb22739c: Status 404 returned error can't find the container with id 12efc661db1e1523c53c0aeb1bc810575e03ed832c17627b31642c73fb22739c
Jul 30 15:23:06.120398 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:06.120371    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr"
Jul 30 15:23:06.232614 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:06.232589    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-7ba3a489-c66996d9b-54n8g"]
Jul 30 15:23:06.234581 ip-10-0-133-117 kubenswrapper[2578]: W0730 15:23:06.234546    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5513da55_7e86_4521_abcb_fcccd40cd186.slice/crio-bcea18ba740e7a96d21ac60e6e136be3b3000caa71be87af2b8e25b53716c589 WatchSource:0}: Error finding container bcea18ba740e7a96d21ac60e6e136be3b3000caa71be87af2b8e25b53716c589: Status 404 returned error can't find the container with id bcea18ba740e7a96d21ac60e6e136be3b3000caa71be87af2b8e25b53716c589
Jul 30 15:23:06.270132 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:06.270110    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-7ba3a489-8554d5d454-ljsln"]
Jul 30 15:23:06.274423 ip-10-0-133-117 kubenswrapper[2578]: W0730 15:23:06.274388    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb6388bb_353b_4131_9c51_74a47cf8a61a.slice/crio-7f537aeceffcb16d4325fe7f19433d5e6b2ce559aec49b3e54b6583f337db8bc WatchSource:0}: Error finding container 7f537aeceffcb16d4325fe7f19433d5e6b2ce559aec49b3e54b6583f337db8bc: Status 404 returned error can't find the container with id 7f537aeceffcb16d4325fe7f19433d5e6b2ce559aec49b3e54b6583f337db8bc
Jul 30 15:23:06.274668 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:06.274652    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/190c779f-555e-4c6b-9d2b-2261442c56e1-istio-podinfo\") pod \"190c779f-555e-4c6b-9d2b-2261442c56e1\" (UID: \"190c779f-555e-4c6b-9d2b-2261442c56e1\") "
Jul 30 15:23:06.274734 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:06.274697    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/190c779f-555e-4c6b-9d2b-2261442c56e1-credential-socket\") pod \"190c779f-555e-4c6b-9d2b-2261442c56e1\" (UID: \"190c779f-555e-4c6b-9d2b-2261442c56e1\") "
Jul 30 15:23:06.274734 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:06.274722    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/190c779f-555e-4c6b-9d2b-2261442c56e1-istio-envoy\") pod \"190c779f-555e-4c6b-9d2b-2261442c56e1\" (UID: \"190c779f-555e-4c6b-9d2b-2261442c56e1\") "
Jul 30 15:23:06.274841 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:06.274758    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/190c779f-555e-4c6b-9d2b-2261442c56e1-istio-token\") pod \"190c779f-555e-4c6b-9d2b-2261442c56e1\" (UID: \"190c779f-555e-4c6b-9d2b-2261442c56e1\") "
Jul 30 15:23:06.274841 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:06.274798    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/190c779f-555e-4c6b-9d2b-2261442c56e1-workload-socket\") pod \"190c779f-555e-4c6b-9d2b-2261442c56e1\" (UID: \"190c779f-555e-4c6b-9d2b-2261442c56e1\") "
Jul 30 15:23:06.274945 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:06.274839    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/190c779f-555e-4c6b-9d2b-2261442c56e1-istiod-ca-cert\") pod \"190c779f-555e-4c6b-9d2b-2261442c56e1\" (UID: \"190c779f-555e-4c6b-9d2b-2261442c56e1\") "
Jul 30 15:23:06.274945 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:06.274915    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/190c779f-555e-4c6b-9d2b-2261442c56e1-istio-data\") pod \"190c779f-555e-4c6b-9d2b-2261442c56e1\" (UID: \"190c779f-555e-4c6b-9d2b-2261442c56e1\") "
Jul 30 15:23:06.274945 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:06.274929    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/190c779f-555e-4c6b-9d2b-2261442c56e1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "190c779f-555e-4c6b-9d2b-2261442c56e1" (UID: "190c779f-555e-4c6b-9d2b-2261442c56e1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:23:06.275100 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:06.274978    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cv9qg\" (UniqueName: \"kubernetes.io/projected/190c779f-555e-4c6b-9d2b-2261442c56e1-kube-api-access-cv9qg\") pod \"190c779f-555e-4c6b-9d2b-2261442c56e1\" (UID: \"190c779f-555e-4c6b-9d2b-2261442c56e1\") "
Jul 30 15:23:06.275100 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:06.275035    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/190c779f-555e-4c6b-9d2b-2261442c56e1-workload-certs\") pod \"190c779f-555e-4c6b-9d2b-2261442c56e1\" (UID: \"190c779f-555e-4c6b-9d2b-2261442c56e1\") "
Jul 30 15:23:06.275713 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:06.275308    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/190c779f-555e-4c6b-9d2b-2261442c56e1-credential-socket\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:23:06.275713 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:06.275522    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/190c779f-555e-4c6b-9d2b-2261442c56e1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "190c779f-555e-4c6b-9d2b-2261442c56e1" (UID: "190c779f-555e-4c6b-9d2b-2261442c56e1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:23:06.275713 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:06.275530    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/190c779f-555e-4c6b-9d2b-2261442c56e1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "190c779f-555e-4c6b-9d2b-2261442c56e1" (UID: "190c779f-555e-4c6b-9d2b-2261442c56e1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:23:06.275713 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:06.275681    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/190c779f-555e-4c6b-9d2b-2261442c56e1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "190c779f-555e-4c6b-9d2b-2261442c56e1" (UID: "190c779f-555e-4c6b-9d2b-2261442c56e1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:23:06.277027 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:06.276872    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/190c779f-555e-4c6b-9d2b-2261442c56e1-istio-data" (OuterVolumeSpecName: "istio-data") pod "190c779f-555e-4c6b-9d2b-2261442c56e1" (UID: "190c779f-555e-4c6b-9d2b-2261442c56e1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:23:06.279012 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:06.278141    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/190c779f-555e-4c6b-9d2b-2261442c56e1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "190c779f-555e-4c6b-9d2b-2261442c56e1" (UID: "190c779f-555e-4c6b-9d2b-2261442c56e1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:23:06.279012 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:06.278703    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/190c779f-555e-4c6b-9d2b-2261442c56e1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "190c779f-555e-4c6b-9d2b-2261442c56e1" (UID: "190c779f-555e-4c6b-9d2b-2261442c56e1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 15:23:06.279012 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:06.278812    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/190c779f-555e-4c6b-9d2b-2261442c56e1-istio-token" (OuterVolumeSpecName: "istio-token") pod "190c779f-555e-4c6b-9d2b-2261442c56e1" (UID: "190c779f-555e-4c6b-9d2b-2261442c56e1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:23:06.279661 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:06.279637    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/190c779f-555e-4c6b-9d2b-2261442c56e1-kube-api-access-cv9qg" (OuterVolumeSpecName: "kube-api-access-cv9qg") pod "190c779f-555e-4c6b-9d2b-2261442c56e1" (UID: "190c779f-555e-4c6b-9d2b-2261442c56e1"). InnerVolumeSpecName "kube-api-access-cv9qg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:23:06.376132 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:06.376055    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/190c779f-555e-4c6b-9d2b-2261442c56e1-istio-data\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:23:06.376132 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:06.376089    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cv9qg\" (UniqueName: \"kubernetes.io/projected/190c779f-555e-4c6b-9d2b-2261442c56e1-kube-api-access-cv9qg\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:23:06.376132 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:06.376100    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/190c779f-555e-4c6b-9d2b-2261442c56e1-workload-certs\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:23:06.376132 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:06.376108    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/190c779f-555e-4c6b-9d2b-2261442c56e1-istio-podinfo\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:23:06.376132 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:06.376116    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/190c779f-555e-4c6b-9d2b-2261442c56e1-istio-envoy\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:23:06.376132 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:06.376124    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/190c779f-555e-4c6b-9d2b-2261442c56e1-istio-token\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:23:06.376132 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:06.376132    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/190c779f-555e-4c6b-9d2b-2261442c56e1-workload-socket\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:23:06.376132 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:06.376139    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/190c779f-555e-4c6b-9d2b-2261442c56e1-istiod-ca-cert\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:23:06.482260 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:06.482218    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-7ba3a489-c66996d9b-54n8g" event={"ID":"5513da55-7e86-4521-abcb-fcccd40cd186","Type":"ContainerStarted","Data":"6f86c48b87463a20c403f1a740c397344250bc3b60f55efbbc9b6dd3a1187128"}
Jul 30 15:23:06.482446 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:06.482282    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-7ba3a489-c66996d9b-54n8g" event={"ID":"5513da55-7e86-4521-abcb-fcccd40cd186","Type":"ContainerStarted","Data":"bcea18ba740e7a96d21ac60e6e136be3b3000caa71be87af2b8e25b53716c589"}
Jul 30 15:23:06.482446 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:06.482404    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-7ba3a489-c66996d9b-54n8g"
Jul 30 15:23:06.483938 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:06.483913    2578 generic.go:356] "Generic (PLEG): container finished" podID="190c779f-555e-4c6b-9d2b-2261442c56e1" containerID="91ee894b01ec38d50fbd69827310d56427baa605110a1d1eadd992324e66c7b2" exitCode=0
Jul 30 15:23:06.484052 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:06.483956    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr"
Jul 30 15:23:06.484052 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:06.483994    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr" event={"ID":"190c779f-555e-4c6b-9d2b-2261442c56e1","Type":"ContainerDied","Data":"91ee894b01ec38d50fbd69827310d56427baa605110a1d1eadd992324e66c7b2"}
Jul 30 15:23:06.484052 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:06.484041    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr" event={"ID":"190c779f-555e-4c6b-9d2b-2261442c56e1","Type":"ContainerDied","Data":"9c538584a229b68bcbb1c35ca0378fc2dad0c14fffa96bc2fc8359370b2ecbfd"}
Jul 30 15:23:06.484208 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:06.484073    2578 scope.go:117] "RemoveContainer" containerID="91ee894b01ec38d50fbd69827310d56427baa605110a1d1eadd992324e66c7b2"
Jul 30 15:23:06.485814 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:06.485789    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-7ba3a489-8554d5d454-ljsln" event={"ID":"fb6388bb-353b-4131-9c51-74a47cf8a61a","Type":"ContainerStarted","Data":"280bde81ccc96a9caa0afdf3ba6920d452bc31da36a7a9470a8114795cbbfd92"}
Jul 30 15:23:06.485937 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:06.485836    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-7ba3a489-8554d5d454-ljsln" event={"ID":"fb6388bb-353b-4131-9c51-74a47cf8a61a","Type":"ContainerStarted","Data":"7f537aeceffcb16d4325fe7f19433d5e6b2ce559aec49b3e54b6583f337db8bc"}
Jul 30 15:23:06.485937 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:06.485891    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-7ba3a489-8554d5d454-ljsln"
Jul 30 15:23:06.487290 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:06.487137    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7ba3a489-658697c88-s7xl2" event={"ID":"05f00a28-d937-41d2-b175-7a8cfbfedafc","Type":"ContainerStarted","Data":"12efc661db1e1523c53c0aeb1bc810575e03ed832c17627b31642c73fb22739c"}
Jul 30 15:23:06.493857 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:06.493836    2578 scope.go:117] "RemoveContainer" containerID="e53e7b3647645b68e6a8c3bd4653c8f8c864f4dc45934851167110442c0f29f0"
Jul 30 15:23:06.498239 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:06.498184    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-7ba3a489-c66996d9b-54n8g" podStartSLOduration=1.498131027 podStartE2EDuration="1.498131027s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:23:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:23:06.495642431 +0000 UTC m=+2098.024580906" watchObservedRunningTime="2026-07-30 15:23:06.498131027 +0000 UTC m=+2098.027069556"
Jul 30 15:23:06.502744 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:06.502723    2578 scope.go:117] "RemoveContainer" containerID="91ee894b01ec38d50fbd69827310d56427baa605110a1d1eadd992324e66c7b2"
Jul 30 15:23:06.503037 ip-10-0-133-117 kubenswrapper[2578]: E0730 15:23:06.503013    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91ee894b01ec38d50fbd69827310d56427baa605110a1d1eadd992324e66c7b2\": container with ID starting with 91ee894b01ec38d50fbd69827310d56427baa605110a1d1eadd992324e66c7b2 not found: ID does not exist" containerID="91ee894b01ec38d50fbd69827310d56427baa605110a1d1eadd992324e66c7b2"
Jul 30 15:23:06.503115 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:06.503050    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91ee894b01ec38d50fbd69827310d56427baa605110a1d1eadd992324e66c7b2"} err="failed to get container status \"91ee894b01ec38d50fbd69827310d56427baa605110a1d1eadd992324e66c7b2\": rpc error: code = NotFound desc = could not find container \"91ee894b01ec38d50fbd69827310d56427baa605110a1d1eadd992324e66c7b2\": container with ID starting with 91ee894b01ec38d50fbd69827310d56427baa605110a1d1eadd992324e66c7b2 not found: ID does not exist"
Jul 30 15:23:06.503115 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:06.503072    2578 scope.go:117] "RemoveContainer" containerID="e53e7b3647645b68e6a8c3bd4653c8f8c864f4dc45934851167110442c0f29f0"
Jul 30 15:23:06.503469 ip-10-0-133-117 kubenswrapper[2578]: E0730 15:23:06.503451    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e53e7b3647645b68e6a8c3bd4653c8f8c864f4dc45934851167110442c0f29f0\": container with ID starting with e53e7b3647645b68e6a8c3bd4653c8f8c864f4dc45934851167110442c0f29f0 not found: ID does not exist" containerID="e53e7b3647645b68e6a8c3bd4653c8f8c864f4dc45934851167110442c0f29f0"
Jul 30 15:23:06.503533 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:06.503473    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e53e7b3647645b68e6a8c3bd4653c8f8c864f4dc45934851167110442c0f29f0"} err="failed to get container status \"e53e7b3647645b68e6a8c3bd4653c8f8c864f4dc45934851167110442c0f29f0\": rpc error: code = NotFound desc = could not find container \"e53e7b3647645b68e6a8c3bd4653c8f8c864f4dc45934851167110442c0f29f0\": container with ID starting with e53e7b3647645b68e6a8c3bd4653c8f8c864f4dc45934851167110442c0f29f0 not found: ID does not exist"
Jul 30 15:23:06.511095 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:06.511050    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-7ba3a489-8554d5d454-ljsln" podStartSLOduration=1.5110362670000002 podStartE2EDuration="1.511036267s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:23:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:23:06.5091182 +0000 UTC m=+2098.038056695" watchObservedRunningTime="2026-07-30 15:23:06.511036267 +0000 UTC m=+2098.039974740"
Jul 30 15:23:06.522808 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:06.522782    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr"]
Jul 30 15:23:06.528214 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:06.528190    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-73521211-openshift-default-549976f7f6-xj8nr"]
Jul 30 15:23:06.980302 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:06.980271    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="190c779f-555e-4c6b-9d2b-2261442c56e1" path="/var/lib/kubelet/pods/190c779f-555e-4c6b-9d2b-2261442c56e1/volumes"
Jul 30 15:23:07.494586 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:07.494549    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7ba3a489-658697c88-s7xl2" event={"ID":"05f00a28-d937-41d2-b175-7a8cfbfedafc","Type":"ContainerStarted","Data":"c80794218599793a6ec7bfc364dde3bec20bf7e6bc24df792fee24fe250f4351"}
Jul 30 15:23:07.495241 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:07.494860    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7ba3a489-658697c88-s7xl2"
Jul 30 15:23:07.514827 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:07.514751    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7ba3a489-658697c88-s7xl2" podStartSLOduration=2.692701186 podStartE2EDuration="3.514736089s" totalImagesPullingTime="822.034903ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:23:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:23:07.512458135 +0000 UTC m=+2099.041396609" watchObservedRunningTime="2026-07-30 15:23:07.514736089 +0000 UTC m=+2099.043674563"
Jul 30 15:23:09.059518 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:09.059483    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4mjmp_de241220-5f27-4b0b-b7c6-6d4b350e24e1/console-operator/1.log"
Jul 30 15:23:09.065703 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:09.065681    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4mjmp_de241220-5f27-4b0b-b7c6-6d4b350e24e1/console-operator/1.log"
Jul 30 15:23:13.505365 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:13.505337    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7ba3a489-658697c88-s7xl2"
Jul 30 15:23:17.496962 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:17.496927    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-7ba3a489-8554d5d454-ljsln"
Jul 30 15:23:17.497455 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:17.497428    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-7ba3a489-c66996d9b-54n8g"
Jul 30 15:23:19.472861 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:19.472822    2578 patch_prober.go:28] interesting pod/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 30 15:23:19.473309 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:19.472881    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx" podUID="56e83def-04be-4211-adb9-ca44eb58654c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 30 15:23:19.555535 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:19.555505    2578 generic.go:356] "Generic (PLEG): container finished" podID="56e83def-04be-4211-adb9-ca44eb58654c" containerID="754b5ea589d4ed4ac7a4a685055c2d85aebd7b6f38b30d516ed4f4b47a526905" exitCode=0
Jul 30 15:23:19.555703 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:19.555579    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx" event={"ID":"56e83def-04be-4211-adb9-ca44eb58654c","Type":"ContainerDied","Data":"754b5ea589d4ed4ac7a4a685055c2d85aebd7b6f38b30d516ed4f4b47a526905"}
Jul 30 15:23:19.555989 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:19.555974    2578 scope.go:117] "RemoveContainer" containerID="754b5ea589d4ed4ac7a4a685055c2d85aebd7b6f38b30d516ed4f4b47a526905"
Jul 30 15:23:19.556959 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:19.556933    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:23:19.557008 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:19.556997    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:23:19.557065 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:19.557053    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:23:20.562266 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:20.562232    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx" event={"ID":"56e83def-04be-4211-adb9-ca44eb58654c","Type":"ContainerStarted","Data":"dc951eda3ced81fea302e452d62c9e4f68cfb38f210bae42ac932c4e090f61aa"}
Jul 30 15:23:20.940247 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:20.940133    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx"
Jul 30 15:23:20.940435 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:20.940412    2578 patch_prober.go:28] interesting pod/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 30 15:23:20.940512 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:20.940492    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx" podUID="56e83def-04be-4211-adb9-ca44eb58654c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 30 15:23:21.940713 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:21.940662    2578 patch_prober.go:28] interesting pod/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 30 15:23:21.941285 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:21.940735    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx" podUID="56e83def-04be-4211-adb9-ca44eb58654c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 30 15:23:22.940766 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:22.940725    2578 patch_prober.go:28] interesting pod/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 30 15:23:22.941606 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:22.941574    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx" podUID="56e83def-04be-4211-adb9-ca44eb58654c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 30 15:23:23.940927 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:23.940880    2578 patch_prober.go:28] interesting pod/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 30 15:23:23.941457 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:23.940949    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx" podUID="56e83def-04be-4211-adb9-ca44eb58654c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 30 15:23:24.940579 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:24.940532    2578 patch_prober.go:28] interesting pod/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 30 15:23:24.940795 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:24.940607    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx" podUID="56e83def-04be-4211-adb9-ca44eb58654c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 30 15:23:25.944344 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:25.944307    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx"
Jul 30 15:23:25.944734 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:25.944668    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx"
Jul 30 15:23:25.945312 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:25.945293    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx"
Jul 30 15:23:35.463230 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:35.463196    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7ba3a489-658697c88-s7xl2"]
Jul 30 15:23:35.463889 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:35.463414    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7ba3a489-658697c88-s7xl2" podUID="05f00a28-d937-41d2-b175-7a8cfbfedafc" containerName="maas-api" containerID="cri-o://c80794218599793a6ec7bfc364dde3bec20bf7e6bc24df792fee24fe250f4351" gracePeriod=30
Jul 30 15:23:35.521005 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:35.520970    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-7ba3a489-8554d5d454-ljsln"]
Jul 30 15:23:35.521280 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:35.521252    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-7ba3a489-8554d5d454-ljsln" podUID="fb6388bb-353b-4131-9c51-74a47cf8a61a" containerName="payload-processing" containerID="cri-o://280bde81ccc96a9caa0afdf3ba6920d452bc31da36a7a9470a8114795cbbfd92" gracePeriod=30
Jul 30 15:23:35.530001 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:35.529972    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-7ba3a489-c66996d9b-54n8g"]
Jul 30 15:23:35.530245 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:35.530213    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-7ba3a489-c66996d9b-54n8g" podUID="5513da55-7e86-4521-abcb-fcccd40cd186" containerName="payload-pre-processing" containerID="cri-o://6f86c48b87463a20c403f1a740c397344250bc3b60f55efbbc9b6dd3a1187128" gracePeriod=30
Jul 30 15:23:35.635222 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:35.635194    2578 generic.go:356] "Generic (PLEG): container finished" podID="05f00a28-d937-41d2-b175-7a8cfbfedafc" containerID="c80794218599793a6ec7bfc364dde3bec20bf7e6bc24df792fee24fe250f4351" exitCode=0
Jul 30 15:23:35.635386 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:35.635272    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7ba3a489-658697c88-s7xl2" event={"ID":"05f00a28-d937-41d2-b175-7a8cfbfedafc","Type":"ContainerDied","Data":"c80794218599793a6ec7bfc364dde3bec20bf7e6bc24df792fee24fe250f4351"}
Jul 30 15:23:35.644124 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:35.644094    2578 generic.go:356] "Generic (PLEG): container finished" podID="fb6388bb-353b-4131-9c51-74a47cf8a61a" containerID="280bde81ccc96a9caa0afdf3ba6920d452bc31da36a7a9470a8114795cbbfd92" exitCode=0
Jul 30 15:23:35.644349 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:35.644132    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-7ba3a489-8554d5d454-ljsln" event={"ID":"fb6388bb-353b-4131-9c51-74a47cf8a61a","Type":"ContainerDied","Data":"280bde81ccc96a9caa0afdf3ba6920d452bc31da36a7a9470a8114795cbbfd92"}
Jul 30 15:23:35.820373 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:35.820333    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-7ba3a489-8554d5d454-ljsln"
Jul 30 15:23:35.827749 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:35.827716    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx"]
Jul 30 15:23:35.828007 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:35.827958    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx" podUID="56e83def-04be-4211-adb9-ca44eb58654c" containerName="istio-proxy" containerID="cri-o://dc951eda3ced81fea302e452d62c9e4f68cfb38f210bae42ac932c4e090f61aa" gracePeriod=30
Jul 30 15:23:35.834529 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:35.834507    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7ba3a489-658697c88-s7xl2"
Jul 30 15:23:35.846827 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:35.846806    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-7ba3a489-c66996d9b-54n8g"
Jul 30 15:23:35.872416 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:35.872390    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fb6388bb-353b-4131-9c51-74a47cf8a61a-plugins-config-volume\") pod \"fb6388bb-353b-4131-9c51-74a47cf8a61a\" (UID: \"fb6388bb-353b-4131-9c51-74a47cf8a61a\") "
Jul 30 15:23:35.872568 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:35.872437    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kg4lt\" (UniqueName: \"kubernetes.io/projected/fb6388bb-353b-4131-9c51-74a47cf8a61a-kube-api-access-kg4lt\") pod \"fb6388bb-353b-4131-9c51-74a47cf8a61a\" (UID: \"fb6388bb-353b-4131-9c51-74a47cf8a61a\") "
Jul 30 15:23:35.872734 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:35.872709    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb6388bb-353b-4131-9c51-74a47cf8a61a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fb6388bb-353b-4131-9c51-74a47cf8a61a" (UID: "fb6388bb-353b-4131-9c51-74a47cf8a61a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:23:35.874426 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:35.874400    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb6388bb-353b-4131-9c51-74a47cf8a61a-kube-api-access-kg4lt" (OuterVolumeSpecName: "kube-api-access-kg4lt") pod "fb6388bb-353b-4131-9c51-74a47cf8a61a" (UID: "fb6388bb-353b-4131-9c51-74a47cf8a61a"). InnerVolumeSpecName "kube-api-access-kg4lt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:23:35.972921 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:35.972883    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jnxv2\" (UniqueName: \"kubernetes.io/projected/5513da55-7e86-4521-abcb-fcccd40cd186-kube-api-access-jnxv2\") pod \"5513da55-7e86-4521-abcb-fcccd40cd186\" (UID: \"5513da55-7e86-4521-abcb-fcccd40cd186\") "
Jul 30 15:23:35.973089 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:35.972943    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5513da55-7e86-4521-abcb-fcccd40cd186-plugins-config-volume\") pod \"5513da55-7e86-4521-abcb-fcccd40cd186\" (UID: \"5513da55-7e86-4521-abcb-fcccd40cd186\") "
Jul 30 15:23:35.973089 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:35.972990    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5fth4\" (UniqueName: \"kubernetes.io/projected/05f00a28-d937-41d2-b175-7a8cfbfedafc-kube-api-access-5fth4\") pod \"05f00a28-d937-41d2-b175-7a8cfbfedafc\" (UID: \"05f00a28-d937-41d2-b175-7a8cfbfedafc\") "
Jul 30 15:23:35.973089 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:35.973034    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05f00a28-d937-41d2-b175-7a8cfbfedafc-openshift-service-ca\") pod \"05f00a28-d937-41d2-b175-7a8cfbfedafc\" (UID: \"05f00a28-d937-41d2-b175-7a8cfbfedafc\") "
Jul 30 15:23:35.973089 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:35.973068    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05f00a28-d937-41d2-b175-7a8cfbfedafc-maas-api-tls\") pod \"05f00a28-d937-41d2-b175-7a8cfbfedafc\" (UID: \"05f00a28-d937-41d2-b175-7a8cfbfedafc\") "
Jul 30 15:23:35.973408 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:35.973385    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fb6388bb-353b-4131-9c51-74a47cf8a61a-plugins-config-volume\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:23:35.973486 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:35.973413    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kg4lt\" (UniqueName: \"kubernetes.io/projected/fb6388bb-353b-4131-9c51-74a47cf8a61a-kube-api-access-kg4lt\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:23:35.973486 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:35.973427    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5513da55-7e86-4521-abcb-fcccd40cd186-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5513da55-7e86-4521-abcb-fcccd40cd186" (UID: "5513da55-7e86-4521-abcb-fcccd40cd186"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:23:35.973486 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:35.973458    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05f00a28-d937-41d2-b175-7a8cfbfedafc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "05f00a28-d937-41d2-b175-7a8cfbfedafc" (UID: "05f00a28-d937-41d2-b175-7a8cfbfedafc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:23:35.975290 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:35.975265    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05f00a28-d937-41d2-b175-7a8cfbfedafc-kube-api-access-5fth4" (OuterVolumeSpecName: "kube-api-access-5fth4") pod "05f00a28-d937-41d2-b175-7a8cfbfedafc" (UID: "05f00a28-d937-41d2-b175-7a8cfbfedafc"). InnerVolumeSpecName "kube-api-access-5fth4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:23:35.975368 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:35.975265    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05f00a28-d937-41d2-b175-7a8cfbfedafc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "05f00a28-d937-41d2-b175-7a8cfbfedafc" (UID: "05f00a28-d937-41d2-b175-7a8cfbfedafc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 15:23:35.975368 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:35.975322    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5513da55-7e86-4521-abcb-fcccd40cd186-kube-api-access-jnxv2" (OuterVolumeSpecName: "kube-api-access-jnxv2") pod "5513da55-7e86-4521-abcb-fcccd40cd186" (UID: "5513da55-7e86-4521-abcb-fcccd40cd186"). InnerVolumeSpecName "kube-api-access-jnxv2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:23:36.074118 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:36.074083    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5513da55-7e86-4521-abcb-fcccd40cd186-plugins-config-volume\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:23:36.074118 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:36.074114    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5fth4\" (UniqueName: \"kubernetes.io/projected/05f00a28-d937-41d2-b175-7a8cfbfedafc-kube-api-access-5fth4\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:23:36.074323 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:36.074127    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05f00a28-d937-41d2-b175-7a8cfbfedafc-openshift-service-ca\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:23:36.074323 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:36.074139    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05f00a28-d937-41d2-b175-7a8cfbfedafc-maas-api-tls\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:23:36.074323 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:36.074165    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jnxv2\" (UniqueName: \"kubernetes.io/projected/5513da55-7e86-4521-abcb-fcccd40cd186-kube-api-access-jnxv2\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:23:36.651213 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:36.651176    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7ba3a489-658697c88-s7xl2" event={"ID":"05f00a28-d937-41d2-b175-7a8cfbfedafc","Type":"ContainerDied","Data":"12efc661db1e1523c53c0aeb1bc810575e03ed832c17627b31642c73fb22739c"}
Jul 30 15:23:36.651650 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:36.651218    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7ba3a489-658697c88-s7xl2"
Jul 30 15:23:36.651650 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:36.651243    2578 scope.go:117] "RemoveContainer" containerID="c80794218599793a6ec7bfc364dde3bec20bf7e6bc24df792fee24fe250f4351"
Jul 30 15:23:36.652560 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:36.652535    2578 generic.go:356] "Generic (PLEG): container finished" podID="5513da55-7e86-4521-abcb-fcccd40cd186" containerID="6f86c48b87463a20c403f1a740c397344250bc3b60f55efbbc9b6dd3a1187128" exitCode=0
Jul 30 15:23:36.652638 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:36.652570    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-7ba3a489-c66996d9b-54n8g"
Jul 30 15:23:36.652638 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:36.652618    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-7ba3a489-c66996d9b-54n8g" event={"ID":"5513da55-7e86-4521-abcb-fcccd40cd186","Type":"ContainerDied","Data":"6f86c48b87463a20c403f1a740c397344250bc3b60f55efbbc9b6dd3a1187128"}
Jul 30 15:23:36.652755 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:36.652669    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-7ba3a489-c66996d9b-54n8g" event={"ID":"5513da55-7e86-4521-abcb-fcccd40cd186","Type":"ContainerDied","Data":"bcea18ba740e7a96d21ac60e6e136be3b3000caa71be87af2b8e25b53716c589"}
Jul 30 15:23:36.654112 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:36.654084    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-7ba3a489-8554d5d454-ljsln" event={"ID":"fb6388bb-353b-4131-9c51-74a47cf8a61a","Type":"ContainerDied","Data":"7f537aeceffcb16d4325fe7f19433d5e6b2ce559aec49b3e54b6583f337db8bc"}
Jul 30 15:23:36.654233 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:36.654130    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-7ba3a489-8554d5d454-ljsln"
Jul 30 15:23:36.661702 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:36.661678    2578 scope.go:117] "RemoveContainer" containerID="6f86c48b87463a20c403f1a740c397344250bc3b60f55efbbc9b6dd3a1187128"
Jul 30 15:23:36.671612 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:36.671598    2578 scope.go:117] "RemoveContainer" containerID="6f86c48b87463a20c403f1a740c397344250bc3b60f55efbbc9b6dd3a1187128"
Jul 30 15:23:36.671860 ip-10-0-133-117 kubenswrapper[2578]: E0730 15:23:36.671842    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f86c48b87463a20c403f1a740c397344250bc3b60f55efbbc9b6dd3a1187128\": container with ID starting with 6f86c48b87463a20c403f1a740c397344250bc3b60f55efbbc9b6dd3a1187128 not found: ID does not exist" containerID="6f86c48b87463a20c403f1a740c397344250bc3b60f55efbbc9b6dd3a1187128"
Jul 30 15:23:36.671901 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:36.671869    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f86c48b87463a20c403f1a740c397344250bc3b60f55efbbc9b6dd3a1187128"} err="failed to get container status \"6f86c48b87463a20c403f1a740c397344250bc3b60f55efbbc9b6dd3a1187128\": rpc error: code = NotFound desc = could not find container \"6f86c48b87463a20c403f1a740c397344250bc3b60f55efbbc9b6dd3a1187128\": container with ID starting with 6f86c48b87463a20c403f1a740c397344250bc3b60f55efbbc9b6dd3a1187128 not found: ID does not exist"
Jul 30 15:23:36.671901 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:36.671887    2578 scope.go:117] "RemoveContainer" containerID="280bde81ccc96a9caa0afdf3ba6920d452bc31da36a7a9470a8114795cbbfd92"
Jul 30 15:23:36.679467 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:36.679446    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7ba3a489-658697c88-s7xl2"]
Jul 30 15:23:36.685694 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:36.685673    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7ba3a489-658697c88-s7xl2"]
Jul 30 15:23:36.698815 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:36.698795    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-7ba3a489-c66996d9b-54n8g"]
Jul 30 15:23:36.702723 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:36.702706    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-7ba3a489-c66996d9b-54n8g"]
Jul 30 15:23:36.714026 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:36.714004    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-7ba3a489-8554d5d454-ljsln"]
Jul 30 15:23:36.719879 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:36.719859    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-7ba3a489-8554d5d454-ljsln"]
Jul 30 15:23:36.978128 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:36.978098    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05f00a28-d937-41d2-b175-7a8cfbfedafc" path="/var/lib/kubelet/pods/05f00a28-d937-41d2-b175-7a8cfbfedafc/volumes"
Jul 30 15:23:36.978507 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:36.978493    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5513da55-7e86-4521-abcb-fcccd40cd186" path="/var/lib/kubelet/pods/5513da55-7e86-4521-abcb-fcccd40cd186/volumes"
Jul 30 15:23:36.979080 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:36.979060    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb6388bb-353b-4131-9c51-74a47cf8a61a" path="/var/lib/kubelet/pods/fb6388bb-353b-4131-9c51-74a47cf8a61a/volumes"
Jul 30 15:23:40.946678 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:40.946621    2578 patch_prober.go:28] interesting pod/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 30 15:23:40.946917 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:40.946685    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx" podUID="56e83def-04be-4211-adb9-ca44eb58654c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 30 15:23:41.083653 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:41.083631    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx"
Jul 30 15:23:41.219837 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:41.219753    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56e83def-04be-4211-adb9-ca44eb58654c-credential-socket\") pod \"56e83def-04be-4211-adb9-ca44eb58654c\" (UID: \"56e83def-04be-4211-adb9-ca44eb58654c\") "
Jul 30 15:23:41.219837 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:41.219811    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56e83def-04be-4211-adb9-ca44eb58654c-istio-token\") pod \"56e83def-04be-4211-adb9-ca44eb58654c\" (UID: \"56e83def-04be-4211-adb9-ca44eb58654c\") "
Jul 30 15:23:41.220027 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:41.219843    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56e83def-04be-4211-adb9-ca44eb58654c-istio-envoy\") pod \"56e83def-04be-4211-adb9-ca44eb58654c\" (UID: \"56e83def-04be-4211-adb9-ca44eb58654c\") "
Jul 30 15:23:41.220027 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:41.219886    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56e83def-04be-4211-adb9-ca44eb58654c-istiod-ca-cert\") pod \"56e83def-04be-4211-adb9-ca44eb58654c\" (UID: \"56e83def-04be-4211-adb9-ca44eb58654c\") "
Jul 30 15:23:41.220027 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:41.219921    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56e83def-04be-4211-adb9-ca44eb58654c-workload-socket\") pod \"56e83def-04be-4211-adb9-ca44eb58654c\" (UID: \"56e83def-04be-4211-adb9-ca44eb58654c\") "
Jul 30 15:23:41.220027 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:41.219950    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56e83def-04be-4211-adb9-ca44eb58654c-workload-certs\") pod \"56e83def-04be-4211-adb9-ca44eb58654c\" (UID: \"56e83def-04be-4211-adb9-ca44eb58654c\") "
Jul 30 15:23:41.220027 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:41.219983    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56e83def-04be-4211-adb9-ca44eb58654c-istio-data\") pod \"56e83def-04be-4211-adb9-ca44eb58654c\" (UID: \"56e83def-04be-4211-adb9-ca44eb58654c\") "
Jul 30 15:23:41.220027 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:41.220010    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fw8lk\" (UniqueName: \"kubernetes.io/projected/56e83def-04be-4211-adb9-ca44eb58654c-kube-api-access-fw8lk\") pod \"56e83def-04be-4211-adb9-ca44eb58654c\" (UID: \"56e83def-04be-4211-adb9-ca44eb58654c\") "
Jul 30 15:23:41.220268 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:41.220053    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56e83def-04be-4211-adb9-ca44eb58654c-istio-podinfo\") pod \"56e83def-04be-4211-adb9-ca44eb58654c\" (UID: \"56e83def-04be-4211-adb9-ca44eb58654c\") "
Jul 30 15:23:41.220268 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:41.220047    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56e83def-04be-4211-adb9-ca44eb58654c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "56e83def-04be-4211-adb9-ca44eb58654c" (UID: "56e83def-04be-4211-adb9-ca44eb58654c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:23:41.220384 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:41.220370    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56e83def-04be-4211-adb9-ca44eb58654c-credential-socket\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:23:41.220441 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:41.220392    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56e83def-04be-4211-adb9-ca44eb58654c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "56e83def-04be-4211-adb9-ca44eb58654c" (UID: "56e83def-04be-4211-adb9-ca44eb58654c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:23:41.220603 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:41.220579    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56e83def-04be-4211-adb9-ca44eb58654c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "56e83def-04be-4211-adb9-ca44eb58654c" (UID: "56e83def-04be-4211-adb9-ca44eb58654c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:23:41.220881 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:41.220856    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56e83def-04be-4211-adb9-ca44eb58654c-istio-data" (OuterVolumeSpecName: "istio-data") pod "56e83def-04be-4211-adb9-ca44eb58654c" (UID: "56e83def-04be-4211-adb9-ca44eb58654c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:23:41.220983 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:41.220927    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56e83def-04be-4211-adb9-ca44eb58654c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "56e83def-04be-4211-adb9-ca44eb58654c" (UID: "56e83def-04be-4211-adb9-ca44eb58654c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:23:41.222337 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:41.222314    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56e83def-04be-4211-adb9-ca44eb58654c-kube-api-access-fw8lk" (OuterVolumeSpecName: "kube-api-access-fw8lk") pod "56e83def-04be-4211-adb9-ca44eb58654c" (UID: "56e83def-04be-4211-adb9-ca44eb58654c"). InnerVolumeSpecName "kube-api-access-fw8lk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:23:41.222556 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:41.222530    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56e83def-04be-4211-adb9-ca44eb58654c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "56e83def-04be-4211-adb9-ca44eb58654c" (UID: "56e83def-04be-4211-adb9-ca44eb58654c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:23:41.222630 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:41.222587    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56e83def-04be-4211-adb9-ca44eb58654c-istio-token" (OuterVolumeSpecName: "istio-token") pod "56e83def-04be-4211-adb9-ca44eb58654c" (UID: "56e83def-04be-4211-adb9-ca44eb58654c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:23:41.222712 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:41.222696    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/56e83def-04be-4211-adb9-ca44eb58654c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "56e83def-04be-4211-adb9-ca44eb58654c" (UID: "56e83def-04be-4211-adb9-ca44eb58654c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 15:23:41.321506 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:41.321472    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56e83def-04be-4211-adb9-ca44eb58654c-istio-podinfo\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:23:41.321506 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:41.321501    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56e83def-04be-4211-adb9-ca44eb58654c-istio-token\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:23:41.321506 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:41.321510    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56e83def-04be-4211-adb9-ca44eb58654c-istio-envoy\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:23:41.321737 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:41.321518    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56e83def-04be-4211-adb9-ca44eb58654c-istiod-ca-cert\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:23:41.321737 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:41.321527    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56e83def-04be-4211-adb9-ca44eb58654c-workload-socket\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:23:41.321737 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:41.321535    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56e83def-04be-4211-adb9-ca44eb58654c-workload-certs\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:23:41.321737 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:41.321542    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56e83def-04be-4211-adb9-ca44eb58654c-istio-data\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:23:41.321737 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:41.321550    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fw8lk\" (UniqueName: \"kubernetes.io/projected/56e83def-04be-4211-adb9-ca44eb58654c-kube-api-access-fw8lk\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:23:41.682045 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:41.682015    2578 generic.go:356] "Generic (PLEG): container finished" podID="56e83def-04be-4211-adb9-ca44eb58654c" containerID="dc951eda3ced81fea302e452d62c9e4f68cfb38f210bae42ac932c4e090f61aa" exitCode=0
Jul 30 15:23:41.682265 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:41.682069    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx"
Jul 30 15:23:41.682265 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:41.682111    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx" event={"ID":"56e83def-04be-4211-adb9-ca44eb58654c","Type":"ContainerDied","Data":"dc951eda3ced81fea302e452d62c9e4f68cfb38f210bae42ac932c4e090f61aa"}
Jul 30 15:23:41.682265 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:41.682196    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx" event={"ID":"56e83def-04be-4211-adb9-ca44eb58654c","Type":"ContainerDied","Data":"9a5ee474fa82daf26d0b4a9621bb862dc241c1fb593eddeeb2bae973b7d521b3"}
Jul 30 15:23:41.682265 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:41.682242    2578 scope.go:117] "RemoveContainer" containerID="dc951eda3ced81fea302e452d62c9e4f68cfb38f210bae42ac932c4e090f61aa"
Jul 30 15:23:41.692192 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:41.692172    2578 scope.go:117] "RemoveContainer" containerID="754b5ea589d4ed4ac7a4a685055c2d85aebd7b6f38b30d516ed4f4b47a526905"
Jul 30 15:23:41.700045 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:41.700028    2578 scope.go:117] "RemoveContainer" containerID="dc951eda3ced81fea302e452d62c9e4f68cfb38f210bae42ac932c4e090f61aa"
Jul 30 15:23:41.700335 ip-10-0-133-117 kubenswrapper[2578]: E0730 15:23:41.700305    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc951eda3ced81fea302e452d62c9e4f68cfb38f210bae42ac932c4e090f61aa\": container with ID starting with dc951eda3ced81fea302e452d62c9e4f68cfb38f210bae42ac932c4e090f61aa not found: ID does not exist" containerID="dc951eda3ced81fea302e452d62c9e4f68cfb38f210bae42ac932c4e090f61aa"
Jul 30 15:23:41.700920 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:41.700345    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc951eda3ced81fea302e452d62c9e4f68cfb38f210bae42ac932c4e090f61aa"} err="failed to get container status \"dc951eda3ced81fea302e452d62c9e4f68cfb38f210bae42ac932c4e090f61aa\": rpc error: code = NotFound desc = could not find container \"dc951eda3ced81fea302e452d62c9e4f68cfb38f210bae42ac932c4e090f61aa\": container with ID starting with dc951eda3ced81fea302e452d62c9e4f68cfb38f210bae42ac932c4e090f61aa not found: ID does not exist"
Jul 30 15:23:41.700920 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:41.700477    2578 scope.go:117] "RemoveContainer" containerID="754b5ea589d4ed4ac7a4a685055c2d85aebd7b6f38b30d516ed4f4b47a526905"
Jul 30 15:23:41.700920 ip-10-0-133-117 kubenswrapper[2578]: E0730 15:23:41.700889    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"754b5ea589d4ed4ac7a4a685055c2d85aebd7b6f38b30d516ed4f4b47a526905\": container with ID starting with 754b5ea589d4ed4ac7a4a685055c2d85aebd7b6f38b30d516ed4f4b47a526905 not found: ID does not exist" containerID="754b5ea589d4ed4ac7a4a685055c2d85aebd7b6f38b30d516ed4f4b47a526905"
Jul 30 15:23:41.701101 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:41.700916    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"754b5ea589d4ed4ac7a4a685055c2d85aebd7b6f38b30d516ed4f4b47a526905"} err="failed to get container status \"754b5ea589d4ed4ac7a4a685055c2d85aebd7b6f38b30d516ed4f4b47a526905\": rpc error: code = NotFound desc = could not find container \"754b5ea589d4ed4ac7a4a685055c2d85aebd7b6f38b30d516ed4f4b47a526905\": container with ID starting with 754b5ea589d4ed4ac7a4a685055c2d85aebd7b6f38b30d516ed4f4b47a526905 not found: ID does not exist"
Jul 30 15:23:41.702645 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:41.702626    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx"]
Jul 30 15:23:41.707087 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:41.707067    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-7ba3a489-openshift-default-697cbb7784-mwsrx"]
Jul 30 15:23:42.979197 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:23:42.979138    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56e83def-04be-4211-adb9-ca44eb58654c" path="/var/lib/kubelet/pods/56e83def-04be-4211-adb9-ca44eb58654c/volumes"
Jul 30 15:25:46.309856 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:46.309782    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g"]
Jul 30 15:25:46.310316 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:46.310159    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56e83def-04be-4211-adb9-ca44eb58654c" containerName="istio-proxy"
Jul 30 15:25:46.310316 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:46.310174    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="56e83def-04be-4211-adb9-ca44eb58654c" containerName="istio-proxy"
Jul 30 15:25:46.310316 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:46.310188    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5513da55-7e86-4521-abcb-fcccd40cd186" containerName="payload-pre-processing"
Jul 30 15:25:46.310316 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:46.310194    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5513da55-7e86-4521-abcb-fcccd40cd186" containerName="payload-pre-processing"
Jul 30 15:25:46.310316 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:46.310206    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb6388bb-353b-4131-9c51-74a47cf8a61a" containerName="payload-processing"
Jul 30 15:25:46.310316 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:46.310211    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb6388bb-353b-4131-9c51-74a47cf8a61a" containerName="payload-processing"
Jul 30 15:25:46.310316 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:46.310219    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="190c779f-555e-4c6b-9d2b-2261442c56e1" containerName="istio-proxy"
Jul 30 15:25:46.310316 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:46.310224    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="190c779f-555e-4c6b-9d2b-2261442c56e1" containerName="istio-proxy"
Jul 30 15:25:46.310316 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:46.310236    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="190c779f-555e-4c6b-9d2b-2261442c56e1" containerName="istio-proxy"
Jul 30 15:25:46.310316 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:46.310243    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="190c779f-555e-4c6b-9d2b-2261442c56e1" containerName="istio-proxy"
Jul 30 15:25:46.310316 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:46.310252    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05f00a28-d937-41d2-b175-7a8cfbfedafc" containerName="maas-api"
Jul 30 15:25:46.310316 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:46.310259    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="05f00a28-d937-41d2-b175-7a8cfbfedafc" containerName="maas-api"
Jul 30 15:25:46.310316 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:46.310311    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="56e83def-04be-4211-adb9-ca44eb58654c" containerName="istio-proxy"
Jul 30 15:25:46.310316 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:46.310321    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="05f00a28-d937-41d2-b175-7a8cfbfedafc" containerName="maas-api"
Jul 30 15:25:46.310747 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:46.310331    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="190c779f-555e-4c6b-9d2b-2261442c56e1" containerName="istio-proxy"
Jul 30 15:25:46.310747 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:46.310338    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb6388bb-353b-4131-9c51-74a47cf8a61a" containerName="payload-processing"
Jul 30 15:25:46.310747 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:46.310344    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5513da55-7e86-4521-abcb-fcccd40cd186" containerName="payload-pre-processing"
Jul 30 15:25:46.313396 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:46.313379    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g"
Jul 30 15:25:46.315365 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:46.315338    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 30 15:25:46.315492 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:46.315442    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-0ea8d0fa-openshift-default-dockercfg-wp2s8\""
Jul 30 15:25:46.340407 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:46.340380    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g"]
Jul 30 15:25:46.464168 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:46.464125    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e60ab3cd-df3e-4e80-94d2-95a4be986bfc-istio-data\") pod \"e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g\" (UID: \"e60ab3cd-df3e-4e80-94d2-95a4be986bfc\") " pod="openshift-ingress/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g"
Jul 30 15:25:46.464352 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:46.464230    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e60ab3cd-df3e-4e80-94d2-95a4be986bfc-istio-token\") pod \"e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g\" (UID: \"e60ab3cd-df3e-4e80-94d2-95a4be986bfc\") " pod="openshift-ingress/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g"
Jul 30 15:25:46.464352 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:46.464287    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e60ab3cd-df3e-4e80-94d2-95a4be986bfc-credential-socket\") pod \"e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g\" (UID: \"e60ab3cd-df3e-4e80-94d2-95a4be986bfc\") " pod="openshift-ingress/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g"
Jul 30 15:25:46.464352 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:46.464316    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhrh5\" (UniqueName: \"kubernetes.io/projected/e60ab3cd-df3e-4e80-94d2-95a4be986bfc-kube-api-access-xhrh5\") pod \"e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g\" (UID: \"e60ab3cd-df3e-4e80-94d2-95a4be986bfc\") " pod="openshift-ingress/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g"
Jul 30 15:25:46.464477 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:46.464369    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e60ab3cd-df3e-4e80-94d2-95a4be986bfc-workload-socket\") pod \"e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g\" (UID: \"e60ab3cd-df3e-4e80-94d2-95a4be986bfc\") " pod="openshift-ingress/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g"
Jul 30 15:25:46.464477 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:46.464434    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e60ab3cd-df3e-4e80-94d2-95a4be986bfc-workload-certs\") pod \"e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g\" (UID: \"e60ab3cd-df3e-4e80-94d2-95a4be986bfc\") " pod="openshift-ingress/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g"
Jul 30 15:25:46.464477 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:46.464455    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e60ab3cd-df3e-4e80-94d2-95a4be986bfc-istio-envoy\") pod \"e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g\" (UID: \"e60ab3cd-df3e-4e80-94d2-95a4be986bfc\") " pod="openshift-ingress/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g"
Jul 30 15:25:46.464588 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:46.464478    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e60ab3cd-df3e-4e80-94d2-95a4be986bfc-istiod-ca-cert\") pod \"e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g\" (UID: \"e60ab3cd-df3e-4e80-94d2-95a4be986bfc\") " pod="openshift-ingress/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g"
Jul 30 15:25:46.464588 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:46.464496    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e60ab3cd-df3e-4e80-94d2-95a4be986bfc-istio-podinfo\") pod \"e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g\" (UID: \"e60ab3cd-df3e-4e80-94d2-95a4be986bfc\") " pod="openshift-ingress/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g"
Jul 30 15:25:46.565977 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:46.565897    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e60ab3cd-df3e-4e80-94d2-95a4be986bfc-workload-certs\") pod \"e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g\" (UID: \"e60ab3cd-df3e-4e80-94d2-95a4be986bfc\") " pod="openshift-ingress/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g"
Jul 30 15:25:46.565977 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:46.565937    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e60ab3cd-df3e-4e80-94d2-95a4be986bfc-istio-envoy\") pod \"e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g\" (UID: \"e60ab3cd-df3e-4e80-94d2-95a4be986bfc\") " pod="openshift-ingress/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g"
Jul 30 15:25:46.565977 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:46.565966    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e60ab3cd-df3e-4e80-94d2-95a4be986bfc-istiod-ca-cert\") pod \"e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g\" (UID: \"e60ab3cd-df3e-4e80-94d2-95a4be986bfc\") " pod="openshift-ingress/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g"
Jul 30 15:25:46.566274 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:46.565993    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e60ab3cd-df3e-4e80-94d2-95a4be986bfc-istio-podinfo\") pod \"e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g\" (UID: \"e60ab3cd-df3e-4e80-94d2-95a4be986bfc\") " pod="openshift-ingress/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g"
Jul 30 15:25:46.566274 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:46.566069    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e60ab3cd-df3e-4e80-94d2-95a4be986bfc-istio-data\") pod \"e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g\" (UID: \"e60ab3cd-df3e-4e80-94d2-95a4be986bfc\") " pod="openshift-ingress/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g"
Jul 30 15:25:46.566274 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:46.566115    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e60ab3cd-df3e-4e80-94d2-95a4be986bfc-istio-token\") pod \"e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g\" (UID: \"e60ab3cd-df3e-4e80-94d2-95a4be986bfc\") " pod="openshift-ingress/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g"
Jul 30 15:25:46.566274 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:46.566167    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e60ab3cd-df3e-4e80-94d2-95a4be986bfc-credential-socket\") pod \"e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g\" (UID: \"e60ab3cd-df3e-4e80-94d2-95a4be986bfc\") " pod="openshift-ingress/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g"
Jul 30 15:25:46.566485 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:46.566284    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhrh5\" (UniqueName: \"kubernetes.io/projected/e60ab3cd-df3e-4e80-94d2-95a4be986bfc-kube-api-access-xhrh5\") pod \"e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g\" (UID: \"e60ab3cd-df3e-4e80-94d2-95a4be986bfc\") " pod="openshift-ingress/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g"
Jul 30 15:25:46.566485 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:46.566322    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e60ab3cd-df3e-4e80-94d2-95a4be986bfc-workload-socket\") pod \"e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g\" (UID: \"e60ab3cd-df3e-4e80-94d2-95a4be986bfc\") " pod="openshift-ingress/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g"
Jul 30 15:25:46.566485 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:46.566462    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e60ab3cd-df3e-4e80-94d2-95a4be986bfc-workload-certs\") pod \"e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g\" (UID: \"e60ab3cd-df3e-4e80-94d2-95a4be986bfc\") " pod="openshift-ingress/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g"
Jul 30 15:25:46.566646 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:46.566567    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e60ab3cd-df3e-4e80-94d2-95a4be986bfc-istio-data\") pod \"e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g\" (UID: \"e60ab3cd-df3e-4e80-94d2-95a4be986bfc\") " pod="openshift-ingress/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g"
Jul 30 15:25:46.566646 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:46.566629    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e60ab3cd-df3e-4e80-94d2-95a4be986bfc-workload-socket\") pod \"e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g\" (UID: \"e60ab3cd-df3e-4e80-94d2-95a4be986bfc\") " pod="openshift-ingress/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g"
Jul 30 15:25:46.566760 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:46.566679    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e60ab3cd-df3e-4e80-94d2-95a4be986bfc-credential-socket\") pod \"e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g\" (UID: \"e60ab3cd-df3e-4e80-94d2-95a4be986bfc\") " pod="openshift-ingress/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g"
Jul 30 15:25:46.566871 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:46.566849    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e60ab3cd-df3e-4e80-94d2-95a4be986bfc-istiod-ca-cert\") pod \"e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g\" (UID: \"e60ab3cd-df3e-4e80-94d2-95a4be986bfc\") " pod="openshift-ingress/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g"
Jul 30 15:25:46.568246 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:46.568224    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e60ab3cd-df3e-4e80-94d2-95a4be986bfc-istio-envoy\") pod \"e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g\" (UID: \"e60ab3cd-df3e-4e80-94d2-95a4be986bfc\") " pod="openshift-ingress/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g"
Jul 30 15:25:46.568750 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:46.568728    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e60ab3cd-df3e-4e80-94d2-95a4be986bfc-istio-podinfo\") pod \"e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g\" (UID: \"e60ab3cd-df3e-4e80-94d2-95a4be986bfc\") " pod="openshift-ingress/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g"
Jul 30 15:25:46.573718 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:46.573700    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e60ab3cd-df3e-4e80-94d2-95a4be986bfc-istio-token\") pod \"e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g\" (UID: \"e60ab3cd-df3e-4e80-94d2-95a4be986bfc\") " pod="openshift-ingress/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g"
Jul 30 15:25:46.573968 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:46.573951    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhrh5\" (UniqueName: \"kubernetes.io/projected/e60ab3cd-df3e-4e80-94d2-95a4be986bfc-kube-api-access-xhrh5\") pod \"e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g\" (UID: \"e60ab3cd-df3e-4e80-94d2-95a4be986bfc\") " pod="openshift-ingress/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g"
Jul 30 15:25:46.624058 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:46.624026    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g"
Jul 30 15:25:46.761381 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:46.761352    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g"]
Jul 30 15:25:46.763308 ip-10-0-133-117 kubenswrapper[2578]: W0730 15:25:46.763283    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode60ab3cd_df3e_4e80_94d2_95a4be986bfc.slice/crio-2e2efaf9d4b3281ec9d84dd8301e5a09110833b45b10307b11a6ae0b47343bac WatchSource:0}: Error finding container 2e2efaf9d4b3281ec9d84dd8301e5a09110833b45b10307b11a6ae0b47343bac: Status 404 returned error can't find the container with id 2e2efaf9d4b3281ec9d84dd8301e5a09110833b45b10307b11a6ae0b47343bac
Jul 30 15:25:46.765392 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:46.765363    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:25:46.765468 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:46.765426    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:25:46.765468 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:46.765456    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:25:47.274003 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:47.273919    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g" event={"ID":"e60ab3cd-df3e-4e80-94d2-95a4be986bfc","Type":"ContainerStarted","Data":"e88412fe8ea707d620906aeeb482c37baffade87682bed4bf4b732d16a78b3cb"}
Jul 30 15:25:47.274003 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:47.273969    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g" event={"ID":"e60ab3cd-df3e-4e80-94d2-95a4be986bfc","Type":"ContainerStarted","Data":"2e2efaf9d4b3281ec9d84dd8301e5a09110833b45b10307b11a6ae0b47343bac"}
Jul 30 15:25:47.297784 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:47.297733    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g" podStartSLOduration=1.29771458 podStartE2EDuration="1.29771458s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:25:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:25:47.295372284 +0000 UTC m=+2258.824310758" watchObservedRunningTime="2026-07-30 15:25:47.29771458 +0000 UTC m=+2258.826653054"
Jul 30 15:25:47.625293 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:47.625260    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g"
Jul 30 15:25:47.630188 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:47.630164    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g"
Jul 30 15:25:48.279557 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:48.279523    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g"
Jul 30 15:25:48.280431 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:48.280409    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g"
Jul 30 15:25:50.443920 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:50.443885    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0ea8d0fa-7756d7d5f8-jf8gw"]
Jul 30 15:25:50.444397 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:50.444302    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56e83def-04be-4211-adb9-ca44eb58654c" containerName="istio-proxy"
Jul 30 15:25:50.444397 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:50.444318    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="56e83def-04be-4211-adb9-ca44eb58654c" containerName="istio-proxy"
Jul 30 15:25:50.444397 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:50.444384    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="56e83def-04be-4211-adb9-ca44eb58654c" containerName="istio-proxy"
Jul 30 15:25:50.444572 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:50.444398    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="190c779f-555e-4c6b-9d2b-2261442c56e1" containerName="istio-proxy"
Jul 30 15:25:50.448462 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:50.448440    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0ea8d0fa-7756d7d5f8-jf8gw"
Jul 30 15:25:50.450544 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:50.450517    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-0ea8d0fa-serving-cert\""
Jul 30 15:25:50.459673 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:50.459645    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0ea8d0fa-7756d7d5f8-jf8gw"]
Jul 30 15:25:50.604381 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:50.604346    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/177e5c1d-eb6f-448e-849e-cdae394c2635-maas-api-tls\") pod \"maas-api-e2e-mt-0ea8d0fa-7756d7d5f8-jf8gw\" (UID: \"177e5c1d-eb6f-448e-849e-cdae394c2635\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0ea8d0fa-7756d7d5f8-jf8gw"
Jul 30 15:25:50.604381 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:50.604386    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/177e5c1d-eb6f-448e-849e-cdae394c2635-openshift-service-ca\") pod \"maas-api-e2e-mt-0ea8d0fa-7756d7d5f8-jf8gw\" (UID: \"177e5c1d-eb6f-448e-849e-cdae394c2635\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0ea8d0fa-7756d7d5f8-jf8gw"
Jul 30 15:25:50.604601 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:50.604426    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2x62\" (UniqueName: \"kubernetes.io/projected/177e5c1d-eb6f-448e-849e-cdae394c2635-kube-api-access-b2x62\") pod \"maas-api-e2e-mt-0ea8d0fa-7756d7d5f8-jf8gw\" (UID: \"177e5c1d-eb6f-448e-849e-cdae394c2635\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0ea8d0fa-7756d7d5f8-jf8gw"
Jul 30 15:25:50.706046 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:50.705957    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/177e5c1d-eb6f-448e-849e-cdae394c2635-maas-api-tls\") pod \"maas-api-e2e-mt-0ea8d0fa-7756d7d5f8-jf8gw\" (UID: \"177e5c1d-eb6f-448e-849e-cdae394c2635\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0ea8d0fa-7756d7d5f8-jf8gw"
Jul 30 15:25:50.706046 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:50.706006    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/177e5c1d-eb6f-448e-849e-cdae394c2635-openshift-service-ca\") pod \"maas-api-e2e-mt-0ea8d0fa-7756d7d5f8-jf8gw\" (UID: \"177e5c1d-eb6f-448e-849e-cdae394c2635\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0ea8d0fa-7756d7d5f8-jf8gw"
Jul 30 15:25:50.706293 ip-10-0-133-117 kubenswrapper[2578]: E0730 15:25:50.706113    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-0ea8d0fa-serving-cert: secret "maas-api-e2e-mt-0ea8d0fa-serving-cert" not found
Jul 30 15:25:50.706293 ip-10-0-133-117 kubenswrapper[2578]: E0730 15:25:50.706187    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/177e5c1d-eb6f-448e-849e-cdae394c2635-maas-api-tls podName:177e5c1d-eb6f-448e-849e-cdae394c2635 nodeName:}" failed. No retries permitted until 2026-07-30 15:25:51.206168577 +0000 UTC m=+2262.735107029 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/177e5c1d-eb6f-448e-849e-cdae394c2635-maas-api-tls") pod "maas-api-e2e-mt-0ea8d0fa-7756d7d5f8-jf8gw" (UID: "177e5c1d-eb6f-448e-849e-cdae394c2635") : secret "maas-api-e2e-mt-0ea8d0fa-serving-cert" not found
Jul 30 15:25:50.706293 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:50.706183    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2x62\" (UniqueName: \"kubernetes.io/projected/177e5c1d-eb6f-448e-849e-cdae394c2635-kube-api-access-b2x62\") pod \"maas-api-e2e-mt-0ea8d0fa-7756d7d5f8-jf8gw\" (UID: \"177e5c1d-eb6f-448e-849e-cdae394c2635\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0ea8d0fa-7756d7d5f8-jf8gw"
Jul 30 15:25:50.706833 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:50.706813    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/177e5c1d-eb6f-448e-849e-cdae394c2635-openshift-service-ca\") pod \"maas-api-e2e-mt-0ea8d0fa-7756d7d5f8-jf8gw\" (UID: \"177e5c1d-eb6f-448e-849e-cdae394c2635\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0ea8d0fa-7756d7d5f8-jf8gw"
Jul 30 15:25:50.715938 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:50.715911    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2x62\" (UniqueName: \"kubernetes.io/projected/177e5c1d-eb6f-448e-849e-cdae394c2635-kube-api-access-b2x62\") pod \"maas-api-e2e-mt-0ea8d0fa-7756d7d5f8-jf8gw\" (UID: \"177e5c1d-eb6f-448e-849e-cdae394c2635\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0ea8d0fa-7756d7d5f8-jf8gw"
Jul 30 15:25:51.089029 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:51.088991    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-0ea8d0fa-6b57d6b488-62v77"]
Jul 30 15:25:51.092225 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:51.092203    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-0ea8d0fa-6b57d6b488-62v77"
Jul 30 15:25:51.093948 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:51.093927    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-0ea8d0fa\""
Jul 30 15:25:51.094170 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:51.094130    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-0ea8d0fa-dockercfg-tqbq7\""
Jul 30 15:25:51.106837 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:51.106817    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-0ea8d0fa-6b57d6b488-62v77"]
Jul 30 15:25:51.112100 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:51.112077    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-0ea8d0fa-65c79dbd9d-2s786"]
Jul 30 15:25:51.115184 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:51.115168    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-0ea8d0fa-65c79dbd9d-2s786"
Jul 30 15:25:51.129453 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:51.129429    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-0ea8d0fa-65c79dbd9d-2s786"]
Jul 30 15:25:51.211002 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:51.210968    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/177e5c1d-eb6f-448e-849e-cdae394c2635-maas-api-tls\") pod \"maas-api-e2e-mt-0ea8d0fa-7756d7d5f8-jf8gw\" (UID: \"177e5c1d-eb6f-448e-849e-cdae394c2635\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0ea8d0fa-7756d7d5f8-jf8gw"
Jul 30 15:25:51.211204 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:51.211008    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cead90e8-4167-4119-81c7-4900c3f9aa79-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-0ea8d0fa-6b57d6b488-62v77\" (UID: \"cead90e8-4167-4119-81c7-4900c3f9aa79\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-0ea8d0fa-6b57d6b488-62v77"
Jul 30 15:25:51.211204 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:51.211029    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6xhn\" (UniqueName: \"kubernetes.io/projected/cead90e8-4167-4119-81c7-4900c3f9aa79-kube-api-access-w6xhn\") pod \"payload-pre-processing-e2e-mt-0ea8d0fa-6b57d6b488-62v77\" (UID: \"cead90e8-4167-4119-81c7-4900c3f9aa79\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-0ea8d0fa-6b57d6b488-62v77"
Jul 30 15:25:51.211204 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:51.211051    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d11ff9b5-9651-4c81-a554-b350e22d1694-plugins-config-volume\") pod \"payload-processing-e2e-mt-0ea8d0fa-65c79dbd9d-2s786\" (UID: \"d11ff9b5-9651-4c81-a554-b350e22d1694\") " pod="openshift-ingress/payload-processing-e2e-mt-0ea8d0fa-65c79dbd9d-2s786"
Jul 30 15:25:51.211204 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:51.211116    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgm2z\" (UniqueName: \"kubernetes.io/projected/d11ff9b5-9651-4c81-a554-b350e22d1694-kube-api-access-wgm2z\") pod \"payload-processing-e2e-mt-0ea8d0fa-65c79dbd9d-2s786\" (UID: \"d11ff9b5-9651-4c81-a554-b350e22d1694\") " pod="openshift-ingress/payload-processing-e2e-mt-0ea8d0fa-65c79dbd9d-2s786"
Jul 30 15:25:51.213417 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:51.213388    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/177e5c1d-eb6f-448e-849e-cdae394c2635-maas-api-tls\") pod \"maas-api-e2e-mt-0ea8d0fa-7756d7d5f8-jf8gw\" (UID: \"177e5c1d-eb6f-448e-849e-cdae394c2635\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0ea8d0fa-7756d7d5f8-jf8gw"
Jul 30 15:25:51.312057 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:51.312015    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cead90e8-4167-4119-81c7-4900c3f9aa79-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-0ea8d0fa-6b57d6b488-62v77\" (UID: \"cead90e8-4167-4119-81c7-4900c3f9aa79\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-0ea8d0fa-6b57d6b488-62v77"
Jul 30 15:25:51.312057 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:51.312059    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6xhn\" (UniqueName: \"kubernetes.io/projected/cead90e8-4167-4119-81c7-4900c3f9aa79-kube-api-access-w6xhn\") pod \"payload-pre-processing-e2e-mt-0ea8d0fa-6b57d6b488-62v77\" (UID: \"cead90e8-4167-4119-81c7-4900c3f9aa79\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-0ea8d0fa-6b57d6b488-62v77"
Jul 30 15:25:51.312329 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:51.312194    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d11ff9b5-9651-4c81-a554-b350e22d1694-plugins-config-volume\") pod \"payload-processing-e2e-mt-0ea8d0fa-65c79dbd9d-2s786\" (UID: \"d11ff9b5-9651-4c81-a554-b350e22d1694\") " pod="openshift-ingress/payload-processing-e2e-mt-0ea8d0fa-65c79dbd9d-2s786"
Jul 30 15:25:51.312329 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:51.312280    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgm2z\" (UniqueName: \"kubernetes.io/projected/d11ff9b5-9651-4c81-a554-b350e22d1694-kube-api-access-wgm2z\") pod \"payload-processing-e2e-mt-0ea8d0fa-65c79dbd9d-2s786\" (UID: \"d11ff9b5-9651-4c81-a554-b350e22d1694\") " pod="openshift-ingress/payload-processing-e2e-mt-0ea8d0fa-65c79dbd9d-2s786"
Jul 30 15:25:51.312854 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:51.312835    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cead90e8-4167-4119-81c7-4900c3f9aa79-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-0ea8d0fa-6b57d6b488-62v77\" (UID: \"cead90e8-4167-4119-81c7-4900c3f9aa79\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-0ea8d0fa-6b57d6b488-62v77"
Jul 30 15:25:51.312908 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:51.312892    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d11ff9b5-9651-4c81-a554-b350e22d1694-plugins-config-volume\") pod \"payload-processing-e2e-mt-0ea8d0fa-65c79dbd9d-2s786\" (UID: \"d11ff9b5-9651-4c81-a554-b350e22d1694\") " pod="openshift-ingress/payload-processing-e2e-mt-0ea8d0fa-65c79dbd9d-2s786"
Jul 30 15:25:51.321371 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:51.321344    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6xhn\" (UniqueName: \"kubernetes.io/projected/cead90e8-4167-4119-81c7-4900c3f9aa79-kube-api-access-w6xhn\") pod \"payload-pre-processing-e2e-mt-0ea8d0fa-6b57d6b488-62v77\" (UID: \"cead90e8-4167-4119-81c7-4900c3f9aa79\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-0ea8d0fa-6b57d6b488-62v77"
Jul 30 15:25:51.321487 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:51.321404    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgm2z\" (UniqueName: \"kubernetes.io/projected/d11ff9b5-9651-4c81-a554-b350e22d1694-kube-api-access-wgm2z\") pod \"payload-processing-e2e-mt-0ea8d0fa-65c79dbd9d-2s786\" (UID: \"d11ff9b5-9651-4c81-a554-b350e22d1694\") " pod="openshift-ingress/payload-processing-e2e-mt-0ea8d0fa-65c79dbd9d-2s786"
Jul 30 15:25:51.361061 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:51.360993    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0ea8d0fa-7756d7d5f8-jf8gw"
Jul 30 15:25:51.403537 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:51.403507    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-0ea8d0fa-6b57d6b488-62v77"
Jul 30 15:25:51.426019 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:51.425808    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-0ea8d0fa-65c79dbd9d-2s786"
Jul 30 15:25:51.508589 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:51.508517    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0ea8d0fa-7756d7d5f8-jf8gw"]
Jul 30 15:25:51.511219 ip-10-0-133-117 kubenswrapper[2578]: W0730 15:25:51.511192    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod177e5c1d_eb6f_448e_849e_cdae394c2635.slice/crio-9cd8e25a816727acff0fd5206f67c388f0f03ab5955a296513efb460abdec808 WatchSource:0}: Error finding container 9cd8e25a816727acff0fd5206f67c388f0f03ab5955a296513efb460abdec808: Status 404 returned error can't find the container with id 9cd8e25a816727acff0fd5206f67c388f0f03ab5955a296513efb460abdec808
Jul 30 15:25:51.568364 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:51.568329    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-0ea8d0fa-6b57d6b488-62v77"]
Jul 30 15:25:51.570793 ip-10-0-133-117 kubenswrapper[2578]: W0730 15:25:51.570768    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcead90e8_4167_4119_81c7_4900c3f9aa79.slice/crio-f965b2af726d597c93896d7665d6bbbb356657f52bc39c8414f8197bb6bce145 WatchSource:0}: Error finding container f965b2af726d597c93896d7665d6bbbb356657f52bc39c8414f8197bb6bce145: Status 404 returned error can't find the container with id f965b2af726d597c93896d7665d6bbbb356657f52bc39c8414f8197bb6bce145
Jul 30 15:25:51.595312 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:51.595290    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-0ea8d0fa-65c79dbd9d-2s786"]
Jul 30 15:25:51.605317 ip-10-0-133-117 kubenswrapper[2578]: W0730 15:25:51.605293    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd11ff9b5_9651_4c81_a554_b350e22d1694.slice/crio-9d7bbe1104afb84fd97977ad207b881be62f9b7a7695a5b69b97b081c1bd47a7 WatchSource:0}: Error finding container 9d7bbe1104afb84fd97977ad207b881be62f9b7a7695a5b69b97b081c1bd47a7: Status 404 returned error can't find the container with id 9d7bbe1104afb84fd97977ad207b881be62f9b7a7695a5b69b97b081c1bd47a7
Jul 30 15:25:52.299576 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:52.299538    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-0ea8d0fa-65c79dbd9d-2s786" event={"ID":"d11ff9b5-9651-4c81-a554-b350e22d1694","Type":"ContainerStarted","Data":"597591f96a7447fdb9f8ef2a1f4c0415c25f0402e37c5c7cb03925858d13465a"}
Jul 30 15:25:52.299771 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:52.299606    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-0ea8d0fa-65c79dbd9d-2s786" event={"ID":"d11ff9b5-9651-4c81-a554-b350e22d1694","Type":"ContainerStarted","Data":"9d7bbe1104afb84fd97977ad207b881be62f9b7a7695a5b69b97b081c1bd47a7"}
Jul 30 15:25:52.299771 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:52.299725    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-0ea8d0fa-65c79dbd9d-2s786"
Jul 30 15:25:52.300661 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:52.300634    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0ea8d0fa-7756d7d5f8-jf8gw" event={"ID":"177e5c1d-eb6f-448e-849e-cdae394c2635","Type":"ContainerStarted","Data":"9cd8e25a816727acff0fd5206f67c388f0f03ab5955a296513efb460abdec808"}
Jul 30 15:25:52.302112 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:52.302080    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-0ea8d0fa-6b57d6b488-62v77" event={"ID":"cead90e8-4167-4119-81c7-4900c3f9aa79","Type":"ContainerStarted","Data":"1c1da17e759c65ddbcb1f676ddfe2d404c8ab02f843a5ae38b9c7f64c5f48df8"}
Jul 30 15:25:52.302235 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:52.302163    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-0ea8d0fa-6b57d6b488-62v77"
Jul 30 15:25:52.302235 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:52.302191    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-0ea8d0fa-6b57d6b488-62v77" event={"ID":"cead90e8-4167-4119-81c7-4900c3f9aa79","Type":"ContainerStarted","Data":"f965b2af726d597c93896d7665d6bbbb356657f52bc39c8414f8197bb6bce145"}
Jul 30 15:25:52.316929 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:52.316879    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-0ea8d0fa-65c79dbd9d-2s786" podStartSLOduration=1.316862594 podStartE2EDuration="1.316862594s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:25:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:25:52.315906194 +0000 UTC m=+2263.844844671" watchObservedRunningTime="2026-07-30 15:25:52.316862594 +0000 UTC m=+2263.845801116"
Jul 30 15:25:52.338795 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:52.338743    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-0ea8d0fa-6b57d6b488-62v77" podStartSLOduration=1.338724457 podStartE2EDuration="1.338724457s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:25:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:25:52.335981291 +0000 UTC m=+2263.864919803" watchObservedRunningTime="2026-07-30 15:25:52.338724457 +0000 UTC m=+2263.867662934"
Jul 30 15:25:53.308053 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:53.308011    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0ea8d0fa-7756d7d5f8-jf8gw" event={"ID":"177e5c1d-eb6f-448e-849e-cdae394c2635","Type":"ContainerStarted","Data":"f5969c28b4a7844aaaa9107756c63a08455c2f03b4d4286cec7c5a70449ac5e7"}
Jul 30 15:25:53.308502 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:53.308172    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0ea8d0fa-7756d7d5f8-jf8gw"
Jul 30 15:25:53.326632 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:53.326588    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0ea8d0fa-7756d7d5f8-jf8gw" podStartSLOduration=2.2695401 podStartE2EDuration="3.326573145s" totalImagesPullingTime="1.057033045s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:25:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:25:53.323998272 +0000 UTC m=+2264.852936745" watchObservedRunningTime="2026-07-30 15:25:53.326573145 +0000 UTC m=+2264.855511616"
Jul 30 15:25:59.319451 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:25:59.319419    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0ea8d0fa-7756d7d5f8-jf8gw"
Jul 30 15:26:01.348228 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:01.348197    2578 generic.go:356] "Generic (PLEG): container finished" podID="e60ab3cd-df3e-4e80-94d2-95a4be986bfc" containerID="e88412fe8ea707d620906aeeb482c37baffade87682bed4bf4b732d16a78b3cb" exitCode=0
Jul 30 15:26:01.348659 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:01.348276    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g" event={"ID":"e60ab3cd-df3e-4e80-94d2-95a4be986bfc","Type":"ContainerDied","Data":"e88412fe8ea707d620906aeeb482c37baffade87682bed4bf4b732d16a78b3cb"}
Jul 30 15:26:01.348659 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:01.348653    2578 scope.go:117] "RemoveContainer" containerID="e88412fe8ea707d620906aeeb482c37baffade87682bed4bf4b732d16a78b3cb"
Jul 30 15:26:01.349754 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:01.349729    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:26:01.349825 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:01.349796    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:26:01.349862 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:01.349844    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:26:02.358220 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:02.358177    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g" event={"ID":"e60ab3cd-df3e-4e80-94d2-95a4be986bfc","Type":"ContainerStarted","Data":"649e4e7a5b9306c4bc4ad05f5bf17ef30dc60d018906a0bab2f7da3a41bbd2a7"}
Jul 30 15:26:02.624298 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:02.624213    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g"
Jul 30 15:26:02.625120 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:02.625096    2578 patch_prober.go:28] interesting pod/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Jul 30 15:26:02.625207 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:02.625182    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g" podUID="e60ab3cd-df3e-4e80-94d2-95a4be986bfc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Jul 30 15:26:03.279968 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:03.279937    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g"
Jul 30 15:26:03.310769 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:03.310744    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-0ea8d0fa-6b57d6b488-62v77"
Jul 30 15:26:03.310942 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:03.310901    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-0ea8d0fa-65c79dbd9d-2s786"
Jul 30 15:26:03.625161 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:03.625116    2578 patch_prober.go:28] interesting pod/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Jul 30 15:26:03.625547 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:03.625183    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g" podUID="e60ab3cd-df3e-4e80-94d2-95a4be986bfc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Jul 30 15:26:04.624489 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:04.624452    2578 patch_prober.go:28] interesting pod/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Jul 30 15:26:04.624662 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:04.624512    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g" podUID="e60ab3cd-df3e-4e80-94d2-95a4be986bfc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Jul 30 15:26:05.625141 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:05.625103    2578 patch_prober.go:28] interesting pod/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Jul 30 15:26:05.625554 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:05.625181    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g" podUID="e60ab3cd-df3e-4e80-94d2-95a4be986bfc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Jul 30 15:26:06.628384 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:06.628350    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g"
Jul 30 15:26:06.629393 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:06.629371    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g"
Jul 30 15:26:32.501549 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:32.501511    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0ea8d0fa-7756d7d5f8-jf8gw"]
Jul 30 15:26:32.502203 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:32.501760    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0ea8d0fa-7756d7d5f8-jf8gw" podUID="177e5c1d-eb6f-448e-849e-cdae394c2635" containerName="maas-api" containerID="cri-o://f5969c28b4a7844aaaa9107756c63a08455c2f03b4d4286cec7c5a70449ac5e7" gracePeriod=30
Jul 30 15:26:32.557229 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:32.557197    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-0ea8d0fa-65c79dbd9d-2s786"]
Jul 30 15:26:32.557434 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:32.557410    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-0ea8d0fa-65c79dbd9d-2s786" podUID="d11ff9b5-9651-4c81-a554-b350e22d1694" containerName="payload-processing" containerID="cri-o://597591f96a7447fdb9f8ef2a1f4c0415c25f0402e37c5c7cb03925858d13465a" gracePeriod=30
Jul 30 15:26:32.580884 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:32.580837    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-0ea8d0fa-6b57d6b488-62v77"]
Jul 30 15:26:32.581502 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:32.581219    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-0ea8d0fa-6b57d6b488-62v77" podUID="cead90e8-4167-4119-81c7-4900c3f9aa79" containerName="payload-pre-processing" containerID="cri-o://1c1da17e759c65ddbcb1f676ddfe2d404c8ab02f843a5ae38b9c7f64c5f48df8" gracePeriod=30
Jul 30 15:26:32.762865 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:32.762824    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0ea8d0fa-7756d7d5f8-jf8gw"
Jul 30 15:26:32.839068 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:32.839042    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-0ea8d0fa-6b57d6b488-62v77"
Jul 30 15:26:32.860379 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:32.860352    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-0ea8d0fa-65c79dbd9d-2s786"
Jul 30 15:26:32.933506 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:32.933475    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b2x62\" (UniqueName: \"kubernetes.io/projected/177e5c1d-eb6f-448e-849e-cdae394c2635-kube-api-access-b2x62\") pod \"177e5c1d-eb6f-448e-849e-cdae394c2635\" (UID: \"177e5c1d-eb6f-448e-849e-cdae394c2635\") "
Jul 30 15:26:32.933673 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:32.933527    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/177e5c1d-eb6f-448e-849e-cdae394c2635-openshift-service-ca\") pod \"177e5c1d-eb6f-448e-849e-cdae394c2635\" (UID: \"177e5c1d-eb6f-448e-849e-cdae394c2635\") "
Jul 30 15:26:32.933673 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:32.933554    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cead90e8-4167-4119-81c7-4900c3f9aa79-plugins-config-volume\") pod \"cead90e8-4167-4119-81c7-4900c3f9aa79\" (UID: \"cead90e8-4167-4119-81c7-4900c3f9aa79\") "
Jul 30 15:26:32.933673 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:32.933660    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/177e5c1d-eb6f-448e-849e-cdae394c2635-maas-api-tls\") pod \"177e5c1d-eb6f-448e-849e-cdae394c2635\" (UID: \"177e5c1d-eb6f-448e-849e-cdae394c2635\") "
Jul 30 15:26:32.933885 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:32.933686    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w6xhn\" (UniqueName: \"kubernetes.io/projected/cead90e8-4167-4119-81c7-4900c3f9aa79-kube-api-access-w6xhn\") pod \"cead90e8-4167-4119-81c7-4900c3f9aa79\" (UID: \"cead90e8-4167-4119-81c7-4900c3f9aa79\") "
Jul 30 15:26:32.933988 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:32.933948    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/177e5c1d-eb6f-448e-849e-cdae394c2635-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "177e5c1d-eb6f-448e-849e-cdae394c2635" (UID: "177e5c1d-eb6f-448e-849e-cdae394c2635"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:26:32.934059 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:32.934031    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cead90e8-4167-4119-81c7-4900c3f9aa79-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cead90e8-4167-4119-81c7-4900c3f9aa79" (UID: "cead90e8-4167-4119-81c7-4900c3f9aa79"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:26:32.935720 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:32.935688    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/177e5c1d-eb6f-448e-849e-cdae394c2635-kube-api-access-b2x62" (OuterVolumeSpecName: "kube-api-access-b2x62") pod "177e5c1d-eb6f-448e-849e-cdae394c2635" (UID: "177e5c1d-eb6f-448e-849e-cdae394c2635"). InnerVolumeSpecName "kube-api-access-b2x62". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:26:32.935720 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:32.935702    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/177e5c1d-eb6f-448e-849e-cdae394c2635-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "177e5c1d-eb6f-448e-849e-cdae394c2635" (UID: "177e5c1d-eb6f-448e-849e-cdae394c2635"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 15:26:32.935853 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:32.935743    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cead90e8-4167-4119-81c7-4900c3f9aa79-kube-api-access-w6xhn" (OuterVolumeSpecName: "kube-api-access-w6xhn") pod "cead90e8-4167-4119-81c7-4900c3f9aa79" (UID: "cead90e8-4167-4119-81c7-4900c3f9aa79"). InnerVolumeSpecName "kube-api-access-w6xhn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:26:33.034957 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:33.034896    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d11ff9b5-9651-4c81-a554-b350e22d1694-plugins-config-volume\") pod \"d11ff9b5-9651-4c81-a554-b350e22d1694\" (UID: \"d11ff9b5-9651-4c81-a554-b350e22d1694\") "
Jul 30 15:26:33.034957 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:33.034951    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wgm2z\" (UniqueName: \"kubernetes.io/projected/d11ff9b5-9651-4c81-a554-b350e22d1694-kube-api-access-wgm2z\") pod \"d11ff9b5-9651-4c81-a554-b350e22d1694\" (UID: \"d11ff9b5-9651-4c81-a554-b350e22d1694\") "
Jul 30 15:26:33.035108 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:33.035097    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/177e5c1d-eb6f-448e-849e-cdae394c2635-maas-api-tls\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:26:33.035170 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:33.035109    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w6xhn\" (UniqueName: \"kubernetes.io/projected/cead90e8-4167-4119-81c7-4900c3f9aa79-kube-api-access-w6xhn\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:26:33.035170 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:33.035118    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b2x62\" (UniqueName: \"kubernetes.io/projected/177e5c1d-eb6f-448e-849e-cdae394c2635-kube-api-access-b2x62\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:26:33.035170 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:33.035126    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/177e5c1d-eb6f-448e-849e-cdae394c2635-openshift-service-ca\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:26:33.035170 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:33.035134    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cead90e8-4167-4119-81c7-4900c3f9aa79-plugins-config-volume\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:26:33.035345 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:33.035323    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d11ff9b5-9651-4c81-a554-b350e22d1694-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d11ff9b5-9651-4c81-a554-b350e22d1694" (UID: "d11ff9b5-9651-4c81-a554-b350e22d1694"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:26:33.036870 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:33.036852    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d11ff9b5-9651-4c81-a554-b350e22d1694-kube-api-access-wgm2z" (OuterVolumeSpecName: "kube-api-access-wgm2z") pod "d11ff9b5-9651-4c81-a554-b350e22d1694" (UID: "d11ff9b5-9651-4c81-a554-b350e22d1694"). InnerVolumeSpecName "kube-api-access-wgm2z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:26:33.136476 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:33.136441    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d11ff9b5-9651-4c81-a554-b350e22d1694-plugins-config-volume\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:26:33.136476 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:33.136470    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wgm2z\" (UniqueName: \"kubernetes.io/projected/d11ff9b5-9651-4c81-a554-b350e22d1694-kube-api-access-wgm2z\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:26:33.513412 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:33.513378    2578 generic.go:356] "Generic (PLEG): container finished" podID="cead90e8-4167-4119-81c7-4900c3f9aa79" containerID="1c1da17e759c65ddbcb1f676ddfe2d404c8ab02f843a5ae38b9c7f64c5f48df8" exitCode=0
Jul 30 15:26:33.513895 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:33.513425    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-0ea8d0fa-6b57d6b488-62v77"
Jul 30 15:26:33.513895 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:33.513466    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-0ea8d0fa-6b57d6b488-62v77" event={"ID":"cead90e8-4167-4119-81c7-4900c3f9aa79","Type":"ContainerDied","Data":"1c1da17e759c65ddbcb1f676ddfe2d404c8ab02f843a5ae38b9c7f64c5f48df8"}
Jul 30 15:26:33.513895 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:33.513522    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-0ea8d0fa-6b57d6b488-62v77" event={"ID":"cead90e8-4167-4119-81c7-4900c3f9aa79","Type":"ContainerDied","Data":"f965b2af726d597c93896d7665d6bbbb356657f52bc39c8414f8197bb6bce145"}
Jul 30 15:26:33.513895 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:33.513549    2578 scope.go:117] "RemoveContainer" containerID="1c1da17e759c65ddbcb1f676ddfe2d404c8ab02f843a5ae38b9c7f64c5f48df8"
Jul 30 15:26:33.514908 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:33.514825    2578 generic.go:356] "Generic (PLEG): container finished" podID="d11ff9b5-9651-4c81-a554-b350e22d1694" containerID="597591f96a7447fdb9f8ef2a1f4c0415c25f0402e37c5c7cb03925858d13465a" exitCode=0
Jul 30 15:26:33.514908 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:33.514889    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-0ea8d0fa-65c79dbd9d-2s786"
Jul 30 15:26:33.515021 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:33.514960    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-0ea8d0fa-65c79dbd9d-2s786" event={"ID":"d11ff9b5-9651-4c81-a554-b350e22d1694","Type":"ContainerDied","Data":"597591f96a7447fdb9f8ef2a1f4c0415c25f0402e37c5c7cb03925858d13465a"}
Jul 30 15:26:33.515021 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:33.515004    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-0ea8d0fa-65c79dbd9d-2s786" event={"ID":"d11ff9b5-9651-4c81-a554-b350e22d1694","Type":"ContainerDied","Data":"9d7bbe1104afb84fd97977ad207b881be62f9b7a7695a5b69b97b081c1bd47a7"}
Jul 30 15:26:33.516429 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:33.516408    2578 generic.go:356] "Generic (PLEG): container finished" podID="177e5c1d-eb6f-448e-849e-cdae394c2635" containerID="f5969c28b4a7844aaaa9107756c63a08455c2f03b4d4286cec7c5a70449ac5e7" exitCode=0
Jul 30 15:26:33.516557 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:33.516459    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0ea8d0fa-7756d7d5f8-jf8gw" event={"ID":"177e5c1d-eb6f-448e-849e-cdae394c2635","Type":"ContainerDied","Data":"f5969c28b4a7844aaaa9107756c63a08455c2f03b4d4286cec7c5a70449ac5e7"}
Jul 30 15:26:33.516557 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:33.516464    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0ea8d0fa-7756d7d5f8-jf8gw"
Jul 30 15:26:33.516557 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:33.516494    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0ea8d0fa-7756d7d5f8-jf8gw" event={"ID":"177e5c1d-eb6f-448e-849e-cdae394c2635","Type":"ContainerDied","Data":"9cd8e25a816727acff0fd5206f67c388f0f03ab5955a296513efb460abdec808"}
Jul 30 15:26:33.529826 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:33.529807    2578 scope.go:117] "RemoveContainer" containerID="1c1da17e759c65ddbcb1f676ddfe2d404c8ab02f843a5ae38b9c7f64c5f48df8"
Jul 30 15:26:33.530075 ip-10-0-133-117 kubenswrapper[2578]: E0730 15:26:33.530056    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c1da17e759c65ddbcb1f676ddfe2d404c8ab02f843a5ae38b9c7f64c5f48df8\": container with ID starting with 1c1da17e759c65ddbcb1f676ddfe2d404c8ab02f843a5ae38b9c7f64c5f48df8 not found: ID does not exist" containerID="1c1da17e759c65ddbcb1f676ddfe2d404c8ab02f843a5ae38b9c7f64c5f48df8"
Jul 30 15:26:33.530168 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:33.530086    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c1da17e759c65ddbcb1f676ddfe2d404c8ab02f843a5ae38b9c7f64c5f48df8"} err="failed to get container status \"1c1da17e759c65ddbcb1f676ddfe2d404c8ab02f843a5ae38b9c7f64c5f48df8\": rpc error: code = NotFound desc = could not find container \"1c1da17e759c65ddbcb1f676ddfe2d404c8ab02f843a5ae38b9c7f64c5f48df8\": container with ID starting with 1c1da17e759c65ddbcb1f676ddfe2d404c8ab02f843a5ae38b9c7f64c5f48df8 not found: ID does not exist"
Jul 30 15:26:33.530168 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:33.530109    2578 scope.go:117] "RemoveContainer" containerID="597591f96a7447fdb9f8ef2a1f4c0415c25f0402e37c5c7cb03925858d13465a"
Jul 30 15:26:33.537528 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:33.537506    2578 scope.go:117] "RemoveContainer" containerID="597591f96a7447fdb9f8ef2a1f4c0415c25f0402e37c5c7cb03925858d13465a"
Jul 30 15:26:33.537783 ip-10-0-133-117 kubenswrapper[2578]: E0730 15:26:33.537767    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"597591f96a7447fdb9f8ef2a1f4c0415c25f0402e37c5c7cb03925858d13465a\": container with ID starting with 597591f96a7447fdb9f8ef2a1f4c0415c25f0402e37c5c7cb03925858d13465a not found: ID does not exist" containerID="597591f96a7447fdb9f8ef2a1f4c0415c25f0402e37c5c7cb03925858d13465a"
Jul 30 15:26:33.537839 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:33.537787    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"597591f96a7447fdb9f8ef2a1f4c0415c25f0402e37c5c7cb03925858d13465a"} err="failed to get container status \"597591f96a7447fdb9f8ef2a1f4c0415c25f0402e37c5c7cb03925858d13465a\": rpc error: code = NotFound desc = could not find container \"597591f96a7447fdb9f8ef2a1f4c0415c25f0402e37c5c7cb03925858d13465a\": container with ID starting with 597591f96a7447fdb9f8ef2a1f4c0415c25f0402e37c5c7cb03925858d13465a not found: ID does not exist"
Jul 30 15:26:33.537839 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:33.537802    2578 scope.go:117] "RemoveContainer" containerID="f5969c28b4a7844aaaa9107756c63a08455c2f03b4d4286cec7c5a70449ac5e7"
Jul 30 15:26:33.546023 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:33.546000    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0ea8d0fa-7756d7d5f8-jf8gw"]
Jul 30 15:26:33.547006 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:33.546987    2578 scope.go:117] "RemoveContainer" containerID="f5969c28b4a7844aaaa9107756c63a08455c2f03b4d4286cec7c5a70449ac5e7"
Jul 30 15:26:33.547707 ip-10-0-133-117 kubenswrapper[2578]: E0730 15:26:33.547688    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5969c28b4a7844aaaa9107756c63a08455c2f03b4d4286cec7c5a70449ac5e7\": container with ID starting with f5969c28b4a7844aaaa9107756c63a08455c2f03b4d4286cec7c5a70449ac5e7 not found: ID does not exist" containerID="f5969c28b4a7844aaaa9107756c63a08455c2f03b4d4286cec7c5a70449ac5e7"
Jul 30 15:26:33.547800 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:33.547712    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5969c28b4a7844aaaa9107756c63a08455c2f03b4d4286cec7c5a70449ac5e7"} err="failed to get container status \"f5969c28b4a7844aaaa9107756c63a08455c2f03b4d4286cec7c5a70449ac5e7\": rpc error: code = NotFound desc = could not find container \"f5969c28b4a7844aaaa9107756c63a08455c2f03b4d4286cec7c5a70449ac5e7\": container with ID starting with f5969c28b4a7844aaaa9107756c63a08455c2f03b4d4286cec7c5a70449ac5e7 not found: ID does not exist"
Jul 30 15:26:33.548598 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:33.548579    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0ea8d0fa-7756d7d5f8-jf8gw"]
Jul 30 15:26:33.556761 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:33.556741    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-0ea8d0fa-6b57d6b488-62v77"]
Jul 30 15:26:33.561323 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:33.561303    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-0ea8d0fa-6b57d6b488-62v77"]
Jul 30 15:26:33.568496 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:33.568476    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-0ea8d0fa-65c79dbd9d-2s786"]
Jul 30 15:26:33.574240 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:33.574221    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-0ea8d0fa-65c79dbd9d-2s786"]
Jul 30 15:26:34.979234 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:34.979205    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="177e5c1d-eb6f-448e-849e-cdae394c2635" path="/var/lib/kubelet/pods/177e5c1d-eb6f-448e-849e-cdae394c2635/volumes"
Jul 30 15:26:34.979583 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:34.979560    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cead90e8-4167-4119-81c7-4900c3f9aa79" path="/var/lib/kubelet/pods/cead90e8-4167-4119-81c7-4900c3f9aa79/volumes"
Jul 30 15:26:34.979869 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:34.979856    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d11ff9b5-9651-4c81-a554-b350e22d1694" path="/var/lib/kubelet/pods/d11ff9b5-9651-4c81-a554-b350e22d1694/volumes"
Jul 30 15:26:40.018065 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:40.018030    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g"]
Jul 30 15:26:40.018490 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:40.018311    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g" podUID="e60ab3cd-df3e-4e80-94d2-95a4be986bfc" containerName="istio-proxy" containerID="cri-o://649e4e7a5b9306c4bc4ad05f5bf17ef30dc60d018906a0bab2f7da3a41bbd2a7" gracePeriod=30
Jul 30 15:26:45.273328 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:45.273302    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g"
Jul 30 15:26:45.351386 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:45.351348    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e60ab3cd-df3e-4e80-94d2-95a4be986bfc-istio-token\") pod \"e60ab3cd-df3e-4e80-94d2-95a4be986bfc\" (UID: \"e60ab3cd-df3e-4e80-94d2-95a4be986bfc\") "
Jul 30 15:26:45.351546 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:45.351411    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e60ab3cd-df3e-4e80-94d2-95a4be986bfc-istiod-ca-cert\") pod \"e60ab3cd-df3e-4e80-94d2-95a4be986bfc\" (UID: \"e60ab3cd-df3e-4e80-94d2-95a4be986bfc\") "
Jul 30 15:26:45.351546 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:45.351509    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e60ab3cd-df3e-4e80-94d2-95a4be986bfc-credential-socket\") pod \"e60ab3cd-df3e-4e80-94d2-95a4be986bfc\" (UID: \"e60ab3cd-df3e-4e80-94d2-95a4be986bfc\") "
Jul 30 15:26:45.351654 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:45.351544    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e60ab3cd-df3e-4e80-94d2-95a4be986bfc-workload-certs\") pod \"e60ab3cd-df3e-4e80-94d2-95a4be986bfc\" (UID: \"e60ab3cd-df3e-4e80-94d2-95a4be986bfc\") "
Jul 30 15:26:45.351654 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:45.351571    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e60ab3cd-df3e-4e80-94d2-95a4be986bfc-istio-podinfo\") pod \"e60ab3cd-df3e-4e80-94d2-95a4be986bfc\" (UID: \"e60ab3cd-df3e-4e80-94d2-95a4be986bfc\") "
Jul 30 15:26:45.351654 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:45.351619    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e60ab3cd-df3e-4e80-94d2-95a4be986bfc-workload-socket\") pod \"e60ab3cd-df3e-4e80-94d2-95a4be986bfc\" (UID: \"e60ab3cd-df3e-4e80-94d2-95a4be986bfc\") "
Jul 30 15:26:45.351800 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:45.351657    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xhrh5\" (UniqueName: \"kubernetes.io/projected/e60ab3cd-df3e-4e80-94d2-95a4be986bfc-kube-api-access-xhrh5\") pod \"e60ab3cd-df3e-4e80-94d2-95a4be986bfc\" (UID: \"e60ab3cd-df3e-4e80-94d2-95a4be986bfc\") "
Jul 30 15:26:45.351800 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:45.351700    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e60ab3cd-df3e-4e80-94d2-95a4be986bfc-istio-data\") pod \"e60ab3cd-df3e-4e80-94d2-95a4be986bfc\" (UID: \"e60ab3cd-df3e-4e80-94d2-95a4be986bfc\") "
Jul 30 15:26:45.351800 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:45.351752    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e60ab3cd-df3e-4e80-94d2-95a4be986bfc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e60ab3cd-df3e-4e80-94d2-95a4be986bfc" (UID: "e60ab3cd-df3e-4e80-94d2-95a4be986bfc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:26:45.351800 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:45.351764    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e60ab3cd-df3e-4e80-94d2-95a4be986bfc-istio-envoy\") pod \"e60ab3cd-df3e-4e80-94d2-95a4be986bfc\" (UID: \"e60ab3cd-df3e-4e80-94d2-95a4be986bfc\") "
Jul 30 15:26:45.351800 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:45.351779    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e60ab3cd-df3e-4e80-94d2-95a4be986bfc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e60ab3cd-df3e-4e80-94d2-95a4be986bfc" (UID: "e60ab3cd-df3e-4e80-94d2-95a4be986bfc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:26:45.352162 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:45.351879    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e60ab3cd-df3e-4e80-94d2-95a4be986bfc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e60ab3cd-df3e-4e80-94d2-95a4be986bfc" (UID: "e60ab3cd-df3e-4e80-94d2-95a4be986bfc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:26:45.352162 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:45.351893    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e60ab3cd-df3e-4e80-94d2-95a4be986bfc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e60ab3cd-df3e-4e80-94d2-95a4be986bfc" (UID: "e60ab3cd-df3e-4e80-94d2-95a4be986bfc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:26:45.352162 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:45.352132    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e60ab3cd-df3e-4e80-94d2-95a4be986bfc-istiod-ca-cert\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:26:45.352315 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:45.352168    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e60ab3cd-df3e-4e80-94d2-95a4be986bfc-credential-socket\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:26:45.352315 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:45.352182    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e60ab3cd-df3e-4e80-94d2-95a4be986bfc-workload-certs\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:26:45.352315 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:45.352195    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e60ab3cd-df3e-4e80-94d2-95a4be986bfc-workload-socket\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:26:45.352671 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:45.352648    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e60ab3cd-df3e-4e80-94d2-95a4be986bfc-istio-data" (OuterVolumeSpecName: "istio-data") pod "e60ab3cd-df3e-4e80-94d2-95a4be986bfc" (UID: "e60ab3cd-df3e-4e80-94d2-95a4be986bfc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:26:45.353812 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:45.353784    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e60ab3cd-df3e-4e80-94d2-95a4be986bfc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e60ab3cd-df3e-4e80-94d2-95a4be986bfc" (UID: "e60ab3cd-df3e-4e80-94d2-95a4be986bfc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:26:45.353812 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:45.353799    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e60ab3cd-df3e-4e80-94d2-95a4be986bfc-kube-api-access-xhrh5" (OuterVolumeSpecName: "kube-api-access-xhrh5") pod "e60ab3cd-df3e-4e80-94d2-95a4be986bfc" (UID: "e60ab3cd-df3e-4e80-94d2-95a4be986bfc"). InnerVolumeSpecName "kube-api-access-xhrh5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:26:45.353961 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:45.353948    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e60ab3cd-df3e-4e80-94d2-95a4be986bfc-istio-token" (OuterVolumeSpecName: "istio-token") pod "e60ab3cd-df3e-4e80-94d2-95a4be986bfc" (UID: "e60ab3cd-df3e-4e80-94d2-95a4be986bfc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:26:45.354021 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:45.353995    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e60ab3cd-df3e-4e80-94d2-95a4be986bfc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e60ab3cd-df3e-4e80-94d2-95a4be986bfc" (UID: "e60ab3cd-df3e-4e80-94d2-95a4be986bfc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 15:26:45.453610 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:45.453570    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xhrh5\" (UniqueName: \"kubernetes.io/projected/e60ab3cd-df3e-4e80-94d2-95a4be986bfc-kube-api-access-xhrh5\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:26:45.453610 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:45.453608    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e60ab3cd-df3e-4e80-94d2-95a4be986bfc-istio-data\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:26:45.453610 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:45.453626    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e60ab3cd-df3e-4e80-94d2-95a4be986bfc-istio-envoy\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:26:45.453838 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:45.453636    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e60ab3cd-df3e-4e80-94d2-95a4be986bfc-istio-token\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:26:45.453838 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:45.453643    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e60ab3cd-df3e-4e80-94d2-95a4be986bfc-istio-podinfo\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:26:45.578312 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:45.578278    2578 generic.go:356] "Generic (PLEG): container finished" podID="e60ab3cd-df3e-4e80-94d2-95a4be986bfc" containerID="649e4e7a5b9306c4bc4ad05f5bf17ef30dc60d018906a0bab2f7da3a41bbd2a7" exitCode=0
Jul 30 15:26:45.578479 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:45.578332    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g"
Jul 30 15:26:45.578479 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:45.578364    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g" event={"ID":"e60ab3cd-df3e-4e80-94d2-95a4be986bfc","Type":"ContainerDied","Data":"649e4e7a5b9306c4bc4ad05f5bf17ef30dc60d018906a0bab2f7da3a41bbd2a7"}
Jul 30 15:26:45.578479 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:45.578420    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g" event={"ID":"e60ab3cd-df3e-4e80-94d2-95a4be986bfc","Type":"ContainerDied","Data":"2e2efaf9d4b3281ec9d84dd8301e5a09110833b45b10307b11a6ae0b47343bac"}
Jul 30 15:26:45.578479 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:45.578441    2578 scope.go:117] "RemoveContainer" containerID="649e4e7a5b9306c4bc4ad05f5bf17ef30dc60d018906a0bab2f7da3a41bbd2a7"
Jul 30 15:26:45.587878 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:45.587862    2578 scope.go:117] "RemoveContainer" containerID="e88412fe8ea707d620906aeeb482c37baffade87682bed4bf4b732d16a78b3cb"
Jul 30 15:26:45.595810 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:45.595794    2578 scope.go:117] "RemoveContainer" containerID="649e4e7a5b9306c4bc4ad05f5bf17ef30dc60d018906a0bab2f7da3a41bbd2a7"
Jul 30 15:26:45.596060 ip-10-0-133-117 kubenswrapper[2578]: E0730 15:26:45.596043    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"649e4e7a5b9306c4bc4ad05f5bf17ef30dc60d018906a0bab2f7da3a41bbd2a7\": container with ID starting with 649e4e7a5b9306c4bc4ad05f5bf17ef30dc60d018906a0bab2f7da3a41bbd2a7 not found: ID does not exist" containerID="649e4e7a5b9306c4bc4ad05f5bf17ef30dc60d018906a0bab2f7da3a41bbd2a7"
Jul 30 15:26:45.596113 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:45.596069    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"649e4e7a5b9306c4bc4ad05f5bf17ef30dc60d018906a0bab2f7da3a41bbd2a7"} err="failed to get container status \"649e4e7a5b9306c4bc4ad05f5bf17ef30dc60d018906a0bab2f7da3a41bbd2a7\": rpc error: code = NotFound desc = could not find container \"649e4e7a5b9306c4bc4ad05f5bf17ef30dc60d018906a0bab2f7da3a41bbd2a7\": container with ID starting with 649e4e7a5b9306c4bc4ad05f5bf17ef30dc60d018906a0bab2f7da3a41bbd2a7 not found: ID does not exist"
Jul 30 15:26:45.596113 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:45.596086    2578 scope.go:117] "RemoveContainer" containerID="e88412fe8ea707d620906aeeb482c37baffade87682bed4bf4b732d16a78b3cb"
Jul 30 15:26:45.596393 ip-10-0-133-117 kubenswrapper[2578]: E0730 15:26:45.596375    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e88412fe8ea707d620906aeeb482c37baffade87682bed4bf4b732d16a78b3cb\": container with ID starting with e88412fe8ea707d620906aeeb482c37baffade87682bed4bf4b732d16a78b3cb not found: ID does not exist" containerID="e88412fe8ea707d620906aeeb482c37baffade87682bed4bf4b732d16a78b3cb"
Jul 30 15:26:45.596448 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:45.596402    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e88412fe8ea707d620906aeeb482c37baffade87682bed4bf4b732d16a78b3cb"} err="failed to get container status \"e88412fe8ea707d620906aeeb482c37baffade87682bed4bf4b732d16a78b3cb\": rpc error: code = NotFound desc = could not find container \"e88412fe8ea707d620906aeeb482c37baffade87682bed4bf4b732d16a78b3cb\": container with ID starting with e88412fe8ea707d620906aeeb482c37baffade87682bed4bf4b732d16a78b3cb not found: ID does not exist"
Jul 30 15:26:45.599173 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:45.599137    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g"]
Jul 30 15:26:45.605506 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:45.605456    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-0ea8d0fa-openshift-default-5bc4f47988-b2v4g"]
Jul 30 15:26:46.978910 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:26:46.978874    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e60ab3cd-df3e-4e80-94d2-95a4be986bfc" path="/var/lib/kubelet/pods/e60ab3cd-df3e-4e80-94d2-95a4be986bfc/volumes"
Jul 30 15:28:09.086801 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:09.086772    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4mjmp_de241220-5f27-4b0b-b7c6-6d4b350e24e1/console-operator/1.log"
Jul 30 15:28:09.093331 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:09.093307    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4mjmp_de241220-5f27-4b0b-b7c6-6d4b350e24e1/console-operator/1.log"
Jul 30 15:28:56.004208 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:56.004097    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn"]
Jul 30 15:28:56.004763 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:56.004724    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d11ff9b5-9651-4c81-a554-b350e22d1694" containerName="payload-processing"
Jul 30 15:28:56.004763 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:56.004748    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d11ff9b5-9651-4c81-a554-b350e22d1694" containerName="payload-processing"
Jul 30 15:28:56.004887 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:56.004778    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="177e5c1d-eb6f-448e-849e-cdae394c2635" containerName="maas-api"
Jul 30 15:28:56.004887 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:56.004787    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="177e5c1d-eb6f-448e-849e-cdae394c2635" containerName="maas-api"
Jul 30 15:28:56.004887 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:56.004800    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e60ab3cd-df3e-4e80-94d2-95a4be986bfc" containerName="istio-proxy"
Jul 30 15:28:56.004887 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:56.004809    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e60ab3cd-df3e-4e80-94d2-95a4be986bfc" containerName="istio-proxy"
Jul 30 15:28:56.004887 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:56.004819    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cead90e8-4167-4119-81c7-4900c3f9aa79" containerName="payload-pre-processing"
Jul 30 15:28:56.004887 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:56.004828    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="cead90e8-4167-4119-81c7-4900c3f9aa79" containerName="payload-pre-processing"
Jul 30 15:28:56.004887 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:56.004839    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e60ab3cd-df3e-4e80-94d2-95a4be986bfc" containerName="istio-proxy"
Jul 30 15:28:56.004887 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:56.004847    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e60ab3cd-df3e-4e80-94d2-95a4be986bfc" containerName="istio-proxy"
Jul 30 15:28:56.005281 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:56.004931    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d11ff9b5-9651-4c81-a554-b350e22d1694" containerName="payload-processing"
Jul 30 15:28:56.005281 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:56.004947    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="cead90e8-4167-4119-81c7-4900c3f9aa79" containerName="payload-pre-processing"
Jul 30 15:28:56.005281 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:56.004959    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e60ab3cd-df3e-4e80-94d2-95a4be986bfc" containerName="istio-proxy"
Jul 30 15:28:56.005281 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:56.004970    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="177e5c1d-eb6f-448e-849e-cdae394c2635" containerName="maas-api"
Jul 30 15:28:56.005281 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:56.004979    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e60ab3cd-df3e-4e80-94d2-95a4be986bfc" containerName="istio-proxy"
Jul 30 15:28:56.008276 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:56.008255    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn"
Jul 30 15:28:56.010538 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:56.010517    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 30 15:28:56.010643 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:56.010549    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-2fa717fb-openshift-default-dockercfg-h5jms\""
Jul 30 15:28:56.019847 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:56.019827    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn"]
Jul 30 15:28:56.037685 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:56.037637    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d-istio-podinfo\") pod \"e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn\" (UID: \"ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d\") " pod="openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn"
Jul 30 15:28:56.037789 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:56.037728    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d-istiod-ca-cert\") pod \"e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn\" (UID: \"ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d\") " pod="openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn"
Jul 30 15:28:56.037836 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:56.037787    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d-workload-certs\") pod \"e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn\" (UID: \"ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d\") " pod="openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn"
Jul 30 15:28:56.037836 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:56.037817    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d-workload-socket\") pod \"e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn\" (UID: \"ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d\") " pod="openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn"
Jul 30 15:28:56.037915 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:56.037845    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d-istio-token\") pod \"e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn\" (UID: \"ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d\") " pod="openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn"
Jul 30 15:28:56.037984 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:56.037958    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d-istio-envoy\") pod \"e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn\" (UID: \"ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d\") " pod="openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn"
Jul 30 15:28:56.038090 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:56.038072    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d-istio-data\") pod \"e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn\" (UID: \"ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d\") " pod="openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn"
Jul 30 15:28:56.038281 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:56.038263    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d-credential-socket\") pod \"e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn\" (UID: \"ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d\") " pod="openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn"
Jul 30 15:28:56.038393 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:56.038378    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7p44\" (UniqueName: \"kubernetes.io/projected/ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d-kube-api-access-d7p44\") pod \"e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn\" (UID: \"ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d\") " pod="openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn"
Jul 30 15:28:56.139328 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:56.139298    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d-istio-data\") pod \"e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn\" (UID: \"ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d\") " pod="openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn"
Jul 30 15:28:56.139328 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:56.139331    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d-credential-socket\") pod \"e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn\" (UID: \"ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d\") " pod="openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn"
Jul 30 15:28:56.139549 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:56.139348    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7p44\" (UniqueName: \"kubernetes.io/projected/ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d-kube-api-access-d7p44\") pod \"e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn\" (UID: \"ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d\") " pod="openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn"
Jul 30 15:28:56.139549 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:56.139368    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d-istio-podinfo\") pod \"e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn\" (UID: \"ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d\") " pod="openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn"
Jul 30 15:28:56.139549 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:56.139403    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d-istiod-ca-cert\") pod \"e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn\" (UID: \"ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d\") " pod="openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn"
Jul 30 15:28:56.139549 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:56.139436    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d-workload-certs\") pod \"e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn\" (UID: \"ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d\") " pod="openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn"
Jul 30 15:28:56.139549 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:56.139460    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d-workload-socket\") pod \"e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn\" (UID: \"ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d\") " pod="openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn"
Jul 30 15:28:56.139549 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:56.139488    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d-istio-token\") pod \"e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn\" (UID: \"ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d\") " pod="openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn"
Jul 30 15:28:56.139549 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:56.139543    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d-istio-envoy\") pod \"e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn\" (UID: \"ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d\") " pod="openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn"
Jul 30 15:28:56.139911 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:56.139730    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d-istio-data\") pod \"e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn\" (UID: \"ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d\") " pod="openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn"
Jul 30 15:28:56.139911 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:56.139851    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d-credential-socket\") pod \"e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn\" (UID: \"ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d\") " pod="openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn"
Jul 30 15:28:56.140071 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:56.140047    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d-workload-socket\") pod \"e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn\" (UID: \"ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d\") " pod="openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn"
Jul 30 15:28:56.140280 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:56.140253    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d-workload-certs\") pod \"e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn\" (UID: \"ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d\") " pod="openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn"
Jul 30 15:28:56.140556 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:56.140536    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d-istiod-ca-cert\") pod \"e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn\" (UID: \"ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d\") " pod="openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn"
Jul 30 15:28:56.141830 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:56.141809    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d-istio-envoy\") pod \"e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn\" (UID: \"ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d\") " pod="openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn"
Jul 30 15:28:56.142091 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:56.142075    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d-istio-podinfo\") pod \"e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn\" (UID: \"ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d\") " pod="openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn"
Jul 30 15:28:56.146434 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:56.146410    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d-istio-token\") pod \"e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn\" (UID: \"ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d\") " pod="openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn"
Jul 30 15:28:56.146519 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:56.146478    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7p44\" (UniqueName: \"kubernetes.io/projected/ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d-kube-api-access-d7p44\") pod \"e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn\" (UID: \"ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d\") " pod="openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn"
Jul 30 15:28:56.318867 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:56.318836    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn"
Jul 30 15:28:56.453775 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:56.453749    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn"]
Jul 30 15:28:56.455874 ip-10-0-133-117 kubenswrapper[2578]: W0730 15:28:56.455847    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecef9c8d_ff57_46c4_94d7_cffaa6ad2f3d.slice/crio-2e9d55054ab9708ed9c541e73a72cdfed7e941badb7a2e0e0e282264754da49e WatchSource:0}: Error finding container 2e9d55054ab9708ed9c541e73a72cdfed7e941badb7a2e0e0e282264754da49e: Status 404 returned error can't find the container with id 2e9d55054ab9708ed9c541e73a72cdfed7e941badb7a2e0e0e282264754da49e
Jul 30 15:28:56.458386 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:56.458356    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:28:56.458456 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:56.458438    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:28:56.458498 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:56.458472    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:28:57.163973 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:57.163929    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn" event={"ID":"ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d","Type":"ContainerStarted","Data":"4827aecac6e04ab7b3093a4c1c3a998bd63ac784f7c65f3781dab305648207c8"}
Jul 30 15:28:57.164379 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:57.164043    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn" event={"ID":"ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d","Type":"ContainerStarted","Data":"2e9d55054ab9708ed9c541e73a72cdfed7e941badb7a2e0e0e282264754da49e"}
Jul 30 15:28:57.184308 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:57.184261    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn" podStartSLOduration=2.184246312 podStartE2EDuration="2.184246312s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:28:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:28:57.183111529 +0000 UTC m=+2448.712050002" watchObservedRunningTime="2026-07-30 15:28:57.184246312 +0000 UTC m=+2448.713184783"
Jul 30 15:28:57.319912 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:57.319880    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn"
Jul 30 15:28:57.324674 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:57.324652    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn"
Jul 30 15:28:58.168954 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:58.168927    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn"
Jul 30 15:28:58.169844 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:28:58.169824    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn"
Jul 30 15:29:00.239960 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:00.239920    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2fa717fb-bcdcb4597-zqcfj"]
Jul 30 15:29:00.244034 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:00.244004    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa717fb-bcdcb4597-zqcfj"
Jul 30 15:29:00.247098 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:00.246917    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-2fa717fb-serving-cert\""
Jul 30 15:29:00.252430 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:00.252406    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2fa717fb-bcdcb4597-zqcfj"]
Jul 30 15:29:00.280092 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:00.280064    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74b8a938-8751-4fc0-8b2b-ff82e3affb99-maas-api-tls\") pod \"maas-api-e2e-mt-2fa717fb-bcdcb4597-zqcfj\" (UID: \"74b8a938-8751-4fc0-8b2b-ff82e3affb99\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa717fb-bcdcb4597-zqcfj"
Jul 30 15:29:00.280253 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:00.280129    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74b8a938-8751-4fc0-8b2b-ff82e3affb99-openshift-service-ca\") pod \"maas-api-e2e-mt-2fa717fb-bcdcb4597-zqcfj\" (UID: \"74b8a938-8751-4fc0-8b2b-ff82e3affb99\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa717fb-bcdcb4597-zqcfj"
Jul 30 15:29:00.280337 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:00.280312    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzrrg\" (UniqueName: \"kubernetes.io/projected/74b8a938-8751-4fc0-8b2b-ff82e3affb99-kube-api-access-fzrrg\") pod \"maas-api-e2e-mt-2fa717fb-bcdcb4597-zqcfj\" (UID: \"74b8a938-8751-4fc0-8b2b-ff82e3affb99\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa717fb-bcdcb4597-zqcfj"
Jul 30 15:29:00.381354 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:00.381324    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzrrg\" (UniqueName: \"kubernetes.io/projected/74b8a938-8751-4fc0-8b2b-ff82e3affb99-kube-api-access-fzrrg\") pod \"maas-api-e2e-mt-2fa717fb-bcdcb4597-zqcfj\" (UID: \"74b8a938-8751-4fc0-8b2b-ff82e3affb99\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa717fb-bcdcb4597-zqcfj"
Jul 30 15:29:00.381540 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:00.381414    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74b8a938-8751-4fc0-8b2b-ff82e3affb99-maas-api-tls\") pod \"maas-api-e2e-mt-2fa717fb-bcdcb4597-zqcfj\" (UID: \"74b8a938-8751-4fc0-8b2b-ff82e3affb99\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa717fb-bcdcb4597-zqcfj"
Jul 30 15:29:00.381540 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:00.381460    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74b8a938-8751-4fc0-8b2b-ff82e3affb99-openshift-service-ca\") pod \"maas-api-e2e-mt-2fa717fb-bcdcb4597-zqcfj\" (UID: \"74b8a938-8751-4fc0-8b2b-ff82e3affb99\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa717fb-bcdcb4597-zqcfj"
Jul 30 15:29:00.382064 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:00.382043    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74b8a938-8751-4fc0-8b2b-ff82e3affb99-openshift-service-ca\") pod \"maas-api-e2e-mt-2fa717fb-bcdcb4597-zqcfj\" (UID: \"74b8a938-8751-4fc0-8b2b-ff82e3affb99\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa717fb-bcdcb4597-zqcfj"
Jul 30 15:29:00.383837 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:00.383814    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74b8a938-8751-4fc0-8b2b-ff82e3affb99-maas-api-tls\") pod \"maas-api-e2e-mt-2fa717fb-bcdcb4597-zqcfj\" (UID: \"74b8a938-8751-4fc0-8b2b-ff82e3affb99\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa717fb-bcdcb4597-zqcfj"
Jul 30 15:29:00.405231 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:00.405206    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzrrg\" (UniqueName: \"kubernetes.io/projected/74b8a938-8751-4fc0-8b2b-ff82e3affb99-kube-api-access-fzrrg\") pod \"maas-api-e2e-mt-2fa717fb-bcdcb4597-zqcfj\" (UID: \"74b8a938-8751-4fc0-8b2b-ff82e3affb99\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa717fb-bcdcb4597-zqcfj"
Jul 30 15:29:00.559931 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:00.559904    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa717fb-bcdcb4597-zqcfj"
Jul 30 15:29:00.693507 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:00.693480    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2fa717fb-bcdcb4597-zqcfj"]
Jul 30 15:29:00.695342 ip-10-0-133-117 kubenswrapper[2578]: W0730 15:29:00.695312    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8a938_8751_4fc0_8b2b_ff82e3affb99.slice/crio-e69d71c2856046b44a953b1db8bc2c85071235ee10881e571731445c6eb769cb WatchSource:0}: Error finding container e69d71c2856046b44a953b1db8bc2c85071235ee10881e571731445c6eb769cb: Status 404 returned error can't find the container with id e69d71c2856046b44a953b1db8bc2c85071235ee10881e571731445c6eb769cb
Jul 30 15:29:00.696766 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:00.696746    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 15:29:01.036978 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:01.036901    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2fa717fb-5f4cf45f5b-dtk62"]
Jul 30 15:29:01.041518 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:01.041503    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-2fa717fb-5f4cf45f5b-dtk62"
Jul 30 15:29:01.043629 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:01.043605    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-2fa717fb\""
Jul 30 15:29:01.043742 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:01.043718    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-2fa717fb-dockercfg-56s57\""
Jul 30 15:29:01.051257 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:01.051231    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2fa717fb-5f4cf45f5b-dtk62"]
Jul 30 15:29:01.056721 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:01.056702    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2fa717fb-74cddd68f8-szfpr"]
Jul 30 15:29:01.060059 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:01.060043    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-2fa717fb-74cddd68f8-szfpr"
Jul 30 15:29:01.078223 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:01.078196    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2fa717fb-74cddd68f8-szfpr"]
Jul 30 15:29:01.088633 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:01.088611    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/635e6a47-24f8-4870-8ab2-69fd622906e7-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-2fa717fb-5f4cf45f5b-dtk62\" (UID: \"635e6a47-24f8-4870-8ab2-69fd622906e7\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2fa717fb-5f4cf45f5b-dtk62"
Jul 30 15:29:01.088633 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:01.088674    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6gfq\" (UniqueName: \"kubernetes.io/projected/635e6a47-24f8-4870-8ab2-69fd622906e7-kube-api-access-h6gfq\") pod \"payload-pre-processing-e2e-mt-2fa717fb-5f4cf45f5b-dtk62\" (UID: \"635e6a47-24f8-4870-8ab2-69fd622906e7\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2fa717fb-5f4cf45f5b-dtk62"
Jul 30 15:29:01.184681 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:01.184656    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa717fb-bcdcb4597-zqcfj" event={"ID":"74b8a938-8751-4fc0-8b2b-ff82e3affb99","Type":"ContainerStarted","Data":"e69d71c2856046b44a953b1db8bc2c85071235ee10881e571731445c6eb769cb"}
Jul 30 15:29:01.190141 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:01.190116    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpxn9\" (UniqueName: \"kubernetes.io/projected/64d2af35-2264-4579-9686-7986f44aa5c5-kube-api-access-qpxn9\") pod \"payload-processing-e2e-mt-2fa717fb-74cddd68f8-szfpr\" (UID: \"64d2af35-2264-4579-9686-7986f44aa5c5\") " pod="openshift-ingress/payload-processing-e2e-mt-2fa717fb-74cddd68f8-szfpr"
Jul 30 15:29:01.190260 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:01.190185    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/635e6a47-24f8-4870-8ab2-69fd622906e7-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-2fa717fb-5f4cf45f5b-dtk62\" (UID: \"635e6a47-24f8-4870-8ab2-69fd622906e7\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2fa717fb-5f4cf45f5b-dtk62"
Jul 30 15:29:01.190260 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:01.190235    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6gfq\" (UniqueName: \"kubernetes.io/projected/635e6a47-24f8-4870-8ab2-69fd622906e7-kube-api-access-h6gfq\") pod \"payload-pre-processing-e2e-mt-2fa717fb-5f4cf45f5b-dtk62\" (UID: \"635e6a47-24f8-4870-8ab2-69fd622906e7\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2fa717fb-5f4cf45f5b-dtk62"
Jul 30 15:29:01.190365 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:01.190262    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64d2af35-2264-4579-9686-7986f44aa5c5-plugins-config-volume\") pod \"payload-processing-e2e-mt-2fa717fb-74cddd68f8-szfpr\" (UID: \"64d2af35-2264-4579-9686-7986f44aa5c5\") " pod="openshift-ingress/payload-processing-e2e-mt-2fa717fb-74cddd68f8-szfpr"
Jul 30 15:29:01.190852 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:01.190827    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/635e6a47-24f8-4870-8ab2-69fd622906e7-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-2fa717fb-5f4cf45f5b-dtk62\" (UID: \"635e6a47-24f8-4870-8ab2-69fd622906e7\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2fa717fb-5f4cf45f5b-dtk62"
Jul 30 15:29:01.199026 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:01.199006    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6gfq\" (UniqueName: \"kubernetes.io/projected/635e6a47-24f8-4870-8ab2-69fd622906e7-kube-api-access-h6gfq\") pod \"payload-pre-processing-e2e-mt-2fa717fb-5f4cf45f5b-dtk62\" (UID: \"635e6a47-24f8-4870-8ab2-69fd622906e7\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2fa717fb-5f4cf45f5b-dtk62"
Jul 30 15:29:01.291205 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:01.291104    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64d2af35-2264-4579-9686-7986f44aa5c5-plugins-config-volume\") pod \"payload-processing-e2e-mt-2fa717fb-74cddd68f8-szfpr\" (UID: \"64d2af35-2264-4579-9686-7986f44aa5c5\") " pod="openshift-ingress/payload-processing-e2e-mt-2fa717fb-74cddd68f8-szfpr"
Jul 30 15:29:01.291205 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:01.291196    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpxn9\" (UniqueName: \"kubernetes.io/projected/64d2af35-2264-4579-9686-7986f44aa5c5-kube-api-access-qpxn9\") pod \"payload-processing-e2e-mt-2fa717fb-74cddd68f8-szfpr\" (UID: \"64d2af35-2264-4579-9686-7986f44aa5c5\") " pod="openshift-ingress/payload-processing-e2e-mt-2fa717fb-74cddd68f8-szfpr"
Jul 30 15:29:01.291751 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:01.291731    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64d2af35-2264-4579-9686-7986f44aa5c5-plugins-config-volume\") pod \"payload-processing-e2e-mt-2fa717fb-74cddd68f8-szfpr\" (UID: \"64d2af35-2264-4579-9686-7986f44aa5c5\") " pod="openshift-ingress/payload-processing-e2e-mt-2fa717fb-74cddd68f8-szfpr"
Jul 30 15:29:01.298685 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:01.298656    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpxn9\" (UniqueName: \"kubernetes.io/projected/64d2af35-2264-4579-9686-7986f44aa5c5-kube-api-access-qpxn9\") pod \"payload-processing-e2e-mt-2fa717fb-74cddd68f8-szfpr\" (UID: \"64d2af35-2264-4579-9686-7986f44aa5c5\") " pod="openshift-ingress/payload-processing-e2e-mt-2fa717fb-74cddd68f8-szfpr"
Jul 30 15:29:01.353554 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:01.353525    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-2fa717fb-5f4cf45f5b-dtk62"
Jul 30 15:29:01.368827 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:01.368803    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-2fa717fb-74cddd68f8-szfpr"
Jul 30 15:29:01.495431 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:01.495040    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2fa717fb-5f4cf45f5b-dtk62"]
Jul 30 15:29:01.497462 ip-10-0-133-117 kubenswrapper[2578]: W0730 15:29:01.497435    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod635e6a47_24f8_4870_8ab2_69fd622906e7.slice/crio-d06de11d98f155e019a3f15cbb2444991a9d46e56d4997183c193514e265ab37 WatchSource:0}: Error finding container d06de11d98f155e019a3f15cbb2444991a9d46e56d4997183c193514e265ab37: Status 404 returned error can't find the container with id d06de11d98f155e019a3f15cbb2444991a9d46e56d4997183c193514e265ab37
Jul 30 15:29:01.521729 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:01.521695    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2fa717fb-74cddd68f8-szfpr"]
Jul 30 15:29:01.524387 ip-10-0-133-117 kubenswrapper[2578]: W0730 15:29:01.524354    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64d2af35_2264_4579_9686_7986f44aa5c5.slice/crio-e4dba5863bbf34516da63ce0387a37403af8e10ca04728662d58d5aba64797bd WatchSource:0}: Error finding container e4dba5863bbf34516da63ce0387a37403af8e10ca04728662d58d5aba64797bd: Status 404 returned error can't find the container with id e4dba5863bbf34516da63ce0387a37403af8e10ca04728662d58d5aba64797bd
Jul 30 15:29:02.191283 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:02.191246    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-2fa717fb-5f4cf45f5b-dtk62" event={"ID":"635e6a47-24f8-4870-8ab2-69fd622906e7","Type":"ContainerStarted","Data":"5e5b516e03352b548ebd1f45518c39b63c00d1d139c630b364d36ef89a207cb7"}
Jul 30 15:29:02.191489 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:02.191332    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-2fa717fb-5f4cf45f5b-dtk62"
Jul 30 15:29:02.191489 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:02.191358    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-2fa717fb-5f4cf45f5b-dtk62" event={"ID":"635e6a47-24f8-4870-8ab2-69fd622906e7","Type":"ContainerStarted","Data":"d06de11d98f155e019a3f15cbb2444991a9d46e56d4997183c193514e265ab37"}
Jul 30 15:29:02.192802 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:02.192778    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-2fa717fb-74cddd68f8-szfpr" event={"ID":"64d2af35-2264-4579-9686-7986f44aa5c5","Type":"ContainerStarted","Data":"1a3d13fe868ddd43f5fc0fff98377bbf935a5c1de90dcdb74e466e2c4d4f39c0"}
Jul 30 15:29:02.192966 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:02.192820    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-2fa717fb-74cddd68f8-szfpr" event={"ID":"64d2af35-2264-4579-9686-7986f44aa5c5","Type":"ContainerStarted","Data":"e4dba5863bbf34516da63ce0387a37403af8e10ca04728662d58d5aba64797bd"}
Jul 30 15:29:02.192966 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:02.192881    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-2fa717fb-74cddd68f8-szfpr"
Jul 30 15:29:02.194273 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:02.194253    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa717fb-bcdcb4597-zqcfj" event={"ID":"74b8a938-8751-4fc0-8b2b-ff82e3affb99","Type":"ContainerStarted","Data":"0cccce73bd3f169ca141f2cfe00be0d75a775f30da14fae8fdccee62e300b8f7"}
Jul 30 15:29:02.194389 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:02.194376    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa717fb-bcdcb4597-zqcfj"
Jul 30 15:29:02.206933 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:02.206883    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-2fa717fb-5f4cf45f5b-dtk62" podStartSLOduration=1.20686966 podStartE2EDuration="1.20686966s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:29:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:29:02.205541487 +0000 UTC m=+2453.734479984" watchObservedRunningTime="2026-07-30 15:29:02.20686966 +0000 UTC m=+2453.735808111"
Jul 30 15:29:02.222071 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:02.222020    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-2fa717fb-74cddd68f8-szfpr" podStartSLOduration=1.222003723 podStartE2EDuration="1.222003723s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:29:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:29:02.220797611 +0000 UTC m=+2453.749736084" watchObservedRunningTime="2026-07-30 15:29:02.222003723 +0000 UTC m=+2453.750942198"
Jul 30 15:29:02.236570 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:02.236521    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa717fb-bcdcb4597-zqcfj" podStartSLOduration=1.798849965 podStartE2EDuration="2.236506263s" totalImagesPullingTime="437.656298ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:29:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:29:02.235928287 +0000 UTC m=+2453.764866771" watchObservedRunningTime="2026-07-30 15:29:02.236506263 +0000 UTC m=+2453.765444736"
Jul 30 15:29:08.207733 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:08.207704    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa717fb-bcdcb4597-zqcfj"
Jul 30 15:29:13.170611 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:13.170578    2578 patch_prober.go:28] interesting pod/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Jul 30 15:29:13.171046 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:13.170630    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn" podUID="ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Jul 30 15:29:13.203214 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:13.203185    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-2fa717fb-5f4cf45f5b-dtk62"
Jul 30 15:29:13.203376 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:13.203307    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-2fa717fb-74cddd68f8-szfpr"
Jul 30 15:29:13.253651 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:13.253612    2578 generic.go:356] "Generic (PLEG): container finished" podID="ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d" containerID="4827aecac6e04ab7b3093a4c1c3a998bd63ac784f7c65f3781dab305648207c8" exitCode=0
Jul 30 15:29:13.253867 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:13.253699    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn" event={"ID":"ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d","Type":"ContainerDied","Data":"4827aecac6e04ab7b3093a4c1c3a998bd63ac784f7c65f3781dab305648207c8"}
Jul 30 15:29:13.254188 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:13.254166    2578 scope.go:117] "RemoveContainer" containerID="4827aecac6e04ab7b3093a4c1c3a998bd63ac784f7c65f3781dab305648207c8"
Jul 30 15:29:13.255428 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:13.255394    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:29:13.255509 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:13.255482    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:29:13.255552 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:13.255527    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:29:14.259740 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:14.259700    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn" event={"ID":"ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d","Type":"ContainerStarted","Data":"00c6d80a2fda8e7e6e15241c076a0b7138b06cece0f56f862a7ffb09fe4b7fc5"}
Jul 30 15:29:14.319452 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:14.319410    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn"
Jul 30 15:29:14.319742 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:14.319710    2578 patch_prober.go:28] interesting pod/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Jul 30 15:29:14.319869 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:14.319771    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn" podUID="ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Jul 30 15:29:15.320050 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:15.320013    2578 patch_prober.go:28] interesting pod/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Jul 30 15:29:15.320443 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:15.320064    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn" podUID="ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Jul 30 15:29:16.319579 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:16.319542    2578 patch_prober.go:28] interesting pod/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Jul 30 15:29:16.319755 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:16.319598    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn" podUID="ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Jul 30 15:29:17.320020 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:17.319984    2578 patch_prober.go:28] interesting pod/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Jul 30 15:29:17.320414 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:17.320048    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn" podUID="ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Jul 30 15:29:18.319630 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:18.319599    2578 patch_prober.go:28] interesting pod/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Jul 30 15:29:18.319812 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:18.319649    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn" podUID="ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Jul 30 15:29:19.324265 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:19.324226    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn"
Jul 30 15:29:19.324715 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:19.324615    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn"
Jul 30 15:29:19.325132 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:29:19.325110    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn"
Jul 30 15:30:34.956540 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:34.956498    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2fa717fb-bcdcb4597-zqcfj"]
Jul 30 15:30:34.957133 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:34.956797    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa717fb-bcdcb4597-zqcfj" podUID="74b8a938-8751-4fc0-8b2b-ff82e3affb99" containerName="maas-api" containerID="cri-o://0cccce73bd3f169ca141f2cfe00be0d75a775f30da14fae8fdccee62e300b8f7" gracePeriod=30
Jul 30 15:30:35.015059 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:35.015019    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2fa717fb-74cddd68f8-szfpr"]
Jul 30 15:30:35.015335 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:35.015303    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-2fa717fb-74cddd68f8-szfpr" podUID="64d2af35-2264-4579-9686-7986f44aa5c5" containerName="payload-processing" containerID="cri-o://1a3d13fe868ddd43f5fc0fff98377bbf935a5c1de90dcdb74e466e2c4d4f39c0" gracePeriod=30
Jul 30 15:30:35.023519 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:35.023490    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2fa717fb-5f4cf45f5b-dtk62"]
Jul 30 15:30:35.023726 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:35.023702    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-2fa717fb-5f4cf45f5b-dtk62" podUID="635e6a47-24f8-4870-8ab2-69fd622906e7" containerName="payload-pre-processing" containerID="cri-o://5e5b516e03352b548ebd1f45518c39b63c00d1d139c630b364d36ef89a207cb7" gracePeriod=30
Jul 30 15:30:35.281865 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:35.281838    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa717fb-bcdcb4597-zqcfj"
Jul 30 15:30:35.297569 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:35.297537    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-2fa717fb-74cddd68f8-szfpr"
Jul 30 15:30:35.325547 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:35.325524    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-2fa717fb-5f4cf45f5b-dtk62"
Jul 30 15:30:35.343006 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:35.342976    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fzrrg\" (UniqueName: \"kubernetes.io/projected/74b8a938-8751-4fc0-8b2b-ff82e3affb99-kube-api-access-fzrrg\") pod \"74b8a938-8751-4fc0-8b2b-ff82e3affb99\" (UID: \"74b8a938-8751-4fc0-8b2b-ff82e3affb99\") "
Jul 30 15:30:35.343179 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:35.343065    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qpxn9\" (UniqueName: \"kubernetes.io/projected/64d2af35-2264-4579-9686-7986f44aa5c5-kube-api-access-qpxn9\") pod \"64d2af35-2264-4579-9686-7986f44aa5c5\" (UID: \"64d2af35-2264-4579-9686-7986f44aa5c5\") "
Jul 30 15:30:35.343179 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:35.343106    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74b8a938-8751-4fc0-8b2b-ff82e3affb99-openshift-service-ca\") pod \"74b8a938-8751-4fc0-8b2b-ff82e3affb99\" (UID: \"74b8a938-8751-4fc0-8b2b-ff82e3affb99\") "
Jul 30 15:30:35.343179 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:35.343170    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64d2af35-2264-4579-9686-7986f44aa5c5-plugins-config-volume\") pod \"64d2af35-2264-4579-9686-7986f44aa5c5\" (UID: \"64d2af35-2264-4579-9686-7986f44aa5c5\") "
Jul 30 15:30:35.343354 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:35.343214    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74b8a938-8751-4fc0-8b2b-ff82e3affb99-maas-api-tls\") pod \"74b8a938-8751-4fc0-8b2b-ff82e3affb99\" (UID: \"74b8a938-8751-4fc0-8b2b-ff82e3affb99\") "
Jul 30 15:30:35.343776 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:35.343732    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74b8a938-8751-4fc0-8b2b-ff82e3affb99-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "74b8a938-8751-4fc0-8b2b-ff82e3affb99" (UID: "74b8a938-8751-4fc0-8b2b-ff82e3affb99"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:30:35.343909 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:35.343783    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64d2af35-2264-4579-9686-7986f44aa5c5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "64d2af35-2264-4579-9686-7986f44aa5c5" (UID: "64d2af35-2264-4579-9686-7986f44aa5c5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:30:35.345435 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:35.345409    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74b8a938-8751-4fc0-8b2b-ff82e3affb99-kube-api-access-fzrrg" (OuterVolumeSpecName: "kube-api-access-fzrrg") pod "74b8a938-8751-4fc0-8b2b-ff82e3affb99" (UID: "74b8a938-8751-4fc0-8b2b-ff82e3affb99"). InnerVolumeSpecName "kube-api-access-fzrrg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:30:35.345628 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:35.345593    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64d2af35-2264-4579-9686-7986f44aa5c5-kube-api-access-qpxn9" (OuterVolumeSpecName: "kube-api-access-qpxn9") pod "64d2af35-2264-4579-9686-7986f44aa5c5" (UID: "64d2af35-2264-4579-9686-7986f44aa5c5"). InnerVolumeSpecName "kube-api-access-qpxn9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:30:35.345719 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:35.345613    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74b8a938-8751-4fc0-8b2b-ff82e3affb99-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "74b8a938-8751-4fc0-8b2b-ff82e3affb99" (UID: "74b8a938-8751-4fc0-8b2b-ff82e3affb99"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 15:30:35.444363 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:35.444324    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/635e6a47-24f8-4870-8ab2-69fd622906e7-plugins-config-volume\") pod \"635e6a47-24f8-4870-8ab2-69fd622906e7\" (UID: \"635e6a47-24f8-4870-8ab2-69fd622906e7\") "
Jul 30 15:30:35.444530 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:35.444431    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h6gfq\" (UniqueName: \"kubernetes.io/projected/635e6a47-24f8-4870-8ab2-69fd622906e7-kube-api-access-h6gfq\") pod \"635e6a47-24f8-4870-8ab2-69fd622906e7\" (UID: \"635e6a47-24f8-4870-8ab2-69fd622906e7\") "
Jul 30 15:30:35.444596 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:35.444585    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qpxn9\" (UniqueName: \"kubernetes.io/projected/64d2af35-2264-4579-9686-7986f44aa5c5-kube-api-access-qpxn9\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:30:35.444638 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:35.444598    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74b8a938-8751-4fc0-8b2b-ff82e3affb99-openshift-service-ca\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:30:35.444638 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:35.444607    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64d2af35-2264-4579-9686-7986f44aa5c5-plugins-config-volume\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:30:35.444638 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:35.444617    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74b8a938-8751-4fc0-8b2b-ff82e3affb99-maas-api-tls\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:30:35.444638 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:35.444627    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fzrrg\" (UniqueName: \"kubernetes.io/projected/74b8a938-8751-4fc0-8b2b-ff82e3affb99-kube-api-access-fzrrg\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:30:35.444805 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:35.444673    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/635e6a47-24f8-4870-8ab2-69fd622906e7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "635e6a47-24f8-4870-8ab2-69fd622906e7" (UID: "635e6a47-24f8-4870-8ab2-69fd622906e7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:30:35.446504 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:35.446484    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/635e6a47-24f8-4870-8ab2-69fd622906e7-kube-api-access-h6gfq" (OuterVolumeSpecName: "kube-api-access-h6gfq") pod "635e6a47-24f8-4870-8ab2-69fd622906e7" (UID: "635e6a47-24f8-4870-8ab2-69fd622906e7"). InnerVolumeSpecName "kube-api-access-h6gfq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:30:35.545397 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:35.545326    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h6gfq\" (UniqueName: \"kubernetes.io/projected/635e6a47-24f8-4870-8ab2-69fd622906e7-kube-api-access-h6gfq\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:30:35.545397 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:35.545351    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/635e6a47-24f8-4870-8ab2-69fd622906e7-plugins-config-volume\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:30:35.644573 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:35.644542    2578 generic.go:356] "Generic (PLEG): container finished" podID="74b8a938-8751-4fc0-8b2b-ff82e3affb99" containerID="0cccce73bd3f169ca141f2cfe00be0d75a775f30da14fae8fdccee62e300b8f7" exitCode=0
Jul 30 15:30:35.644743 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:35.644588    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa717fb-bcdcb4597-zqcfj"
Jul 30 15:30:35.644743 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:35.644613    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa717fb-bcdcb4597-zqcfj" event={"ID":"74b8a938-8751-4fc0-8b2b-ff82e3affb99","Type":"ContainerDied","Data":"0cccce73bd3f169ca141f2cfe00be0d75a775f30da14fae8fdccee62e300b8f7"}
Jul 30 15:30:35.644743 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:35.644661    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa717fb-bcdcb4597-zqcfj" event={"ID":"74b8a938-8751-4fc0-8b2b-ff82e3affb99","Type":"ContainerDied","Data":"e69d71c2856046b44a953b1db8bc2c85071235ee10881e571731445c6eb769cb"}
Jul 30 15:30:35.644743 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:35.644683    2578 scope.go:117] "RemoveContainer" containerID="0cccce73bd3f169ca141f2cfe00be0d75a775f30da14fae8fdccee62e300b8f7"
Jul 30 15:30:35.645919 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:35.645894    2578 generic.go:356] "Generic (PLEG): container finished" podID="635e6a47-24f8-4870-8ab2-69fd622906e7" containerID="5e5b516e03352b548ebd1f45518c39b63c00d1d139c630b364d36ef89a207cb7" exitCode=0
Jul 30 15:30:35.645997 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:35.645930    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-2fa717fb-5f4cf45f5b-dtk62"
Jul 30 15:30:35.645997 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:35.645978    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-2fa717fb-5f4cf45f5b-dtk62" event={"ID":"635e6a47-24f8-4870-8ab2-69fd622906e7","Type":"ContainerDied","Data":"5e5b516e03352b548ebd1f45518c39b63c00d1d139c630b364d36ef89a207cb7"}
Jul 30 15:30:35.646119 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:35.646023    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-2fa717fb-5f4cf45f5b-dtk62" event={"ID":"635e6a47-24f8-4870-8ab2-69fd622906e7","Type":"ContainerDied","Data":"d06de11d98f155e019a3f15cbb2444991a9d46e56d4997183c193514e265ab37"}
Jul 30 15:30:35.647370 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:35.647343    2578 generic.go:356] "Generic (PLEG): container finished" podID="64d2af35-2264-4579-9686-7986f44aa5c5" containerID="1a3d13fe868ddd43f5fc0fff98377bbf935a5c1de90dcdb74e466e2c4d4f39c0" exitCode=0
Jul 30 15:30:35.647534 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:35.647392    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-2fa717fb-74cddd68f8-szfpr"
Jul 30 15:30:35.647534 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:35.647444    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-2fa717fb-74cddd68f8-szfpr" event={"ID":"64d2af35-2264-4579-9686-7986f44aa5c5","Type":"ContainerDied","Data":"1a3d13fe868ddd43f5fc0fff98377bbf935a5c1de90dcdb74e466e2c4d4f39c0"}
Jul 30 15:30:35.647534 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:35.647503    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-2fa717fb-74cddd68f8-szfpr" event={"ID":"64d2af35-2264-4579-9686-7986f44aa5c5","Type":"ContainerDied","Data":"e4dba5863bbf34516da63ce0387a37403af8e10ca04728662d58d5aba64797bd"}
Jul 30 15:30:35.655720 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:35.655704    2578 scope.go:117] "RemoveContainer" containerID="0cccce73bd3f169ca141f2cfe00be0d75a775f30da14fae8fdccee62e300b8f7"
Jul 30 15:30:35.656033 ip-10-0-133-117 kubenswrapper[2578]: E0730 15:30:35.656005    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0cccce73bd3f169ca141f2cfe00be0d75a775f30da14fae8fdccee62e300b8f7\": container with ID starting with 0cccce73bd3f169ca141f2cfe00be0d75a775f30da14fae8fdccee62e300b8f7 not found: ID does not exist" containerID="0cccce73bd3f169ca141f2cfe00be0d75a775f30da14fae8fdccee62e300b8f7"
Jul 30 15:30:35.656113 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:35.656044    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cccce73bd3f169ca141f2cfe00be0d75a775f30da14fae8fdccee62e300b8f7"} err="failed to get container status \"0cccce73bd3f169ca141f2cfe00be0d75a775f30da14fae8fdccee62e300b8f7\": rpc error: code = NotFound desc = could not find container \"0cccce73bd3f169ca141f2cfe00be0d75a775f30da14fae8fdccee62e300b8f7\": container with ID starting with 0cccce73bd3f169ca141f2cfe00be0d75a775f30da14fae8fdccee62e300b8f7 not found: ID does not exist"
Jul 30 15:30:35.656113 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:35.656070    2578 scope.go:117] "RemoveContainer" containerID="5e5b516e03352b548ebd1f45518c39b63c00d1d139c630b364d36ef89a207cb7"
Jul 30 15:30:35.663971 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:35.663954    2578 scope.go:117] "RemoveContainer" containerID="5e5b516e03352b548ebd1f45518c39b63c00d1d139c630b364d36ef89a207cb7"
Jul 30 15:30:35.664277 ip-10-0-133-117 kubenswrapper[2578]: E0730 15:30:35.664257    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e5b516e03352b548ebd1f45518c39b63c00d1d139c630b364d36ef89a207cb7\": container with ID starting with 5e5b516e03352b548ebd1f45518c39b63c00d1d139c630b364d36ef89a207cb7 not found: ID does not exist" containerID="5e5b516e03352b548ebd1f45518c39b63c00d1d139c630b364d36ef89a207cb7"
Jul 30 15:30:35.664346 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:35.664283    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e5b516e03352b548ebd1f45518c39b63c00d1d139c630b364d36ef89a207cb7"} err="failed to get container status \"5e5b516e03352b548ebd1f45518c39b63c00d1d139c630b364d36ef89a207cb7\": rpc error: code = NotFound desc = could not find container \"5e5b516e03352b548ebd1f45518c39b63c00d1d139c630b364d36ef89a207cb7\": container with ID starting with 5e5b516e03352b548ebd1f45518c39b63c00d1d139c630b364d36ef89a207cb7 not found: ID does not exist"
Jul 30 15:30:35.664346 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:35.664301    2578 scope.go:117] "RemoveContainer" containerID="1a3d13fe868ddd43f5fc0fff98377bbf935a5c1de90dcdb74e466e2c4d4f39c0"
Jul 30 15:30:35.668041 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:35.668019    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2fa717fb-bcdcb4597-zqcfj"]
Jul 30 15:30:35.670053 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:35.670034    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2fa717fb-bcdcb4597-zqcfj"]
Jul 30 15:30:35.673178 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:35.673160    2578 scope.go:117] "RemoveContainer" containerID="1a3d13fe868ddd43f5fc0fff98377bbf935a5c1de90dcdb74e466e2c4d4f39c0"
Jul 30 15:30:35.673437 ip-10-0-133-117 kubenswrapper[2578]: E0730 15:30:35.673419    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a3d13fe868ddd43f5fc0fff98377bbf935a5c1de90dcdb74e466e2c4d4f39c0\": container with ID starting with 1a3d13fe868ddd43f5fc0fff98377bbf935a5c1de90dcdb74e466e2c4d4f39c0 not found: ID does not exist" containerID="1a3d13fe868ddd43f5fc0fff98377bbf935a5c1de90dcdb74e466e2c4d4f39c0"
Jul 30 15:30:35.673538 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:35.673446    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a3d13fe868ddd43f5fc0fff98377bbf935a5c1de90dcdb74e466e2c4d4f39c0"} err="failed to get container status \"1a3d13fe868ddd43f5fc0fff98377bbf935a5c1de90dcdb74e466e2c4d4f39c0\": rpc error: code = NotFound desc = could not find container \"1a3d13fe868ddd43f5fc0fff98377bbf935a5c1de90dcdb74e466e2c4d4f39c0\": container with ID starting with 1a3d13fe868ddd43f5fc0fff98377bbf935a5c1de90dcdb74e466e2c4d4f39c0 not found: ID does not exist"
Jul 30 15:30:35.676886 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:35.676864    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2fa717fb-5f4cf45f5b-dtk62"]
Jul 30 15:30:35.684570 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:35.684551    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2fa717fb-5f4cf45f5b-dtk62"]
Jul 30 15:30:35.694000 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:35.693970    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2fa717fb-74cddd68f8-szfpr"]
Jul 30 15:30:35.695676 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:35.695655    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2fa717fb-74cddd68f8-szfpr"]
Jul 30 15:30:36.978059 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:36.978029    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="635e6a47-24f8-4870-8ab2-69fd622906e7" path="/var/lib/kubelet/pods/635e6a47-24f8-4870-8ab2-69fd622906e7/volumes"
Jul 30 15:30:36.978454 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:36.978423    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64d2af35-2264-4579-9686-7986f44aa5c5" path="/var/lib/kubelet/pods/64d2af35-2264-4579-9686-7986f44aa5c5/volumes"
Jul 30 15:30:36.978763 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:30:36.978750    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74b8a938-8751-4fc0-8b2b-ff82e3affb99" path="/var/lib/kubelet/pods/74b8a938-8751-4fc0-8b2b-ff82e3affb99/volumes"
Jul 30 15:31:41.545681 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:31:41.545593    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn"]
Jul 30 15:31:41.546277 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:31:41.545816    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn" podUID="ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d" containerName="istio-proxy" containerID="cri-o://00c6d80a2fda8e7e6e15241c076a0b7138b06cece0f56f862a7ffb09fe4b7fc5" gracePeriod=30
Jul 30 15:31:46.801189 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:31:46.801135    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn"
Jul 30 15:31:46.908867 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:31:46.908762    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d-workload-certs\") pod \"ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d\" (UID: \"ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d\") "
Jul 30 15:31:46.908867 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:31:46.908816    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d-istio-token\") pod \"ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d\" (UID: \"ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d\") "
Jul 30 15:31:46.908867 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:31:46.908841    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d-workload-socket\") pod \"ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d\" (UID: \"ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d\") "
Jul 30 15:31:46.908867 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:31:46.908868    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d-istiod-ca-cert\") pod \"ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d\" (UID: \"ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d\") "
Jul 30 15:31:46.909279 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:31:46.908898    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d-credential-socket\") pod \"ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d\" (UID: \"ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d\") "
Jul 30 15:31:46.909279 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:31:46.908960    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d7p44\" (UniqueName: \"kubernetes.io/projected/ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d-kube-api-access-d7p44\") pod \"ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d\" (UID: \"ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d\") "
Jul 30 15:31:46.909279 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:31:46.909005    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d-istio-envoy\") pod \"ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d\" (UID: \"ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d\") "
Jul 30 15:31:46.909279 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:31:46.909112    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d-istio-podinfo\") pod \"ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d\" (UID: \"ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d\") "
Jul 30 15:31:46.909279 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:31:46.909175    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d" (UID: "ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:31:46.909279 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:31:46.909200    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d" (UID: "ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:31:46.909279 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:31:46.909185    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d-istio-data\") pod \"ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d\" (UID: \"ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d\") "
Jul 30 15:31:46.909279 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:31:46.909266    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d" (UID: "ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:31:46.909699 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:31:46.909322    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d" (UID: "ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:31:46.909699 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:31:46.909611    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d-workload-socket\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:31:46.909699 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:31:46.909634    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d-istiod-ca-cert\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:31:46.909699 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:31:46.909648    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d-credential-socket\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:31:46.909699 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:31:46.909662    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d-workload-certs\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:31:46.910093 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:31:46.910071    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d-istio-data" (OuterVolumeSpecName: "istio-data") pod "ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d" (UID: "ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:31:46.911259 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:31:46.911226    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d-istio-token" (OuterVolumeSpecName: "istio-token") pod "ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d" (UID: "ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:31:46.911259 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:31:46.911250    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d" (UID: "ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 15:31:46.911498 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:31:46.911483    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d" (UID: "ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:31:46.911619 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:31:46.911602    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d-kube-api-access-d7p44" (OuterVolumeSpecName: "kube-api-access-d7p44") pod "ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d" (UID: "ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d"). InnerVolumeSpecName "kube-api-access-d7p44". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:31:46.988885 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:31:46.988855    2578 generic.go:356] "Generic (PLEG): container finished" podID="ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d" containerID="00c6d80a2fda8e7e6e15241c076a0b7138b06cece0f56f862a7ffb09fe4b7fc5" exitCode=0
Jul 30 15:31:46.989040 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:31:46.988902    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn"
Jul 30 15:31:46.989040 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:31:46.988911    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn" event={"ID":"ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d","Type":"ContainerDied","Data":"00c6d80a2fda8e7e6e15241c076a0b7138b06cece0f56f862a7ffb09fe4b7fc5"}
Jul 30 15:31:46.989040 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:31:46.988966    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn" event={"ID":"ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d","Type":"ContainerDied","Data":"2e9d55054ab9708ed9c541e73a72cdfed7e941badb7a2e0e0e282264754da49e"}
Jul 30 15:31:46.989040 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:31:46.989002    2578 scope.go:117] "RemoveContainer" containerID="00c6d80a2fda8e7e6e15241c076a0b7138b06cece0f56f862a7ffb09fe4b7fc5"
Jul 30 15:31:46.998413 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:31:46.998393    2578 scope.go:117] "RemoveContainer" containerID="4827aecac6e04ab7b3093a4c1c3a998bd63ac784f7c65f3781dab305648207c8"
Jul 30 15:31:47.004752 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:31:47.004724    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn"]
Jul 30 15:31:47.007421 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:31:47.007401    2578 scope.go:117] "RemoveContainer" containerID="00c6d80a2fda8e7e6e15241c076a0b7138b06cece0f56f862a7ffb09fe4b7fc5"
Jul 30 15:31:47.007758 ip-10-0-133-117 kubenswrapper[2578]: E0730 15:31:47.007728    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00c6d80a2fda8e7e6e15241c076a0b7138b06cece0f56f862a7ffb09fe4b7fc5\": container with ID starting with 00c6d80a2fda8e7e6e15241c076a0b7138b06cece0f56f862a7ffb09fe4b7fc5 not found: ID does not exist" containerID="00c6d80a2fda8e7e6e15241c076a0b7138b06cece0f56f862a7ffb09fe4b7fc5"
Jul 30 15:31:47.007839 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:31:47.007788    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00c6d80a2fda8e7e6e15241c076a0b7138b06cece0f56f862a7ffb09fe4b7fc5"} err="failed to get container status \"00c6d80a2fda8e7e6e15241c076a0b7138b06cece0f56f862a7ffb09fe4b7fc5\": rpc error: code = NotFound desc = could not find container \"00c6d80a2fda8e7e6e15241c076a0b7138b06cece0f56f862a7ffb09fe4b7fc5\": container with ID starting with 00c6d80a2fda8e7e6e15241c076a0b7138b06cece0f56f862a7ffb09fe4b7fc5 not found: ID does not exist"
Jul 30 15:31:47.007839 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:31:47.007807    2578 scope.go:117] "RemoveContainer" containerID="4827aecac6e04ab7b3093a4c1c3a998bd63ac784f7c65f3781dab305648207c8"
Jul 30 15:31:47.008194 ip-10-0-133-117 kubenswrapper[2578]: E0730 15:31:47.008161    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4827aecac6e04ab7b3093a4c1c3a998bd63ac784f7c65f3781dab305648207c8\": container with ID starting with 4827aecac6e04ab7b3093a4c1c3a998bd63ac784f7c65f3781dab305648207c8 not found: ID does not exist" containerID="4827aecac6e04ab7b3093a4c1c3a998bd63ac784f7c65f3781dab305648207c8"
Jul 30 15:31:47.008307 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:31:47.008199    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4827aecac6e04ab7b3093a4c1c3a998bd63ac784f7c65f3781dab305648207c8"} err="failed to get container status \"4827aecac6e04ab7b3093a4c1c3a998bd63ac784f7c65f3781dab305648207c8\": rpc error: code = NotFound desc = could not find container \"4827aecac6e04ab7b3093a4c1c3a998bd63ac784f7c65f3781dab305648207c8\": container with ID starting with 4827aecac6e04ab7b3093a4c1c3a998bd63ac784f7c65f3781dab305648207c8 not found: ID does not exist"
Jul 30 15:31:47.010437 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:31:47.010414    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-2fa717fb-openshift-default-548544cf6-8xxcn"]
Jul 30 15:31:47.010852 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:31:47.010829    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d7p44\" (UniqueName: \"kubernetes.io/projected/ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d-kube-api-access-d7p44\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:31:47.010852 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:31:47.010852    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d-istio-envoy\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:31:47.011021 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:31:47.010874    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d-istio-podinfo\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:31:47.011021 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:31:47.010896    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d-istio-data\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:31:47.011021 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:31:47.010909    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d-istio-token\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:31:48.978784 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:31:48.978749    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d" path="/var/lib/kubelet/pods/ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d/volumes"
Jul 30 15:32:58.013741 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:32:58.013708    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9"]
Jul 30 15:32:58.014272 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:32:58.014047    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d" containerName="istio-proxy"
Jul 30 15:32:58.014272 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:32:58.014057    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d" containerName="istio-proxy"
Jul 30 15:32:58.014272 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:32:58.014066    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64d2af35-2264-4579-9686-7986f44aa5c5" containerName="payload-processing"
Jul 30 15:32:58.014272 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:32:58.014072    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="64d2af35-2264-4579-9686-7986f44aa5c5" containerName="payload-processing"
Jul 30 15:32:58.014272 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:32:58.014085    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d" containerName="istio-proxy"
Jul 30 15:32:58.014272 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:32:58.014091    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d" containerName="istio-proxy"
Jul 30 15:32:58.014272 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:32:58.014104    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="635e6a47-24f8-4870-8ab2-69fd622906e7" containerName="payload-pre-processing"
Jul 30 15:32:58.014272 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:32:58.014110    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="635e6a47-24f8-4870-8ab2-69fd622906e7" containerName="payload-pre-processing"
Jul 30 15:32:58.014272 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:32:58.014121    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74b8a938-8751-4fc0-8b2b-ff82e3affb99" containerName="maas-api"
Jul 30 15:32:58.014272 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:32:58.014126    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="74b8a938-8751-4fc0-8b2b-ff82e3affb99" containerName="maas-api"
Jul 30 15:32:58.014272 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:32:58.014214    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="64d2af35-2264-4579-9686-7986f44aa5c5" containerName="payload-processing"
Jul 30 15:32:58.014272 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:32:58.014223    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d" containerName="istio-proxy"
Jul 30 15:32:58.014272 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:32:58.014230    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="ecef9c8d-ff57-46c4-94d7-cffaa6ad2f3d" containerName="istio-proxy"
Jul 30 15:32:58.014272 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:32:58.014236    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="74b8a938-8751-4fc0-8b2b-ff82e3affb99" containerName="maas-api"
Jul 30 15:32:58.014272 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:32:58.014241    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="635e6a47-24f8-4870-8ab2-69fd622906e7" containerName="payload-pre-processing"
Jul 30 15:32:58.017122 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:32:58.017105    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9"
Jul 30 15:32:58.019071 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:32:58.019048    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 30 15:32:58.019205 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:32:58.019048    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-f03573-openshift-default-dockercfg-5hs7x\""
Jul 30 15:32:58.028097 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:32:58.028075    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9"]
Jul 30 15:32:58.164976 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:32:58.164941    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2a022a4-6a44-49c7-81fb-4e70badb77f3-istio-data\") pod \"e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9\" (UID: \"e2a022a4-6a44-49c7-81fb-4e70badb77f3\") " pod="openshift-ingress/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9"
Jul 30 15:32:58.165189 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:32:58.165006    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2a022a4-6a44-49c7-81fb-4e70badb77f3-workload-certs\") pod \"e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9\" (UID: \"e2a022a4-6a44-49c7-81fb-4e70badb77f3\") " pod="openshift-ingress/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9"
Jul 30 15:32:58.165189 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:32:58.165049    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbc9r\" (UniqueName: \"kubernetes.io/projected/e2a022a4-6a44-49c7-81fb-4e70badb77f3-kube-api-access-nbc9r\") pod \"e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9\" (UID: \"e2a022a4-6a44-49c7-81fb-4e70badb77f3\") " pod="openshift-ingress/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9"
Jul 30 15:32:58.165189 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:32:58.165093    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2a022a4-6a44-49c7-81fb-4e70badb77f3-workload-socket\") pod \"e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9\" (UID: \"e2a022a4-6a44-49c7-81fb-4e70badb77f3\") " pod="openshift-ingress/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9"
Jul 30 15:32:58.165189 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:32:58.165136    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2a022a4-6a44-49c7-81fb-4e70badb77f3-istio-envoy\") pod \"e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9\" (UID: \"e2a022a4-6a44-49c7-81fb-4e70badb77f3\") " pod="openshift-ingress/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9"
Jul 30 15:32:58.165189 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:32:58.165167    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2a022a4-6a44-49c7-81fb-4e70badb77f3-istio-token\") pod \"e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9\" (UID: \"e2a022a4-6a44-49c7-81fb-4e70badb77f3\") " pod="openshift-ingress/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9"
Jul 30 15:32:58.165459 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:32:58.165212    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2a022a4-6a44-49c7-81fb-4e70badb77f3-istio-podinfo\") pod \"e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9\" (UID: \"e2a022a4-6a44-49c7-81fb-4e70badb77f3\") " pod="openshift-ingress/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9"
Jul 30 15:32:58.165459 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:32:58.165257    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2a022a4-6a44-49c7-81fb-4e70badb77f3-credential-socket\") pod \"e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9\" (UID: \"e2a022a4-6a44-49c7-81fb-4e70badb77f3\") " pod="openshift-ingress/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9"
Jul 30 15:32:58.165459 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:32:58.165314    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2a022a4-6a44-49c7-81fb-4e70badb77f3-istiod-ca-cert\") pod \"e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9\" (UID: \"e2a022a4-6a44-49c7-81fb-4e70badb77f3\") " pod="openshift-ingress/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9"
Jul 30 15:32:58.266824 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:32:58.266746    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2a022a4-6a44-49c7-81fb-4e70badb77f3-istio-data\") pod \"e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9\" (UID: \"e2a022a4-6a44-49c7-81fb-4e70badb77f3\") " pod="openshift-ingress/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9"
Jul 30 15:32:58.266824 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:32:58.266796    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2a022a4-6a44-49c7-81fb-4e70badb77f3-workload-certs\") pod \"e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9\" (UID: \"e2a022a4-6a44-49c7-81fb-4e70badb77f3\") " pod="openshift-ingress/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9"
Jul 30 15:32:58.267051 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:32:58.266828    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbc9r\" (UniqueName: \"kubernetes.io/projected/e2a022a4-6a44-49c7-81fb-4e70badb77f3-kube-api-access-nbc9r\") pod \"e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9\" (UID: \"e2a022a4-6a44-49c7-81fb-4e70badb77f3\") " pod="openshift-ingress/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9"
Jul 30 15:32:58.267051 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:32:58.266877    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2a022a4-6a44-49c7-81fb-4e70badb77f3-workload-socket\") pod \"e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9\" (UID: \"e2a022a4-6a44-49c7-81fb-4e70badb77f3\") " pod="openshift-ingress/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9"
Jul 30 15:32:58.267051 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:32:58.266911    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2a022a4-6a44-49c7-81fb-4e70badb77f3-istio-envoy\") pod \"e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9\" (UID: \"e2a022a4-6a44-49c7-81fb-4e70badb77f3\") " pod="openshift-ingress/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9"
Jul 30 15:32:58.267259 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:32:58.267037    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2a022a4-6a44-49c7-81fb-4e70badb77f3-istio-token\") pod \"e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9\" (UID: \"e2a022a4-6a44-49c7-81fb-4e70badb77f3\") " pod="openshift-ingress/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9"
Jul 30 15:32:58.267259 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:32:58.267096    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2a022a4-6a44-49c7-81fb-4e70badb77f3-istio-podinfo\") pod \"e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9\" (UID: \"e2a022a4-6a44-49c7-81fb-4e70badb77f3\") " pod="openshift-ingress/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9"
Jul 30 15:32:58.267259 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:32:58.267175    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2a022a4-6a44-49c7-81fb-4e70badb77f3-credential-socket\") pod \"e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9\" (UID: \"e2a022a4-6a44-49c7-81fb-4e70badb77f3\") " pod="openshift-ingress/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9"
Jul 30 15:32:58.267259 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:32:58.267180    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2a022a4-6a44-49c7-81fb-4e70badb77f3-istio-data\") pod \"e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9\" (UID: \"e2a022a4-6a44-49c7-81fb-4e70badb77f3\") " pod="openshift-ingress/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9"
Jul 30 15:32:58.267259 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:32:58.267243    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2a022a4-6a44-49c7-81fb-4e70badb77f3-istiod-ca-cert\") pod \"e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9\" (UID: \"e2a022a4-6a44-49c7-81fb-4e70badb77f3\") " pod="openshift-ingress/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9"
Jul 30 15:32:58.267523 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:32:58.267286    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2a022a4-6a44-49c7-81fb-4e70badb77f3-workload-socket\") pod \"e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9\" (UID: \"e2a022a4-6a44-49c7-81fb-4e70badb77f3\") " pod="openshift-ingress/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9"
Jul 30 15:32:58.267523 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:32:58.267435    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2a022a4-6a44-49c7-81fb-4e70badb77f3-credential-socket\") pod \"e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9\" (UID: \"e2a022a4-6a44-49c7-81fb-4e70badb77f3\") " pod="openshift-ingress/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9"
Jul 30 15:32:58.267523 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:32:58.267478    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2a022a4-6a44-49c7-81fb-4e70badb77f3-workload-certs\") pod \"e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9\" (UID: \"e2a022a4-6a44-49c7-81fb-4e70badb77f3\") " pod="openshift-ingress/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9"
Jul 30 15:32:58.267908 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:32:58.267890    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2a022a4-6a44-49c7-81fb-4e70badb77f3-istiod-ca-cert\") pod \"e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9\" (UID: \"e2a022a4-6a44-49c7-81fb-4e70badb77f3\") " pod="openshift-ingress/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9"
Jul 30 15:32:58.269522 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:32:58.269498    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2a022a4-6a44-49c7-81fb-4e70badb77f3-istio-envoy\") pod \"e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9\" (UID: \"e2a022a4-6a44-49c7-81fb-4e70badb77f3\") " pod="openshift-ingress/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9"
Jul 30 15:32:58.269703 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:32:58.269687    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2a022a4-6a44-49c7-81fb-4e70badb77f3-istio-podinfo\") pod \"e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9\" (UID: \"e2a022a4-6a44-49c7-81fb-4e70badb77f3\") " pod="openshift-ingress/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9"
Jul 30 15:32:58.273981 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:32:58.273960    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2a022a4-6a44-49c7-81fb-4e70badb77f3-istio-token\") pod \"e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9\" (UID: \"e2a022a4-6a44-49c7-81fb-4e70badb77f3\") " pod="openshift-ingress/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9"
Jul 30 15:32:58.274136 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:32:58.274119    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbc9r\" (UniqueName: \"kubernetes.io/projected/e2a022a4-6a44-49c7-81fb-4e70badb77f3-kube-api-access-nbc9r\") pod \"e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9\" (UID: \"e2a022a4-6a44-49c7-81fb-4e70badb77f3\") " pod="openshift-ingress/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9"
Jul 30 15:32:58.328316 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:32:58.328286    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9"
Jul 30 15:32:58.483701 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:32:58.483673    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9"]
Jul 30 15:32:58.485572 ip-10-0-133-117 kubenswrapper[2578]: W0730 15:32:58.485548    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2a022a4_6a44_49c7_81fb_4e70badb77f3.slice/crio-9a961ada0405a80dd00bd380e28883d053ee60d56c65d5123aefe597fef6f431 WatchSource:0}: Error finding container 9a961ada0405a80dd00bd380e28883d053ee60d56c65d5123aefe597fef6f431: Status 404 returned error can't find the container with id 9a961ada0405a80dd00bd380e28883d053ee60d56c65d5123aefe597fef6f431
Jul 30 15:32:58.488924 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:32:58.488207    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:32:58.488924 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:32:58.488296    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:32:58.488924 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:32:58.488338    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:32:59.325901 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:32:59.325859    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9" event={"ID":"e2a022a4-6a44-49c7-81fb-4e70badb77f3","Type":"ContainerStarted","Data":"c2d6d522c6ece646e9683d39d17376d289ad57a2eef9cf160e535856999fa01f"}
Jul 30 15:32:59.326311 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:32:59.325924    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9" event={"ID":"e2a022a4-6a44-49c7-81fb-4e70badb77f3","Type":"ContainerStarted","Data":"9a961ada0405a80dd00bd380e28883d053ee60d56c65d5123aefe597fef6f431"}
Jul 30 15:32:59.328807 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:32:59.328789    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9"
Jul 30 15:32:59.333666 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:32:59.333642    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9"
Jul 30 15:32:59.345074 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:32:59.345025    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9" podStartSLOduration=2.345010449 podStartE2EDuration="2.345010449s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:32:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:32:59.343295361 +0000 UTC m=+2690.872233836" watchObservedRunningTime="2026-07-30 15:32:59.345010449 +0000 UTC m=+2690.873948922"
Jul 30 15:33:00.330279 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:00.330246    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9"
Jul 30 15:33:00.331105 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:00.331087    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9"
Jul 30 15:33:02.503462 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:02.503376    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-f03573-d99d45d55-nw8lp"]
Jul 30 15:33:02.507286 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:02.507265    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f03573-d99d45d55-nw8lp"
Jul 30 15:33:02.509328 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:02.509302    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-f03573-serving-cert\""
Jul 30 15:33:02.519282 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:02.518606    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-f03573-d99d45d55-nw8lp"]
Jul 30 15:33:02.608718 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:02.608669    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rb2j\" (UniqueName: \"kubernetes.io/projected/81c06155-cded-44ca-b41f-d733a6f54fad-kube-api-access-8rb2j\") pod \"maas-api-e2e-inf-a-f03573-d99d45d55-nw8lp\" (UID: \"81c06155-cded-44ca-b41f-d733a6f54fad\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f03573-d99d45d55-nw8lp"
Jul 30 15:33:02.608887 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:02.608801    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81c06155-cded-44ca-b41f-d733a6f54fad-maas-api-tls\") pod \"maas-api-e2e-inf-a-f03573-d99d45d55-nw8lp\" (UID: \"81c06155-cded-44ca-b41f-d733a6f54fad\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f03573-d99d45d55-nw8lp"
Jul 30 15:33:02.608944 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:02.608889    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81c06155-cded-44ca-b41f-d733a6f54fad-openshift-service-ca\") pod \"maas-api-e2e-inf-a-f03573-d99d45d55-nw8lp\" (UID: \"81c06155-cded-44ca-b41f-d733a6f54fad\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f03573-d99d45d55-nw8lp"
Jul 30 15:33:02.710026 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:02.709991    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81c06155-cded-44ca-b41f-d733a6f54fad-maas-api-tls\") pod \"maas-api-e2e-inf-a-f03573-d99d45d55-nw8lp\" (UID: \"81c06155-cded-44ca-b41f-d733a6f54fad\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f03573-d99d45d55-nw8lp"
Jul 30 15:33:02.710204 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:02.710044    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81c06155-cded-44ca-b41f-d733a6f54fad-openshift-service-ca\") pod \"maas-api-e2e-inf-a-f03573-d99d45d55-nw8lp\" (UID: \"81c06155-cded-44ca-b41f-d733a6f54fad\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f03573-d99d45d55-nw8lp"
Jul 30 15:33:02.710204 ip-10-0-133-117 kubenswrapper[2578]: E0730 15:33:02.710169    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-f03573-serving-cert: secret "maas-api-e2e-inf-a-f03573-serving-cert" not found
Jul 30 15:33:02.710338 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:02.710220    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rb2j\" (UniqueName: \"kubernetes.io/projected/81c06155-cded-44ca-b41f-d733a6f54fad-kube-api-access-8rb2j\") pod \"maas-api-e2e-inf-a-f03573-d99d45d55-nw8lp\" (UID: \"81c06155-cded-44ca-b41f-d733a6f54fad\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f03573-d99d45d55-nw8lp"
Jul 30 15:33:02.710338 ip-10-0-133-117 kubenswrapper[2578]: E0730 15:33:02.710250    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81c06155-cded-44ca-b41f-d733a6f54fad-maas-api-tls podName:81c06155-cded-44ca-b41f-d733a6f54fad nodeName:}" failed. No retries permitted until 2026-07-30 15:33:03.210227479 +0000 UTC m=+2694.739165947 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/81c06155-cded-44ca-b41f-d733a6f54fad-maas-api-tls") pod "maas-api-e2e-inf-a-f03573-d99d45d55-nw8lp" (UID: "81c06155-cded-44ca-b41f-d733a6f54fad") : secret "maas-api-e2e-inf-a-f03573-serving-cert" not found
Jul 30 15:33:02.710701 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:02.710683    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81c06155-cded-44ca-b41f-d733a6f54fad-openshift-service-ca\") pod \"maas-api-e2e-inf-a-f03573-d99d45d55-nw8lp\" (UID: \"81c06155-cded-44ca-b41f-d733a6f54fad\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f03573-d99d45d55-nw8lp"
Jul 30 15:33:02.717916 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:02.717895    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rb2j\" (UniqueName: \"kubernetes.io/projected/81c06155-cded-44ca-b41f-d733a6f54fad-kube-api-access-8rb2j\") pod \"maas-api-e2e-inf-a-f03573-d99d45d55-nw8lp\" (UID: \"81c06155-cded-44ca-b41f-d733a6f54fad\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f03573-d99d45d55-nw8lp"
Jul 30 15:33:03.138946 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:03.138914    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-f03573-c74989d7d-rhzsp"]
Jul 30 15:33:03.142442 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:03.142423    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-f03573-c74989d7d-rhzsp"
Jul 30 15:33:03.144410 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:03.144382    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-f03573\""
Jul 30 15:33:03.144525 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:03.144464    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-f03573-dockercfg-5bltg\""
Jul 30 15:33:03.151574 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:03.151548    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-f03573-c74989d7d-rhzsp"]
Jul 30 15:33:03.214811 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:03.214781    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e041fcc6-edac-46d8-b5e3-7875d26ffa13-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-f03573-c74989d7d-rhzsp\" (UID: \"e041fcc6-edac-46d8-b5e3-7875d26ffa13\") " pod="openshift-ingress/payload-processing-e2e-inf-a-f03573-c74989d7d-rhzsp"
Jul 30 15:33:03.214968 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:03.214847    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtzsk\" (UniqueName: \"kubernetes.io/projected/e041fcc6-edac-46d8-b5e3-7875d26ffa13-kube-api-access-qtzsk\") pod \"payload-processing-e2e-inf-a-f03573-c74989d7d-rhzsp\" (UID: \"e041fcc6-edac-46d8-b5e3-7875d26ffa13\") " pod="openshift-ingress/payload-processing-e2e-inf-a-f03573-c74989d7d-rhzsp"
Jul 30 15:33:03.214968 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:03.214898    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81c06155-cded-44ca-b41f-d733a6f54fad-maas-api-tls\") pod \"maas-api-e2e-inf-a-f03573-d99d45d55-nw8lp\" (UID: \"81c06155-cded-44ca-b41f-d733a6f54fad\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f03573-d99d45d55-nw8lp"
Jul 30 15:33:03.217106 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:03.217081    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81c06155-cded-44ca-b41f-d733a6f54fad-maas-api-tls\") pod \"maas-api-e2e-inf-a-f03573-d99d45d55-nw8lp\" (UID: \"81c06155-cded-44ca-b41f-d733a6f54fad\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f03573-d99d45d55-nw8lp"
Jul 30 15:33:03.316327 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:03.316295    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtzsk\" (UniqueName: \"kubernetes.io/projected/e041fcc6-edac-46d8-b5e3-7875d26ffa13-kube-api-access-qtzsk\") pod \"payload-processing-e2e-inf-a-f03573-c74989d7d-rhzsp\" (UID: \"e041fcc6-edac-46d8-b5e3-7875d26ffa13\") " pod="openshift-ingress/payload-processing-e2e-inf-a-f03573-c74989d7d-rhzsp"
Jul 30 15:33:03.316499 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:03.316427    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e041fcc6-edac-46d8-b5e3-7875d26ffa13-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-f03573-c74989d7d-rhzsp\" (UID: \"e041fcc6-edac-46d8-b5e3-7875d26ffa13\") " pod="openshift-ingress/payload-processing-e2e-inf-a-f03573-c74989d7d-rhzsp"
Jul 30 15:33:03.317104 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:03.317085    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e041fcc6-edac-46d8-b5e3-7875d26ffa13-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-f03573-c74989d7d-rhzsp\" (UID: \"e041fcc6-edac-46d8-b5e3-7875d26ffa13\") " pod="openshift-ingress/payload-processing-e2e-inf-a-f03573-c74989d7d-rhzsp"
Jul 30 15:33:03.323373 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:03.323342    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtzsk\" (UniqueName: \"kubernetes.io/projected/e041fcc6-edac-46d8-b5e3-7875d26ffa13-kube-api-access-qtzsk\") pod \"payload-processing-e2e-inf-a-f03573-c74989d7d-rhzsp\" (UID: \"e041fcc6-edac-46d8-b5e3-7875d26ffa13\") " pod="openshift-ingress/payload-processing-e2e-inf-a-f03573-c74989d7d-rhzsp"
Jul 30 15:33:03.436030 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:03.435947    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f03573-d99d45d55-nw8lp"
Jul 30 15:33:03.454190 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:03.454161    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-f03573-c74989d7d-rhzsp"
Jul 30 15:33:03.609086 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:03.609057    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-f03573-c74989d7d-rhzsp"]
Jul 30 15:33:03.610727 ip-10-0-133-117 kubenswrapper[2578]: W0730 15:33:03.610700    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode041fcc6_edac_46d8_b5e3_7875d26ffa13.slice/crio-b184fc27d608318b91d470e0f7584a005ad88c952ac0dc783a25258719017f9c WatchSource:0}: Error finding container b184fc27d608318b91d470e0f7584a005ad88c952ac0dc783a25258719017f9c: Status 404 returned error can't find the container with id b184fc27d608318b91d470e0f7584a005ad88c952ac0dc783a25258719017f9c
Jul 30 15:33:03.791264 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:03.791237    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-f03573-d99d45d55-nw8lp"]
Jul 30 15:33:03.793244 ip-10-0-133-117 kubenswrapper[2578]: W0730 15:33:03.793217    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81c06155_cded_44ca_b41f_d733a6f54fad.slice/crio-3ca5289e54262f614cdab7976b44dbbbe493950f6618a5a02c169c99112de964 WatchSource:0}: Error finding container 3ca5289e54262f614cdab7976b44dbbbe493950f6618a5a02c169c99112de964: Status 404 returned error can't find the container with id 3ca5289e54262f614cdab7976b44dbbbe493950f6618a5a02c169c99112de964
Jul 30 15:33:04.350095 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:04.350039    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-f03573-c74989d7d-rhzsp" event={"ID":"e041fcc6-edac-46d8-b5e3-7875d26ffa13","Type":"ContainerStarted","Data":"2bc4efd0d0bb4d751677e6b9365ccdfdb0f2cda2b23a45283cab2b38a597f785"}
Jul 30 15:33:04.350320 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:04.350211    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-f03573-c74989d7d-rhzsp"
Jul 30 15:33:04.350320 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:04.350266    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-f03573-c74989d7d-rhzsp" event={"ID":"e041fcc6-edac-46d8-b5e3-7875d26ffa13","Type":"ContainerStarted","Data":"b184fc27d608318b91d470e0f7584a005ad88c952ac0dc783a25258719017f9c"}
Jul 30 15:33:04.351583 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:04.351561    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f03573-d99d45d55-nw8lp" event={"ID":"81c06155-cded-44ca-b41f-d733a6f54fad","Type":"ContainerStarted","Data":"974c0c19d5896e53a07df7348270b63716b25d13ea21c727c5878d58f9a31a1d"}
Jul 30 15:33:04.351697 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:04.351595    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f03573-d99d45d55-nw8lp" event={"ID":"81c06155-cded-44ca-b41f-d733a6f54fad","Type":"ContainerStarted","Data":"3ca5289e54262f614cdab7976b44dbbbe493950f6618a5a02c169c99112de964"}
Jul 30 15:33:04.351697 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:04.351630    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f03573-d99d45d55-nw8lp"
Jul 30 15:33:04.365244 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:04.365202    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-f03573-c74989d7d-rhzsp" podStartSLOduration=1.365187548 podStartE2EDuration="1.365187548s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:33:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:33:04.364224833 +0000 UTC m=+2695.893163307" watchObservedRunningTime="2026-07-30 15:33:04.365187548 +0000 UTC m=+2695.894126042"
Jul 30 15:33:04.378774 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:04.378736    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f03573-d99d45d55-nw8lp" podStartSLOduration=1.905726577 podStartE2EDuration="2.378721803s" totalImagesPullingTime="472.995226ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:33:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:33:04.37766991 +0000 UTC m=+2695.906608407" watchObservedRunningTime="2026-07-30 15:33:04.378721803 +0000 UTC m=+2695.907660276"
Jul 30 15:33:09.117295 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:09.117266    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4mjmp_de241220-5f27-4b0b-b7c6-6d4b350e24e1/console-operator/1.log"
Jul 30 15:33:09.124668 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:09.124644    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4mjmp_de241220-5f27-4b0b-b7c6-6d4b350e24e1/console-operator/1.log"
Jul 30 15:33:10.362324 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:10.362292    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f03573-d99d45d55-nw8lp"
Jul 30 15:33:15.359387 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:15.359352    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-f03573-c74989d7d-rhzsp"
Jul 30 15:33:15.412182 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:15.412136    2578 patch_prober.go:28] interesting pod/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": read tcp 10.133.0.2:57156->10.133.0.60:15021: read: connection reset by peer" start-of-body=
Jul 30 15:33:15.412318 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:15.412253    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9" podUID="e2a022a4-6a44-49c7-81fb-4e70badb77f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": read tcp 10.133.0.2:57156->10.133.0.60:15021: read: connection reset by peer"
Jul 30 15:33:16.412240 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:16.412204    2578 generic.go:356] "Generic (PLEG): container finished" podID="e2a022a4-6a44-49c7-81fb-4e70badb77f3" containerID="c2d6d522c6ece646e9683d39d17376d289ad57a2eef9cf160e535856999fa01f" exitCode=0
Jul 30 15:33:16.412621 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:16.412277    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9" event={"ID":"e2a022a4-6a44-49c7-81fb-4e70badb77f3","Type":"ContainerDied","Data":"c2d6d522c6ece646e9683d39d17376d289ad57a2eef9cf160e535856999fa01f"}
Jul 30 15:33:16.412685 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:16.412670    2578 scope.go:117] "RemoveContainer" containerID="c2d6d522c6ece646e9683d39d17376d289ad57a2eef9cf160e535856999fa01f"
Jul 30 15:33:16.413648 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:16.413620    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:33:16.413719 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:16.413688    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:33:16.413760 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:16.413724    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:33:17.420223 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:17.420191    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9" event={"ID":"e2a022a4-6a44-49c7-81fb-4e70badb77f3","Type":"ContainerStarted","Data":"919d07ee14b91d337d3549c9fe316f1537b7e5a15de19f36fe34338772731a4e"}
Jul 30 15:33:18.329211 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:18.329178    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9"
Jul 30 15:33:18.329413 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:18.329385    2578 patch_prober.go:28] interesting pod/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Jul 30 15:33:18.329484 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:18.329429    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9" podUID="e2a022a4-6a44-49c7-81fb-4e70badb77f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Jul 30 15:33:19.328796 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:19.328759    2578 patch_prober.go:28] interesting pod/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Jul 30 15:33:19.329192 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:19.328823    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9" podUID="e2a022a4-6a44-49c7-81fb-4e70badb77f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Jul 30 15:33:20.329545 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:20.329506    2578 patch_prober.go:28] interesting pod/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Jul 30 15:33:20.329943 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:20.329564    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9" podUID="e2a022a4-6a44-49c7-81fb-4e70badb77f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Jul 30 15:33:21.332383 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:21.332347    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9"
Jul 30 15:33:21.332800 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:21.332611    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9"
Jul 30 15:33:21.333417 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:21.333400    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9"
Jul 30 15:33:27.039550 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:27.039508    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl"]
Jul 30 15:33:27.043635 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:27.043612    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl"
Jul 30 15:33:27.045516 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:27.045495    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-a52c4d-openshift-default-dockercfg-jp2gg\""
Jul 30 15:33:27.053619 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:27.053587    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl"]
Jul 30 15:33:27.147909 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:27.147880    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2f543c7-26b5-4424-87b3-27bd324a27ae-workload-certs\") pod \"e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl\" (UID: \"d2f543c7-26b5-4424-87b3-27bd324a27ae\") " pod="openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl"
Jul 30 15:33:27.148087 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:27.147929    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2txnm\" (UniqueName: \"kubernetes.io/projected/d2f543c7-26b5-4424-87b3-27bd324a27ae-kube-api-access-2txnm\") pod \"e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl\" (UID: \"d2f543c7-26b5-4424-87b3-27bd324a27ae\") " pod="openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl"
Jul 30 15:33:27.148087 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:27.147948    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2f543c7-26b5-4424-87b3-27bd324a27ae-istio-podinfo\") pod \"e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl\" (UID: \"d2f543c7-26b5-4424-87b3-27bd324a27ae\") " pod="openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl"
Jul 30 15:33:27.148087 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:27.148000    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2f543c7-26b5-4424-87b3-27bd324a27ae-istio-data\") pod \"e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl\" (UID: \"d2f543c7-26b5-4424-87b3-27bd324a27ae\") " pod="openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl"
Jul 30 15:33:27.148087 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:27.148024    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2f543c7-26b5-4424-87b3-27bd324a27ae-credential-socket\") pod \"e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl\" (UID: \"d2f543c7-26b5-4424-87b3-27bd324a27ae\") " pod="openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl"
Jul 30 15:33:27.148087 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:27.148050    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2f543c7-26b5-4424-87b3-27bd324a27ae-istio-envoy\") pod \"e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl\" (UID: \"d2f543c7-26b5-4424-87b3-27bd324a27ae\") " pod="openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl"
Jul 30 15:33:27.148087 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:27.148076    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2f543c7-26b5-4424-87b3-27bd324a27ae-istiod-ca-cert\") pod \"e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl\" (UID: \"d2f543c7-26b5-4424-87b3-27bd324a27ae\") " pod="openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl"
Jul 30 15:33:27.148329 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:27.148093    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2f543c7-26b5-4424-87b3-27bd324a27ae-workload-socket\") pod \"e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl\" (UID: \"d2f543c7-26b5-4424-87b3-27bd324a27ae\") " pod="openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl"
Jul 30 15:33:27.148329 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:27.148170    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2f543c7-26b5-4424-87b3-27bd324a27ae-istio-token\") pod \"e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl\" (UID: \"d2f543c7-26b5-4424-87b3-27bd324a27ae\") " pod="openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl"
Jul 30 15:33:27.249593 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:27.249555    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2txnm\" (UniqueName: \"kubernetes.io/projected/d2f543c7-26b5-4424-87b3-27bd324a27ae-kube-api-access-2txnm\") pod \"e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl\" (UID: \"d2f543c7-26b5-4424-87b3-27bd324a27ae\") " pod="openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl"
Jul 30 15:33:27.249593 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:27.249593    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2f543c7-26b5-4424-87b3-27bd324a27ae-istio-podinfo\") pod \"e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl\" (UID: \"d2f543c7-26b5-4424-87b3-27bd324a27ae\") " pod="openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl"
Jul 30 15:33:27.249854 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:27.249635    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2f543c7-26b5-4424-87b3-27bd324a27ae-istio-data\") pod \"e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl\" (UID: \"d2f543c7-26b5-4424-87b3-27bd324a27ae\") " pod="openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl"
Jul 30 15:33:27.249854 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:27.249658    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2f543c7-26b5-4424-87b3-27bd324a27ae-credential-socket\") pod \"e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl\" (UID: \"d2f543c7-26b5-4424-87b3-27bd324a27ae\") " pod="openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl"
Jul 30 15:33:27.249854 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:27.249679    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2f543c7-26b5-4424-87b3-27bd324a27ae-istio-envoy\") pod \"e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl\" (UID: \"d2f543c7-26b5-4424-87b3-27bd324a27ae\") " pod="openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl"
Jul 30 15:33:27.249854 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:27.249704    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2f543c7-26b5-4424-87b3-27bd324a27ae-istiod-ca-cert\") pod \"e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl\" (UID: \"d2f543c7-26b5-4424-87b3-27bd324a27ae\") " pod="openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl"
Jul 30 15:33:27.249854 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:27.249747    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2f543c7-26b5-4424-87b3-27bd324a27ae-workload-socket\") pod \"e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl\" (UID: \"d2f543c7-26b5-4424-87b3-27bd324a27ae\") " pod="openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl"
Jul 30 15:33:27.249854 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:27.249804    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2f543c7-26b5-4424-87b3-27bd324a27ae-istio-token\") pod \"e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl\" (UID: \"d2f543c7-26b5-4424-87b3-27bd324a27ae\") " pod="openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl"
Jul 30 15:33:27.250139 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:27.249855    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2f543c7-26b5-4424-87b3-27bd324a27ae-workload-certs\") pod \"e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl\" (UID: \"d2f543c7-26b5-4424-87b3-27bd324a27ae\") " pod="openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl"
Jul 30 15:33:27.250139 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:27.250112    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2f543c7-26b5-4424-87b3-27bd324a27ae-istio-data\") pod \"e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl\" (UID: \"d2f543c7-26b5-4424-87b3-27bd324a27ae\") " pod="openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl"
Jul 30 15:33:27.250350 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:27.250327    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2f543c7-26b5-4424-87b3-27bd324a27ae-credential-socket\") pod \"e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl\" (UID: \"d2f543c7-26b5-4424-87b3-27bd324a27ae\") " pod="openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl"
Jul 30 15:33:27.250595 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:27.250384    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2f543c7-26b5-4424-87b3-27bd324a27ae-workload-socket\") pod \"e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl\" (UID: \"d2f543c7-26b5-4424-87b3-27bd324a27ae\") " pod="openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl"
Jul 30 15:33:27.250697 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:27.250528    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2f543c7-26b5-4424-87b3-27bd324a27ae-workload-certs\") pod \"e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl\" (UID: \"d2f543c7-26b5-4424-87b3-27bd324a27ae\") " pod="openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl"
Jul 30 15:33:27.250697 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:27.250632    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2f543c7-26b5-4424-87b3-27bd324a27ae-istiod-ca-cert\") pod \"e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl\" (UID: \"d2f543c7-26b5-4424-87b3-27bd324a27ae\") " pod="openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl"
Jul 30 15:33:27.252069 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:27.252047    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2f543c7-26b5-4424-87b3-27bd324a27ae-istio-envoy\") pod \"e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl\" (UID: \"d2f543c7-26b5-4424-87b3-27bd324a27ae\") " pod="openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl"
Jul 30 15:33:27.252349 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:27.252329    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2f543c7-26b5-4424-87b3-27bd324a27ae-istio-podinfo\") pod \"e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl\" (UID: \"d2f543c7-26b5-4424-87b3-27bd324a27ae\") " pod="openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl"
Jul 30 15:33:27.256671 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:27.256646    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2f543c7-26b5-4424-87b3-27bd324a27ae-istio-token\") pod \"e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl\" (UID: \"d2f543c7-26b5-4424-87b3-27bd324a27ae\") " pod="openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl"
Jul 30 15:33:27.256671 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:27.256666    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2txnm\" (UniqueName: \"kubernetes.io/projected/d2f543c7-26b5-4424-87b3-27bd324a27ae-kube-api-access-2txnm\") pod \"e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl\" (UID: \"d2f543c7-26b5-4424-87b3-27bd324a27ae\") " pod="openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl"
Jul 30 15:33:27.356089 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:27.356064    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl"
Jul 30 15:33:27.487692 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:27.487666    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl"]
Jul 30 15:33:27.489245 ip-10-0-133-117 kubenswrapper[2578]: W0730 15:33:27.489212    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2f543c7_26b5_4424_87b3_27bd324a27ae.slice/crio-9cfc6a3ffbce987baf05179bc4a4942e621a5b942286f8bee65eefc57a7a02bb WatchSource:0}: Error finding container 9cfc6a3ffbce987baf05179bc4a4942e621a5b942286f8bee65eefc57a7a02bb: Status 404 returned error can't find the container with id 9cfc6a3ffbce987baf05179bc4a4942e621a5b942286f8bee65eefc57a7a02bb
Jul 30 15:33:27.491669 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:27.491626    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:33:27.491769 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:27.491719    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:33:27.491812 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:27.491764    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:33:28.475672 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:28.475631    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl" event={"ID":"d2f543c7-26b5-4424-87b3-27bd324a27ae","Type":"ContainerStarted","Data":"4fc997daac43146b076fbb798ec680bbd63cd4d3dfb83d9ac0d3fd2228cded52"}
Jul 30 15:33:28.476191 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:28.475692    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl" event={"ID":"d2f543c7-26b5-4424-87b3-27bd324a27ae","Type":"ContainerStarted","Data":"9cfc6a3ffbce987baf05179bc4a4942e621a5b942286f8bee65eefc57a7a02bb"}
Jul 30 15:33:28.494890 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:28.494839    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl" podStartSLOduration=1.494819944 podStartE2EDuration="1.494819944s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:33:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:33:28.492660094 +0000 UTC m=+2720.021598567" watchObservedRunningTime="2026-07-30 15:33:28.494819944 +0000 UTC m=+2720.023758483"
Jul 30 15:33:29.357195 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:29.357158    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl"
Jul 30 15:33:29.362194 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:29.362135    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl"
Jul 30 15:33:29.481494 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:29.481464    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl"
Jul 30 15:33:29.482639 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:29.482614    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl"
Jul 30 15:33:32.413334 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:32.413299    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-a52c4d-5d8d9b8d84-p8dwt"]
Jul 30 15:33:32.417066 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:32.417049    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a52c4d-5d8d9b8d84-p8dwt"
Jul 30 15:33:32.419082 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:32.419057    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-a52c4d-dockercfg-26tx5\""
Jul 30 15:33:32.419213 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:32.419057    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-a52c4d\""
Jul 30 15:33:32.426311 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:32.426290    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-a52c4d-5d8d9b8d84-p8dwt"]
Jul 30 15:33:32.432370 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:32.432349    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-a52c4d-55c8d964c4-wzmrq"]
Jul 30 15:33:32.435822 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:32.435758    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-a52c4d-55c8d964c4-wzmrq"
Jul 30 15:33:32.446233 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:32.446207    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-a52c4d-55c8d964c4-wzmrq"]
Jul 30 15:33:32.501457 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:32.501413    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7d6a3cc-a440-45c2-8d24-3678c2966c6b-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-a52c4d-5d8d9b8d84-p8dwt\" (UID: \"f7d6a3cc-a440-45c2-8d24-3678c2966c6b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a52c4d-5d8d9b8d84-p8dwt"
Jul 30 15:33:32.501457 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:32.501453    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gfxc\" (UniqueName: \"kubernetes.io/projected/f7d6a3cc-a440-45c2-8d24-3678c2966c6b-kube-api-access-2gfxc\") pod \"payload-pre-processing-e2e-inf-b-a52c4d-5d8d9b8d84-p8dwt\" (UID: \"f7d6a3cc-a440-45c2-8d24-3678c2966c6b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a52c4d-5d8d9b8d84-p8dwt"
Jul 30 15:33:32.602192 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:32.602141    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7d6a3cc-a440-45c2-8d24-3678c2966c6b-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-a52c4d-5d8d9b8d84-p8dwt\" (UID: \"f7d6a3cc-a440-45c2-8d24-3678c2966c6b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a52c4d-5d8d9b8d84-p8dwt"
Jul 30 15:33:32.602368 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:32.602201    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gfxc\" (UniqueName: \"kubernetes.io/projected/f7d6a3cc-a440-45c2-8d24-3678c2966c6b-kube-api-access-2gfxc\") pod \"payload-pre-processing-e2e-inf-b-a52c4d-5d8d9b8d84-p8dwt\" (UID: \"f7d6a3cc-a440-45c2-8d24-3678c2966c6b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a52c4d-5d8d9b8d84-p8dwt"
Jul 30 15:33:32.602368 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:32.602271    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27ftb\" (UniqueName: \"kubernetes.io/projected/81a4538d-aed5-48cd-82d9-6dea5de19c68-kube-api-access-27ftb\") pod \"payload-processing-e2e-inf-b-a52c4d-55c8d964c4-wzmrq\" (UID: \"81a4538d-aed5-48cd-82d9-6dea5de19c68\") " pod="openshift-ingress/payload-processing-e2e-inf-b-a52c4d-55c8d964c4-wzmrq"
Jul 30 15:33:32.602368 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:32.602324    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/81a4538d-aed5-48cd-82d9-6dea5de19c68-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-a52c4d-55c8d964c4-wzmrq\" (UID: \"81a4538d-aed5-48cd-82d9-6dea5de19c68\") " pod="openshift-ingress/payload-processing-e2e-inf-b-a52c4d-55c8d964c4-wzmrq"
Jul 30 15:33:32.602895 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:32.602874    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7d6a3cc-a440-45c2-8d24-3678c2966c6b-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-a52c4d-5d8d9b8d84-p8dwt\" (UID: \"f7d6a3cc-a440-45c2-8d24-3678c2966c6b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a52c4d-5d8d9b8d84-p8dwt"
Jul 30 15:33:32.613480 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:32.613454    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gfxc\" (UniqueName: \"kubernetes.io/projected/f7d6a3cc-a440-45c2-8d24-3678c2966c6b-kube-api-access-2gfxc\") pod \"payload-pre-processing-e2e-inf-b-a52c4d-5d8d9b8d84-p8dwt\" (UID: \"f7d6a3cc-a440-45c2-8d24-3678c2966c6b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a52c4d-5d8d9b8d84-p8dwt"
Jul 30 15:33:32.702942 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:32.702871    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27ftb\" (UniqueName: \"kubernetes.io/projected/81a4538d-aed5-48cd-82d9-6dea5de19c68-kube-api-access-27ftb\") pod \"payload-processing-e2e-inf-b-a52c4d-55c8d964c4-wzmrq\" (UID: \"81a4538d-aed5-48cd-82d9-6dea5de19c68\") " pod="openshift-ingress/payload-processing-e2e-inf-b-a52c4d-55c8d964c4-wzmrq"
Jul 30 15:33:32.702942 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:32.702929    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/81a4538d-aed5-48cd-82d9-6dea5de19c68-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-a52c4d-55c8d964c4-wzmrq\" (UID: \"81a4538d-aed5-48cd-82d9-6dea5de19c68\") " pod="openshift-ingress/payload-processing-e2e-inf-b-a52c4d-55c8d964c4-wzmrq"
Jul 30 15:33:32.703582 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:32.703562    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/81a4538d-aed5-48cd-82d9-6dea5de19c68-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-a52c4d-55c8d964c4-wzmrq\" (UID: \"81a4538d-aed5-48cd-82d9-6dea5de19c68\") " pod="openshift-ingress/payload-processing-e2e-inf-b-a52c4d-55c8d964c4-wzmrq"
Jul 30 15:33:32.709985 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:32.709964    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27ftb\" (UniqueName: \"kubernetes.io/projected/81a4538d-aed5-48cd-82d9-6dea5de19c68-kube-api-access-27ftb\") pod \"payload-processing-e2e-inf-b-a52c4d-55c8d964c4-wzmrq\" (UID: \"81a4538d-aed5-48cd-82d9-6dea5de19c68\") " pod="openshift-ingress/payload-processing-e2e-inf-b-a52c4d-55c8d964c4-wzmrq"
Jul 30 15:33:32.728956 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:32.728937    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a52c4d-5d8d9b8d84-p8dwt"
Jul 30 15:33:32.746859 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:32.746832    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-a52c4d-55c8d964c4-wzmrq"
Jul 30 15:33:32.878014 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:32.877987    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-a52c4d-5d8d9b8d84-p8dwt"]
Jul 30 15:33:32.881226 ip-10-0-133-117 kubenswrapper[2578]: W0730 15:33:32.881199    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7d6a3cc_a440_45c2_8d24_3678c2966c6b.slice/crio-ec81f64706edce7840735da4b02eac9d17248e5101528d7aeac8d26925e6b0d0 WatchSource:0}: Error finding container ec81f64706edce7840735da4b02eac9d17248e5101528d7aeac8d26925e6b0d0: Status 404 returned error can't find the container with id ec81f64706edce7840735da4b02eac9d17248e5101528d7aeac8d26925e6b0d0
Jul 30 15:33:32.903264 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:32.903228    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-a52c4d-55c8d964c4-wzmrq"]
Jul 30 15:33:32.913183 ip-10-0-133-117 kubenswrapper[2578]: W0730 15:33:32.913123    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81a4538d_aed5_48cd_82d9_6dea5de19c68.slice/crio-b814ed9bd6aae8c8095d46245915668a6b42be03f86a5cfda9669ae32cad9aaa WatchSource:0}: Error finding container b814ed9bd6aae8c8095d46245915668a6b42be03f86a5cfda9669ae32cad9aaa: Status 404 returned error can't find the container with id b814ed9bd6aae8c8095d46245915668a6b42be03f86a5cfda9669ae32cad9aaa
Jul 30 15:33:33.501895 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:33.501854    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-a52c4d-55c8d964c4-wzmrq" event={"ID":"81a4538d-aed5-48cd-82d9-6dea5de19c68","Type":"ContainerStarted","Data":"5e03b58e22e92d11b5fdd41670da2e46bae4d1b445198ac01d96c9428bccc75b"}
Jul 30 15:33:33.502352 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:33.501988    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-a52c4d-55c8d964c4-wzmrq"
Jul 30 15:33:33.502352 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:33.502012    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-a52c4d-55c8d964c4-wzmrq" event={"ID":"81a4538d-aed5-48cd-82d9-6dea5de19c68","Type":"ContainerStarted","Data":"b814ed9bd6aae8c8095d46245915668a6b42be03f86a5cfda9669ae32cad9aaa"}
Jul 30 15:33:33.503344 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:33.503320    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a52c4d-5d8d9b8d84-p8dwt" event={"ID":"f7d6a3cc-a440-45c2-8d24-3678c2966c6b","Type":"ContainerStarted","Data":"f95244425dca5e8cd6b62e00ba54bbbc20566968d0e11826bc107e48c51f4eba"}
Jul 30 15:33:33.503473 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:33.503365    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a52c4d-5d8d9b8d84-p8dwt" event={"ID":"f7d6a3cc-a440-45c2-8d24-3678c2966c6b","Type":"ContainerStarted","Data":"ec81f64706edce7840735da4b02eac9d17248e5101528d7aeac8d26925e6b0d0"}
Jul 30 15:33:33.503473 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:33.503392    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a52c4d-5d8d9b8d84-p8dwt"
Jul 30 15:33:33.518400 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:33.518357    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-a52c4d-55c8d964c4-wzmrq" podStartSLOduration=1.518343665 podStartE2EDuration="1.518343665s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:33:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:33:33.516278749 +0000 UTC m=+2725.045217223" watchObservedRunningTime="2026-07-30 15:33:33.518343665 +0000 UTC m=+2725.047282138"
Jul 30 15:33:33.531315 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:33.531273    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a52c4d-5d8d9b8d84-p8dwt" podStartSLOduration=1.5312599150000001 podStartE2EDuration="1.531259915s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:33:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:33:33.529788954 +0000 UTC m=+2725.058727429" watchObservedRunningTime="2026-07-30 15:33:33.531259915 +0000 UTC m=+2725.060198388"
Jul 30 15:33:44.510622 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:44.510583    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-a52c4d-55c8d964c4-wzmrq"
Jul 30 15:33:44.511030 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:44.510701    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a52c4d-5d8d9b8d84-p8dwt"
Jul 30 15:33:45.523300 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:45.523267    2578 patch_prober.go:28] interesting pod/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 30 15:33:45.524446 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:45.523331    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl" podUID="d2f543c7-26b5-4424-87b3-27bd324a27ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 30 15:33:46.579128 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:46.579095    2578 generic.go:356] "Generic (PLEG): container finished" podID="d2f543c7-26b5-4424-87b3-27bd324a27ae" containerID="4fc997daac43146b076fbb798ec680bbd63cd4d3dfb83d9ac0d3fd2228cded52" exitCode=0
Jul 30 15:33:46.579520 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:46.579177    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl" event={"ID":"d2f543c7-26b5-4424-87b3-27bd324a27ae","Type":"ContainerDied","Data":"4fc997daac43146b076fbb798ec680bbd63cd4d3dfb83d9ac0d3fd2228cded52"}
Jul 30 15:33:46.579588 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:46.579574    2578 scope.go:117] "RemoveContainer" containerID="4fc997daac43146b076fbb798ec680bbd63cd4d3dfb83d9ac0d3fd2228cded52"
Jul 30 15:33:46.580676 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:46.580651    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:33:46.580735 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:46.580716    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:33:46.580771 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:46.580747    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:33:47.586823 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:47.586786    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl" event={"ID":"d2f543c7-26b5-4424-87b3-27bd324a27ae","Type":"ContainerStarted","Data":"75a552dc29b851a5dba00c946848f6dde9c2980a435deca8a49146af357d8d51"}
Jul 30 15:33:48.356344 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:48.356305    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl"
Jul 30 15:33:48.356550 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:48.356529    2578 patch_prober.go:28] interesting pod/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Jul 30 15:33:48.356608 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:48.356576    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl" podUID="d2f543c7-26b5-4424-87b3-27bd324a27ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Jul 30 15:33:49.356584 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:49.356546    2578 patch_prober.go:28] interesting pod/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Jul 30 15:33:49.356983 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:49.356614    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl" podUID="d2f543c7-26b5-4424-87b3-27bd324a27ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Jul 30 15:33:50.356540 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:50.356502    2578 patch_prober.go:28] interesting pod/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Jul 30 15:33:50.356724 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:50.356559    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl" podUID="d2f543c7-26b5-4424-87b3-27bd324a27ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Jul 30 15:33:51.356968 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:51.356920    2578 patch_prober.go:28] interesting pod/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Jul 30 15:33:51.357377 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:51.356991    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl" podUID="d2f543c7-26b5-4424-87b3-27bd324a27ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Jul 30 15:33:52.360617 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:52.360589    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl"
Jul 30 15:33:52.361072 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:52.360958    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl"
Jul 30 15:33:52.361566 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:33:52.361546    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl"
Jul 30 15:35:41.929628 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:41.929580    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-f03573-d99d45d55-nw8lp"]
Jul 30 15:35:41.930227 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:41.929841    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f03573-d99d45d55-nw8lp" podUID="81c06155-cded-44ca-b41f-d733a6f54fad" containerName="maas-api" containerID="cri-o://974c0c19d5896e53a07df7348270b63716b25d13ea21c727c5878d58f9a31a1d" gracePeriod=30
Jul 30 15:35:41.980412 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:41.980373    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-f03573-c74989d7d-rhzsp"]
Jul 30 15:35:41.980926 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:41.980616    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-f03573-c74989d7d-rhzsp" podUID="e041fcc6-edac-46d8-b5e3-7875d26ffa13" containerName="payload-processing" containerID="cri-o://2bc4efd0d0bb4d751677e6b9365ccdfdb0f2cda2b23a45283cab2b38a597f785" gracePeriod=30
Jul 30 15:35:42.170710 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:42.170580    2578 generic.go:356] "Generic (PLEG): container finished" podID="81c06155-cded-44ca-b41f-d733a6f54fad" containerID="974c0c19d5896e53a07df7348270b63716b25d13ea21c727c5878d58f9a31a1d" exitCode=0
Jul 30 15:35:42.170710 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:42.170673    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f03573-d99d45d55-nw8lp" event={"ID":"81c06155-cded-44ca-b41f-d733a6f54fad","Type":"ContainerDied","Data":"974c0c19d5896e53a07df7348270b63716b25d13ea21c727c5878d58f9a31a1d"}
Jul 30 15:35:42.173024 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:42.172903    2578 generic.go:356] "Generic (PLEG): container finished" podID="e041fcc6-edac-46d8-b5e3-7875d26ffa13" containerID="2bc4efd0d0bb4d751677e6b9365ccdfdb0f2cda2b23a45283cab2b38a597f785" exitCode=0
Jul 30 15:35:42.173024 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:42.173004    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-f03573-c74989d7d-rhzsp" event={"ID":"e041fcc6-edac-46d8-b5e3-7875d26ffa13","Type":"ContainerDied","Data":"2bc4efd0d0bb4d751677e6b9365ccdfdb0f2cda2b23a45283cab2b38a597f785"}
Jul 30 15:35:42.239208 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:42.239172    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f03573-d99d45d55-nw8lp"
Jul 30 15:35:42.272044 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:42.272019    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-f03573-c74989d7d-rhzsp"
Jul 30 15:35:42.334985 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:42.334951    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81c06155-cded-44ca-b41f-d733a6f54fad-openshift-service-ca\") pod \"81c06155-cded-44ca-b41f-d733a6f54fad\" (UID: \"81c06155-cded-44ca-b41f-d733a6f54fad\") "
Jul 30 15:35:42.335181 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:42.335010    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e041fcc6-edac-46d8-b5e3-7875d26ffa13-plugins-config-volume\") pod \"e041fcc6-edac-46d8-b5e3-7875d26ffa13\" (UID: \"e041fcc6-edac-46d8-b5e3-7875d26ffa13\") "
Jul 30 15:35:42.335181 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:42.335045    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81c06155-cded-44ca-b41f-d733a6f54fad-maas-api-tls\") pod \"81c06155-cded-44ca-b41f-d733a6f54fad\" (UID: \"81c06155-cded-44ca-b41f-d733a6f54fad\") "
Jul 30 15:35:42.335181 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:42.335098    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qtzsk\" (UniqueName: \"kubernetes.io/projected/e041fcc6-edac-46d8-b5e3-7875d26ffa13-kube-api-access-qtzsk\") pod \"e041fcc6-edac-46d8-b5e3-7875d26ffa13\" (UID: \"e041fcc6-edac-46d8-b5e3-7875d26ffa13\") "
Jul 30 15:35:42.335325 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:42.335280    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8rb2j\" (UniqueName: \"kubernetes.io/projected/81c06155-cded-44ca-b41f-d733a6f54fad-kube-api-access-8rb2j\") pod \"81c06155-cded-44ca-b41f-d733a6f54fad\" (UID: \"81c06155-cded-44ca-b41f-d733a6f54fad\") "
Jul 30 15:35:42.335403 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:42.335374    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81c06155-cded-44ca-b41f-d733a6f54fad-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "81c06155-cded-44ca-b41f-d733a6f54fad" (UID: "81c06155-cded-44ca-b41f-d733a6f54fad"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:35:42.335469 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:42.335425    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e041fcc6-edac-46d8-b5e3-7875d26ffa13-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e041fcc6-edac-46d8-b5e3-7875d26ffa13" (UID: "e041fcc6-edac-46d8-b5e3-7875d26ffa13"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:35:42.335592 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:42.335576    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81c06155-cded-44ca-b41f-d733a6f54fad-openshift-service-ca\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:35:42.335656 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:42.335594    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e041fcc6-edac-46d8-b5e3-7875d26ffa13-plugins-config-volume\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:35:42.337318 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:42.337295    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e041fcc6-edac-46d8-b5e3-7875d26ffa13-kube-api-access-qtzsk" (OuterVolumeSpecName: "kube-api-access-qtzsk") pod "e041fcc6-edac-46d8-b5e3-7875d26ffa13" (UID: "e041fcc6-edac-46d8-b5e3-7875d26ffa13"). InnerVolumeSpecName "kube-api-access-qtzsk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:35:42.337401 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:42.337323    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81c06155-cded-44ca-b41f-d733a6f54fad-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "81c06155-cded-44ca-b41f-d733a6f54fad" (UID: "81c06155-cded-44ca-b41f-d733a6f54fad"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 15:35:42.337401 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:42.337335    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81c06155-cded-44ca-b41f-d733a6f54fad-kube-api-access-8rb2j" (OuterVolumeSpecName: "kube-api-access-8rb2j") pod "81c06155-cded-44ca-b41f-d733a6f54fad" (UID: "81c06155-cded-44ca-b41f-d733a6f54fad"). InnerVolumeSpecName "kube-api-access-8rb2j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:35:42.436335 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:42.436301    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81c06155-cded-44ca-b41f-d733a6f54fad-maas-api-tls\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:35:42.436335 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:42.436327    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qtzsk\" (UniqueName: \"kubernetes.io/projected/e041fcc6-edac-46d8-b5e3-7875d26ffa13-kube-api-access-qtzsk\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:35:42.436543 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:42.436348    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8rb2j\" (UniqueName: \"kubernetes.io/projected/81c06155-cded-44ca-b41f-d733a6f54fad-kube-api-access-8rb2j\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:35:43.179100 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:43.179066    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f03573-d99d45d55-nw8lp"
Jul 30 15:35:43.179100 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:43.179084    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f03573-d99d45d55-nw8lp" event={"ID":"81c06155-cded-44ca-b41f-d733a6f54fad","Type":"ContainerDied","Data":"3ca5289e54262f614cdab7976b44dbbbe493950f6618a5a02c169c99112de964"}
Jul 30 15:35:43.179629 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:43.179172    2578 scope.go:117] "RemoveContainer" containerID="974c0c19d5896e53a07df7348270b63716b25d13ea21c727c5878d58f9a31a1d"
Jul 30 15:35:43.180549 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:43.180529    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-f03573-c74989d7d-rhzsp"
Jul 30 15:35:43.180626 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:43.180551    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-f03573-c74989d7d-rhzsp" event={"ID":"e041fcc6-edac-46d8-b5e3-7875d26ffa13","Type":"ContainerDied","Data":"b184fc27d608318b91d470e0f7584a005ad88c952ac0dc783a25258719017f9c"}
Jul 30 15:35:43.188453 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:43.188430    2578 scope.go:117] "RemoveContainer" containerID="2bc4efd0d0bb4d751677e6b9365ccdfdb0f2cda2b23a45283cab2b38a597f785"
Jul 30 15:35:43.196110 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:43.196060    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-f03573-c74989d7d-rhzsp"]
Jul 30 15:35:43.199487 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:43.199464    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-f03573-c74989d7d-rhzsp"]
Jul 30 15:35:43.211190 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:43.211139    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-f03573-d99d45d55-nw8lp"]
Jul 30 15:35:43.214676 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:43.214656    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-f03573-d99d45d55-nw8lp"]
Jul 30 15:35:44.978607 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:44.978578    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81c06155-cded-44ca-b41f-d733a6f54fad" path="/var/lib/kubelet/pods/81c06155-cded-44ca-b41f-d733a6f54fad/volumes"
Jul 30 15:35:44.978998 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:44.978952    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e041fcc6-edac-46d8-b5e3-7875d26ffa13" path="/var/lib/kubelet/pods/e041fcc6-edac-46d8-b5e3-7875d26ffa13/volumes"
Jul 30 15:35:54.152313 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:54.152274    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9"]
Jul 30 15:35:54.152884 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:54.152523    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9" podUID="e2a022a4-6a44-49c7-81fb-4e70badb77f3" containerName="istio-proxy" containerID="cri-o://919d07ee14b91d337d3549c9fe316f1537b7e5a15de19f36fe34338772731a4e" gracePeriod=30
Jul 30 15:35:59.413793 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:59.413763    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9"
Jul 30 15:35:59.499360 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:59.499283    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2a022a4-6a44-49c7-81fb-4e70badb77f3-istiod-ca-cert\") pod \"e2a022a4-6a44-49c7-81fb-4e70badb77f3\" (UID: \"e2a022a4-6a44-49c7-81fb-4e70badb77f3\") "
Jul 30 15:35:59.499360 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:59.499347    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2a022a4-6a44-49c7-81fb-4e70badb77f3-workload-certs\") pod \"e2a022a4-6a44-49c7-81fb-4e70badb77f3\" (UID: \"e2a022a4-6a44-49c7-81fb-4e70badb77f3\") "
Jul 30 15:35:59.499562 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:59.499414    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2a022a4-6a44-49c7-81fb-4e70badb77f3-workload-socket\") pod \"e2a022a4-6a44-49c7-81fb-4e70badb77f3\" (UID: \"e2a022a4-6a44-49c7-81fb-4e70badb77f3\") "
Jul 30 15:35:59.499562 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:59.499455    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2a022a4-6a44-49c7-81fb-4e70badb77f3-istio-podinfo\") pod \"e2a022a4-6a44-49c7-81fb-4e70badb77f3\" (UID: \"e2a022a4-6a44-49c7-81fb-4e70badb77f3\") "
Jul 30 15:35:59.499562 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:59.499485    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2a022a4-6a44-49c7-81fb-4e70badb77f3-credential-socket\") pod \"e2a022a4-6a44-49c7-81fb-4e70badb77f3\" (UID: \"e2a022a4-6a44-49c7-81fb-4e70badb77f3\") "
Jul 30 15:35:59.499562 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:59.499515    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2a022a4-6a44-49c7-81fb-4e70badb77f3-istio-data\") pod \"e2a022a4-6a44-49c7-81fb-4e70badb77f3\" (UID: \"e2a022a4-6a44-49c7-81fb-4e70badb77f3\") "
Jul 30 15:35:59.499728 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:59.499677    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2a022a4-6a44-49c7-81fb-4e70badb77f3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e2a022a4-6a44-49c7-81fb-4e70badb77f3" (UID: "e2a022a4-6a44-49c7-81fb-4e70badb77f3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:35:59.499728 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:59.499695    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2a022a4-6a44-49c7-81fb-4e70badb77f3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e2a022a4-6a44-49c7-81fb-4e70badb77f3" (UID: "e2a022a4-6a44-49c7-81fb-4e70badb77f3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:35:59.499728 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:59.499682    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2a022a4-6a44-49c7-81fb-4e70badb77f3-istio-token\") pod \"e2a022a4-6a44-49c7-81fb-4e70badb77f3\" (UID: \"e2a022a4-6a44-49c7-81fb-4e70badb77f3\") "
Jul 30 15:35:59.499827 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:59.499751    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nbc9r\" (UniqueName: \"kubernetes.io/projected/e2a022a4-6a44-49c7-81fb-4e70badb77f3-kube-api-access-nbc9r\") pod \"e2a022a4-6a44-49c7-81fb-4e70badb77f3\" (UID: \"e2a022a4-6a44-49c7-81fb-4e70badb77f3\") "
Jul 30 15:35:59.499827 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:59.499784    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2a022a4-6a44-49c7-81fb-4e70badb77f3-istio-envoy\") pod \"e2a022a4-6a44-49c7-81fb-4e70badb77f3\" (UID: \"e2a022a4-6a44-49c7-81fb-4e70badb77f3\") "
Jul 30 15:35:59.500032 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:59.500020    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2a022a4-6a44-49c7-81fb-4e70badb77f3-istiod-ca-cert\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:35:59.500078 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:59.500033    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2a022a4-6a44-49c7-81fb-4e70badb77f3-workload-certs\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:35:59.500268 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:59.500244    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2a022a4-6a44-49c7-81fb-4e70badb77f3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e2a022a4-6a44-49c7-81fb-4e70badb77f3" (UID: "e2a022a4-6a44-49c7-81fb-4e70badb77f3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:35:59.500374 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:59.500262    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2a022a4-6a44-49c7-81fb-4e70badb77f3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e2a022a4-6a44-49c7-81fb-4e70badb77f3" (UID: "e2a022a4-6a44-49c7-81fb-4e70badb77f3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:35:59.500946 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:59.500917    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2a022a4-6a44-49c7-81fb-4e70badb77f3-istio-data" (OuterVolumeSpecName: "istio-data") pod "e2a022a4-6a44-49c7-81fb-4e70badb77f3" (UID: "e2a022a4-6a44-49c7-81fb-4e70badb77f3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:35:59.502249 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:59.502191    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2a022a4-6a44-49c7-81fb-4e70badb77f3-kube-api-access-nbc9r" (OuterVolumeSpecName: "kube-api-access-nbc9r") pod "e2a022a4-6a44-49c7-81fb-4e70badb77f3" (UID: "e2a022a4-6a44-49c7-81fb-4e70badb77f3"). InnerVolumeSpecName "kube-api-access-nbc9r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:35:59.502249 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:59.502239    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2a022a4-6a44-49c7-81fb-4e70badb77f3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e2a022a4-6a44-49c7-81fb-4e70badb77f3" (UID: "e2a022a4-6a44-49c7-81fb-4e70badb77f3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:35:59.502491 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:59.502261    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2a022a4-6a44-49c7-81fb-4e70badb77f3-istio-token" (OuterVolumeSpecName: "istio-token") pod "e2a022a4-6a44-49c7-81fb-4e70badb77f3" (UID: "e2a022a4-6a44-49c7-81fb-4e70badb77f3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:35:59.502491 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:59.502333    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e2a022a4-6a44-49c7-81fb-4e70badb77f3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e2a022a4-6a44-49c7-81fb-4e70badb77f3" (UID: "e2a022a4-6a44-49c7-81fb-4e70badb77f3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 15:35:59.601564 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:59.601528    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2a022a4-6a44-49c7-81fb-4e70badb77f3-workload-socket\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:35:59.601564 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:59.601559    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2a022a4-6a44-49c7-81fb-4e70badb77f3-istio-podinfo\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:35:59.601564 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:59.601568    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2a022a4-6a44-49c7-81fb-4e70badb77f3-credential-socket\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:35:59.601799 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:59.601578    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2a022a4-6a44-49c7-81fb-4e70badb77f3-istio-data\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:35:59.601799 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:59.601587    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2a022a4-6a44-49c7-81fb-4e70badb77f3-istio-token\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:35:59.601799 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:59.601596    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nbc9r\" (UniqueName: \"kubernetes.io/projected/e2a022a4-6a44-49c7-81fb-4e70badb77f3-kube-api-access-nbc9r\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:35:59.601799 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:59.601604    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2a022a4-6a44-49c7-81fb-4e70badb77f3-istio-envoy\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:35:59.894762 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:59.894719    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-a52c4d-55c8d964c4-wzmrq"]
Jul 30 15:35:59.895006 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:59.894972    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-a52c4d-55c8d964c4-wzmrq" podUID="81a4538d-aed5-48cd-82d9-6dea5de19c68" containerName="payload-processing" containerID="cri-o://5e03b58e22e92d11b5fdd41670da2e46bae4d1b445198ac01d96c9428bccc75b" gracePeriod=30
Jul 30 15:35:59.901381 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:59.901336    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-a52c4d-5d8d9b8d84-p8dwt"]
Jul 30 15:35:59.901622 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:35:59.901570    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a52c4d-5d8d9b8d84-p8dwt" podUID="f7d6a3cc-a440-45c2-8d24-3678c2966c6b" containerName="payload-pre-processing" containerID="cri-o://f95244425dca5e8cd6b62e00ba54bbbc20566968d0e11826bc107e48c51f4eba" gracePeriod=30
Jul 30 15:36:00.161956 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:00.161927    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-a52c4d-55c8d964c4-wzmrq"
Jul 30 15:36:00.171175 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:00.171129    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a52c4d-5d8d9b8d84-p8dwt"
Jul 30 15:36:00.274768 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:00.274732    2578 generic.go:356] "Generic (PLEG): container finished" podID="81a4538d-aed5-48cd-82d9-6dea5de19c68" containerID="5e03b58e22e92d11b5fdd41670da2e46bae4d1b445198ac01d96c9428bccc75b" exitCode=0
Jul 30 15:36:00.274953 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:00.274785    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-a52c4d-55c8d964c4-wzmrq"
Jul 30 15:36:00.274953 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:00.274812    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-a52c4d-55c8d964c4-wzmrq" event={"ID":"81a4538d-aed5-48cd-82d9-6dea5de19c68","Type":"ContainerDied","Data":"5e03b58e22e92d11b5fdd41670da2e46bae4d1b445198ac01d96c9428bccc75b"}
Jul 30 15:36:00.274953 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:00.274891    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-a52c4d-55c8d964c4-wzmrq" event={"ID":"81a4538d-aed5-48cd-82d9-6dea5de19c68","Type":"ContainerDied","Data":"b814ed9bd6aae8c8095d46245915668a6b42be03f86a5cfda9669ae32cad9aaa"}
Jul 30 15:36:00.274953 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:00.274920    2578 scope.go:117] "RemoveContainer" containerID="5e03b58e22e92d11b5fdd41670da2e46bae4d1b445198ac01d96c9428bccc75b"
Jul 30 15:36:00.276222 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:00.276171    2578 generic.go:356] "Generic (PLEG): container finished" podID="f7d6a3cc-a440-45c2-8d24-3678c2966c6b" containerID="f95244425dca5e8cd6b62e00ba54bbbc20566968d0e11826bc107e48c51f4eba" exitCode=0
Jul 30 15:36:00.276287 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:00.276226    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a52c4d-5d8d9b8d84-p8dwt"
Jul 30 15:36:00.276287 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:00.276254    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a52c4d-5d8d9b8d84-p8dwt" event={"ID":"f7d6a3cc-a440-45c2-8d24-3678c2966c6b","Type":"ContainerDied","Data":"f95244425dca5e8cd6b62e00ba54bbbc20566968d0e11826bc107e48c51f4eba"}
Jul 30 15:36:00.276378 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:00.276291    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a52c4d-5d8d9b8d84-p8dwt" event={"ID":"f7d6a3cc-a440-45c2-8d24-3678c2966c6b","Type":"ContainerDied","Data":"ec81f64706edce7840735da4b02eac9d17248e5101528d7aeac8d26925e6b0d0"}
Jul 30 15:36:00.277872 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:00.277838    2578 generic.go:356] "Generic (PLEG): container finished" podID="e2a022a4-6a44-49c7-81fb-4e70badb77f3" containerID="919d07ee14b91d337d3549c9fe316f1537b7e5a15de19f36fe34338772731a4e" exitCode=0
Jul 30 15:36:00.277980 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:00.277887    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9" event={"ID":"e2a022a4-6a44-49c7-81fb-4e70badb77f3","Type":"ContainerDied","Data":"919d07ee14b91d337d3549c9fe316f1537b7e5a15de19f36fe34338772731a4e"}
Jul 30 15:36:00.277980 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:00.277927    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9" event={"ID":"e2a022a4-6a44-49c7-81fb-4e70badb77f3","Type":"ContainerDied","Data":"9a961ada0405a80dd00bd380e28883d053ee60d56c65d5123aefe597fef6f431"}
Jul 30 15:36:00.277980 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:00.277897    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9"
Jul 30 15:36:00.283555 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:00.283535    2578 scope.go:117] "RemoveContainer" containerID="5e03b58e22e92d11b5fdd41670da2e46bae4d1b445198ac01d96c9428bccc75b"
Jul 30 15:36:00.283819 ip-10-0-133-117 kubenswrapper[2578]: E0730 15:36:00.283795    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e03b58e22e92d11b5fdd41670da2e46bae4d1b445198ac01d96c9428bccc75b\": container with ID starting with 5e03b58e22e92d11b5fdd41670da2e46bae4d1b445198ac01d96c9428bccc75b not found: ID does not exist" containerID="5e03b58e22e92d11b5fdd41670da2e46bae4d1b445198ac01d96c9428bccc75b"
Jul 30 15:36:00.283924 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:00.283828    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e03b58e22e92d11b5fdd41670da2e46bae4d1b445198ac01d96c9428bccc75b"} err="failed to get container status \"5e03b58e22e92d11b5fdd41670da2e46bae4d1b445198ac01d96c9428bccc75b\": rpc error: code = NotFound desc = could not find container \"5e03b58e22e92d11b5fdd41670da2e46bae4d1b445198ac01d96c9428bccc75b\": container with ID starting with 5e03b58e22e92d11b5fdd41670da2e46bae4d1b445198ac01d96c9428bccc75b not found: ID does not exist"
Jul 30 15:36:00.283924 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:00.283851    2578 scope.go:117] "RemoveContainer" containerID="f95244425dca5e8cd6b62e00ba54bbbc20566968d0e11826bc107e48c51f4eba"
Jul 30 15:36:00.291740 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:00.291723    2578 scope.go:117] "RemoveContainer" containerID="f95244425dca5e8cd6b62e00ba54bbbc20566968d0e11826bc107e48c51f4eba"
Jul 30 15:36:00.291998 ip-10-0-133-117 kubenswrapper[2578]: E0730 15:36:00.291978    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f95244425dca5e8cd6b62e00ba54bbbc20566968d0e11826bc107e48c51f4eba\": container with ID starting with f95244425dca5e8cd6b62e00ba54bbbc20566968d0e11826bc107e48c51f4eba not found: ID does not exist" containerID="f95244425dca5e8cd6b62e00ba54bbbc20566968d0e11826bc107e48c51f4eba"
Jul 30 15:36:00.292043 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:00.292005    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f95244425dca5e8cd6b62e00ba54bbbc20566968d0e11826bc107e48c51f4eba"} err="failed to get container status \"f95244425dca5e8cd6b62e00ba54bbbc20566968d0e11826bc107e48c51f4eba\": rpc error: code = NotFound desc = could not find container \"f95244425dca5e8cd6b62e00ba54bbbc20566968d0e11826bc107e48c51f4eba\": container with ID starting with f95244425dca5e8cd6b62e00ba54bbbc20566968d0e11826bc107e48c51f4eba not found: ID does not exist"
Jul 30 15:36:00.292043 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:00.292021    2578 scope.go:117] "RemoveContainer" containerID="919d07ee14b91d337d3549c9fe316f1537b7e5a15de19f36fe34338772731a4e"
Jul 30 15:36:00.299186 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:00.299160    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9"]
Jul 30 15:36:00.300395 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:00.300379    2578 scope.go:117] "RemoveContainer" containerID="c2d6d522c6ece646e9683d39d17376d289ad57a2eef9cf160e535856999fa01f"
Jul 30 15:36:00.303322 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:00.303301    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-f03573-openshift-default-9966d59f4-49rs9"]
Jul 30 15:36:00.308305 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:00.308285    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-27ftb\" (UniqueName: \"kubernetes.io/projected/81a4538d-aed5-48cd-82d9-6dea5de19c68-kube-api-access-27ftb\") pod \"81a4538d-aed5-48cd-82d9-6dea5de19c68\" (UID: \"81a4538d-aed5-48cd-82d9-6dea5de19c68\") "
Jul 30 15:36:00.308411 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:00.308370    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/81a4538d-aed5-48cd-82d9-6dea5de19c68-plugins-config-volume\") pod \"81a4538d-aed5-48cd-82d9-6dea5de19c68\" (UID: \"81a4538d-aed5-48cd-82d9-6dea5de19c68\") "
Jul 30 15:36:00.308481 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:00.308454    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7d6a3cc-a440-45c2-8d24-3678c2966c6b-plugins-config-volume\") pod \"f7d6a3cc-a440-45c2-8d24-3678c2966c6b\" (UID: \"f7d6a3cc-a440-45c2-8d24-3678c2966c6b\") "
Jul 30 15:36:00.308537 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:00.308501    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2gfxc\" (UniqueName: \"kubernetes.io/projected/f7d6a3cc-a440-45c2-8d24-3678c2966c6b-kube-api-access-2gfxc\") pod \"f7d6a3cc-a440-45c2-8d24-3678c2966c6b\" (UID: \"f7d6a3cc-a440-45c2-8d24-3678c2966c6b\") "
Jul 30 15:36:00.308763 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:00.308737    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81a4538d-aed5-48cd-82d9-6dea5de19c68-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "81a4538d-aed5-48cd-82d9-6dea5de19c68" (UID: "81a4538d-aed5-48cd-82d9-6dea5de19c68"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:36:00.308838 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:00.308740    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7d6a3cc-a440-45c2-8d24-3678c2966c6b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f7d6a3cc-a440-45c2-8d24-3678c2966c6b" (UID: "f7d6a3cc-a440-45c2-8d24-3678c2966c6b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:36:00.309136 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:00.309118    2578 scope.go:117] "RemoveContainer" containerID="919d07ee14b91d337d3549c9fe316f1537b7e5a15de19f36fe34338772731a4e"
Jul 30 15:36:00.309450 ip-10-0-133-117 kubenswrapper[2578]: E0730 15:36:00.309427    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"919d07ee14b91d337d3549c9fe316f1537b7e5a15de19f36fe34338772731a4e\": container with ID starting with 919d07ee14b91d337d3549c9fe316f1537b7e5a15de19f36fe34338772731a4e not found: ID does not exist" containerID="919d07ee14b91d337d3549c9fe316f1537b7e5a15de19f36fe34338772731a4e"
Jul 30 15:36:00.309528 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:00.309460    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"919d07ee14b91d337d3549c9fe316f1537b7e5a15de19f36fe34338772731a4e"} err="failed to get container status \"919d07ee14b91d337d3549c9fe316f1537b7e5a15de19f36fe34338772731a4e\": rpc error: code = NotFound desc = could not find container \"919d07ee14b91d337d3549c9fe316f1537b7e5a15de19f36fe34338772731a4e\": container with ID starting with 919d07ee14b91d337d3549c9fe316f1537b7e5a15de19f36fe34338772731a4e not found: ID does not exist"
Jul 30 15:36:00.309528 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:00.309484    2578 scope.go:117] "RemoveContainer" containerID="c2d6d522c6ece646e9683d39d17376d289ad57a2eef9cf160e535856999fa01f"
Jul 30 15:36:00.309773 ip-10-0-133-117 kubenswrapper[2578]: E0730 15:36:00.309750    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2d6d522c6ece646e9683d39d17376d289ad57a2eef9cf160e535856999fa01f\": container with ID starting with c2d6d522c6ece646e9683d39d17376d289ad57a2eef9cf160e535856999fa01f not found: ID does not exist" containerID="c2d6d522c6ece646e9683d39d17376d289ad57a2eef9cf160e535856999fa01f"
Jul 30 15:36:00.309858 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:00.309777    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2d6d522c6ece646e9683d39d17376d289ad57a2eef9cf160e535856999fa01f"} err="failed to get container status \"c2d6d522c6ece646e9683d39d17376d289ad57a2eef9cf160e535856999fa01f\": rpc error: code = NotFound desc = could not find container \"c2d6d522c6ece646e9683d39d17376d289ad57a2eef9cf160e535856999fa01f\": container with ID starting with c2d6d522c6ece646e9683d39d17376d289ad57a2eef9cf160e535856999fa01f not found: ID does not exist"
Jul 30 15:36:00.310479 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:00.310458    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7d6a3cc-a440-45c2-8d24-3678c2966c6b-kube-api-access-2gfxc" (OuterVolumeSpecName: "kube-api-access-2gfxc") pod "f7d6a3cc-a440-45c2-8d24-3678c2966c6b" (UID: "f7d6a3cc-a440-45c2-8d24-3678c2966c6b"). InnerVolumeSpecName "kube-api-access-2gfxc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:36:00.310565 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:00.310490    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81a4538d-aed5-48cd-82d9-6dea5de19c68-kube-api-access-27ftb" (OuterVolumeSpecName: "kube-api-access-27ftb") pod "81a4538d-aed5-48cd-82d9-6dea5de19c68" (UID: "81a4538d-aed5-48cd-82d9-6dea5de19c68"). InnerVolumeSpecName "kube-api-access-27ftb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:36:00.410103 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:00.410016    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7d6a3cc-a440-45c2-8d24-3678c2966c6b-plugins-config-volume\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:36:00.410103 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:00.410046    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2gfxc\" (UniqueName: \"kubernetes.io/projected/f7d6a3cc-a440-45c2-8d24-3678c2966c6b-kube-api-access-2gfxc\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:36:00.410103 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:00.410057    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-27ftb\" (UniqueName: \"kubernetes.io/projected/81a4538d-aed5-48cd-82d9-6dea5de19c68-kube-api-access-27ftb\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:36:00.410103 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:00.410068    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/81a4538d-aed5-48cd-82d9-6dea5de19c68-plugins-config-volume\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:36:00.599453 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:00.599427    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-a52c4d-55c8d964c4-wzmrq"]
Jul 30 15:36:00.603336 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:00.603313    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-a52c4d-55c8d964c4-wzmrq"]
Jul 30 15:36:00.614882 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:00.614859    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-a52c4d-5d8d9b8d84-p8dwt"]
Jul 30 15:36:00.618397 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:00.618378    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-a52c4d-5d8d9b8d84-p8dwt"]
Jul 30 15:36:00.978662 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:00.978628    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81a4538d-aed5-48cd-82d9-6dea5de19c68" path="/var/lib/kubelet/pods/81a4538d-aed5-48cd-82d9-6dea5de19c68/volumes"
Jul 30 15:36:00.979099 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:00.979085    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2a022a4-6a44-49c7-81fb-4e70badb77f3" path="/var/lib/kubelet/pods/e2a022a4-6a44-49c7-81fb-4e70badb77f3/volumes"
Jul 30 15:36:00.979483 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:00.979470    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7d6a3cc-a440-45c2-8d24-3678c2966c6b" path="/var/lib/kubelet/pods/f7d6a3cc-a440-45c2-8d24-3678c2966c6b/volumes"
Jul 30 15:36:06.800106 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:06.800066    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl"]
Jul 30 15:36:06.800523 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:06.800382    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl" podUID="d2f543c7-26b5-4424-87b3-27bd324a27ae" containerName="istio-proxy" containerID="cri-o://75a552dc29b851a5dba00c946848f6dde9c2980a435deca8a49146af357d8d51" gracePeriod=30
Jul 30 15:36:07.363670 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:07.363629    2578 patch_prober.go:28] interesting pod/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 30 15:36:07.363839 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:07.363714    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl" podUID="d2f543c7-26b5-4424-87b3-27bd324a27ae" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 30 15:36:07.706700 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:07.706618    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc"]
Jul 30 15:36:07.707007 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:07.706994    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81c06155-cded-44ca-b41f-d733a6f54fad" containerName="maas-api"
Jul 30 15:36:07.707047 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:07.707009    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="81c06155-cded-44ca-b41f-d733a6f54fad" containerName="maas-api"
Jul 30 15:36:07.707047 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:07.707027    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2a022a4-6a44-49c7-81fb-4e70badb77f3" containerName="istio-proxy"
Jul 30 15:36:07.707047 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:07.707035    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2a022a4-6a44-49c7-81fb-4e70badb77f3" containerName="istio-proxy"
Jul 30 15:36:07.707047 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:07.707042    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7d6a3cc-a440-45c2-8d24-3678c2966c6b" containerName="payload-pre-processing"
Jul 30 15:36:07.707202 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:07.707051    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7d6a3cc-a440-45c2-8d24-3678c2966c6b" containerName="payload-pre-processing"
Jul 30 15:36:07.707202 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:07.707063    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2a022a4-6a44-49c7-81fb-4e70badb77f3" containerName="istio-proxy"
Jul 30 15:36:07.707202 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:07.707069    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2a022a4-6a44-49c7-81fb-4e70badb77f3" containerName="istio-proxy"
Jul 30 15:36:07.707202 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:07.707074    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81a4538d-aed5-48cd-82d9-6dea5de19c68" containerName="payload-processing"
Jul 30 15:36:07.707202 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:07.707080    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="81a4538d-aed5-48cd-82d9-6dea5de19c68" containerName="payload-processing"
Jul 30 15:36:07.707202 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:07.707095    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e041fcc6-edac-46d8-b5e3-7875d26ffa13" containerName="payload-processing"
Jul 30 15:36:07.707202 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:07.707101    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e041fcc6-edac-46d8-b5e3-7875d26ffa13" containerName="payload-processing"
Jul 30 15:36:07.707202 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:07.707189    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e041fcc6-edac-46d8-b5e3-7875d26ffa13" containerName="payload-processing"
Jul 30 15:36:07.707202 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:07.707200    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2a022a4-6a44-49c7-81fb-4e70badb77f3" containerName="istio-proxy"
Jul 30 15:36:07.707491 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:07.707208    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="81a4538d-aed5-48cd-82d9-6dea5de19c68" containerName="payload-processing"
Jul 30 15:36:07.707491 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:07.707220    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7d6a3cc-a440-45c2-8d24-3678c2966c6b" containerName="payload-pre-processing"
Jul 30 15:36:07.707491 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:07.707227    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="81c06155-cded-44ca-b41f-d733a6f54fad" containerName="maas-api"
Jul 30 15:36:07.711933 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:07.711911    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc"
Jul 30 15:36:07.713960 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:07.713942    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-a1c473-openshift-default-dockercfg-9xn9w\""
Jul 30 15:36:07.722764 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:07.722741    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc"]
Jul 30 15:36:07.879310 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:07.879276    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c595b020-052f-4b1c-8281-ccee6116b418-istiod-ca-cert\") pod \"e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc\" (UID: \"c595b020-052f-4b1c-8281-ccee6116b418\") " pod="openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc"
Jul 30 15:36:07.879731 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:07.879328    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c595b020-052f-4b1c-8281-ccee6116b418-workload-socket\") pod \"e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc\" (UID: \"c595b020-052f-4b1c-8281-ccee6116b418\") " pod="openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc"
Jul 30 15:36:07.879731 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:07.879397    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c595b020-052f-4b1c-8281-ccee6116b418-workload-certs\") pod \"e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc\" (UID: \"c595b020-052f-4b1c-8281-ccee6116b418\") " pod="openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc"
Jul 30 15:36:07.879731 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:07.879430    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c595b020-052f-4b1c-8281-ccee6116b418-istio-podinfo\") pod \"e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc\" (UID: \"c595b020-052f-4b1c-8281-ccee6116b418\") " pod="openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc"
Jul 30 15:36:07.879731 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:07.879519    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c595b020-052f-4b1c-8281-ccee6116b418-istio-token\") pod \"e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc\" (UID: \"c595b020-052f-4b1c-8281-ccee6116b418\") " pod="openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc"
Jul 30 15:36:07.879731 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:07.879549    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c595b020-052f-4b1c-8281-ccee6116b418-istio-data\") pod \"e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc\" (UID: \"c595b020-052f-4b1c-8281-ccee6116b418\") " pod="openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc"
Jul 30 15:36:07.879731 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:07.879569    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c595b020-052f-4b1c-8281-ccee6116b418-credential-socket\") pod \"e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc\" (UID: \"c595b020-052f-4b1c-8281-ccee6116b418\") " pod="openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc"
Jul 30 15:36:07.879731 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:07.879615    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zj6tk\" (UniqueName: \"kubernetes.io/projected/c595b020-052f-4b1c-8281-ccee6116b418-kube-api-access-zj6tk\") pod \"e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc\" (UID: \"c595b020-052f-4b1c-8281-ccee6116b418\") " pod="openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc"
Jul 30 15:36:07.879731 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:07.879683    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c595b020-052f-4b1c-8281-ccee6116b418-istio-envoy\") pod \"e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc\" (UID: \"c595b020-052f-4b1c-8281-ccee6116b418\") " pod="openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc"
Jul 30 15:36:07.980613 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:07.980513    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c595b020-052f-4b1c-8281-ccee6116b418-istiod-ca-cert\") pod \"e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc\" (UID: \"c595b020-052f-4b1c-8281-ccee6116b418\") " pod="openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc"
Jul 30 15:36:07.980613 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:07.980565    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c595b020-052f-4b1c-8281-ccee6116b418-workload-socket\") pod \"e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc\" (UID: \"c595b020-052f-4b1c-8281-ccee6116b418\") " pod="openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc"
Jul 30 15:36:07.980613 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:07.980601    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c595b020-052f-4b1c-8281-ccee6116b418-workload-certs\") pod \"e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc\" (UID: \"c595b020-052f-4b1c-8281-ccee6116b418\") " pod="openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc"
Jul 30 15:36:07.980885 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:07.980627    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c595b020-052f-4b1c-8281-ccee6116b418-istio-podinfo\") pod \"e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc\" (UID: \"c595b020-052f-4b1c-8281-ccee6116b418\") " pod="openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc"
Jul 30 15:36:07.980885 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:07.980704    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c595b020-052f-4b1c-8281-ccee6116b418-istio-token\") pod \"e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc\" (UID: \"c595b020-052f-4b1c-8281-ccee6116b418\") " pod="openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc"
Jul 30 15:36:07.980885 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:07.980748    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c595b020-052f-4b1c-8281-ccee6116b418-istio-data\") pod \"e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc\" (UID: \"c595b020-052f-4b1c-8281-ccee6116b418\") " pod="openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc"
Jul 30 15:36:07.980885 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:07.980777    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c595b020-052f-4b1c-8281-ccee6116b418-credential-socket\") pod \"e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc\" (UID: \"c595b020-052f-4b1c-8281-ccee6116b418\") " pod="openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc"
Jul 30 15:36:07.980885 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:07.980852    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zj6tk\" (UniqueName: \"kubernetes.io/projected/c595b020-052f-4b1c-8281-ccee6116b418-kube-api-access-zj6tk\") pod \"e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc\" (UID: \"c595b020-052f-4b1c-8281-ccee6116b418\") " pod="openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc"
Jul 30 15:36:07.981125 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:07.980913    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c595b020-052f-4b1c-8281-ccee6116b418-istio-envoy\") pod \"e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc\" (UID: \"c595b020-052f-4b1c-8281-ccee6116b418\") " pod="openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc"
Jul 30 15:36:07.981125 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:07.981057    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c595b020-052f-4b1c-8281-ccee6116b418-workload-socket\") pod \"e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc\" (UID: \"c595b020-052f-4b1c-8281-ccee6116b418\") " pod="openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc"
Jul 30 15:36:07.981125 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:07.981076    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c595b020-052f-4b1c-8281-ccee6116b418-workload-certs\") pod \"e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc\" (UID: \"c595b020-052f-4b1c-8281-ccee6116b418\") " pod="openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc"
Jul 30 15:36:07.981328 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:07.981135    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c595b020-052f-4b1c-8281-ccee6116b418-istio-data\") pod \"e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc\" (UID: \"c595b020-052f-4b1c-8281-ccee6116b418\") " pod="openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc"
Jul 30 15:36:07.981403 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:07.981372    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c595b020-052f-4b1c-8281-ccee6116b418-credential-socket\") pod \"e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc\" (UID: \"c595b020-052f-4b1c-8281-ccee6116b418\") " pod="openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc"
Jul 30 15:36:07.981403 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:07.981396    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c595b020-052f-4b1c-8281-ccee6116b418-istiod-ca-cert\") pod \"e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc\" (UID: \"c595b020-052f-4b1c-8281-ccee6116b418\") " pod="openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc"
Jul 30 15:36:07.983103 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:07.983078    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c595b020-052f-4b1c-8281-ccee6116b418-istio-envoy\") pod \"e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc\" (UID: \"c595b020-052f-4b1c-8281-ccee6116b418\") " pod="openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc"
Jul 30 15:36:07.983355 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:07.983336    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c595b020-052f-4b1c-8281-ccee6116b418-istio-podinfo\") pod \"e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc\" (UID: \"c595b020-052f-4b1c-8281-ccee6116b418\") " pod="openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc"
Jul 30 15:36:07.988009 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:07.987981    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c595b020-052f-4b1c-8281-ccee6116b418-istio-token\") pod \"e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc\" (UID: \"c595b020-052f-4b1c-8281-ccee6116b418\") " pod="openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc"
Jul 30 15:36:07.988319 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:07.988303    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zj6tk\" (UniqueName: \"kubernetes.io/projected/c595b020-052f-4b1c-8281-ccee6116b418-kube-api-access-zj6tk\") pod \"e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc\" (UID: \"c595b020-052f-4b1c-8281-ccee6116b418\") " pod="openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc"
Jul 30 15:36:08.023059 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:08.023027    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc"
Jul 30 15:36:08.162736 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:08.162708    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc"]
Jul 30 15:36:08.164634 ip-10-0-133-117 kubenswrapper[2578]: W0730 15:36:08.164601    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc595b020_052f_4b1c_8281_ccee6116b418.slice/crio-ac1cb73b7b228e7e1faccb81f257f15c5d362d32e71956b32d863be50d81051a WatchSource:0}: Error finding container ac1cb73b7b228e7e1faccb81f257f15c5d362d32e71956b32d863be50d81051a: Status 404 returned error can't find the container with id ac1cb73b7b228e7e1faccb81f257f15c5d362d32e71956b32d863be50d81051a
Jul 30 15:36:08.166870 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:08.166836    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:36:08.166961 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:08.166906    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:36:08.166961 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:08.166933    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:36:08.321829 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:08.321785    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc" event={"ID":"c595b020-052f-4b1c-8281-ccee6116b418","Type":"ContainerStarted","Data":"281576b0a017fc6f71d903777bd127d5eac370d397e7db65b2d0649a89825ceb"}
Jul 30 15:36:08.322041 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:08.321849    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc" event={"ID":"c595b020-052f-4b1c-8281-ccee6116b418","Type":"ContainerStarted","Data":"ac1cb73b7b228e7e1faccb81f257f15c5d362d32e71956b32d863be50d81051a"}
Jul 30 15:36:08.341966 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:08.341915    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc" podStartSLOduration=1.341891813 podStartE2EDuration="1.341891813s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:36:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:36:08.339610967 +0000 UTC m=+2879.868549442" watchObservedRunningTime="2026-07-30 15:36:08.341891813 +0000 UTC m=+2879.870830286"
Jul 30 15:36:09.023500 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:09.023461    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc"
Jul 30 15:36:09.028521 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:09.028497    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc"
Jul 30 15:36:09.326284 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:09.326255    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc"
Jul 30 15:36:09.327075 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:09.327059    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc"
Jul 30 15:36:12.080008 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.079978    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl"
Jul 30 15:36:12.120746 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.120708    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2f543c7-26b5-4424-87b3-27bd324a27ae-istio-envoy\") pod \"d2f543c7-26b5-4424-87b3-27bd324a27ae\" (UID: \"d2f543c7-26b5-4424-87b3-27bd324a27ae\") "
Jul 30 15:36:12.120945 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.120766    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2f543c7-26b5-4424-87b3-27bd324a27ae-istio-data\") pod \"d2f543c7-26b5-4424-87b3-27bd324a27ae\" (UID: \"d2f543c7-26b5-4424-87b3-27bd324a27ae\") "
Jul 30 15:36:12.120945 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.120809    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2f543c7-26b5-4424-87b3-27bd324a27ae-istio-podinfo\") pod \"d2f543c7-26b5-4424-87b3-27bd324a27ae\" (UID: \"d2f543c7-26b5-4424-87b3-27bd324a27ae\") "
Jul 30 15:36:12.120945 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.120847    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2txnm\" (UniqueName: \"kubernetes.io/projected/d2f543c7-26b5-4424-87b3-27bd324a27ae-kube-api-access-2txnm\") pod \"d2f543c7-26b5-4424-87b3-27bd324a27ae\" (UID: \"d2f543c7-26b5-4424-87b3-27bd324a27ae\") "
Jul 30 15:36:12.120945 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.120879    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2f543c7-26b5-4424-87b3-27bd324a27ae-workload-socket\") pod \"d2f543c7-26b5-4424-87b3-27bd324a27ae\" (UID: \"d2f543c7-26b5-4424-87b3-27bd324a27ae\") "
Jul 30 15:36:12.121198 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.121173    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2f543c7-26b5-4424-87b3-27bd324a27ae-credential-socket\") pod \"d2f543c7-26b5-4424-87b3-27bd324a27ae\" (UID: \"d2f543c7-26b5-4424-87b3-27bd324a27ae\") "
Jul 30 15:36:12.121280 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.121230    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2f543c7-26b5-4424-87b3-27bd324a27ae-istiod-ca-cert\") pod \"d2f543c7-26b5-4424-87b3-27bd324a27ae\" (UID: \"d2f543c7-26b5-4424-87b3-27bd324a27ae\") "
Jul 30 15:36:12.121662 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.121634    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2f543c7-26b5-4424-87b3-27bd324a27ae-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d2f543c7-26b5-4424-87b3-27bd324a27ae" (UID: "d2f543c7-26b5-4424-87b3-27bd324a27ae"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:36:12.121828 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.121716    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2f543c7-26b5-4424-87b3-27bd324a27ae-istio-data" (OuterVolumeSpecName: "istio-data") pod "d2f543c7-26b5-4424-87b3-27bd324a27ae" (UID: "d2f543c7-26b5-4424-87b3-27bd324a27ae"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:36:12.122022 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.121971    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2f543c7-26b5-4424-87b3-27bd324a27ae-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d2f543c7-26b5-4424-87b3-27bd324a27ae" (UID: "d2f543c7-26b5-4424-87b3-27bd324a27ae"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:36:12.122022 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.121973    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2f543c7-26b5-4424-87b3-27bd324a27ae-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d2f543c7-26b5-4424-87b3-27bd324a27ae" (UID: "d2f543c7-26b5-4424-87b3-27bd324a27ae"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:36:12.122673 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.122648    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-a1c473-74659688b8-gr8sp"]
Jul 30 15:36:12.123283 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.123264    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2f543c7-26b5-4424-87b3-27bd324a27ae" containerName="istio-proxy"
Jul 30 15:36:12.123374 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.123288    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2f543c7-26b5-4424-87b3-27bd324a27ae" containerName="istio-proxy"
Jul 30 15:36:12.123374 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.123302    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2f543c7-26b5-4424-87b3-27bd324a27ae" containerName="istio-proxy"
Jul 30 15:36:12.123374 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.123313    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2f543c7-26b5-4424-87b3-27bd324a27ae" containerName="istio-proxy"
Jul 30 15:36:12.123529 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.123397    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2f543c7-26b5-4424-87b3-27bd324a27ae" containerName="istio-proxy"
Jul 30 15:36:12.123529 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.123415    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2a022a4-6a44-49c7-81fb-4e70badb77f3" containerName="istio-proxy"
Jul 30 15:36:12.123529 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.123424    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2f543c7-26b5-4424-87b3-27bd324a27ae" containerName="istio-proxy"
Jul 30 15:36:12.123529 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.123478    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2f543c7-26b5-4424-87b3-27bd324a27ae-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d2f543c7-26b5-4424-87b3-27bd324a27ae" (UID: "d2f543c7-26b5-4424-87b3-27bd324a27ae"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:36:12.123529 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.123501    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2f543c7-26b5-4424-87b3-27bd324a27ae-kube-api-access-2txnm" (OuterVolumeSpecName: "kube-api-access-2txnm") pod "d2f543c7-26b5-4424-87b3-27bd324a27ae" (UID: "d2f543c7-26b5-4424-87b3-27bd324a27ae"). InnerVolumeSpecName "kube-api-access-2txnm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:36:12.124535 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.124508    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d2f543c7-26b5-4424-87b3-27bd324a27ae-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d2f543c7-26b5-4424-87b3-27bd324a27ae" (UID: "d2f543c7-26b5-4424-87b3-27bd324a27ae"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 15:36:12.126933 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.126910    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a1c473-74659688b8-gr8sp"
Jul 30 15:36:12.128923 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.128899    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-a1c473-dockercfg-mcjdm\""
Jul 30 15:36:12.129022 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.128947    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-a1c473\""
Jul 30 15:36:12.133762 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.133732    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-a1c473-74659688b8-gr8sp"]
Jul 30 15:36:12.159964 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.159933    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-a1c473-7ff64bc75d-t7sc9"]
Jul 30 15:36:12.163394 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.163376    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-a1c473-7ff64bc75d-t7sc9"
Jul 30 15:36:12.172409 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.172383    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-a1c473-7ff64bc75d-t7sc9"]
Jul 30 15:36:12.222448 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.222415    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2f543c7-26b5-4424-87b3-27bd324a27ae-workload-certs\") pod \"d2f543c7-26b5-4424-87b3-27bd324a27ae\" (UID: \"d2f543c7-26b5-4424-87b3-27bd324a27ae\") "
Jul 30 15:36:12.222631 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.222466    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2f543c7-26b5-4424-87b3-27bd324a27ae-istio-token\") pod \"d2f543c7-26b5-4424-87b3-27bd324a27ae\" (UID: \"d2f543c7-26b5-4424-87b3-27bd324a27ae\") "
Jul 30 15:36:12.222631 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.222599    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2305a153-2af1-417f-aecb-6f6ef6988e4e-plugins-config-volume\") pod \"payload-processing-e2e-api-a-a1c473-7ff64bc75d-t7sc9\" (UID: \"2305a153-2af1-417f-aecb-6f6ef6988e4e\") " pod="openshift-ingress/payload-processing-e2e-api-a-a1c473-7ff64bc75d-t7sc9"
Jul 30 15:36:12.222723 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.222641    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49668e3d-600b-4e5c-bdc2-6231711f3b49-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-a1c473-74659688b8-gr8sp\" (UID: \"49668e3d-600b-4e5c-bdc2-6231711f3b49\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-a1c473-74659688b8-gr8sp"
Jul 30 15:36:12.222723 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.222672    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2f543c7-26b5-4424-87b3-27bd324a27ae-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d2f543c7-26b5-4424-87b3-27bd324a27ae" (UID: "d2f543c7-26b5-4424-87b3-27bd324a27ae"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:36:12.222793 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.222725    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-595jv\" (UniqueName: \"kubernetes.io/projected/2305a153-2af1-417f-aecb-6f6ef6988e4e-kube-api-access-595jv\") pod \"payload-processing-e2e-api-a-a1c473-7ff64bc75d-t7sc9\" (UID: \"2305a153-2af1-417f-aecb-6f6ef6988e4e\") " pod="openshift-ingress/payload-processing-e2e-api-a-a1c473-7ff64bc75d-t7sc9"
Jul 30 15:36:12.222793 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.222752    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgv5s\" (UniqueName: \"kubernetes.io/projected/49668e3d-600b-4e5c-bdc2-6231711f3b49-kube-api-access-fgv5s\") pod \"payload-pre-processing-e2e-api-a-a1c473-74659688b8-gr8sp\" (UID: \"49668e3d-600b-4e5c-bdc2-6231711f3b49\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-a1c473-74659688b8-gr8sp"
Jul 30 15:36:12.222867 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.222796    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2f543c7-26b5-4424-87b3-27bd324a27ae-istio-podinfo\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:36:12.222867 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.222811    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2txnm\" (UniqueName: \"kubernetes.io/projected/d2f543c7-26b5-4424-87b3-27bd324a27ae-kube-api-access-2txnm\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:36:12.222867 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.222821    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2f543c7-26b5-4424-87b3-27bd324a27ae-workload-socket\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:36:12.222867 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.222830    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2f543c7-26b5-4424-87b3-27bd324a27ae-credential-socket\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:36:12.222867 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.222838    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2f543c7-26b5-4424-87b3-27bd324a27ae-istiod-ca-cert\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:36:12.222867 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.222846    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2f543c7-26b5-4424-87b3-27bd324a27ae-workload-certs\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:36:12.222867 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.222853    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2f543c7-26b5-4424-87b3-27bd324a27ae-istio-envoy\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:36:12.222867 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.222860    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2f543c7-26b5-4424-87b3-27bd324a27ae-istio-data\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:36:12.224574 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.224548    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2f543c7-26b5-4424-87b3-27bd324a27ae-istio-token" (OuterVolumeSpecName: "istio-token") pod "d2f543c7-26b5-4424-87b3-27bd324a27ae" (UID: "d2f543c7-26b5-4424-87b3-27bd324a27ae"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:36:12.323468 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.323375    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-595jv\" (UniqueName: \"kubernetes.io/projected/2305a153-2af1-417f-aecb-6f6ef6988e4e-kube-api-access-595jv\") pod \"payload-processing-e2e-api-a-a1c473-7ff64bc75d-t7sc9\" (UID: \"2305a153-2af1-417f-aecb-6f6ef6988e4e\") " pod="openshift-ingress/payload-processing-e2e-api-a-a1c473-7ff64bc75d-t7sc9"
Jul 30 15:36:12.323468 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.323420    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgv5s\" (UniqueName: \"kubernetes.io/projected/49668e3d-600b-4e5c-bdc2-6231711f3b49-kube-api-access-fgv5s\") pod \"payload-pre-processing-e2e-api-a-a1c473-74659688b8-gr8sp\" (UID: \"49668e3d-600b-4e5c-bdc2-6231711f3b49\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-a1c473-74659688b8-gr8sp"
Jul 30 15:36:12.323679 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.323489    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2305a153-2af1-417f-aecb-6f6ef6988e4e-plugins-config-volume\") pod \"payload-processing-e2e-api-a-a1c473-7ff64bc75d-t7sc9\" (UID: \"2305a153-2af1-417f-aecb-6f6ef6988e4e\") " pod="openshift-ingress/payload-processing-e2e-api-a-a1c473-7ff64bc75d-t7sc9"
Jul 30 15:36:12.323679 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.323541    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49668e3d-600b-4e5c-bdc2-6231711f3b49-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-a1c473-74659688b8-gr8sp\" (UID: \"49668e3d-600b-4e5c-bdc2-6231711f3b49\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-a1c473-74659688b8-gr8sp"
Jul 30 15:36:12.323679 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.323619    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2f543c7-26b5-4424-87b3-27bd324a27ae-istio-token\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:36:12.324294 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.324277    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49668e3d-600b-4e5c-bdc2-6231711f3b49-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-a1c473-74659688b8-gr8sp\" (UID: \"49668e3d-600b-4e5c-bdc2-6231711f3b49\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-a1c473-74659688b8-gr8sp"
Jul 30 15:36:12.324388 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.324286    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2305a153-2af1-417f-aecb-6f6ef6988e4e-plugins-config-volume\") pod \"payload-processing-e2e-api-a-a1c473-7ff64bc75d-t7sc9\" (UID: \"2305a153-2af1-417f-aecb-6f6ef6988e4e\") " pod="openshift-ingress/payload-processing-e2e-api-a-a1c473-7ff64bc75d-t7sc9"
Jul 30 15:36:12.331132 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.331104    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgv5s\" (UniqueName: \"kubernetes.io/projected/49668e3d-600b-4e5c-bdc2-6231711f3b49-kube-api-access-fgv5s\") pod \"payload-pre-processing-e2e-api-a-a1c473-74659688b8-gr8sp\" (UID: \"49668e3d-600b-4e5c-bdc2-6231711f3b49\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-a1c473-74659688b8-gr8sp"
Jul 30 15:36:12.331132 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.331121    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-595jv\" (UniqueName: \"kubernetes.io/projected/2305a153-2af1-417f-aecb-6f6ef6988e4e-kube-api-access-595jv\") pod \"payload-processing-e2e-api-a-a1c473-7ff64bc75d-t7sc9\" (UID: \"2305a153-2af1-417f-aecb-6f6ef6988e4e\") " pod="openshift-ingress/payload-processing-e2e-api-a-a1c473-7ff64bc75d-t7sc9"
Jul 30 15:36:12.343726 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.343684    2578 generic.go:356] "Generic (PLEG): container finished" podID="d2f543c7-26b5-4424-87b3-27bd324a27ae" containerID="75a552dc29b851a5dba00c946848f6dde9c2980a435deca8a49146af357d8d51" exitCode=0
Jul 30 15:36:12.343888 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.343764    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl"
Jul 30 15:36:12.343888 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.343765    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl" event={"ID":"d2f543c7-26b5-4424-87b3-27bd324a27ae","Type":"ContainerDied","Data":"75a552dc29b851a5dba00c946848f6dde9c2980a435deca8a49146af357d8d51"}
Jul 30 15:36:12.343978 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.343887    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl" event={"ID":"d2f543c7-26b5-4424-87b3-27bd324a27ae","Type":"ContainerDied","Data":"9cfc6a3ffbce987baf05179bc4a4942e621a5b942286f8bee65eefc57a7a02bb"}
Jul 30 15:36:12.343978 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.343914    2578 scope.go:117] "RemoveContainer" containerID="75a552dc29b851a5dba00c946848f6dde9c2980a435deca8a49146af357d8d51"
Jul 30 15:36:12.353090 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.353072    2578 scope.go:117] "RemoveContainer" containerID="4fc997daac43146b076fbb798ec680bbd63cd4d3dfb83d9ac0d3fd2228cded52"
Jul 30 15:36:12.361888 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.361870    2578 scope.go:117] "RemoveContainer" containerID="75a552dc29b851a5dba00c946848f6dde9c2980a435deca8a49146af357d8d51"
Jul 30 15:36:12.362158 ip-10-0-133-117 kubenswrapper[2578]: E0730 15:36:12.362126    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75a552dc29b851a5dba00c946848f6dde9c2980a435deca8a49146af357d8d51\": container with ID starting with 75a552dc29b851a5dba00c946848f6dde9c2980a435deca8a49146af357d8d51 not found: ID does not exist" containerID="75a552dc29b851a5dba00c946848f6dde9c2980a435deca8a49146af357d8d51"
Jul 30 15:36:12.362208 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.362178    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75a552dc29b851a5dba00c946848f6dde9c2980a435deca8a49146af357d8d51"} err="failed to get container status \"75a552dc29b851a5dba00c946848f6dde9c2980a435deca8a49146af357d8d51\": rpc error: code = NotFound desc = could not find container \"75a552dc29b851a5dba00c946848f6dde9c2980a435deca8a49146af357d8d51\": container with ID starting with 75a552dc29b851a5dba00c946848f6dde9c2980a435deca8a49146af357d8d51 not found: ID does not exist"
Jul 30 15:36:12.362208 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.362197    2578 scope.go:117] "RemoveContainer" containerID="4fc997daac43146b076fbb798ec680bbd63cd4d3dfb83d9ac0d3fd2228cded52"
Jul 30 15:36:12.362427 ip-10-0-133-117 kubenswrapper[2578]: E0730 15:36:12.362411    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4fc997daac43146b076fbb798ec680bbd63cd4d3dfb83d9ac0d3fd2228cded52\": container with ID starting with 4fc997daac43146b076fbb798ec680bbd63cd4d3dfb83d9ac0d3fd2228cded52 not found: ID does not exist" containerID="4fc997daac43146b076fbb798ec680bbd63cd4d3dfb83d9ac0d3fd2228cded52"
Jul 30 15:36:12.362469 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.362432    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fc997daac43146b076fbb798ec680bbd63cd4d3dfb83d9ac0d3fd2228cded52"} err="failed to get container status \"4fc997daac43146b076fbb798ec680bbd63cd4d3dfb83d9ac0d3fd2228cded52\": rpc error: code = NotFound desc = could not find container \"4fc997daac43146b076fbb798ec680bbd63cd4d3dfb83d9ac0d3fd2228cded52\": container with ID starting with 4fc997daac43146b076fbb798ec680bbd63cd4d3dfb83d9ac0d3fd2228cded52 not found: ID does not exist"
Jul 30 15:36:12.366374 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.366350    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl"]
Jul 30 15:36:12.370179 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.370126    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-a52c4d-openshift-default-78dfd7fc86-8mzhl"]
Jul 30 15:36:12.452901 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.452874    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a1c473-74659688b8-gr8sp"
Jul 30 15:36:12.474303 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.474273    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-a1c473-7ff64bc75d-t7sc9"
Jul 30 15:36:12.603425 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.603394    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-a1c473-74659688b8-gr8sp"]
Jul 30 15:36:12.604813 ip-10-0-133-117 kubenswrapper[2578]: W0730 15:36:12.604780    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49668e3d_600b_4e5c_bdc2_6231711f3b49.slice/crio-069ab4238451f90ef567998b1341e62859eea590bc23e53143185ab366d7f7e6 WatchSource:0}: Error finding container 069ab4238451f90ef567998b1341e62859eea590bc23e53143185ab366d7f7e6: Status 404 returned error can't find the container with id 069ab4238451f90ef567998b1341e62859eea590bc23e53143185ab366d7f7e6
Jul 30 15:36:12.629394 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.629369    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-a1c473-7ff64bc75d-t7sc9"]
Jul 30 15:36:12.642226 ip-10-0-133-117 kubenswrapper[2578]: W0730 15:36:12.642196    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2305a153_2af1_417f_aecb_6f6ef6988e4e.slice/crio-d4614fa1a3a65e9d387fdbe517a2b04f278e8a84631f6aca7bbc94fa92c364e1 WatchSource:0}: Error finding container d4614fa1a3a65e9d387fdbe517a2b04f278e8a84631f6aca7bbc94fa92c364e1: Status 404 returned error can't find the container with id d4614fa1a3a65e9d387fdbe517a2b04f278e8a84631f6aca7bbc94fa92c364e1
Jul 30 15:36:12.981260 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:12.981224    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2f543c7-26b5-4424-87b3-27bd324a27ae" path="/var/lib/kubelet/pods/d2f543c7-26b5-4424-87b3-27bd324a27ae/volumes"
Jul 30 15:36:13.353624 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:13.353580    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-a1c473-7ff64bc75d-t7sc9" event={"ID":"2305a153-2af1-417f-aecb-6f6ef6988e4e","Type":"ContainerStarted","Data":"9b4b1b7071a7b1b2946cba7916523370dd18cdfe0dbccab2d147d72009e01aee"}
Jul 30 15:36:13.354097 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:13.353648    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-a1c473-7ff64bc75d-t7sc9" event={"ID":"2305a153-2af1-417f-aecb-6f6ef6988e4e","Type":"ContainerStarted","Data":"d4614fa1a3a65e9d387fdbe517a2b04f278e8a84631f6aca7bbc94fa92c364e1"}
Jul 30 15:36:13.354097 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:13.353689    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-a1c473-7ff64bc75d-t7sc9"
Jul 30 15:36:13.355227 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:13.355201    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a1c473-74659688b8-gr8sp" event={"ID":"49668e3d-600b-4e5c-bdc2-6231711f3b49","Type":"ContainerStarted","Data":"2cab01c797fd2f5867bf20c9b38a6f2b54e2a12870843e1f443ac9ce22c2602a"}
Jul 30 15:36:13.355348 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:13.355247    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a1c473-74659688b8-gr8sp" event={"ID":"49668e3d-600b-4e5c-bdc2-6231711f3b49","Type":"ContainerStarted","Data":"069ab4238451f90ef567998b1341e62859eea590bc23e53143185ab366d7f7e6"}
Jul 30 15:36:13.355348 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:13.355281    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a1c473-74659688b8-gr8sp"
Jul 30 15:36:13.369724 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:13.369669    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-a1c473-7ff64bc75d-t7sc9" podStartSLOduration=1.369651191 podStartE2EDuration="1.369651191s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:36:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:36:13.367716008 +0000 UTC m=+2884.896654508" watchObservedRunningTime="2026-07-30 15:36:13.369651191 +0000 UTC m=+2884.898589672"
Jul 30 15:36:13.384504 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:13.384444    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a1c473-74659688b8-gr8sp" podStartSLOduration=1.384426406 podStartE2EDuration="1.384426406s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:36:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:36:13.381873727 +0000 UTC m=+2884.910812226" watchObservedRunningTime="2026-07-30 15:36:13.384426406 +0000 UTC m=+2884.913364870"
Jul 30 15:36:24.327912 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:24.327880    2578 patch_prober.go:28] interesting pod/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Jul 30 15:36:24.328335 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:24.327938    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc" podUID="c595b020-052f-4b1c-8281-ccee6116b418" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Jul 30 15:36:24.362069 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:24.362030    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a1c473-74659688b8-gr8sp"
Jul 30 15:36:24.362301 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:24.362281    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-a1c473-7ff64bc75d-t7sc9"
Jul 30 15:36:24.421893 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:24.421861    2578 generic.go:356] "Generic (PLEG): container finished" podID="c595b020-052f-4b1c-8281-ccee6116b418" containerID="281576b0a017fc6f71d903777bd127d5eac370d397e7db65b2d0649a89825ceb" exitCode=0
Jul 30 15:36:24.422065 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:24.421909    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc" event={"ID":"c595b020-052f-4b1c-8281-ccee6116b418","Type":"ContainerDied","Data":"281576b0a017fc6f71d903777bd127d5eac370d397e7db65b2d0649a89825ceb"}
Jul 30 15:36:24.422356 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:24.422340    2578 scope.go:117] "RemoveContainer" containerID="281576b0a017fc6f71d903777bd127d5eac370d397e7db65b2d0649a89825ceb"
Jul 30 15:36:24.423795 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:24.423747    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:36:24.424389 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:24.424112    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:36:24.424389 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:24.424197    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:36:25.428757 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:25.428721    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc" event={"ID":"c595b020-052f-4b1c-8281-ccee6116b418","Type":"ContainerStarted","Data":"43fb8ce6074abfe78cd688cef816d1cda916c6ebed9dab58b846d0b625de40b0"}
Jul 30 15:36:26.024204 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:26.024174    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc"
Jul 30 15:36:26.024421 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:26.024401    2578 patch_prober.go:28] interesting pod/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Jul 30 15:36:26.024479 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:26.024456    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc" podUID="c595b020-052f-4b1c-8281-ccee6116b418" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Jul 30 15:36:27.023467 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:27.023425    2578 patch_prober.go:28] interesting pod/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Jul 30 15:36:27.023946 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:27.023502    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc" podUID="c595b020-052f-4b1c-8281-ccee6116b418" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Jul 30 15:36:28.023513 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:28.023473    2578 patch_prober.go:28] interesting pod/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Jul 30 15:36:28.023932 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:28.023547    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc" podUID="c595b020-052f-4b1c-8281-ccee6116b418" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Jul 30 15:36:29.023546 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:29.023507    2578 patch_prober.go:28] interesting pod/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Jul 30 15:36:29.023944 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:29.023562    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc" podUID="c595b020-052f-4b1c-8281-ccee6116b418" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Jul 30 15:36:30.028426 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:30.028393    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc"
Jul 30 15:36:30.028796 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:30.028645    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc"
Jul 30 15:36:30.029234 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:30.029215    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc"
Jul 30 15:36:36.223602 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:36.223566    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz"]
Jul 30 15:36:36.227264 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:36.227240    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz"
Jul 30 15:36:36.229236 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:36.229214    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-e2257c-openshift-default-dockercfg-5qj6r\""
Jul 30 15:36:36.239246 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:36.239219    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz"]
Jul 30 15:36:36.365066 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:36.365029    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36e1cb08-f75a-44e9-9914-fd1124432ab3-istio-envoy\") pod \"e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz\" (UID: \"36e1cb08-f75a-44e9-9914-fd1124432ab3\") " pod="openshift-ingress/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz"
Jul 30 15:36:36.365248 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:36.365093    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvlt2\" (UniqueName: \"kubernetes.io/projected/36e1cb08-f75a-44e9-9914-fd1124432ab3-kube-api-access-fvlt2\") pod \"e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz\" (UID: \"36e1cb08-f75a-44e9-9914-fd1124432ab3\") " pod="openshift-ingress/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz"
Jul 30 15:36:36.365248 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:36.365131    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36e1cb08-f75a-44e9-9914-fd1124432ab3-workload-certs\") pod \"e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz\" (UID: \"36e1cb08-f75a-44e9-9914-fd1124432ab3\") " pod="openshift-ingress/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz"
Jul 30 15:36:36.365248 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:36.365181    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36e1cb08-f75a-44e9-9914-fd1124432ab3-istiod-ca-cert\") pod \"e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz\" (UID: \"36e1cb08-f75a-44e9-9914-fd1124432ab3\") " pod="openshift-ingress/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz"
Jul 30 15:36:36.365248 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:36.365205    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36e1cb08-f75a-44e9-9914-fd1124432ab3-istio-data\") pod \"e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz\" (UID: \"36e1cb08-f75a-44e9-9914-fd1124432ab3\") " pod="openshift-ingress/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz"
Jul 30 15:36:36.365248 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:36.365236    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36e1cb08-f75a-44e9-9914-fd1124432ab3-workload-socket\") pod \"e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz\" (UID: \"36e1cb08-f75a-44e9-9914-fd1124432ab3\") " pod="openshift-ingress/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz"
Jul 30 15:36:36.365432 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:36.365376    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36e1cb08-f75a-44e9-9914-fd1124432ab3-credential-socket\") pod \"e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz\" (UID: \"36e1cb08-f75a-44e9-9914-fd1124432ab3\") " pod="openshift-ingress/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz"
Jul 30 15:36:36.365468 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:36.365444    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36e1cb08-f75a-44e9-9914-fd1124432ab3-istio-podinfo\") pod \"e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz\" (UID: \"36e1cb08-f75a-44e9-9914-fd1124432ab3\") " pod="openshift-ingress/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz"
Jul 30 15:36:36.365502 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:36.365470    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36e1cb08-f75a-44e9-9914-fd1124432ab3-istio-token\") pod \"e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz\" (UID: \"36e1cb08-f75a-44e9-9914-fd1124432ab3\") " pod="openshift-ingress/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz"
Jul 30 15:36:36.466624 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:36.466588    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36e1cb08-f75a-44e9-9914-fd1124432ab3-credential-socket\") pod \"e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz\" (UID: \"36e1cb08-f75a-44e9-9914-fd1124432ab3\") " pod="openshift-ingress/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz"
Jul 30 15:36:36.466785 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:36.466639    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36e1cb08-f75a-44e9-9914-fd1124432ab3-istio-podinfo\") pod \"e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz\" (UID: \"36e1cb08-f75a-44e9-9914-fd1124432ab3\") " pod="openshift-ingress/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz"
Jul 30 15:36:36.466785 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:36.466750    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36e1cb08-f75a-44e9-9914-fd1124432ab3-istio-token\") pod \"e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz\" (UID: \"36e1cb08-f75a-44e9-9914-fd1124432ab3\") " pod="openshift-ingress/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz"
Jul 30 15:36:36.466881 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:36.466796    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36e1cb08-f75a-44e9-9914-fd1124432ab3-istio-envoy\") pod \"e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz\" (UID: \"36e1cb08-f75a-44e9-9914-fd1124432ab3\") " pod="openshift-ingress/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz"
Jul 30 15:36:36.466881 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:36.466867    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvlt2\" (UniqueName: \"kubernetes.io/projected/36e1cb08-f75a-44e9-9914-fd1124432ab3-kube-api-access-fvlt2\") pod \"e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz\" (UID: \"36e1cb08-f75a-44e9-9914-fd1124432ab3\") " pod="openshift-ingress/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz"
Jul 30 15:36:36.466978 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:36.466910    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36e1cb08-f75a-44e9-9914-fd1124432ab3-workload-certs\") pod \"e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz\" (UID: \"36e1cb08-f75a-44e9-9914-fd1124432ab3\") " pod="openshift-ingress/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz"
Jul 30 15:36:36.466978 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:36.466947    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36e1cb08-f75a-44e9-9914-fd1124432ab3-istiod-ca-cert\") pod \"e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz\" (UID: \"36e1cb08-f75a-44e9-9914-fd1124432ab3\") " pod="openshift-ingress/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz"
Jul 30 15:36:36.467088 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:36.466976    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36e1cb08-f75a-44e9-9914-fd1124432ab3-istio-data\") pod \"e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz\" (UID: \"36e1cb08-f75a-44e9-9914-fd1124432ab3\") " pod="openshift-ingress/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz"
Jul 30 15:36:36.467088 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:36.466999    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36e1cb08-f75a-44e9-9914-fd1124432ab3-credential-socket\") pod \"e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz\" (UID: \"36e1cb08-f75a-44e9-9914-fd1124432ab3\") " pod="openshift-ingress/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz"
Jul 30 15:36:36.467239 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:36.467085    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36e1cb08-f75a-44e9-9914-fd1124432ab3-workload-socket\") pod \"e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz\" (UID: \"36e1cb08-f75a-44e9-9914-fd1124432ab3\") " pod="openshift-ingress/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz"
Jul 30 15:36:36.467446 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:36.467421    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36e1cb08-f75a-44e9-9914-fd1124432ab3-workload-socket\") pod \"e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz\" (UID: \"36e1cb08-f75a-44e9-9914-fd1124432ab3\") " pod="openshift-ingress/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz"
Jul 30 15:36:36.467600 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:36.467500    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36e1cb08-f75a-44e9-9914-fd1124432ab3-workload-certs\") pod \"e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz\" (UID: \"36e1cb08-f75a-44e9-9914-fd1124432ab3\") " pod="openshift-ingress/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz"
Jul 30 15:36:36.467666 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:36.467646    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36e1cb08-f75a-44e9-9914-fd1124432ab3-istio-data\") pod \"e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz\" (UID: \"36e1cb08-f75a-44e9-9914-fd1124432ab3\") " pod="openshift-ingress/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz"
Jul 30 15:36:36.467840 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:36.467817    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36e1cb08-f75a-44e9-9914-fd1124432ab3-istiod-ca-cert\") pod \"e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz\" (UID: \"36e1cb08-f75a-44e9-9914-fd1124432ab3\") " pod="openshift-ingress/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz"
Jul 30 15:36:36.469065 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:36.469042    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36e1cb08-f75a-44e9-9914-fd1124432ab3-istio-envoy\") pod \"e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz\" (UID: \"36e1cb08-f75a-44e9-9914-fd1124432ab3\") " pod="openshift-ingress/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz"
Jul 30 15:36:36.469265 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:36.469245    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36e1cb08-f75a-44e9-9914-fd1124432ab3-istio-podinfo\") pod \"e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz\" (UID: \"36e1cb08-f75a-44e9-9914-fd1124432ab3\") " pod="openshift-ingress/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz"
Jul 30 15:36:36.484030 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:36.483969    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36e1cb08-f75a-44e9-9914-fd1124432ab3-istio-token\") pod \"e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz\" (UID: \"36e1cb08-f75a-44e9-9914-fd1124432ab3\") " pod="openshift-ingress/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz"
Jul 30 15:36:36.484173 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:36.484078    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvlt2\" (UniqueName: \"kubernetes.io/projected/36e1cb08-f75a-44e9-9914-fd1124432ab3-kube-api-access-fvlt2\") pod \"e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz\" (UID: \"36e1cb08-f75a-44e9-9914-fd1124432ab3\") " pod="openshift-ingress/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz"
Jul 30 15:36:36.540904 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:36.540873    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz"
Jul 30 15:36:36.686814 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:36.686787    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz"]
Jul 30 15:36:36.688516 ip-10-0-133-117 kubenswrapper[2578]: W0730 15:36:36.688490    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36e1cb08_f75a_44e9_9914_fd1124432ab3.slice/crio-b6818d984cb64013e23325bf937eec80bfea552391756cec66a93b284a4c7e3c WatchSource:0}: Error finding container b6818d984cb64013e23325bf937eec80bfea552391756cec66a93b284a4c7e3c: Status 404 returned error can't find the container with id b6818d984cb64013e23325bf937eec80bfea552391756cec66a93b284a4c7e3c
Jul 30 15:36:36.690637 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:36.690602    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:36:36.690732 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:36.690680    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:36:36.690732 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:36.690723    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:36:37.506795 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:37.506758    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz" event={"ID":"36e1cb08-f75a-44e9-9914-fd1124432ab3","Type":"ContainerStarted","Data":"838b0a659dd4aaca98822ad64acdf55cfac6eff9e959fcfbc97b7821d37b42de"}
Jul 30 15:36:37.507182 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:37.506810    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz" event={"ID":"36e1cb08-f75a-44e9-9914-fd1124432ab3","Type":"ContainerStarted","Data":"b6818d984cb64013e23325bf937eec80bfea552391756cec66a93b284a4c7e3c"}
Jul 30 15:36:37.525743 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:37.525658    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz" podStartSLOduration=1.5256375530000001 podStartE2EDuration="1.525637553s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:36:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:36:37.52502424 +0000 UTC m=+2909.053962740" watchObservedRunningTime="2026-07-30 15:36:37.525637553 +0000 UTC m=+2909.054576028"
Jul 30 15:36:37.541044 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:37.541013    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz"
Jul 30 15:36:37.546297 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:37.546273    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz"
Jul 30 15:36:38.512394 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:38.512364    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz"
Jul 30 15:36:38.513480 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:38.513460    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz"
Jul 30 15:36:41.263077 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:41.263036    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-e2257c-78bf5bc898-57mhn"]
Jul 30 15:36:41.271156 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:41.271127    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-e2257c-78bf5bc898-57mhn"
Jul 30 15:36:41.273280 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:41.273255    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-e2257c\""
Jul 30 15:36:41.273399 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:41.273367    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-e2257c-dockercfg-djv97\""
Jul 30 15:36:41.278470 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:41.278441    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-e2257c-78bf5bc898-57mhn"]
Jul 30 15:36:41.417589 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:41.417553    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1c7f6ab-ab23-4144-ba0b-0bb517a75bce-plugins-config-volume\") pod \"payload-processing-e2e-api-b-e2257c-78bf5bc898-57mhn\" (UID: \"d1c7f6ab-ab23-4144-ba0b-0bb517a75bce\") " pod="openshift-ingress/payload-processing-e2e-api-b-e2257c-78bf5bc898-57mhn"
Jul 30 15:36:41.417769 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:41.417605    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmvw5\" (UniqueName: \"kubernetes.io/projected/d1c7f6ab-ab23-4144-ba0b-0bb517a75bce-kube-api-access-fmvw5\") pod \"payload-processing-e2e-api-b-e2257c-78bf5bc898-57mhn\" (UID: \"d1c7f6ab-ab23-4144-ba0b-0bb517a75bce\") " pod="openshift-ingress/payload-processing-e2e-api-b-e2257c-78bf5bc898-57mhn"
Jul 30 15:36:41.518651 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:41.518567    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1c7f6ab-ab23-4144-ba0b-0bb517a75bce-plugins-config-volume\") pod \"payload-processing-e2e-api-b-e2257c-78bf5bc898-57mhn\" (UID: \"d1c7f6ab-ab23-4144-ba0b-0bb517a75bce\") " pod="openshift-ingress/payload-processing-e2e-api-b-e2257c-78bf5bc898-57mhn"
Jul 30 15:36:41.518651 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:41.518614    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmvw5\" (UniqueName: \"kubernetes.io/projected/d1c7f6ab-ab23-4144-ba0b-0bb517a75bce-kube-api-access-fmvw5\") pod \"payload-processing-e2e-api-b-e2257c-78bf5bc898-57mhn\" (UID: \"d1c7f6ab-ab23-4144-ba0b-0bb517a75bce\") " pod="openshift-ingress/payload-processing-e2e-api-b-e2257c-78bf5bc898-57mhn"
Jul 30 15:36:41.519282 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:41.519263    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1c7f6ab-ab23-4144-ba0b-0bb517a75bce-plugins-config-volume\") pod \"payload-processing-e2e-api-b-e2257c-78bf5bc898-57mhn\" (UID: \"d1c7f6ab-ab23-4144-ba0b-0bb517a75bce\") " pod="openshift-ingress/payload-processing-e2e-api-b-e2257c-78bf5bc898-57mhn"
Jul 30 15:36:41.530228 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:41.530201    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmvw5\" (UniqueName: \"kubernetes.io/projected/d1c7f6ab-ab23-4144-ba0b-0bb517a75bce-kube-api-access-fmvw5\") pod \"payload-processing-e2e-api-b-e2257c-78bf5bc898-57mhn\" (UID: \"d1c7f6ab-ab23-4144-ba0b-0bb517a75bce\") " pod="openshift-ingress/payload-processing-e2e-api-b-e2257c-78bf5bc898-57mhn"
Jul 30 15:36:41.584380 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:41.584346    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-e2257c-78bf5bc898-57mhn"
Jul 30 15:36:41.727116 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:41.727090    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-e2257c-78bf5bc898-57mhn"]
Jul 30 15:36:41.728950 ip-10-0-133-117 kubenswrapper[2578]: W0730 15:36:41.728919    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1c7f6ab_ab23_4144_ba0b_0bb517a75bce.slice/crio-429d2f8b0ac2c991ae97900c0d570913548e9326a4290385995271bcc941534d WatchSource:0}: Error finding container 429d2f8b0ac2c991ae97900c0d570913548e9326a4290385995271bcc941534d: Status 404 returned error can't find the container with id 429d2f8b0ac2c991ae97900c0d570913548e9326a4290385995271bcc941534d
Jul 30 15:36:42.533998 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:42.533960    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-e2257c-78bf5bc898-57mhn" event={"ID":"d1c7f6ab-ab23-4144-ba0b-0bb517a75bce","Type":"ContainerStarted","Data":"ae5b3359557b004c06ae258c0f9d91a096761932a77a4ef5ec3bcf61d6c58c80"}
Jul 30 15:36:42.534466 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:42.534015    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-e2257c-78bf5bc898-57mhn" event={"ID":"d1c7f6ab-ab23-4144-ba0b-0bb517a75bce","Type":"ContainerStarted","Data":"429d2f8b0ac2c991ae97900c0d570913548e9326a4290385995271bcc941534d"}
Jul 30 15:36:42.534466 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:42.534102    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-e2257c-78bf5bc898-57mhn"
Jul 30 15:36:42.550134 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:42.550083    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-e2257c-78bf5bc898-57mhn" podStartSLOduration=1.550068999 podStartE2EDuration="1.550068999s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:36:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:36:42.547708248 +0000 UTC m=+2914.076646725" watchObservedRunningTime="2026-07-30 15:36:42.550068999 +0000 UTC m=+2914.079007473"
Jul 30 15:36:53.541373 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:53.541337    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-e2257c-78bf5bc898-57mhn"
Jul 30 15:36:54.554348 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:54.554298    2578 patch_prober.go:28] interesting pod/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 30 15:36:54.554753 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:54.554373    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz" podUID="36e1cb08-f75a-44e9-9914-fd1124432ab3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 30 15:36:55.603440 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:55.603409    2578 generic.go:356] "Generic (PLEG): container finished" podID="36e1cb08-f75a-44e9-9914-fd1124432ab3" containerID="838b0a659dd4aaca98822ad64acdf55cfac6eff9e959fcfbc97b7821d37b42de" exitCode=0
Jul 30 15:36:55.603852 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:55.603484    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz" event={"ID":"36e1cb08-f75a-44e9-9914-fd1124432ab3","Type":"ContainerDied","Data":"838b0a659dd4aaca98822ad64acdf55cfac6eff9e959fcfbc97b7821d37b42de"}
Jul 30 15:36:55.603916 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:55.603900    2578 scope.go:117] "RemoveContainer" containerID="838b0a659dd4aaca98822ad64acdf55cfac6eff9e959fcfbc97b7821d37b42de"
Jul 30 15:36:55.604897 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:55.604870    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:36:55.604954 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:55.604938    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:36:55.604994 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:55.604969    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:36:56.609721 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:56.609677    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz" event={"ID":"36e1cb08-f75a-44e9-9914-fd1124432ab3","Type":"ContainerStarted","Data":"d40e7b9e2c1e0377287e96c501c94a8b1e7d9bfe64c5d4895e85f802fe7fa519"}
Jul 30 15:36:57.541556 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:57.541513    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz"
Jul 30 15:36:57.541940 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:57.541914    2578 patch_prober.go:28] interesting pod/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Jul 30 15:36:57.542165 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:57.542112    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz" podUID="36e1cb08-f75a-44e9-9914-fd1124432ab3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Jul 30 15:36:58.541939 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:58.541891    2578 patch_prober.go:28] interesting pod/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Jul 30 15:36:58.542465 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:58.541963    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz" podUID="36e1cb08-f75a-44e9-9914-fd1124432ab3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Jul 30 15:36:59.542101 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:59.542053    2578 patch_prober.go:28] interesting pod/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Jul 30 15:36:59.542631 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:36:59.542125    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz" podUID="36e1cb08-f75a-44e9-9914-fd1124432ab3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Jul 30 15:37:00.544609 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:00.544579    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz"
Jul 30 15:37:00.545042 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:00.544829    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz"
Jul 30 15:37:00.545535 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:00.545518    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz"
Jul 30 15:37:23.582667 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:23.582623    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-a1c473-7ff64bc75d-t7sc9"]
Jul 30 15:37:23.583134 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:23.582885    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-a1c473-7ff64bc75d-t7sc9" podUID="2305a153-2af1-417f-aecb-6f6ef6988e4e" containerName="payload-processing" containerID="cri-o://9b4b1b7071a7b1b2946cba7916523370dd18cdfe0dbccab2d147d72009e01aee" gracePeriod=30
Jul 30 15:37:23.588425 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:23.588396    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-a1c473-74659688b8-gr8sp"]
Jul 30 15:37:23.588681 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:23.588631    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a1c473-74659688b8-gr8sp" podUID="49668e3d-600b-4e5c-bdc2-6231711f3b49" containerName="payload-pre-processing" containerID="cri-o://2cab01c797fd2f5867bf20c9b38a6f2b54e2a12870843e1f443ac9ce22c2602a" gracePeriod=30
Jul 30 15:37:23.744177 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:23.744050    2578 generic.go:356] "Generic (PLEG): container finished" podID="2305a153-2af1-417f-aecb-6f6ef6988e4e" containerID="9b4b1b7071a7b1b2946cba7916523370dd18cdfe0dbccab2d147d72009e01aee" exitCode=0
Jul 30 15:37:23.744177 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:23.744131    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-a1c473-7ff64bc75d-t7sc9" event={"ID":"2305a153-2af1-417f-aecb-6f6ef6988e4e","Type":"ContainerDied","Data":"9b4b1b7071a7b1b2946cba7916523370dd18cdfe0dbccab2d147d72009e01aee"}
Jul 30 15:37:23.745995 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:23.745969    2578 generic.go:356] "Generic (PLEG): container finished" podID="49668e3d-600b-4e5c-bdc2-6231711f3b49" containerID="2cab01c797fd2f5867bf20c9b38a6f2b54e2a12870843e1f443ac9ce22c2602a" exitCode=0
Jul 30 15:37:23.746159 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:23.746047    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a1c473-74659688b8-gr8sp" event={"ID":"49668e3d-600b-4e5c-bdc2-6231711f3b49","Type":"ContainerDied","Data":"2cab01c797fd2f5867bf20c9b38a6f2b54e2a12870843e1f443ac9ce22c2602a"}
Jul 30 15:37:23.845100 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:23.845037    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-a1c473-7ff64bc75d-t7sc9"
Jul 30 15:37:23.878137 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:23.878112    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a1c473-74659688b8-gr8sp"
Jul 30 15:37:24.028495 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:24.028461    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2305a153-2af1-417f-aecb-6f6ef6988e4e-plugins-config-volume\") pod \"2305a153-2af1-417f-aecb-6f6ef6988e4e\" (UID: \"2305a153-2af1-417f-aecb-6f6ef6988e4e\") "
Jul 30 15:37:24.028696 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:24.028523    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49668e3d-600b-4e5c-bdc2-6231711f3b49-plugins-config-volume\") pod \"49668e3d-600b-4e5c-bdc2-6231711f3b49\" (UID: \"49668e3d-600b-4e5c-bdc2-6231711f3b49\") "
Jul 30 15:37:24.028696 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:24.028577    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-595jv\" (UniqueName: \"kubernetes.io/projected/2305a153-2af1-417f-aecb-6f6ef6988e4e-kube-api-access-595jv\") pod \"2305a153-2af1-417f-aecb-6f6ef6988e4e\" (UID: \"2305a153-2af1-417f-aecb-6f6ef6988e4e\") "
Jul 30 15:37:24.028696 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:24.028664    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fgv5s\" (UniqueName: \"kubernetes.io/projected/49668e3d-600b-4e5c-bdc2-6231711f3b49-kube-api-access-fgv5s\") pod \"49668e3d-600b-4e5c-bdc2-6231711f3b49\" (UID: \"49668e3d-600b-4e5c-bdc2-6231711f3b49\") "
Jul 30 15:37:24.028942 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:24.028917    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2305a153-2af1-417f-aecb-6f6ef6988e4e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2305a153-2af1-417f-aecb-6f6ef6988e4e" (UID: "2305a153-2af1-417f-aecb-6f6ef6988e4e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:37:24.029003 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:24.028916    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49668e3d-600b-4e5c-bdc2-6231711f3b49-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "49668e3d-600b-4e5c-bdc2-6231711f3b49" (UID: "49668e3d-600b-4e5c-bdc2-6231711f3b49"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:37:24.030676 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:24.030650    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49668e3d-600b-4e5c-bdc2-6231711f3b49-kube-api-access-fgv5s" (OuterVolumeSpecName: "kube-api-access-fgv5s") pod "49668e3d-600b-4e5c-bdc2-6231711f3b49" (UID: "49668e3d-600b-4e5c-bdc2-6231711f3b49"). InnerVolumeSpecName "kube-api-access-fgv5s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:37:24.030754 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:24.030735    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2305a153-2af1-417f-aecb-6f6ef6988e4e-kube-api-access-595jv" (OuterVolumeSpecName: "kube-api-access-595jv") pod "2305a153-2af1-417f-aecb-6f6ef6988e4e" (UID: "2305a153-2af1-417f-aecb-6f6ef6988e4e"). InnerVolumeSpecName "kube-api-access-595jv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:37:24.130030 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:24.129934    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fgv5s\" (UniqueName: \"kubernetes.io/projected/49668e3d-600b-4e5c-bdc2-6231711f3b49-kube-api-access-fgv5s\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:37:24.130030 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:24.129977    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2305a153-2af1-417f-aecb-6f6ef6988e4e-plugins-config-volume\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:37:24.130030 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:24.129987    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49668e3d-600b-4e5c-bdc2-6231711f3b49-plugins-config-volume\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:37:24.130030 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:24.129996    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-595jv\" (UniqueName: \"kubernetes.io/projected/2305a153-2af1-417f-aecb-6f6ef6988e4e-kube-api-access-595jv\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:37:24.752555 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:24.752521    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-a1c473-7ff64bc75d-t7sc9"
Jul 30 15:37:24.753009 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:24.752524    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-a1c473-7ff64bc75d-t7sc9" event={"ID":"2305a153-2af1-417f-aecb-6f6ef6988e4e","Type":"ContainerDied","Data":"d4614fa1a3a65e9d387fdbe517a2b04f278e8a84631f6aca7bbc94fa92c364e1"}
Jul 30 15:37:24.753009 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:24.752643    2578 scope.go:117] "RemoveContainer" containerID="9b4b1b7071a7b1b2946cba7916523370dd18cdfe0dbccab2d147d72009e01aee"
Jul 30 15:37:24.753973 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:24.753931    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a1c473-74659688b8-gr8sp"
Jul 30 15:37:24.753973 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:24.753945    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a1c473-74659688b8-gr8sp" event={"ID":"49668e3d-600b-4e5c-bdc2-6231711f3b49","Type":"ContainerDied","Data":"069ab4238451f90ef567998b1341e62859eea590bc23e53143185ab366d7f7e6"}
Jul 30 15:37:24.761384 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:24.761365    2578 scope.go:117] "RemoveContainer" containerID="2cab01c797fd2f5867bf20c9b38a6f2b54e2a12870843e1f443ac9ce22c2602a"
Jul 30 15:37:24.774100 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:24.774079    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-a1c473-7ff64bc75d-t7sc9"]
Jul 30 15:37:24.778229 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:24.778207    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-a1c473-7ff64bc75d-t7sc9"]
Jul 30 15:37:24.787399 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:24.787378    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-a1c473-74659688b8-gr8sp"]
Jul 30 15:37:24.791217 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:24.791196    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-a1c473-74659688b8-gr8sp"]
Jul 30 15:37:24.978180 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:24.978091    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2305a153-2af1-417f-aecb-6f6ef6988e4e" path="/var/lib/kubelet/pods/2305a153-2af1-417f-aecb-6f6ef6988e4e/volumes"
Jul 30 15:37:24.978482 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:24.978469    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49668e3d-600b-4e5c-bdc2-6231711f3b49" path="/var/lib/kubelet/pods/49668e3d-600b-4e5c-bdc2-6231711f3b49/volumes"
Jul 30 15:37:30.671816 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:30.671780    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc"]
Jul 30 15:37:30.672262 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:30.672033    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc" podUID="c595b020-052f-4b1c-8281-ccee6116b418" containerName="istio-proxy" containerID="cri-o://43fb8ce6074abfe78cd688cef816d1cda916c6ebed9dab58b846d0b625de40b0" gracePeriod=30
Jul 30 15:37:35.809620 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:35.809580    2578 generic.go:356] "Generic (PLEG): container finished" podID="c595b020-052f-4b1c-8281-ccee6116b418" containerID="43fb8ce6074abfe78cd688cef816d1cda916c6ebed9dab58b846d0b625de40b0" exitCode=0
Jul 30 15:37:35.810068 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:35.809638    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc" event={"ID":"c595b020-052f-4b1c-8281-ccee6116b418","Type":"ContainerDied","Data":"43fb8ce6074abfe78cd688cef816d1cda916c6ebed9dab58b846d0b625de40b0"}
Jul 30 15:37:35.810068 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:35.809690    2578 scope.go:117] "RemoveContainer" containerID="281576b0a017fc6f71d903777bd127d5eac370d397e7db65b2d0649a89825ceb"
Jul 30 15:37:35.920065 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:35.920042    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc"
Jul 30 15:37:35.942059 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:35.941990    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c595b020-052f-4b1c-8281-ccee6116b418-istio-podinfo\") pod \"c595b020-052f-4b1c-8281-ccee6116b418\" (UID: \"c595b020-052f-4b1c-8281-ccee6116b418\") "
Jul 30 15:37:35.942310 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:35.942059    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c595b020-052f-4b1c-8281-ccee6116b418-istio-envoy\") pod \"c595b020-052f-4b1c-8281-ccee6116b418\" (UID: \"c595b020-052f-4b1c-8281-ccee6116b418\") "
Jul 30 15:37:35.942310 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:35.942100    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c595b020-052f-4b1c-8281-ccee6116b418-workload-certs\") pod \"c595b020-052f-4b1c-8281-ccee6116b418\" (UID: \"c595b020-052f-4b1c-8281-ccee6116b418\") "
Jul 30 15:37:35.942310 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:35.942190    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c595b020-052f-4b1c-8281-ccee6116b418-istiod-ca-cert\") pod \"c595b020-052f-4b1c-8281-ccee6116b418\" (UID: \"c595b020-052f-4b1c-8281-ccee6116b418\") "
Jul 30 15:37:35.942310 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:35.942252    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c595b020-052f-4b1c-8281-ccee6116b418-istio-token\") pod \"c595b020-052f-4b1c-8281-ccee6116b418\" (UID: \"c595b020-052f-4b1c-8281-ccee6116b418\") "
Jul 30 15:37:35.942310 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:35.942285    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zj6tk\" (UniqueName: \"kubernetes.io/projected/c595b020-052f-4b1c-8281-ccee6116b418-kube-api-access-zj6tk\") pod \"c595b020-052f-4b1c-8281-ccee6116b418\" (UID: \"c595b020-052f-4b1c-8281-ccee6116b418\") "
Jul 30 15:37:35.942604 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:35.942313    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c595b020-052f-4b1c-8281-ccee6116b418-credential-socket\") pod \"c595b020-052f-4b1c-8281-ccee6116b418\" (UID: \"c595b020-052f-4b1c-8281-ccee6116b418\") "
Jul 30 15:37:35.942604 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:35.942339    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c595b020-052f-4b1c-8281-ccee6116b418-workload-socket\") pod \"c595b020-052f-4b1c-8281-ccee6116b418\" (UID: \"c595b020-052f-4b1c-8281-ccee6116b418\") "
Jul 30 15:37:35.942604 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:35.942367    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c595b020-052f-4b1c-8281-ccee6116b418-istio-data\") pod \"c595b020-052f-4b1c-8281-ccee6116b418\" (UID: \"c595b020-052f-4b1c-8281-ccee6116b418\") "
Jul 30 15:37:35.943620 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:35.943473    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c595b020-052f-4b1c-8281-ccee6116b418-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c595b020-052f-4b1c-8281-ccee6116b418" (UID: "c595b020-052f-4b1c-8281-ccee6116b418"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:37:35.943620 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:35.943537    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c595b020-052f-4b1c-8281-ccee6116b418-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c595b020-052f-4b1c-8281-ccee6116b418" (UID: "c595b020-052f-4b1c-8281-ccee6116b418"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:37:35.943620 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:35.943596    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c595b020-052f-4b1c-8281-ccee6116b418-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c595b020-052f-4b1c-8281-ccee6116b418" (UID: "c595b020-052f-4b1c-8281-ccee6116b418"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:37:35.943843 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:35.943674    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c595b020-052f-4b1c-8281-ccee6116b418-istio-data" (OuterVolumeSpecName: "istio-data") pod "c595b020-052f-4b1c-8281-ccee6116b418" (UID: "c595b020-052f-4b1c-8281-ccee6116b418"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:37:35.943843 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:35.943719    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c595b020-052f-4b1c-8281-ccee6116b418-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c595b020-052f-4b1c-8281-ccee6116b418" (UID: "c595b020-052f-4b1c-8281-ccee6116b418"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:37:35.944560 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:35.944536    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c595b020-052f-4b1c-8281-ccee6116b418-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c595b020-052f-4b1c-8281-ccee6116b418" (UID: "c595b020-052f-4b1c-8281-ccee6116b418"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 15:37:35.945186 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:35.945124    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c595b020-052f-4b1c-8281-ccee6116b418-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c595b020-052f-4b1c-8281-ccee6116b418" (UID: "c595b020-052f-4b1c-8281-ccee6116b418"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:37:35.945265 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:35.945226    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c595b020-052f-4b1c-8281-ccee6116b418-istio-token" (OuterVolumeSpecName: "istio-token") pod "c595b020-052f-4b1c-8281-ccee6116b418" (UID: "c595b020-052f-4b1c-8281-ccee6116b418"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:37:35.945936 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:35.945911    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c595b020-052f-4b1c-8281-ccee6116b418-kube-api-access-zj6tk" (OuterVolumeSpecName: "kube-api-access-zj6tk") pod "c595b020-052f-4b1c-8281-ccee6116b418" (UID: "c595b020-052f-4b1c-8281-ccee6116b418"). InnerVolumeSpecName "kube-api-access-zj6tk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:37:36.043675 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:36.043646    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c595b020-052f-4b1c-8281-ccee6116b418-workload-certs\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:37:36.043675 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:36.043671    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c595b020-052f-4b1c-8281-ccee6116b418-istiod-ca-cert\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:37:36.043675 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:36.043680    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c595b020-052f-4b1c-8281-ccee6116b418-istio-token\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:37:36.043897 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:36.043692    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zj6tk\" (UniqueName: \"kubernetes.io/projected/c595b020-052f-4b1c-8281-ccee6116b418-kube-api-access-zj6tk\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:37:36.043897 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:36.043701    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c595b020-052f-4b1c-8281-ccee6116b418-credential-socket\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:37:36.043897 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:36.043710    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c595b020-052f-4b1c-8281-ccee6116b418-workload-socket\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:37:36.043897 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:36.043719    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c595b020-052f-4b1c-8281-ccee6116b418-istio-data\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:37:36.043897 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:36.043726    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c595b020-052f-4b1c-8281-ccee6116b418-istio-podinfo\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:37:36.043897 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:36.043734    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c595b020-052f-4b1c-8281-ccee6116b418-istio-envoy\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:37:36.821027 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:36.820994    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc"
Jul 30 15:37:36.821464 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:36.821034    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc" event={"ID":"c595b020-052f-4b1c-8281-ccee6116b418","Type":"ContainerDied","Data":"ac1cb73b7b228e7e1faccb81f257f15c5d362d32e71956b32d863be50d81051a"}
Jul 30 15:37:36.821464 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:36.821112    2578 scope.go:117] "RemoveContainer" containerID="43fb8ce6074abfe78cd688cef816d1cda916c6ebed9dab58b846d0b625de40b0"
Jul 30 15:37:36.840963 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:36.840935    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc"]
Jul 30 15:37:36.844542 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:36.844520    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-a1c473-openshift-default-5ff475db94-w8cnc"]
Jul 30 15:37:36.978385 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:36.978353    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c595b020-052f-4b1c-8281-ccee6116b418" path="/var/lib/kubelet/pods/c595b020-052f-4b1c-8281-ccee6116b418/volumes"
Jul 30 15:37:46.600245 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:46.600209    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-e2257c-78bf5bc898-57mhn"]
Jul 30 15:37:46.600800 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:46.600460    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-e2257c-78bf5bc898-57mhn" podUID="d1c7f6ab-ab23-4144-ba0b-0bb517a75bce" containerName="payload-processing" containerID="cri-o://ae5b3359557b004c06ae258c0f9d91a096761932a77a4ef5ec3bcf61d6c58c80" gracePeriod=30
Jul 30 15:37:46.866324 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:46.866299    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-e2257c-78bf5bc898-57mhn"
Jul 30 15:37:46.870924 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:46.870899    2578 generic.go:356] "Generic (PLEG): container finished" podID="d1c7f6ab-ab23-4144-ba0b-0bb517a75bce" containerID="ae5b3359557b004c06ae258c0f9d91a096761932a77a4ef5ec3bcf61d6c58c80" exitCode=0
Jul 30 15:37:46.871058 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:46.870937    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-e2257c-78bf5bc898-57mhn"
Jul 30 15:37:46.871058 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:46.870983    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-e2257c-78bf5bc898-57mhn" event={"ID":"d1c7f6ab-ab23-4144-ba0b-0bb517a75bce","Type":"ContainerDied","Data":"ae5b3359557b004c06ae258c0f9d91a096761932a77a4ef5ec3bcf61d6c58c80"}
Jul 30 15:37:46.871058 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:46.871048    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-e2257c-78bf5bc898-57mhn" event={"ID":"d1c7f6ab-ab23-4144-ba0b-0bb517a75bce","Type":"ContainerDied","Data":"429d2f8b0ac2c991ae97900c0d570913548e9326a4290385995271bcc941534d"}
Jul 30 15:37:46.871229 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:46.871072    2578 scope.go:117] "RemoveContainer" containerID="ae5b3359557b004c06ae258c0f9d91a096761932a77a4ef5ec3bcf61d6c58c80"
Jul 30 15:37:46.878951 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:46.878937    2578 scope.go:117] "RemoveContainer" containerID="ae5b3359557b004c06ae258c0f9d91a096761932a77a4ef5ec3bcf61d6c58c80"
Jul 30 15:37:46.879233 ip-10-0-133-117 kubenswrapper[2578]: E0730 15:37:46.879217    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae5b3359557b004c06ae258c0f9d91a096761932a77a4ef5ec3bcf61d6c58c80\": container with ID starting with ae5b3359557b004c06ae258c0f9d91a096761932a77a4ef5ec3bcf61d6c58c80 not found: ID does not exist" containerID="ae5b3359557b004c06ae258c0f9d91a096761932a77a4ef5ec3bcf61d6c58c80"
Jul 30 15:37:46.879294 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:46.879240    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae5b3359557b004c06ae258c0f9d91a096761932a77a4ef5ec3bcf61d6c58c80"} err="failed to get container status \"ae5b3359557b004c06ae258c0f9d91a096761932a77a4ef5ec3bcf61d6c58c80\": rpc error: code = NotFound desc = could not find container \"ae5b3359557b004c06ae258c0f9d91a096761932a77a4ef5ec3bcf61d6c58c80\": container with ID starting with ae5b3359557b004c06ae258c0f9d91a096761932a77a4ef5ec3bcf61d6c58c80 not found: ID does not exist"
Jul 30 15:37:46.950127 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:46.950093    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fmvw5\" (UniqueName: \"kubernetes.io/projected/d1c7f6ab-ab23-4144-ba0b-0bb517a75bce-kube-api-access-fmvw5\") pod \"d1c7f6ab-ab23-4144-ba0b-0bb517a75bce\" (UID: \"d1c7f6ab-ab23-4144-ba0b-0bb517a75bce\") "
Jul 30 15:37:46.950294 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:46.950156    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1c7f6ab-ab23-4144-ba0b-0bb517a75bce-plugins-config-volume\") pod \"d1c7f6ab-ab23-4144-ba0b-0bb517a75bce\" (UID: \"d1c7f6ab-ab23-4144-ba0b-0bb517a75bce\") "
Jul 30 15:37:46.950549 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:46.950528    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1c7f6ab-ab23-4144-ba0b-0bb517a75bce-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d1c7f6ab-ab23-4144-ba0b-0bb517a75bce" (UID: "d1c7f6ab-ab23-4144-ba0b-0bb517a75bce"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:37:46.952295 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:46.952267    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1c7f6ab-ab23-4144-ba0b-0bb517a75bce-kube-api-access-fmvw5" (OuterVolumeSpecName: "kube-api-access-fmvw5") pod "d1c7f6ab-ab23-4144-ba0b-0bb517a75bce" (UID: "d1c7f6ab-ab23-4144-ba0b-0bb517a75bce"). InnerVolumeSpecName "kube-api-access-fmvw5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:37:47.051217 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:47.051190    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1c7f6ab-ab23-4144-ba0b-0bb517a75bce-plugins-config-volume\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:37:47.051217 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:47.051214    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fmvw5\" (UniqueName: \"kubernetes.io/projected/d1c7f6ab-ab23-4144-ba0b-0bb517a75bce-kube-api-access-fmvw5\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:37:47.192879 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:47.192805    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-e2257c-78bf5bc898-57mhn"]
Jul 30 15:37:47.199465 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:47.199433    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-e2257c-78bf5bc898-57mhn"]
Jul 30 15:37:48.984900 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:48.984867    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1c7f6ab-ab23-4144-ba0b-0bb517a75bce" path="/var/lib/kubelet/pods/d1c7f6ab-ab23-4144-ba0b-0bb517a75bce/volumes"
Jul 30 15:37:53.567113 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:53.567081    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz"]
Jul 30 15:37:53.567608 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:53.567314    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz" podUID="36e1cb08-f75a-44e9-9914-fd1124432ab3" containerName="istio-proxy" containerID="cri-o://d40e7b9e2c1e0377287e96c501c94a8b1e7d9bfe64c5d4895e85f802fe7fa519" gracePeriod=30
Jul 30 15:37:54.613394 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:54.613358    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl"]
Jul 30 15:37:54.613880 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:54.613718    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1c7f6ab-ab23-4144-ba0b-0bb517a75bce" containerName="payload-processing"
Jul 30 15:37:54.613880 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:54.613730    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1c7f6ab-ab23-4144-ba0b-0bb517a75bce" containerName="payload-processing"
Jul 30 15:37:54.613880 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:54.613740    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c595b020-052f-4b1c-8281-ccee6116b418" containerName="istio-proxy"
Jul 30 15:37:54.613880 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:54.613745    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c595b020-052f-4b1c-8281-ccee6116b418" containerName="istio-proxy"
Jul 30 15:37:54.613880 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:54.613762    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2305a153-2af1-417f-aecb-6f6ef6988e4e" containerName="payload-processing"
Jul 30 15:37:54.613880 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:54.613768    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2305a153-2af1-417f-aecb-6f6ef6988e4e" containerName="payload-processing"
Jul 30 15:37:54.613880 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:54.613776    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49668e3d-600b-4e5c-bdc2-6231711f3b49" containerName="payload-pre-processing"
Jul 30 15:37:54.613880 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:54.613782    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="49668e3d-600b-4e5c-bdc2-6231711f3b49" containerName="payload-pre-processing"
Jul 30 15:37:54.613880 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:54.613853    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1c7f6ab-ab23-4144-ba0b-0bb517a75bce" containerName="payload-processing"
Jul 30 15:37:54.613880 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:54.613860    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c595b020-052f-4b1c-8281-ccee6116b418" containerName="istio-proxy"
Jul 30 15:37:54.613880 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:54.613866    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="49668e3d-600b-4e5c-bdc2-6231711f3b49" containerName="payload-pre-processing"
Jul 30 15:37:54.613880 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:54.613873    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="2305a153-2af1-417f-aecb-6f6ef6988e4e" containerName="payload-processing"
Jul 30 15:37:54.616844 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:54.616827    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl"
Jul 30 15:37:54.618802 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:54.618779    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-845a26-openshift-default-dockercfg-vzwqr\""
Jul 30 15:37:54.630223 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:54.630197    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl"]
Jul 30 15:37:54.726580 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:54.726547    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea2aceba-fd54-43a6-8707-05f92f63ab75-istio-token\") pod \"e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl\" (UID: \"ea2aceba-fd54-43a6-8707-05f92f63ab75\") " pod="openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl"
Jul 30 15:37:54.726755 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:54.726593    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea2aceba-fd54-43a6-8707-05f92f63ab75-istio-envoy\") pod \"e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl\" (UID: \"ea2aceba-fd54-43a6-8707-05f92f63ab75\") " pod="openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl"
Jul 30 15:37:54.726755 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:54.726654    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fn9q\" (UniqueName: \"kubernetes.io/projected/ea2aceba-fd54-43a6-8707-05f92f63ab75-kube-api-access-5fn9q\") pod \"e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl\" (UID: \"ea2aceba-fd54-43a6-8707-05f92f63ab75\") " pod="openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl"
Jul 30 15:37:54.726755 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:54.726688    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea2aceba-fd54-43a6-8707-05f92f63ab75-workload-socket\") pod \"e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl\" (UID: \"ea2aceba-fd54-43a6-8707-05f92f63ab75\") " pod="openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl"
Jul 30 15:37:54.726755 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:54.726721    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea2aceba-fd54-43a6-8707-05f92f63ab75-istiod-ca-cert\") pod \"e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl\" (UID: \"ea2aceba-fd54-43a6-8707-05f92f63ab75\") " pod="openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl"
Jul 30 15:37:54.726755 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:54.726747    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea2aceba-fd54-43a6-8707-05f92f63ab75-istio-podinfo\") pod \"e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl\" (UID: \"ea2aceba-fd54-43a6-8707-05f92f63ab75\") " pod="openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl"
Jul 30 15:37:54.726985 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:54.726795    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea2aceba-fd54-43a6-8707-05f92f63ab75-workload-certs\") pod \"e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl\" (UID: \"ea2aceba-fd54-43a6-8707-05f92f63ab75\") " pod="openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl"
Jul 30 15:37:54.726985 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:54.726825    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea2aceba-fd54-43a6-8707-05f92f63ab75-istio-data\") pod \"e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl\" (UID: \"ea2aceba-fd54-43a6-8707-05f92f63ab75\") " pod="openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl"
Jul 30 15:37:54.726985 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:54.726883    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea2aceba-fd54-43a6-8707-05f92f63ab75-credential-socket\") pod \"e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl\" (UID: \"ea2aceba-fd54-43a6-8707-05f92f63ab75\") " pod="openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl"
Jul 30 15:37:54.827617 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:54.827587    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea2aceba-fd54-43a6-8707-05f92f63ab75-istio-data\") pod \"e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl\" (UID: \"ea2aceba-fd54-43a6-8707-05f92f63ab75\") " pod="openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl"
Jul 30 15:37:54.827789 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:54.827633    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea2aceba-fd54-43a6-8707-05f92f63ab75-credential-socket\") pod \"e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl\" (UID: \"ea2aceba-fd54-43a6-8707-05f92f63ab75\") " pod="openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl"
Jul 30 15:37:54.827789 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:54.827679    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea2aceba-fd54-43a6-8707-05f92f63ab75-istio-token\") pod \"e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl\" (UID: \"ea2aceba-fd54-43a6-8707-05f92f63ab75\") " pod="openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl"
Jul 30 15:37:54.827789 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:54.827711    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea2aceba-fd54-43a6-8707-05f92f63ab75-istio-envoy\") pod \"e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl\" (UID: \"ea2aceba-fd54-43a6-8707-05f92f63ab75\") " pod="openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl"
Jul 30 15:37:54.827789 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:54.827737    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fn9q\" (UniqueName: \"kubernetes.io/projected/ea2aceba-fd54-43a6-8707-05f92f63ab75-kube-api-access-5fn9q\") pod \"e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl\" (UID: \"ea2aceba-fd54-43a6-8707-05f92f63ab75\") " pod="openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl"
Jul 30 15:37:54.828026 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:54.827884    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea2aceba-fd54-43a6-8707-05f92f63ab75-workload-socket\") pod \"e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl\" (UID: \"ea2aceba-fd54-43a6-8707-05f92f63ab75\") " pod="openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl"
Jul 30 15:37:54.828026 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:54.827939    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea2aceba-fd54-43a6-8707-05f92f63ab75-istiod-ca-cert\") pod \"e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl\" (UID: \"ea2aceba-fd54-43a6-8707-05f92f63ab75\") " pod="openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl"
Jul 30 15:37:54.828026 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:54.827976    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea2aceba-fd54-43a6-8707-05f92f63ab75-istio-podinfo\") pod \"e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl\" (UID: \"ea2aceba-fd54-43a6-8707-05f92f63ab75\") " pod="openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl"
Jul 30 15:37:54.828026 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:54.828020    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea2aceba-fd54-43a6-8707-05f92f63ab75-workload-certs\") pod \"e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl\" (UID: \"ea2aceba-fd54-43a6-8707-05f92f63ab75\") " pod="openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl"
Jul 30 15:37:54.828283 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:54.828017    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea2aceba-fd54-43a6-8707-05f92f63ab75-istio-data\") pod \"e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl\" (UID: \"ea2aceba-fd54-43a6-8707-05f92f63ab75\") " pod="openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl"
Jul 30 15:37:54.828283 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:54.828087    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea2aceba-fd54-43a6-8707-05f92f63ab75-credential-socket\") pod \"e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl\" (UID: \"ea2aceba-fd54-43a6-8707-05f92f63ab75\") " pod="openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl"
Jul 30 15:37:54.828283 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:54.828241    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea2aceba-fd54-43a6-8707-05f92f63ab75-workload-socket\") pod \"e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl\" (UID: \"ea2aceba-fd54-43a6-8707-05f92f63ab75\") " pod="openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl"
Jul 30 15:37:54.828446 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:54.828401    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea2aceba-fd54-43a6-8707-05f92f63ab75-workload-certs\") pod \"e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl\" (UID: \"ea2aceba-fd54-43a6-8707-05f92f63ab75\") " pod="openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl"
Jul 30 15:37:54.828723 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:54.828700    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea2aceba-fd54-43a6-8707-05f92f63ab75-istiod-ca-cert\") pod \"e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl\" (UID: \"ea2aceba-fd54-43a6-8707-05f92f63ab75\") " pod="openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl"
Jul 30 15:37:54.830013 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:54.829979    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea2aceba-fd54-43a6-8707-05f92f63ab75-istio-envoy\") pod \"e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl\" (UID: \"ea2aceba-fd54-43a6-8707-05f92f63ab75\") " pod="openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl"
Jul 30 15:37:54.830345 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:54.830328    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea2aceba-fd54-43a6-8707-05f92f63ab75-istio-podinfo\") pod \"e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl\" (UID: \"ea2aceba-fd54-43a6-8707-05f92f63ab75\") " pod="openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl"
Jul 30 15:37:54.834750 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:54.834725    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fn9q\" (UniqueName: \"kubernetes.io/projected/ea2aceba-fd54-43a6-8707-05f92f63ab75-kube-api-access-5fn9q\") pod \"e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl\" (UID: \"ea2aceba-fd54-43a6-8707-05f92f63ab75\") " pod="openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl"
Jul 30 15:37:54.834875 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:54.834806    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea2aceba-fd54-43a6-8707-05f92f63ab75-istio-token\") pod \"e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl\" (UID: \"ea2aceba-fd54-43a6-8707-05f92f63ab75\") " pod="openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl"
Jul 30 15:37:54.927781 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:54.927712    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl"
Jul 30 15:37:55.065799 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:55.065773    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl"]
Jul 30 15:37:55.067560 ip-10-0-133-117 kubenswrapper[2578]: W0730 15:37:55.067533    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea2aceba_fd54_43a6_8707_05f92f63ab75.slice/crio-2caaba0f33e13ec7be402601aad2f0bf0fb4bcaddce19b8d28efc8cd41cd49d8 WatchSource:0}: Error finding container 2caaba0f33e13ec7be402601aad2f0bf0fb4bcaddce19b8d28efc8cd41cd49d8: Status 404 returned error can't find the container with id 2caaba0f33e13ec7be402601aad2f0bf0fb4bcaddce19b8d28efc8cd41cd49d8
Jul 30 15:37:55.069688 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:55.069659    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:37:55.069796 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:55.069731    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:37:55.069796 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:55.069778    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:37:55.914639 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:55.914605    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl" event={"ID":"ea2aceba-fd54-43a6-8707-05f92f63ab75","Type":"ContainerStarted","Data":"6886cfb66beda6c4c58bf49990c29d12f7e45db6d1d4fa618f5a69044390abe5"}
Jul 30 15:37:55.915029 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:55.914669    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl" event={"ID":"ea2aceba-fd54-43a6-8707-05f92f63ab75","Type":"ContainerStarted","Data":"2caaba0f33e13ec7be402601aad2f0bf0fb4bcaddce19b8d28efc8cd41cd49d8"}
Jul 30 15:37:55.928495 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:55.928473    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl"
Jul 30 15:37:55.933485 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:55.933440    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl" podStartSLOduration=1.9334265419999999 podStartE2EDuration="1.933426542s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:37:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:37:55.932860355 +0000 UTC m=+2987.461798828" watchObservedRunningTime="2026-07-30 15:37:55.933426542 +0000 UTC m=+2987.462365015"
Jul 30 15:37:56.933728 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:56.933696    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl"
Jul 30 15:37:57.924678 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:57.924646    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl"
Jul 30 15:37:57.925661 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:57.925642    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl"
Jul 30 15:37:58.821694 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:58.821664    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz"
Jul 30 15:37:58.930519 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:58.930443    2578 generic.go:356] "Generic (PLEG): container finished" podID="36e1cb08-f75a-44e9-9914-fd1124432ab3" containerID="d40e7b9e2c1e0377287e96c501c94a8b1e7d9bfe64c5d4895e85f802fe7fa519" exitCode=0
Jul 30 15:37:58.930519 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:58.930508    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz"
Jul 30 15:37:58.930736 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:58.930520    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz" event={"ID":"36e1cb08-f75a-44e9-9914-fd1124432ab3","Type":"ContainerDied","Data":"d40e7b9e2c1e0377287e96c501c94a8b1e7d9bfe64c5d4895e85f802fe7fa519"}
Jul 30 15:37:58.930736 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:58.930570    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz" event={"ID":"36e1cb08-f75a-44e9-9914-fd1124432ab3","Type":"ContainerDied","Data":"b6818d984cb64013e23325bf937eec80bfea552391756cec66a93b284a4c7e3c"}
Jul 30 15:37:58.930736 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:58.930595    2578 scope.go:117] "RemoveContainer" containerID="d40e7b9e2c1e0377287e96c501c94a8b1e7d9bfe64c5d4895e85f802fe7fa519"
Jul 30 15:37:58.939235 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:58.939222    2578 scope.go:117] "RemoveContainer" containerID="838b0a659dd4aaca98822ad64acdf55cfac6eff9e959fcfbc97b7821d37b42de"
Jul 30 15:37:58.948987 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:58.948970    2578 scope.go:117] "RemoveContainer" containerID="d40e7b9e2c1e0377287e96c501c94a8b1e7d9bfe64c5d4895e85f802fe7fa519"
Jul 30 15:37:58.949283 ip-10-0-133-117 kubenswrapper[2578]: E0730 15:37:58.949267    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d40e7b9e2c1e0377287e96c501c94a8b1e7d9bfe64c5d4895e85f802fe7fa519\": container with ID starting with d40e7b9e2c1e0377287e96c501c94a8b1e7d9bfe64c5d4895e85f802fe7fa519 not found: ID does not exist" containerID="d40e7b9e2c1e0377287e96c501c94a8b1e7d9bfe64c5d4895e85f802fe7fa519"
Jul 30 15:37:58.949355 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:58.949290    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d40e7b9e2c1e0377287e96c501c94a8b1e7d9bfe64c5d4895e85f802fe7fa519"} err="failed to get container status \"d40e7b9e2c1e0377287e96c501c94a8b1e7d9bfe64c5d4895e85f802fe7fa519\": rpc error: code = NotFound desc = could not find container \"d40e7b9e2c1e0377287e96c501c94a8b1e7d9bfe64c5d4895e85f802fe7fa519\": container with ID starting with d40e7b9e2c1e0377287e96c501c94a8b1e7d9bfe64c5d4895e85f802fe7fa519 not found: ID does not exist"
Jul 30 15:37:58.949355 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:58.949306    2578 scope.go:117] "RemoveContainer" containerID="838b0a659dd4aaca98822ad64acdf55cfac6eff9e959fcfbc97b7821d37b42de"
Jul 30 15:37:58.949528 ip-10-0-133-117 kubenswrapper[2578]: E0730 15:37:58.949513    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"838b0a659dd4aaca98822ad64acdf55cfac6eff9e959fcfbc97b7821d37b42de\": container with ID starting with 838b0a659dd4aaca98822ad64acdf55cfac6eff9e959fcfbc97b7821d37b42de not found: ID does not exist" containerID="838b0a659dd4aaca98822ad64acdf55cfac6eff9e959fcfbc97b7821d37b42de"
Jul 30 15:37:58.949573 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:58.949530    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"838b0a659dd4aaca98822ad64acdf55cfac6eff9e959fcfbc97b7821d37b42de"} err="failed to get container status \"838b0a659dd4aaca98822ad64acdf55cfac6eff9e959fcfbc97b7821d37b42de\": rpc error: code = NotFound desc = could not find container \"838b0a659dd4aaca98822ad64acdf55cfac6eff9e959fcfbc97b7821d37b42de\": container with ID starting with 838b0a659dd4aaca98822ad64acdf55cfac6eff9e959fcfbc97b7821d37b42de not found: ID does not exist"
Jul 30 15:37:58.975136 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:58.975112    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36e1cb08-f75a-44e9-9914-fd1124432ab3-istio-data\") pod \"36e1cb08-f75a-44e9-9914-fd1124432ab3\" (UID: \"36e1cb08-f75a-44e9-9914-fd1124432ab3\") "
Jul 30 15:37:58.975261 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:58.975159    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36e1cb08-f75a-44e9-9914-fd1124432ab3-credential-socket\") pod \"36e1cb08-f75a-44e9-9914-fd1124432ab3\" (UID: \"36e1cb08-f75a-44e9-9914-fd1124432ab3\") "
Jul 30 15:37:58.975261 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:58.975192    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36e1cb08-f75a-44e9-9914-fd1124432ab3-workload-certs\") pod \"36e1cb08-f75a-44e9-9914-fd1124432ab3\" (UID: \"36e1cb08-f75a-44e9-9914-fd1124432ab3\") "
Jul 30 15:37:58.975261 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:58.975214    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36e1cb08-f75a-44e9-9914-fd1124432ab3-workload-socket\") pod \"36e1cb08-f75a-44e9-9914-fd1124432ab3\" (UID: \"36e1cb08-f75a-44e9-9914-fd1124432ab3\") "
Jul 30 15:37:58.975261 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:58.975228    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36e1cb08-f75a-44e9-9914-fd1124432ab3-istio-envoy\") pod \"36e1cb08-f75a-44e9-9914-fd1124432ab3\" (UID: \"36e1cb08-f75a-44e9-9914-fd1124432ab3\") "
Jul 30 15:37:58.975486 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:58.975265    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36e1cb08-f75a-44e9-9914-fd1124432ab3-istiod-ca-cert\") pod \"36e1cb08-f75a-44e9-9914-fd1124432ab3\" (UID: \"36e1cb08-f75a-44e9-9914-fd1124432ab3\") "
Jul 30 15:37:58.975486 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:58.975320    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36e1cb08-f75a-44e9-9914-fd1124432ab3-istio-podinfo\") pod \"36e1cb08-f75a-44e9-9914-fd1124432ab3\" (UID: \"36e1cb08-f75a-44e9-9914-fd1124432ab3\") "
Jul 30 15:37:58.975486 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:58.975379    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fvlt2\" (UniqueName: \"kubernetes.io/projected/36e1cb08-f75a-44e9-9914-fd1124432ab3-kube-api-access-fvlt2\") pod \"36e1cb08-f75a-44e9-9914-fd1124432ab3\" (UID: \"36e1cb08-f75a-44e9-9914-fd1124432ab3\") "
Jul 30 15:37:58.975486 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:58.975410    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36e1cb08-f75a-44e9-9914-fd1124432ab3-istio-token\") pod \"36e1cb08-f75a-44e9-9914-fd1124432ab3\" (UID: \"36e1cb08-f75a-44e9-9914-fd1124432ab3\") "
Jul 30 15:37:58.975682 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:58.975537    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36e1cb08-f75a-44e9-9914-fd1124432ab3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "36e1cb08-f75a-44e9-9914-fd1124432ab3" (UID: "36e1cb08-f75a-44e9-9914-fd1124432ab3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:37:58.975682 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:58.975676    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36e1cb08-f75a-44e9-9914-fd1124432ab3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "36e1cb08-f75a-44e9-9914-fd1124432ab3" (UID: "36e1cb08-f75a-44e9-9914-fd1124432ab3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:37:58.975809 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:58.975781    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36e1cb08-f75a-44e9-9914-fd1124432ab3-credential-socket\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:37:58.975809 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:58.975805    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36e1cb08-f75a-44e9-9914-fd1124432ab3-workload-certs\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:37:58.975906 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:58.975877    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36e1cb08-f75a-44e9-9914-fd1124432ab3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "36e1cb08-f75a-44e9-9914-fd1124432ab3" (UID: "36e1cb08-f75a-44e9-9914-fd1124432ab3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:37:58.976158 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:58.976106    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36e1cb08-f75a-44e9-9914-fd1124432ab3-istio-data" (OuterVolumeSpecName: "istio-data") pod "36e1cb08-f75a-44e9-9914-fd1124432ab3" (UID: "36e1cb08-f75a-44e9-9914-fd1124432ab3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:37:58.976413 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:58.976392    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36e1cb08-f75a-44e9-9914-fd1124432ab3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "36e1cb08-f75a-44e9-9914-fd1124432ab3" (UID: "36e1cb08-f75a-44e9-9914-fd1124432ab3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:37:58.977987 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:58.977961    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36e1cb08-f75a-44e9-9914-fd1124432ab3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "36e1cb08-f75a-44e9-9914-fd1124432ab3" (UID: "36e1cb08-f75a-44e9-9914-fd1124432ab3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:37:58.978069 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:58.978045    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/36e1cb08-f75a-44e9-9914-fd1124432ab3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "36e1cb08-f75a-44e9-9914-fd1124432ab3" (UID: "36e1cb08-f75a-44e9-9914-fd1124432ab3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 15:37:58.978328 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:58.978299    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36e1cb08-f75a-44e9-9914-fd1124432ab3-kube-api-access-fvlt2" (OuterVolumeSpecName: "kube-api-access-fvlt2") pod "36e1cb08-f75a-44e9-9914-fd1124432ab3" (UID: "36e1cb08-f75a-44e9-9914-fd1124432ab3"). InnerVolumeSpecName "kube-api-access-fvlt2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:37:58.978453 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:58.978435    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36e1cb08-f75a-44e9-9914-fd1124432ab3-istio-token" (OuterVolumeSpecName: "istio-token") pod "36e1cb08-f75a-44e9-9914-fd1124432ab3" (UID: "36e1cb08-f75a-44e9-9914-fd1124432ab3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:37:59.076851 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:59.076704    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36e1cb08-f75a-44e9-9914-fd1124432ab3-istio-data\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:37:59.076851 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:59.076738    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36e1cb08-f75a-44e9-9914-fd1124432ab3-workload-socket\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:37:59.076851 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:59.076752    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36e1cb08-f75a-44e9-9914-fd1124432ab3-istio-envoy\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:37:59.076851 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:59.076766    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36e1cb08-f75a-44e9-9914-fd1124432ab3-istiod-ca-cert\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:37:59.076851 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:59.076779    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36e1cb08-f75a-44e9-9914-fd1124432ab3-istio-podinfo\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:37:59.076851 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:59.076791    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fvlt2\" (UniqueName: \"kubernetes.io/projected/36e1cb08-f75a-44e9-9914-fd1124432ab3-kube-api-access-fvlt2\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:37:59.076851 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:59.076806    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36e1cb08-f75a-44e9-9914-fd1124432ab3-istio-token\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:37:59.080964 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:59.080934    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-845a26-7c7c6fbb89-6h6qp"]
Jul 30 15:37:59.081504 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:59.081485    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36e1cb08-f75a-44e9-9914-fd1124432ab3" containerName="istio-proxy"
Jul 30 15:37:59.081504 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:59.081504    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="36e1cb08-f75a-44e9-9914-fd1124432ab3" containerName="istio-proxy"
Jul 30 15:37:59.081654 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:59.081534    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c595b020-052f-4b1c-8281-ccee6116b418" containerName="istio-proxy"
Jul 30 15:37:59.081654 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:59.081544    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c595b020-052f-4b1c-8281-ccee6116b418" containerName="istio-proxy"
Jul 30 15:37:59.081654 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:59.081638    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c595b020-052f-4b1c-8281-ccee6116b418" containerName="istio-proxy"
Jul 30 15:37:59.081654 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:59.081649    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="36e1cb08-f75a-44e9-9914-fd1124432ab3" containerName="istio-proxy"
Jul 30 15:37:59.086626 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:59.086606    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-845a26-7c7c6fbb89-6h6qp"
Jul 30 15:37:59.088388 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:59.088363    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-845a26-serving-cert\""
Jul 30 15:37:59.095277 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:59.095251    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-845a26-7c7c6fbb89-6h6qp"]
Jul 30 15:37:59.245632 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:59.245549    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz"]
Jul 30 15:37:59.248682 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:59.248657    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-e2257c-openshift-default-85ff5cd58-kf2qz"]
Jul 30 15:37:59.278942 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:59.278911    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b059341-28e1-4546-b960-2dd668984f4b-openshift-service-ca\") pod \"maas-api-e2e-shared-a-845a26-7c7c6fbb89-6h6qp\" (UID: \"7b059341-28e1-4546-b960-2dd668984f4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-845a26-7c7c6fbb89-6h6qp"
Jul 30 15:37:59.279098 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:59.278965    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b059341-28e1-4546-b960-2dd668984f4b-maas-api-tls\") pod \"maas-api-e2e-shared-a-845a26-7c7c6fbb89-6h6qp\" (UID: \"7b059341-28e1-4546-b960-2dd668984f4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-845a26-7c7c6fbb89-6h6qp"
Jul 30 15:37:59.279098 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:59.279083    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hn9ks\" (UniqueName: \"kubernetes.io/projected/7b059341-28e1-4546-b960-2dd668984f4b-kube-api-access-hn9ks\") pod \"maas-api-e2e-shared-a-845a26-7c7c6fbb89-6h6qp\" (UID: \"7b059341-28e1-4546-b960-2dd668984f4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-845a26-7c7c6fbb89-6h6qp"
Jul 30 15:37:59.380171 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:59.380110    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b059341-28e1-4546-b960-2dd668984f4b-openshift-service-ca\") pod \"maas-api-e2e-shared-a-845a26-7c7c6fbb89-6h6qp\" (UID: \"7b059341-28e1-4546-b960-2dd668984f4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-845a26-7c7c6fbb89-6h6qp"
Jul 30 15:37:59.380322 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:59.380207    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b059341-28e1-4546-b960-2dd668984f4b-maas-api-tls\") pod \"maas-api-e2e-shared-a-845a26-7c7c6fbb89-6h6qp\" (UID: \"7b059341-28e1-4546-b960-2dd668984f4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-845a26-7c7c6fbb89-6h6qp"
Jul 30 15:37:59.380322 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:59.380280    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hn9ks\" (UniqueName: \"kubernetes.io/projected/7b059341-28e1-4546-b960-2dd668984f4b-kube-api-access-hn9ks\") pod \"maas-api-e2e-shared-a-845a26-7c7c6fbb89-6h6qp\" (UID: \"7b059341-28e1-4546-b960-2dd668984f4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-845a26-7c7c6fbb89-6h6qp"
Jul 30 15:37:59.380755 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:59.380731    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b059341-28e1-4546-b960-2dd668984f4b-openshift-service-ca\") pod \"maas-api-e2e-shared-a-845a26-7c7c6fbb89-6h6qp\" (UID: \"7b059341-28e1-4546-b960-2dd668984f4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-845a26-7c7c6fbb89-6h6qp"
Jul 30 15:37:59.382554 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:59.382531    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b059341-28e1-4546-b960-2dd668984f4b-maas-api-tls\") pod \"maas-api-e2e-shared-a-845a26-7c7c6fbb89-6h6qp\" (UID: \"7b059341-28e1-4546-b960-2dd668984f4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-845a26-7c7c6fbb89-6h6qp"
Jul 30 15:37:59.387450 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:59.387429    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hn9ks\" (UniqueName: \"kubernetes.io/projected/7b059341-28e1-4546-b960-2dd668984f4b-kube-api-access-hn9ks\") pod \"maas-api-e2e-shared-a-845a26-7c7c6fbb89-6h6qp\" (UID: \"7b059341-28e1-4546-b960-2dd668984f4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-845a26-7c7c6fbb89-6h6qp"
Jul 30 15:37:59.400266 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:59.400241    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-845a26-7c7c6fbb89-6h6qp"
Jul 30 15:37:59.531701 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:59.531656    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-845a26-7c7c6fbb89-6h6qp"]
Jul 30 15:37:59.532828 ip-10-0-133-117 kubenswrapper[2578]: W0730 15:37:59.532801    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b059341_28e1_4546_b960_2dd668984f4b.slice/crio-d6a89287ae4ee801588506a3dd032e6c80903d0fcb149326143946fb0ccbc010 WatchSource:0}: Error finding container d6a89287ae4ee801588506a3dd032e6c80903d0fcb149326143946fb0ccbc010: Status 404 returned error can't find the container with id d6a89287ae4ee801588506a3dd032e6c80903d0fcb149326143946fb0ccbc010
Jul 30 15:37:59.534414 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:59.534392    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 15:37:59.798458 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:59.798382    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-845a26-74cffd7c8d-gcnzh"]
Jul 30 15:37:59.798795 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:59.798783    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36e1cb08-f75a-44e9-9914-fd1124432ab3" containerName="istio-proxy"
Jul 30 15:37:59.798847 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:59.798806    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="36e1cb08-f75a-44e9-9914-fd1124432ab3" containerName="istio-proxy"
Jul 30 15:37:59.798884 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:59.798868    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="36e1cb08-f75a-44e9-9914-fd1124432ab3" containerName="istio-proxy"
Jul 30 15:37:59.803060 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:59.803039    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-845a26-74cffd7c8d-gcnzh"
Jul 30 15:37:59.804931 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:59.804911    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-845a26\""
Jul 30 15:37:59.805021 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:59.804920    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-845a26-dockercfg-gkvvq\""
Jul 30 15:37:59.812021 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:59.811998    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-845a26-74cffd7c8d-gcnzh"]
Jul 30 15:37:59.841342 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:59.841307    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-845a26-cdf79787d-k4thx"]
Jul 30 15:37:59.844856 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:59.844835    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-845a26-cdf79787d-k4thx"
Jul 30 15:37:59.854288 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:59.854246    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-845a26-cdf79787d-k4thx"]
Jul 30 15:37:59.883818 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:59.883778    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z89mb\" (UniqueName: \"kubernetes.io/projected/fff1cd8b-5cd0-447d-8f23-6f4a7f86aa57-kube-api-access-z89mb\") pod \"payload-processing-e2e-shared-a-845a26-cdf79787d-k4thx\" (UID: \"fff1cd8b-5cd0-447d-8f23-6f4a7f86aa57\") " pod="openshift-ingress/payload-processing-e2e-shared-a-845a26-cdf79787d-k4thx"
Jul 30 15:37:59.883953 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:59.883883    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9a2a6a2-e909-4d96-ac32-0f0bb8626d74-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-845a26-74cffd7c8d-gcnzh\" (UID: \"b9a2a6a2-e909-4d96-ac32-0f0bb8626d74\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-845a26-74cffd7c8d-gcnzh"
Jul 30 15:37:59.883953 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:59.883911    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dgv6\" (UniqueName: \"kubernetes.io/projected/b9a2a6a2-e909-4d96-ac32-0f0bb8626d74-kube-api-access-5dgv6\") pod \"payload-pre-processing-e2e-shared-a-845a26-74cffd7c8d-gcnzh\" (UID: \"b9a2a6a2-e909-4d96-ac32-0f0bb8626d74\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-845a26-74cffd7c8d-gcnzh"
Jul 30 15:37:59.884063 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:59.883986    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fff1cd8b-5cd0-447d-8f23-6f4a7f86aa57-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-845a26-cdf79787d-k4thx\" (UID: \"fff1cd8b-5cd0-447d-8f23-6f4a7f86aa57\") " pod="openshift-ingress/payload-processing-e2e-shared-a-845a26-cdf79787d-k4thx"
Jul 30 15:37:59.939172 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:59.939128    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-845a26-7c7c6fbb89-6h6qp" event={"ID":"7b059341-28e1-4546-b960-2dd668984f4b","Type":"ContainerStarted","Data":"d6a89287ae4ee801588506a3dd032e6c80903d0fcb149326143946fb0ccbc010"}
Jul 30 15:37:59.984865 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:59.984837    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fff1cd8b-5cd0-447d-8f23-6f4a7f86aa57-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-845a26-cdf79787d-k4thx\" (UID: \"fff1cd8b-5cd0-447d-8f23-6f4a7f86aa57\") " pod="openshift-ingress/payload-processing-e2e-shared-a-845a26-cdf79787d-k4thx"
Jul 30 15:37:59.985100 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:59.985071    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z89mb\" (UniqueName: \"kubernetes.io/projected/fff1cd8b-5cd0-447d-8f23-6f4a7f86aa57-kube-api-access-z89mb\") pod \"payload-processing-e2e-shared-a-845a26-cdf79787d-k4thx\" (UID: \"fff1cd8b-5cd0-447d-8f23-6f4a7f86aa57\") " pod="openshift-ingress/payload-processing-e2e-shared-a-845a26-cdf79787d-k4thx"
Jul 30 15:37:59.985263 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:59.985244    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9a2a6a2-e909-4d96-ac32-0f0bb8626d74-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-845a26-74cffd7c8d-gcnzh\" (UID: \"b9a2a6a2-e909-4d96-ac32-0f0bb8626d74\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-845a26-74cffd7c8d-gcnzh"
Jul 30 15:37:59.985365 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:59.985280    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dgv6\" (UniqueName: \"kubernetes.io/projected/b9a2a6a2-e909-4d96-ac32-0f0bb8626d74-kube-api-access-5dgv6\") pod \"payload-pre-processing-e2e-shared-a-845a26-74cffd7c8d-gcnzh\" (UID: \"b9a2a6a2-e909-4d96-ac32-0f0bb8626d74\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-845a26-74cffd7c8d-gcnzh"
Jul 30 15:37:59.985967 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:59.985941    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fff1cd8b-5cd0-447d-8f23-6f4a7f86aa57-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-845a26-cdf79787d-k4thx\" (UID: \"fff1cd8b-5cd0-447d-8f23-6f4a7f86aa57\") " pod="openshift-ingress/payload-processing-e2e-shared-a-845a26-cdf79787d-k4thx"
Jul 30 15:37:59.986077 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:59.986061    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9a2a6a2-e909-4d96-ac32-0f0bb8626d74-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-845a26-74cffd7c8d-gcnzh\" (UID: \"b9a2a6a2-e909-4d96-ac32-0f0bb8626d74\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-845a26-74cffd7c8d-gcnzh"
Jul 30 15:37:59.992739 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:59.992715    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dgv6\" (UniqueName: \"kubernetes.io/projected/b9a2a6a2-e909-4d96-ac32-0f0bb8626d74-kube-api-access-5dgv6\") pod \"payload-pre-processing-e2e-shared-a-845a26-74cffd7c8d-gcnzh\" (UID: \"b9a2a6a2-e909-4d96-ac32-0f0bb8626d74\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-845a26-74cffd7c8d-gcnzh"
Jul 30 15:37:59.993444 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:37:59.993423    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z89mb\" (UniqueName: \"kubernetes.io/projected/fff1cd8b-5cd0-447d-8f23-6f4a7f86aa57-kube-api-access-z89mb\") pod \"payload-processing-e2e-shared-a-845a26-cdf79787d-k4thx\" (UID: \"fff1cd8b-5cd0-447d-8f23-6f4a7f86aa57\") " pod="openshift-ingress/payload-processing-e2e-shared-a-845a26-cdf79787d-k4thx"
Jul 30 15:38:00.115429 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:00.115392    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-845a26-74cffd7c8d-gcnzh"
Jul 30 15:38:00.156314 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:00.156279    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-845a26-cdf79787d-k4thx"
Jul 30 15:38:00.266919 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:00.266888    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-845a26-74cffd7c8d-gcnzh"]
Jul 30 15:38:00.267393 ip-10-0-133-117 kubenswrapper[2578]: W0730 15:38:00.267367    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9a2a6a2_e909_4d96_ac32_0f0bb8626d74.slice/crio-c5e84ddeba6630b0b0f88d8a545289677158f2f4c6c04adaaefa413e5f00f46d WatchSource:0}: Error finding container c5e84ddeba6630b0b0f88d8a545289677158f2f4c6c04adaaefa413e5f00f46d: Status 404 returned error can't find the container with id c5e84ddeba6630b0b0f88d8a545289677158f2f4c6c04adaaefa413e5f00f46d
Jul 30 15:38:00.307467 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:00.307441    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-845a26-cdf79787d-k4thx"]
Jul 30 15:38:00.309734 ip-10-0-133-117 kubenswrapper[2578]: W0730 15:38:00.309710    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfff1cd8b_5cd0_447d_8f23_6f4a7f86aa57.slice/crio-97c97a48bf7d13006d64b9614ef4715e547a6bcef3ac6562aed97d5c49f0bc5e WatchSource:0}: Error finding container 97c97a48bf7d13006d64b9614ef4715e547a6bcef3ac6562aed97d5c49f0bc5e: Status 404 returned error can't find the container with id 97c97a48bf7d13006d64b9614ef4715e547a6bcef3ac6562aed97d5c49f0bc5e
Jul 30 15:38:00.946306 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:00.946254    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-845a26-74cffd7c8d-gcnzh" event={"ID":"b9a2a6a2-e909-4d96-ac32-0f0bb8626d74","Type":"ContainerStarted","Data":"90d96689fc46a37e3fbb1af635ddc913f3a5e25d640a10afcd8b9fbcd8e23557"}
Jul 30 15:38:00.946806 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:00.946316    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-845a26-74cffd7c8d-gcnzh" event={"ID":"b9a2a6a2-e909-4d96-ac32-0f0bb8626d74","Type":"ContainerStarted","Data":"c5e84ddeba6630b0b0f88d8a545289677158f2f4c6c04adaaefa413e5f00f46d"}
Jul 30 15:38:00.946806 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:00.946344    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-845a26-74cffd7c8d-gcnzh"
Jul 30 15:38:00.947954 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:00.947927    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-845a26-7c7c6fbb89-6h6qp" event={"ID":"7b059341-28e1-4546-b960-2dd668984f4b","Type":"ContainerStarted","Data":"71bd91b2962beaf6f74a2e20823a3bceb3ec50d6ea2515a1fa52d21d44a8afe3"}
Jul 30 15:38:00.948069 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:00.947982    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-845a26-7c7c6fbb89-6h6qp"
Jul 30 15:38:00.949533 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:00.949507    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-845a26-cdf79787d-k4thx" event={"ID":"fff1cd8b-5cd0-447d-8f23-6f4a7f86aa57","Type":"ContainerStarted","Data":"f8314eecface2ace481ee04a6ac0771fee9f281949743d91e05b9530cc44f1b0"}
Jul 30 15:38:00.949641 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:00.949554    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-845a26-cdf79787d-k4thx" event={"ID":"fff1cd8b-5cd0-447d-8f23-6f4a7f86aa57","Type":"ContainerStarted","Data":"97c97a48bf7d13006d64b9614ef4715e547a6bcef3ac6562aed97d5c49f0bc5e"}
Jul 30 15:38:00.949641 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:00.949583    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-845a26-cdf79787d-k4thx"
Jul 30 15:38:00.979234 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:00.979195    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36e1cb08-f75a-44e9-9914-fd1124432ab3" path="/var/lib/kubelet/pods/36e1cb08-f75a-44e9-9914-fd1124432ab3/volumes"
Jul 30 15:38:00.979679 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:00.979638    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-845a26-7c7c6fbb89-6h6qp" podStartSLOduration=1.592437657 podStartE2EDuration="1.979625246s" totalImagesPullingTime="387.187589ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:37:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:38:00.978417463 +0000 UTC m=+2992.507355938" watchObservedRunningTime="2026-07-30 15:38:00.979625246 +0000 UTC m=+2992.508563720"
Jul 30 15:38:00.979999 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:00.979974    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-845a26-74cffd7c8d-gcnzh" podStartSLOduration=1.9799661469999998 podStartE2EDuration="1.979966147s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:37:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:38:00.960890528 +0000 UTC m=+2992.489829003" watchObservedRunningTime="2026-07-30 15:38:00.979966147 +0000 UTC m=+2992.508904656"
Jul 30 15:38:00.994951 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:00.994892    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-845a26-cdf79787d-k4thx" podStartSLOduration=1.994874543 podStartE2EDuration="1.994874543s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:37:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:38:00.992735376 +0000 UTC m=+2992.521673850" watchObservedRunningTime="2026-07-30 15:38:00.994874543 +0000 UTC m=+2992.523813019"
Jul 30 15:38:06.962941 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:06.962911    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-845a26-7c7c6fbb89-6h6qp"
Jul 30 15:38:09.147656 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:09.147626    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4mjmp_de241220-5f27-4b0b-b7c6-6d4b350e24e1/console-operator/1.log"
Jul 30 15:38:09.173480 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:09.173452    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4mjmp_de241220-5f27-4b0b-b7c6-6d4b350e24e1/console-operator/1.log"
Jul 30 15:38:11.003857 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:11.003827    2578 generic.go:356] "Generic (PLEG): container finished" podID="ea2aceba-fd54-43a6-8707-05f92f63ab75" containerID="6886cfb66beda6c4c58bf49990c29d12f7e45db6d1d4fa618f5a69044390abe5" exitCode=0
Jul 30 15:38:11.004284 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:11.003900    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl" event={"ID":"ea2aceba-fd54-43a6-8707-05f92f63ab75","Type":"ContainerDied","Data":"6886cfb66beda6c4c58bf49990c29d12f7e45db6d1d4fa618f5a69044390abe5"}
Jul 30 15:38:11.004340 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:11.004329    2578 scope.go:117] "RemoveContainer" containerID="6886cfb66beda6c4c58bf49990c29d12f7e45db6d1d4fa618f5a69044390abe5"
Jul 30 15:38:11.005364 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:11.005336    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:38:11.005434 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:11.005417    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:38:11.005480 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:11.005465    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:38:11.958207 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:11.958137    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-845a26-74cffd7c8d-gcnzh"
Jul 30 15:38:11.958416 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:11.958240    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-845a26-cdf79787d-k4thx"
Jul 30 15:38:12.011344 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:12.011312    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl" event={"ID":"ea2aceba-fd54-43a6-8707-05f92f63ab75","Type":"ContainerStarted","Data":"5b09f9116a3ae3d6ea329a5ec98588248b8c11f8ddc1737108cc5d9fe883cb95"}
Jul 30 15:38:12.925567 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:12.925512    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl"
Jul 30 15:38:12.928700 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:12.928683    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl"
Jul 30 15:38:12.929909 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:12.929887    2578 patch_prober.go:28] interesting pod/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Jul 30 15:38:12.929999 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:12.929957    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl" podUID="ea2aceba-fd54-43a6-8707-05f92f63ab75" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Jul 30 15:38:13.928443 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:13.928400    2578 patch_prober.go:28] interesting pod/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Jul 30 15:38:13.928844 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:13.928467    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl" podUID="ea2aceba-fd54-43a6-8707-05f92f63ab75" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Jul 30 15:38:14.928766 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:14.928733    2578 patch_prober.go:28] interesting pod/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Jul 30 15:38:14.929172 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:14.928818    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl" podUID="ea2aceba-fd54-43a6-8707-05f92f63ab75" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Jul 30 15:38:15.928523 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:15.928490    2578 patch_prober.go:28] interesting pod/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Jul 30 15:38:15.928696 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:15.928545    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl" podUID="ea2aceba-fd54-43a6-8707-05f92f63ab75" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Jul 30 15:38:16.932108 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:16.932077    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl"
Jul 30 15:38:16.933079 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:16.933058    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl"
Jul 30 15:38:23.806764 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:23.806726    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s"]
Jul 30 15:38:23.810305 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:23.810284    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s"
Jul 30 15:38:23.812325 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:23.812303    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-2b1f1f-openshift-default-dockercfg-7pd9q\""
Jul 30 15:38:23.821004 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:23.820980    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s"]
Jul 30 15:38:23.921103 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:23.921067    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efe00480-68c7-4658-a34a-53ccefe2ba3e-istio-podinfo\") pod \"e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s\" (UID: \"efe00480-68c7-4658-a34a-53ccefe2ba3e\") " pod="openshift-ingress/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s"
Jul 30 15:38:23.921103 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:23.921105    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efe00480-68c7-4658-a34a-53ccefe2ba3e-istio-data\") pod \"e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s\" (UID: \"efe00480-68c7-4658-a34a-53ccefe2ba3e\") " pod="openshift-ingress/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s"
Jul 30 15:38:23.921356 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:23.921193    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efe00480-68c7-4658-a34a-53ccefe2ba3e-workload-certs\") pod \"e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s\" (UID: \"efe00480-68c7-4658-a34a-53ccefe2ba3e\") " pod="openshift-ingress/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s"
Jul 30 15:38:23.921356 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:23.921215    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efe00480-68c7-4658-a34a-53ccefe2ba3e-credential-socket\") pod \"e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s\" (UID: \"efe00480-68c7-4658-a34a-53ccefe2ba3e\") " pod="openshift-ingress/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s"
Jul 30 15:38:23.921356 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:23.921267    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efe00480-68c7-4658-a34a-53ccefe2ba3e-istio-envoy\") pod \"e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s\" (UID: \"efe00480-68c7-4658-a34a-53ccefe2ba3e\") " pod="openshift-ingress/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s"
Jul 30 15:38:23.921356 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:23.921293    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efe00480-68c7-4658-a34a-53ccefe2ba3e-istiod-ca-cert\") pod \"e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s\" (UID: \"efe00480-68c7-4658-a34a-53ccefe2ba3e\") " pod="openshift-ingress/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s"
Jul 30 15:38:23.921506 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:23.921357    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efe00480-68c7-4658-a34a-53ccefe2ba3e-workload-socket\") pod \"e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s\" (UID: \"efe00480-68c7-4658-a34a-53ccefe2ba3e\") " pod="openshift-ingress/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s"
Jul 30 15:38:23.921506 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:23.921382    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/efe00480-68c7-4658-a34a-53ccefe2ba3e-istio-token\") pod \"e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s\" (UID: \"efe00480-68c7-4658-a34a-53ccefe2ba3e\") " pod="openshift-ingress/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s"
Jul 30 15:38:23.921506 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:23.921407    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7vhs\" (UniqueName: \"kubernetes.io/projected/efe00480-68c7-4658-a34a-53ccefe2ba3e-kube-api-access-p7vhs\") pod \"e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s\" (UID: \"efe00480-68c7-4658-a34a-53ccefe2ba3e\") " pod="openshift-ingress/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s"
Jul 30 15:38:24.022516 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:24.022481    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efe00480-68c7-4658-a34a-53ccefe2ba3e-istio-envoy\") pod \"e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s\" (UID: \"efe00480-68c7-4658-a34a-53ccefe2ba3e\") " pod="openshift-ingress/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s"
Jul 30 15:38:24.022516 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:24.022518    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efe00480-68c7-4658-a34a-53ccefe2ba3e-istiod-ca-cert\") pod \"e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s\" (UID: \"efe00480-68c7-4658-a34a-53ccefe2ba3e\") " pod="openshift-ingress/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s"
Jul 30 15:38:24.022762 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:24.022537    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efe00480-68c7-4658-a34a-53ccefe2ba3e-workload-socket\") pod \"e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s\" (UID: \"efe00480-68c7-4658-a34a-53ccefe2ba3e\") " pod="openshift-ingress/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s"
Jul 30 15:38:24.022762 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:24.022650    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/efe00480-68c7-4658-a34a-53ccefe2ba3e-istio-token\") pod \"e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s\" (UID: \"efe00480-68c7-4658-a34a-53ccefe2ba3e\") " pod="openshift-ingress/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s"
Jul 30 15:38:24.022762 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:24.022698    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7vhs\" (UniqueName: \"kubernetes.io/projected/efe00480-68c7-4658-a34a-53ccefe2ba3e-kube-api-access-p7vhs\") pod \"e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s\" (UID: \"efe00480-68c7-4658-a34a-53ccefe2ba3e\") " pod="openshift-ingress/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s"
Jul 30 15:38:24.022762 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:24.022747    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efe00480-68c7-4658-a34a-53ccefe2ba3e-istio-podinfo\") pod \"e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s\" (UID: \"efe00480-68c7-4658-a34a-53ccefe2ba3e\") " pod="openshift-ingress/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s"
Jul 30 15:38:24.022995 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:24.022789    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efe00480-68c7-4658-a34a-53ccefe2ba3e-istio-data\") pod \"e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s\" (UID: \"efe00480-68c7-4658-a34a-53ccefe2ba3e\") " pod="openshift-ingress/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s"
Jul 30 15:38:24.023098 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:24.023070    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efe00480-68c7-4658-a34a-53ccefe2ba3e-istio-data\") pod \"e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s\" (UID: \"efe00480-68c7-4658-a34a-53ccefe2ba3e\") " pod="openshift-ingress/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s"
Jul 30 15:38:24.023253 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:24.022987    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efe00480-68c7-4658-a34a-53ccefe2ba3e-workload-socket\") pod \"e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s\" (UID: \"efe00480-68c7-4658-a34a-53ccefe2ba3e\") " pod="openshift-ingress/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s"
Jul 30 15:38:24.023253 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:24.023202    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efe00480-68c7-4658-a34a-53ccefe2ba3e-workload-certs\") pod \"e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s\" (UID: \"efe00480-68c7-4658-a34a-53ccefe2ba3e\") " pod="openshift-ingress/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s"
Jul 30 15:38:24.023253 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:24.023240    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efe00480-68c7-4658-a34a-53ccefe2ba3e-credential-socket\") pod \"e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s\" (UID: \"efe00480-68c7-4658-a34a-53ccefe2ba3e\") " pod="openshift-ingress/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s"
Jul 30 15:38:24.023537 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:24.023513    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efe00480-68c7-4658-a34a-53ccefe2ba3e-workload-certs\") pod \"e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s\" (UID: \"efe00480-68c7-4658-a34a-53ccefe2ba3e\") " pod="openshift-ingress/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s"
Jul 30 15:38:24.023652 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:24.023632    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efe00480-68c7-4658-a34a-53ccefe2ba3e-credential-socket\") pod \"e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s\" (UID: \"efe00480-68c7-4658-a34a-53ccefe2ba3e\") " pod="openshift-ingress/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s"
Jul 30 15:38:24.023719 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:24.023633    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efe00480-68c7-4658-a34a-53ccefe2ba3e-istiod-ca-cert\") pod \"e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s\" (UID: \"efe00480-68c7-4658-a34a-53ccefe2ba3e\") " pod="openshift-ingress/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s"
Jul 30 15:38:24.024871 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:24.024851    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efe00480-68c7-4658-a34a-53ccefe2ba3e-istio-envoy\") pod \"e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s\" (UID: \"efe00480-68c7-4658-a34a-53ccefe2ba3e\") " pod="openshift-ingress/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s"
Jul 30 15:38:24.025231 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:24.025213    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efe00480-68c7-4658-a34a-53ccefe2ba3e-istio-podinfo\") pod \"e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s\" (UID: \"efe00480-68c7-4658-a34a-53ccefe2ba3e\") " pod="openshift-ingress/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s"
Jul 30 15:38:24.029717 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:24.029691    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/efe00480-68c7-4658-a34a-53ccefe2ba3e-istio-token\") pod \"e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s\" (UID: \"efe00480-68c7-4658-a34a-53ccefe2ba3e\") " pod="openshift-ingress/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s"
Jul 30 15:38:24.029799 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:24.029780    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7vhs\" (UniqueName: \"kubernetes.io/projected/efe00480-68c7-4658-a34a-53ccefe2ba3e-kube-api-access-p7vhs\") pod \"e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s\" (UID: \"efe00480-68c7-4658-a34a-53ccefe2ba3e\") " pod="openshift-ingress/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s"
Jul 30 15:38:24.121846 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:24.121755    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s"
Jul 30 15:38:24.257567 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:24.257526    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s"]
Jul 30 15:38:24.259417 ip-10-0-133-117 kubenswrapper[2578]: W0730 15:38:24.259390    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefe00480_68c7_4658_a34a_53ccefe2ba3e.slice/crio-a98171d0aa7ff260e8e20654e6c9d95af139cbe72423c0ac74790bcdfda1dbf2 WatchSource:0}: Error finding container a98171d0aa7ff260e8e20654e6c9d95af139cbe72423c0ac74790bcdfda1dbf2: Status 404 returned error can't find the container with id a98171d0aa7ff260e8e20654e6c9d95af139cbe72423c0ac74790bcdfda1dbf2
Jul 30 15:38:24.261630 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:24.261593    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:38:24.261702 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:24.261662    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:38:24.261702 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:24.261691    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:38:25.076836 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:25.076793    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s" event={"ID":"efe00480-68c7-4658-a34a-53ccefe2ba3e","Type":"ContainerStarted","Data":"0a1a0f8f7874632415fd76be600aaddba52cc5d4bf0e9c246f2c2f113f1925bc"}
Jul 30 15:38:25.077357 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:25.076861    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s" event={"ID":"efe00480-68c7-4658-a34a-53ccefe2ba3e","Type":"ContainerStarted","Data":"a98171d0aa7ff260e8e20654e6c9d95af139cbe72423c0ac74790bcdfda1dbf2"}
Jul 30 15:38:25.095761 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:25.095690    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s" podStartSLOduration=2.095675127 podStartE2EDuration="2.095675127s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:38:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:38:25.093981916 +0000 UTC m=+3016.622920389" watchObservedRunningTime="2026-07-30 15:38:25.095675127 +0000 UTC m=+3016.624613678"
Jul 30 15:38:25.121948 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:25.121921    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s"
Jul 30 15:38:25.126754 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:25.126731    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s"
Jul 30 15:38:26.082021 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:26.081988    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s"
Jul 30 15:38:26.082847 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:26.082828    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s"
Jul 30 15:38:28.904526 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:28.904492    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-2b1f1f-5c54dc7758-wfdx7"]
Jul 30 15:38:28.908354 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:28.908334    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-2b1f1f-5c54dc7758-wfdx7"
Jul 30 15:38:28.910287 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:28.910261    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-2b1f1f\""
Jul 30 15:38:28.910399 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:28.910307    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-2b1f1f-dockercfg-49xbx\""
Jul 30 15:38:28.918876 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:28.918848    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-2b1f1f-5c54dc7758-wfdx7"]
Jul 30 15:38:29.070716 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:29.070674    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a2c0115-cc11-4717-b200-42d925bcd9ae-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-2b1f1f-5c54dc7758-wfdx7\" (UID: \"2a2c0115-cc11-4717-b200-42d925bcd9ae\") " pod="openshift-ingress/payload-processing-e2e-shared-b-2b1f1f-5c54dc7758-wfdx7"
Jul 30 15:38:29.070918 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:29.070756    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppkm6\" (UniqueName: \"kubernetes.io/projected/2a2c0115-cc11-4717-b200-42d925bcd9ae-kube-api-access-ppkm6\") pod \"payload-processing-e2e-shared-b-2b1f1f-5c54dc7758-wfdx7\" (UID: \"2a2c0115-cc11-4717-b200-42d925bcd9ae\") " pod="openshift-ingress/payload-processing-e2e-shared-b-2b1f1f-5c54dc7758-wfdx7"
Jul 30 15:38:29.171730 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:29.171632    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a2c0115-cc11-4717-b200-42d925bcd9ae-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-2b1f1f-5c54dc7758-wfdx7\" (UID: \"2a2c0115-cc11-4717-b200-42d925bcd9ae\") " pod="openshift-ingress/payload-processing-e2e-shared-b-2b1f1f-5c54dc7758-wfdx7"
Jul 30 15:38:29.171730 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:29.171688    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppkm6\" (UniqueName: \"kubernetes.io/projected/2a2c0115-cc11-4717-b200-42d925bcd9ae-kube-api-access-ppkm6\") pod \"payload-processing-e2e-shared-b-2b1f1f-5c54dc7758-wfdx7\" (UID: \"2a2c0115-cc11-4717-b200-42d925bcd9ae\") " pod="openshift-ingress/payload-processing-e2e-shared-b-2b1f1f-5c54dc7758-wfdx7"
Jul 30 15:38:29.172462 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:29.172443    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a2c0115-cc11-4717-b200-42d925bcd9ae-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-2b1f1f-5c54dc7758-wfdx7\" (UID: \"2a2c0115-cc11-4717-b200-42d925bcd9ae\") " pod="openshift-ingress/payload-processing-e2e-shared-b-2b1f1f-5c54dc7758-wfdx7"
Jul 30 15:38:29.182716 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:29.182683    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppkm6\" (UniqueName: \"kubernetes.io/projected/2a2c0115-cc11-4717-b200-42d925bcd9ae-kube-api-access-ppkm6\") pod \"payload-processing-e2e-shared-b-2b1f1f-5c54dc7758-wfdx7\" (UID: \"2a2c0115-cc11-4717-b200-42d925bcd9ae\") " pod="openshift-ingress/payload-processing-e2e-shared-b-2b1f1f-5c54dc7758-wfdx7"
Jul 30 15:38:29.220695 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:29.220666    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-2b1f1f-5c54dc7758-wfdx7"
Jul 30 15:38:29.356417 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:29.356389    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-2b1f1f-5c54dc7758-wfdx7"]
Jul 30 15:38:29.358642 ip-10-0-133-117 kubenswrapper[2578]: W0730 15:38:29.358611    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a2c0115_cc11_4717_b200_42d925bcd9ae.slice/crio-fffdb8a82f2e5d1c05425aea3915d33c81fefb71620476ffb327a1e5f22bb29c WatchSource:0}: Error finding container fffdb8a82f2e5d1c05425aea3915d33c81fefb71620476ffb327a1e5f22bb29c: Status 404 returned error can't find the container with id fffdb8a82f2e5d1c05425aea3915d33c81fefb71620476ffb327a1e5f22bb29c
Jul 30 15:38:30.102566 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:30.102528    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-2b1f1f-5c54dc7758-wfdx7" event={"ID":"2a2c0115-cc11-4717-b200-42d925bcd9ae","Type":"ContainerStarted","Data":"f3990364922e8d96e9edd10d6a4facb089aae99a77d85493592fa2ae6bf7aae2"}
Jul 30 15:38:30.102987 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:30.102607    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-2b1f1f-5c54dc7758-wfdx7" event={"ID":"2a2c0115-cc11-4717-b200-42d925bcd9ae","Type":"ContainerStarted","Data":"fffdb8a82f2e5d1c05425aea3915d33c81fefb71620476ffb327a1e5f22bb29c"}
Jul 30 15:38:30.102987 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:30.102759    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-2b1f1f-5c54dc7758-wfdx7"
Jul 30 15:38:30.118356 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:30.118307    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-2b1f1f-5c54dc7758-wfdx7" podStartSLOduration=2.118292154 podStartE2EDuration="2.118292154s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:38:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:38:30.117455413 +0000 UTC m=+3021.646393886" watchObservedRunningTime="2026-07-30 15:38:30.118292154 +0000 UTC m=+3021.647230664"
Jul 30 15:38:41.108972 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:41.108929    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-2b1f1f-5c54dc7758-wfdx7"
Jul 30 15:38:42.124288 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:42.124251    2578 patch_prober.go:28] interesting pod/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 30 15:38:42.124674 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:42.124318    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s" podUID="efe00480-68c7-4658-a34a-53ccefe2ba3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 30 15:38:43.175631 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:43.175599    2578 generic.go:356] "Generic (PLEG): container finished" podID="efe00480-68c7-4658-a34a-53ccefe2ba3e" containerID="0a1a0f8f7874632415fd76be600aaddba52cc5d4bf0e9c246f2c2f113f1925bc" exitCode=0
Jul 30 15:38:43.176034 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:43.175672    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s" event={"ID":"efe00480-68c7-4658-a34a-53ccefe2ba3e","Type":"ContainerDied","Data":"0a1a0f8f7874632415fd76be600aaddba52cc5d4bf0e9c246f2c2f113f1925bc"}
Jul 30 15:38:43.176076 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:43.176059    2578 scope.go:117] "RemoveContainer" containerID="0a1a0f8f7874632415fd76be600aaddba52cc5d4bf0e9c246f2c2f113f1925bc"
Jul 30 15:38:43.177077 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:43.177050    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:38:43.177165 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:43.177115    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:38:43.177165 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:43.177158    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 30 15:38:44.182702 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:44.182666    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s" event={"ID":"efe00480-68c7-4658-a34a-53ccefe2ba3e","Type":"ContainerStarted","Data":"5ee8c35aa05e2466d7e5efbefbc22062fd92eb3e837c644637ff2f1f83e2a4a5"}
Jul 30 15:38:45.122230 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:45.122195    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s"
Jul 30 15:38:45.122467 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:45.122443    2578 patch_prober.go:28] interesting pod/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Jul 30 15:38:45.122571 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:45.122494    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s" podUID="efe00480-68c7-4658-a34a-53ccefe2ba3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Jul 30 15:38:46.122920 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:46.122883    2578 patch_prober.go:28] interesting pod/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Jul 30 15:38:46.123343 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:46.122938    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s" podUID="efe00480-68c7-4658-a34a-53ccefe2ba3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Jul 30 15:38:47.122331 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:47.122301    2578 patch_prober.go:28] interesting pod/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Jul 30 15:38:47.122529 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:47.122351    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s" podUID="efe00480-68c7-4658-a34a-53ccefe2ba3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Jul 30 15:38:48.126088 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:48.126060    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s"
Jul 30 15:38:48.126503 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:48.126481    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s"
Jul 30 15:38:48.127331 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:38:48.127310    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s"
Jul 30 15:43:09.178577 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:43:09.178548    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4mjmp_de241220-5f27-4b0b-b7c6-6d4b350e24e1/console-operator/1.log"
Jul 30 15:43:09.204702 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:43:09.204678    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4mjmp_de241220-5f27-4b0b-b7c6-6d4b350e24e1/console-operator/1.log"
Jul 30 15:45:03.487473 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:45:03.487371    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29757060-g2z7x"]
Jul 30 15:45:03.491321 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:45:03.491291    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29757060-g2z7x"]
Jul 30 15:45:04.978949 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:45:04.978915    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8d2ee95-3d18-4385-9dcf-e67c75850a73" path="/var/lib/kubelet/pods/f8d2ee95-3d18-4385-9dcf-e67c75850a73/volumes"
Jul 30 15:45:09.397952 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:45:09.397921    2578 scope.go:117] "RemoveContainer" containerID="5a9b19883d174b14465418ee526d59d6ad18435eab8243c0f56e796db9f46207"
Jul 30 15:48:09.210392 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:48:09.210364    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4mjmp_de241220-5f27-4b0b-b7c6-6d4b350e24e1/console-operator/1.log"
Jul 30 15:48:09.234612 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:48:09.234589    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4mjmp_de241220-5f27-4b0b-b7c6-6d4b350e24e1/console-operator/1.log"
Jul 30 15:50:28.529884 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:50:28.529848    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-0d4073-7468ccbd48-wmkl7"]
Jul 30 15:50:28.533528 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:50:28.533508    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0d4073-7468ccbd48-wmkl7"
Jul 30 15:50:28.535659 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:50:28.535622    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-0d4073-dockercfg-mxwbf\""
Jul 30 15:50:28.535784 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:50:28.535704    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-0d4073\""
Jul 30 15:50:28.541783 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:50:28.541344    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-0d4073-7468ccbd48-wmkl7"]
Jul 30 15:50:28.545975 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:50:28.545950    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-0d4073-5f86d56468-h49zr"]
Jul 30 15:50:28.549563 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:50:28.549543    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-0d4073-5f86d56468-h49zr"
Jul 30 15:50:28.559740 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:50:28.559719    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-0d4073-5f86d56468-h49zr"]
Jul 30 15:50:28.664781 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:50:28.664737    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4191ac92-c0bc-4c8d-9b52-22e11641fcd7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-0d4073-7468ccbd48-wmkl7\" (UID: \"4191ac92-c0bc-4c8d-9b52-22e11641fcd7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0d4073-7468ccbd48-wmkl7"
Jul 30 15:50:28.664965 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:50:28.664887    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chmqb\" (UniqueName: \"kubernetes.io/projected/4191ac92-c0bc-4c8d-9b52-22e11641fcd7-kube-api-access-chmqb\") pod \"payload-pre-processing-e2e-ipp-b-0d4073-7468ccbd48-wmkl7\" (UID: \"4191ac92-c0bc-4c8d-9b52-22e11641fcd7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0d4073-7468ccbd48-wmkl7"
Jul 30 15:50:28.664965 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:50:28.664943    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4djz2\" (UniqueName: \"kubernetes.io/projected/62142c95-4d70-4c88-8b5b-c479850ce466-kube-api-access-4djz2\") pod \"payload-processing-e2e-ipp-b-0d4073-5f86d56468-h49zr\" (UID: \"62142c95-4d70-4c88-8b5b-c479850ce466\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-0d4073-5f86d56468-h49zr"
Jul 30 15:50:28.665058 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:50:28.664978    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/62142c95-4d70-4c88-8b5b-c479850ce466-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-0d4073-5f86d56468-h49zr\" (UID: \"62142c95-4d70-4c88-8b5b-c479850ce466\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-0d4073-5f86d56468-h49zr"
Jul 30 15:50:28.766099 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:50:28.766059    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4djz2\" (UniqueName: \"kubernetes.io/projected/62142c95-4d70-4c88-8b5b-c479850ce466-kube-api-access-4djz2\") pod \"payload-processing-e2e-ipp-b-0d4073-5f86d56468-h49zr\" (UID: \"62142c95-4d70-4c88-8b5b-c479850ce466\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-0d4073-5f86d56468-h49zr"
Jul 30 15:50:28.766332 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:50:28.766117    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/62142c95-4d70-4c88-8b5b-c479850ce466-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-0d4073-5f86d56468-h49zr\" (UID: \"62142c95-4d70-4c88-8b5b-c479850ce466\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-0d4073-5f86d56468-h49zr"
Jul 30 15:50:28.766332 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:50:28.766195    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4191ac92-c0bc-4c8d-9b52-22e11641fcd7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-0d4073-7468ccbd48-wmkl7\" (UID: \"4191ac92-c0bc-4c8d-9b52-22e11641fcd7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0d4073-7468ccbd48-wmkl7"
Jul 30 15:50:28.766332 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:50:28.766298    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chmqb\" (UniqueName: \"kubernetes.io/projected/4191ac92-c0bc-4c8d-9b52-22e11641fcd7-kube-api-access-chmqb\") pod \"payload-pre-processing-e2e-ipp-b-0d4073-7468ccbd48-wmkl7\" (UID: \"4191ac92-c0bc-4c8d-9b52-22e11641fcd7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0d4073-7468ccbd48-wmkl7"
Jul 30 15:50:28.766904 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:50:28.766868    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4191ac92-c0bc-4c8d-9b52-22e11641fcd7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-0d4073-7468ccbd48-wmkl7\" (UID: \"4191ac92-c0bc-4c8d-9b52-22e11641fcd7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0d4073-7468ccbd48-wmkl7"
Jul 30 15:50:28.766973 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:50:28.766874    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/62142c95-4d70-4c88-8b5b-c479850ce466-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-0d4073-5f86d56468-h49zr\" (UID: \"62142c95-4d70-4c88-8b5b-c479850ce466\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-0d4073-5f86d56468-h49zr"
Jul 30 15:50:28.773528 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:50:28.773499    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4djz2\" (UniqueName: \"kubernetes.io/projected/62142c95-4d70-4c88-8b5b-c479850ce466-kube-api-access-4djz2\") pod \"payload-processing-e2e-ipp-b-0d4073-5f86d56468-h49zr\" (UID: \"62142c95-4d70-4c88-8b5b-c479850ce466\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-0d4073-5f86d56468-h49zr"
Jul 30 15:50:28.773678 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:50:28.773510    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chmqb\" (UniqueName: \"kubernetes.io/projected/4191ac92-c0bc-4c8d-9b52-22e11641fcd7-kube-api-access-chmqb\") pod \"payload-pre-processing-e2e-ipp-b-0d4073-7468ccbd48-wmkl7\" (UID: \"4191ac92-c0bc-4c8d-9b52-22e11641fcd7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0d4073-7468ccbd48-wmkl7"
Jul 30 15:50:28.846531 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:50:28.846499    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0d4073-7468ccbd48-wmkl7"
Jul 30 15:50:28.861352 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:50:28.861324    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-0d4073-5f86d56468-h49zr"
Jul 30 15:50:29.012996 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:50:29.012850    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-0d4073-7468ccbd48-wmkl7"]
Jul 30 15:50:29.015619 ip-10-0-133-117 kubenswrapper[2578]: W0730 15:50:29.015568    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4191ac92_c0bc_4c8d_9b52_22e11641fcd7.slice/crio-1fdbae20589f804ec6ee6346a77d3bf39e20d172850aea533e092ae49c586291 WatchSource:0}: Error finding container 1fdbae20589f804ec6ee6346a77d3bf39e20d172850aea533e092ae49c586291: Status 404 returned error can't find the container with id 1fdbae20589f804ec6ee6346a77d3bf39e20d172850aea533e092ae49c586291
Jul 30 15:50:29.043025 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:50:29.042991    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-0d4073-5f86d56468-h49zr"]
Jul 30 15:50:29.053223 ip-10-0-133-117 kubenswrapper[2578]: W0730 15:50:29.053195    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62142c95_4d70_4c88_8b5b_c479850ce466.slice/crio-418ebdaf5ea3941194e28ff327cee95db77e005de180eb67cd1c8b4d3cfefe28 WatchSource:0}: Error finding container 418ebdaf5ea3941194e28ff327cee95db77e005de180eb67cd1c8b4d3cfefe28: Status 404 returned error can't find the container with id 418ebdaf5ea3941194e28ff327cee95db77e005de180eb67cd1c8b4d3cfefe28
Jul 30 15:50:29.631085 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:50:29.631038    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-0d4073-5f86d56468-h49zr" event={"ID":"62142c95-4d70-4c88-8b5b-c479850ce466","Type":"ContainerStarted","Data":"8ba2e138ae2a7fde051539d90baa6fff53df235c38663fbfbe450ddb40fb505a"}
Jul 30 15:50:29.631565 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:50:29.631113    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-0d4073-5f86d56468-h49zr"
Jul 30 15:50:29.631565 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:50:29.631140    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-0d4073-5f86d56468-h49zr" event={"ID":"62142c95-4d70-4c88-8b5b-c479850ce466","Type":"ContainerStarted","Data":"418ebdaf5ea3941194e28ff327cee95db77e005de180eb67cd1c8b4d3cfefe28"}
Jul 30 15:50:29.632668 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:50:29.632639    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0d4073-7468ccbd48-wmkl7" event={"ID":"4191ac92-c0bc-4c8d-9b52-22e11641fcd7","Type":"ContainerStarted","Data":"df4dec5601e86de764a39d8d530f59449848ac9a1b4009988414bec25f34d0e7"}
Jul 30 15:50:29.632777 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:50:29.632688    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0d4073-7468ccbd48-wmkl7" event={"ID":"4191ac92-c0bc-4c8d-9b52-22e11641fcd7","Type":"ContainerStarted","Data":"1fdbae20589f804ec6ee6346a77d3bf39e20d172850aea533e092ae49c586291"}
Jul 30 15:50:29.632777 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:50:29.632712    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0d4073-7468ccbd48-wmkl7"
Jul 30 15:50:29.651696 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:50:29.651647    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-0d4073-5f86d56468-h49zr" podStartSLOduration=1.6516325809999999 podStartE2EDuration="1.651632581s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:50:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:50:29.64868824 +0000 UTC m=+3741.177626712" watchObservedRunningTime="2026-07-30 15:50:29.651632581 +0000 UTC m=+3741.180571055"
Jul 30 15:50:29.666733 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:50:29.666689    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0d4073-7468ccbd48-wmkl7" podStartSLOduration=1.666675287 podStartE2EDuration="1.666675287s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:50:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:50:29.664360672 +0000 UTC m=+3741.193299145" watchObservedRunningTime="2026-07-30 15:50:29.666675287 +0000 UTC m=+3741.195613760"
Jul 30 15:50:40.641336 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:50:40.641305    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0d4073-7468ccbd48-wmkl7"
Jul 30 15:50:40.641769 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:50:40.641377    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-0d4073-5f86d56468-h49zr"
Jul 30 15:50:58.917521 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:50:58.917481    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-bjkzn_2cb2ce43-84fb-480e-9b51-ea6c36a181b2/payload-processing/0.log"
Jul 30 15:50:59.047679 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:50:59.047652    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-0d4073-5f86d56468-h49zr_62142c95-4d70-4c88-8b5b-c479850ce466/payload-processing/0.log"
Jul 30 15:52:04.690359 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:04.690328    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-bjkzn_2cb2ce43-84fb-480e-9b51-ea6c36a181b2/payload-processing/0.log"
Jul 30 15:52:04.839704 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:04.839671    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-0d4073-5f86d56468-h49zr_62142c95-4d70-4c88-8b5b-c479850ce466/payload-processing/0.log"
Jul 30 15:52:09.231267 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:09.231232    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2d48ec-7b59468cbd-qf4cn"]
Jul 30 15:52:09.234810 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:09.234785    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2d48ec-7b59468cbd-qf4cn"
Jul 30 15:52:09.236771 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:09.236744    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-2d48ec-serving-cert\""
Jul 30 15:52:09.249244 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:09.249217    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2d48ec-7b59468cbd-qf4cn"]
Jul 30 15:52:09.356319 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:09.356281    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf4b2e33-34b0-4d2b-9e30-57d132f17cb3-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-2d48ec-7b59468cbd-qf4cn\" (UID: \"bf4b2e33-34b0-4d2b-9e30-57d132f17cb3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2d48ec-7b59468cbd-qf4cn"
Jul 30 15:52:09.356492 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:09.356327    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqnxf\" (UniqueName: \"kubernetes.io/projected/bf4b2e33-34b0-4d2b-9e30-57d132f17cb3-kube-api-access-gqnxf\") pod \"maas-api-e2e-ipp-cleanup-2d48ec-7b59468cbd-qf4cn\" (UID: \"bf4b2e33-34b0-4d2b-9e30-57d132f17cb3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2d48ec-7b59468cbd-qf4cn"
Jul 30 15:52:09.356492 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:09.356445    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf4b2e33-34b0-4d2b-9e30-57d132f17cb3-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-2d48ec-7b59468cbd-qf4cn\" (UID: \"bf4b2e33-34b0-4d2b-9e30-57d132f17cb3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2d48ec-7b59468cbd-qf4cn"
Jul 30 15:52:09.457218 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:09.457178    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqnxf\" (UniqueName: \"kubernetes.io/projected/bf4b2e33-34b0-4d2b-9e30-57d132f17cb3-kube-api-access-gqnxf\") pod \"maas-api-e2e-ipp-cleanup-2d48ec-7b59468cbd-qf4cn\" (UID: \"bf4b2e33-34b0-4d2b-9e30-57d132f17cb3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2d48ec-7b59468cbd-qf4cn"
Jul 30 15:52:09.457412 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:09.457294    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf4b2e33-34b0-4d2b-9e30-57d132f17cb3-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-2d48ec-7b59468cbd-qf4cn\" (UID: \"bf4b2e33-34b0-4d2b-9e30-57d132f17cb3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2d48ec-7b59468cbd-qf4cn"
Jul 30 15:52:09.457412 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:09.457345    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf4b2e33-34b0-4d2b-9e30-57d132f17cb3-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-2d48ec-7b59468cbd-qf4cn\" (UID: \"bf4b2e33-34b0-4d2b-9e30-57d132f17cb3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2d48ec-7b59468cbd-qf4cn"
Jul 30 15:52:09.457969 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:09.457946    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf4b2e33-34b0-4d2b-9e30-57d132f17cb3-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-2d48ec-7b59468cbd-qf4cn\" (UID: \"bf4b2e33-34b0-4d2b-9e30-57d132f17cb3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2d48ec-7b59468cbd-qf4cn"
Jul 30 15:52:09.460082 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:09.460055    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf4b2e33-34b0-4d2b-9e30-57d132f17cb3-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-2d48ec-7b59468cbd-qf4cn\" (UID: \"bf4b2e33-34b0-4d2b-9e30-57d132f17cb3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2d48ec-7b59468cbd-qf4cn"
Jul 30 15:52:09.471060 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:09.471023    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqnxf\" (UniqueName: \"kubernetes.io/projected/bf4b2e33-34b0-4d2b-9e30-57d132f17cb3-kube-api-access-gqnxf\") pod \"maas-api-e2e-ipp-cleanup-2d48ec-7b59468cbd-qf4cn\" (UID: \"bf4b2e33-34b0-4d2b-9e30-57d132f17cb3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2d48ec-7b59468cbd-qf4cn"
Jul 30 15:52:09.552312 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:09.552280    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2d48ec-7b59468cbd-qf4cn"
Jul 30 15:52:09.698015 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:09.697986    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2d48ec-7b59468cbd-qf4cn"]
Jul 30 15:52:09.700286 ip-10-0-133-117 kubenswrapper[2578]: W0730 15:52:09.700256    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf4b2e33_34b0_4d2b_9e30_57d132f17cb3.slice/crio-020cda492243f3bfeb49feea5ade8914e81a7a943020fcc4cc4094e12f6c56f6 WatchSource:0}: Error finding container 020cda492243f3bfeb49feea5ade8914e81a7a943020fcc4cc4094e12f6c56f6: Status 404 returned error can't find the container with id 020cda492243f3bfeb49feea5ade8914e81a7a943020fcc4cc4094e12f6c56f6
Jul 30 15:52:09.702297 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:09.702275    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 15:52:09.898140 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:09.898054    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-2d48ec-58df6cff6d-slf95"]
Jul 30 15:52:09.902633 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:09.902616    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-2d48ec-58df6cff6d-slf95"
Jul 30 15:52:09.904670 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:09.904652    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-2d48ec\""
Jul 30 15:52:09.910441 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:09.910416    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-2d48ec-58df6cff6d-slf95"]
Jul 30 15:52:09.937059 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:09.937025    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-2d48ec-69c575c68d-hhpls"]
Jul 30 15:52:09.940717 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:09.940692    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-2d48ec-69c575c68d-hhpls"
Jul 30 15:52:09.942475 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:09.942452    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-2d48ec-dockercfg-mcs2x\""
Jul 30 15:52:09.949482 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:09.949455    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-2d48ec-69c575c68d-hhpls"]
Jul 30 15:52:09.962431 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:09.962403    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9n7h\" (UniqueName: \"kubernetes.io/projected/5d9bfd52-6865-4f15-b84f-56748f8f4f58-kube-api-access-c9n7h\") pod \"payload-pre-processing-e2e-ipp-cleanup-2d48ec-58df6cff6d-slf95\" (UID: \"5d9bfd52-6865-4f15-b84f-56748f8f4f58\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-2d48ec-58df6cff6d-slf95"
Jul 30 15:52:09.962578 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:09.962509    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d9bfd52-6865-4f15-b84f-56748f8f4f58-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-2d48ec-58df6cff6d-slf95\" (UID: \"5d9bfd52-6865-4f15-b84f-56748f8f4f58\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-2d48ec-58df6cff6d-slf95"
Jul 30 15:52:10.064130 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:10.064089    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7vhs\" (UniqueName: \"kubernetes.io/projected/7ff6a146-1368-4741-8f31-9cb1f5814fcb-kube-api-access-t7vhs\") pod \"payload-processing-e2e-ipp-cleanup-2d48ec-69c575c68d-hhpls\" (UID: \"7ff6a146-1368-4741-8f31-9cb1f5814fcb\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-2d48ec-69c575c68d-hhpls"
Jul 30 15:52:10.064329 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:10.064175    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ff6a146-1368-4741-8f31-9cb1f5814fcb-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-2d48ec-69c575c68d-hhpls\" (UID: \"7ff6a146-1368-4741-8f31-9cb1f5814fcb\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-2d48ec-69c575c68d-hhpls"
Jul 30 15:52:10.064329 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:10.064201    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9n7h\" (UniqueName: \"kubernetes.io/projected/5d9bfd52-6865-4f15-b84f-56748f8f4f58-kube-api-access-c9n7h\") pod \"payload-pre-processing-e2e-ipp-cleanup-2d48ec-58df6cff6d-slf95\" (UID: \"5d9bfd52-6865-4f15-b84f-56748f8f4f58\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-2d48ec-58df6cff6d-slf95"
Jul 30 15:52:10.064487 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:10.064358    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d9bfd52-6865-4f15-b84f-56748f8f4f58-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-2d48ec-58df6cff6d-slf95\" (UID: \"5d9bfd52-6865-4f15-b84f-56748f8f4f58\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-2d48ec-58df6cff6d-slf95"
Jul 30 15:52:10.065008 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:10.064989    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d9bfd52-6865-4f15-b84f-56748f8f4f58-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-2d48ec-58df6cff6d-slf95\" (UID: \"5d9bfd52-6865-4f15-b84f-56748f8f4f58\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-2d48ec-58df6cff6d-slf95"
Jul 30 15:52:10.073129 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:10.073100    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9n7h\" (UniqueName: \"kubernetes.io/projected/5d9bfd52-6865-4f15-b84f-56748f8f4f58-kube-api-access-c9n7h\") pod \"payload-pre-processing-e2e-ipp-cleanup-2d48ec-58df6cff6d-slf95\" (UID: \"5d9bfd52-6865-4f15-b84f-56748f8f4f58\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-2d48ec-58df6cff6d-slf95"
Jul 30 15:52:10.145223 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:10.145184    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2d48ec-7b59468cbd-qf4cn" event={"ID":"bf4b2e33-34b0-4d2b-9e30-57d132f17cb3","Type":"ContainerStarted","Data":"020cda492243f3bfeb49feea5ade8914e81a7a943020fcc4cc4094e12f6c56f6"}
Jul 30 15:52:10.164889 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:10.164861    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7vhs\" (UniqueName: \"kubernetes.io/projected/7ff6a146-1368-4741-8f31-9cb1f5814fcb-kube-api-access-t7vhs\") pod \"payload-processing-e2e-ipp-cleanup-2d48ec-69c575c68d-hhpls\" (UID: \"7ff6a146-1368-4741-8f31-9cb1f5814fcb\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-2d48ec-69c575c68d-hhpls"
Jul 30 15:52:10.165006 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:10.164918    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ff6a146-1368-4741-8f31-9cb1f5814fcb-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-2d48ec-69c575c68d-hhpls\" (UID: \"7ff6a146-1368-4741-8f31-9cb1f5814fcb\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-2d48ec-69c575c68d-hhpls"
Jul 30 15:52:10.165609 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:10.165589    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ff6a146-1368-4741-8f31-9cb1f5814fcb-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-2d48ec-69c575c68d-hhpls\" (UID: \"7ff6a146-1368-4741-8f31-9cb1f5814fcb\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-2d48ec-69c575c68d-hhpls"
Jul 30 15:52:10.176823 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:10.176801    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7vhs\" (UniqueName: \"kubernetes.io/projected/7ff6a146-1368-4741-8f31-9cb1f5814fcb-kube-api-access-t7vhs\") pod \"payload-processing-e2e-ipp-cleanup-2d48ec-69c575c68d-hhpls\" (UID: \"7ff6a146-1368-4741-8f31-9cb1f5814fcb\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-2d48ec-69c575c68d-hhpls"
Jul 30 15:52:10.214884 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:10.214855    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-2d48ec-58df6cff6d-slf95"
Jul 30 15:52:10.252803 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:10.252769    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-2d48ec-69c575c68d-hhpls"
Jul 30 15:52:10.581729 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:10.581698    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-2d48ec-58df6cff6d-slf95"]
Jul 30 15:52:10.582629 ip-10-0-133-117 kubenswrapper[2578]: W0730 15:52:10.582600    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d9bfd52_6865_4f15_b84f_56748f8f4f58.slice/crio-05530374be44725765a88a78bf568e22c478fd61eff51da5b3c1d22a8f0ce960 WatchSource:0}: Error finding container 05530374be44725765a88a78bf568e22c478fd61eff51da5b3c1d22a8f0ce960: Status 404 returned error can't find the container with id 05530374be44725765a88a78bf568e22c478fd61eff51da5b3c1d22a8f0ce960
Jul 30 15:52:10.622885 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:10.622857    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-2d48ec-69c575c68d-hhpls"]
Jul 30 15:52:10.624192 ip-10-0-133-117 kubenswrapper[2578]: W0730 15:52:10.624162    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ff6a146_1368_4741_8f31_9cb1f5814fcb.slice/crio-34aafa3e3e3ff9ebe904c286cb9cd872eb136300f44563ef15f21f6db2cb6df5 WatchSource:0}: Error finding container 34aafa3e3e3ff9ebe904c286cb9cd872eb136300f44563ef15f21f6db2cb6df5: Status 404 returned error can't find the container with id 34aafa3e3e3ff9ebe904c286cb9cd872eb136300f44563ef15f21f6db2cb6df5
Jul 30 15:52:11.152518 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:11.152455    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2d48ec-7b59468cbd-qf4cn" event={"ID":"bf4b2e33-34b0-4d2b-9e30-57d132f17cb3","Type":"ContainerStarted","Data":"8ad27c946efb8946d7b12ebf6a9418557dbc6f0fd763a05d6edde2fe51baafb0"}
Jul 30 15:52:11.152775 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:11.152752    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2d48ec-7b59468cbd-qf4cn"
Jul 30 15:52:11.154184 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:11.154136    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-2d48ec-69c575c68d-hhpls" event={"ID":"7ff6a146-1368-4741-8f31-9cb1f5814fcb","Type":"ContainerStarted","Data":"f947f27908eb1d8e653a2078544114d96d0bcb76d16097b5b2c8f263d89ae1cb"}
Jul 30 15:52:11.154324 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:11.154206    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-2d48ec-69c575c68d-hhpls" event={"ID":"7ff6a146-1368-4741-8f31-9cb1f5814fcb","Type":"ContainerStarted","Data":"34aafa3e3e3ff9ebe904c286cb9cd872eb136300f44563ef15f21f6db2cb6df5"}
Jul 30 15:52:11.154324 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:11.154235    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-2d48ec-69c575c68d-hhpls"
Jul 30 15:52:11.155776 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:11.155746    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-2d48ec-58df6cff6d-slf95" event={"ID":"5d9bfd52-6865-4f15-b84f-56748f8f4f58","Type":"ContainerStarted","Data":"8b3234cb44e5d038bd8ebb6e3c720d5a187f2d4cffd179a237830b8e2f3c3370"}
Jul 30 15:52:11.155884 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:11.155789    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-2d48ec-58df6cff6d-slf95" event={"ID":"5d9bfd52-6865-4f15-b84f-56748f8f4f58","Type":"ContainerStarted","Data":"05530374be44725765a88a78bf568e22c478fd61eff51da5b3c1d22a8f0ce960"}
Jul 30 15:52:11.155884 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:11.155831    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-2d48ec-58df6cff6d-slf95"
Jul 30 15:52:11.169365 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:11.169320    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2d48ec-7b59468cbd-qf4cn" podStartSLOduration=1.725450526 podStartE2EDuration="2.169304489s" totalImagesPullingTime="443.853963ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:52:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:52:11.167410679 +0000 UTC m=+3842.696349154" watchObservedRunningTime="2026-07-30 15:52:11.169304489 +0000 UTC m=+3842.698242963"
Jul 30 15:52:11.182811 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:11.182759    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-2d48ec-58df6cff6d-slf95" podStartSLOduration=2.182741376 podStartE2EDuration="2.182741376s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:52:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:52:11.181925451 +0000 UTC m=+3842.710863925" watchObservedRunningTime="2026-07-30 15:52:11.182741376 +0000 UTC m=+3842.711679856"
Jul 30 15:52:11.198246 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:11.198198    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-2d48ec-69c575c68d-hhpls" podStartSLOduration=2.198183436 podStartE2EDuration="2.198183436s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:52:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:52:11.196541392 +0000 UTC m=+3842.725479880" watchObservedRunningTime="2026-07-30 15:52:11.198183436 +0000 UTC m=+3842.727121910"
Jul 30 15:52:17.167859 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:17.167829    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2d48ec-7b59468cbd-qf4cn"
Jul 30 15:52:22.164847 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:22.164809    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-2d48ec-69c575c68d-hhpls"
Jul 30 15:52:22.165247 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:22.164985    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-2d48ec-58df6cff6d-slf95"
Jul 30 15:52:39.757772 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:39.757685    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2d48ec-7b59468cbd-qf4cn"]
Jul 30 15:52:39.758361 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:39.757923    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2d48ec-7b59468cbd-qf4cn" podUID="bf4b2e33-34b0-4d2b-9e30-57d132f17cb3" containerName="maas-api" containerID="cri-o://8ad27c946efb8946d7b12ebf6a9418557dbc6f0fd763a05d6edde2fe51baafb0" gracePeriod=30
Jul 30 15:52:39.816496 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:39.816464    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-2d48ec-69c575c68d-hhpls"]
Jul 30 15:52:39.816750 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:39.816723    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-2d48ec-69c575c68d-hhpls" podUID="7ff6a146-1368-4741-8f31-9cb1f5814fcb" containerName="payload-processing" containerID="cri-o://f947f27908eb1d8e653a2078544114d96d0bcb76d16097b5b2c8f263d89ae1cb" gracePeriod=30
Jul 30 15:52:39.823320 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:39.823291    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-2d48ec-58df6cff6d-slf95"]
Jul 30 15:52:39.823538 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:39.823504    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-2d48ec-58df6cff6d-slf95" podUID="5d9bfd52-6865-4f15-b84f-56748f8f4f58" containerName="payload-pre-processing" containerID="cri-o://8b3234cb44e5d038bd8ebb6e3c720d5a187f2d4cffd179a237830b8e2f3c3370" gracePeriod=30
Jul 30 15:52:40.101703 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:40.101668    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-2d48ec-69c575c68d-hhpls"
Jul 30 15:52:40.121807 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:40.121779    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2d48ec-7b59468cbd-qf4cn"
Jul 30 15:52:40.144267 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:40.144240    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-2d48ec-58df6cff6d-slf95"
Jul 30 15:52:40.167195 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:40.167170    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t7vhs\" (UniqueName: \"kubernetes.io/projected/7ff6a146-1368-4741-8f31-9cb1f5814fcb-kube-api-access-t7vhs\") pod \"7ff6a146-1368-4741-8f31-9cb1f5814fcb\" (UID: \"7ff6a146-1368-4741-8f31-9cb1f5814fcb\") "
Jul 30 15:52:40.167357 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:40.167275    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gqnxf\" (UniqueName: \"kubernetes.io/projected/bf4b2e33-34b0-4d2b-9e30-57d132f17cb3-kube-api-access-gqnxf\") pod \"bf4b2e33-34b0-4d2b-9e30-57d132f17cb3\" (UID: \"bf4b2e33-34b0-4d2b-9e30-57d132f17cb3\") "
Jul 30 15:52:40.167357 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:40.167304    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf4b2e33-34b0-4d2b-9e30-57d132f17cb3-maas-api-tls\") pod \"bf4b2e33-34b0-4d2b-9e30-57d132f17cb3\" (UID: \"bf4b2e33-34b0-4d2b-9e30-57d132f17cb3\") "
Jul 30 15:52:40.167481 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:40.167374    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf4b2e33-34b0-4d2b-9e30-57d132f17cb3-openshift-service-ca\") pod \"bf4b2e33-34b0-4d2b-9e30-57d132f17cb3\" (UID: \"bf4b2e33-34b0-4d2b-9e30-57d132f17cb3\") "
Jul 30 15:52:40.167481 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:40.167430    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ff6a146-1368-4741-8f31-9cb1f5814fcb-plugins-config-volume\") pod \"7ff6a146-1368-4741-8f31-9cb1f5814fcb\" (UID: \"7ff6a146-1368-4741-8f31-9cb1f5814fcb\") "
Jul 30 15:52:40.168045 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:40.167922    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf4b2e33-34b0-4d2b-9e30-57d132f17cb3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bf4b2e33-34b0-4d2b-9e30-57d132f17cb3" (UID: "bf4b2e33-34b0-4d2b-9e30-57d132f17cb3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:52:40.168045 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:40.168008    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ff6a146-1368-4741-8f31-9cb1f5814fcb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7ff6a146-1368-4741-8f31-9cb1f5814fcb" (UID: "7ff6a146-1368-4741-8f31-9cb1f5814fcb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:52:40.169364 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:40.169337    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ff6a146-1368-4741-8f31-9cb1f5814fcb-kube-api-access-t7vhs" (OuterVolumeSpecName: "kube-api-access-t7vhs") pod "7ff6a146-1368-4741-8f31-9cb1f5814fcb" (UID: "7ff6a146-1368-4741-8f31-9cb1f5814fcb"). InnerVolumeSpecName "kube-api-access-t7vhs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:52:40.169529 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:40.169511    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf4b2e33-34b0-4d2b-9e30-57d132f17cb3-kube-api-access-gqnxf" (OuterVolumeSpecName: "kube-api-access-gqnxf") pod "bf4b2e33-34b0-4d2b-9e30-57d132f17cb3" (UID: "bf4b2e33-34b0-4d2b-9e30-57d132f17cb3"). InnerVolumeSpecName "kube-api-access-gqnxf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:52:40.169769 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:40.169750    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf4b2e33-34b0-4d2b-9e30-57d132f17cb3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bf4b2e33-34b0-4d2b-9e30-57d132f17cb3" (UID: "bf4b2e33-34b0-4d2b-9e30-57d132f17cb3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 15:52:40.268786 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:40.268756    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c9n7h\" (UniqueName: \"kubernetes.io/projected/5d9bfd52-6865-4f15-b84f-56748f8f4f58-kube-api-access-c9n7h\") pod \"5d9bfd52-6865-4f15-b84f-56748f8f4f58\" (UID: \"5d9bfd52-6865-4f15-b84f-56748f8f4f58\") "
Jul 30 15:52:40.268946 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:40.268913    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d9bfd52-6865-4f15-b84f-56748f8f4f58-plugins-config-volume\") pod \"5d9bfd52-6865-4f15-b84f-56748f8f4f58\" (UID: \"5d9bfd52-6865-4f15-b84f-56748f8f4f58\") "
Jul 30 15:52:40.269273 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:40.269252    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf4b2e33-34b0-4d2b-9e30-57d132f17cb3-openshift-service-ca\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:52:40.269273 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:40.269263    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d9bfd52-6865-4f15-b84f-56748f8f4f58-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5d9bfd52-6865-4f15-b84f-56748f8f4f58" (UID: "5d9bfd52-6865-4f15-b84f-56748f8f4f58"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:52:40.269438 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:40.269272    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ff6a146-1368-4741-8f31-9cb1f5814fcb-plugins-config-volume\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:52:40.269438 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:40.269305    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t7vhs\" (UniqueName: \"kubernetes.io/projected/7ff6a146-1368-4741-8f31-9cb1f5814fcb-kube-api-access-t7vhs\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:52:40.269438 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:40.269316    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gqnxf\" (UniqueName: \"kubernetes.io/projected/bf4b2e33-34b0-4d2b-9e30-57d132f17cb3-kube-api-access-gqnxf\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:52:40.269438 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:40.269326    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf4b2e33-34b0-4d2b-9e30-57d132f17cb3-maas-api-tls\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:52:40.270837 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:40.270813    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d9bfd52-6865-4f15-b84f-56748f8f4f58-kube-api-access-c9n7h" (OuterVolumeSpecName: "kube-api-access-c9n7h") pod "5d9bfd52-6865-4f15-b84f-56748f8f4f58" (UID: "5d9bfd52-6865-4f15-b84f-56748f8f4f58"). InnerVolumeSpecName "kube-api-access-c9n7h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:52:40.310200 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:40.310173    2578 generic.go:356] "Generic (PLEG): container finished" podID="bf4b2e33-34b0-4d2b-9e30-57d132f17cb3" containerID="8ad27c946efb8946d7b12ebf6a9418557dbc6f0fd763a05d6edde2fe51baafb0" exitCode=0
Jul 30 15:52:40.310339 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:40.310213    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2d48ec-7b59468cbd-qf4cn"
Jul 30 15:52:40.310339 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:40.310253    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2d48ec-7b59468cbd-qf4cn" event={"ID":"bf4b2e33-34b0-4d2b-9e30-57d132f17cb3","Type":"ContainerDied","Data":"8ad27c946efb8946d7b12ebf6a9418557dbc6f0fd763a05d6edde2fe51baafb0"}
Jul 30 15:52:40.310339 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:40.310303    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2d48ec-7b59468cbd-qf4cn" event={"ID":"bf4b2e33-34b0-4d2b-9e30-57d132f17cb3","Type":"ContainerDied","Data":"020cda492243f3bfeb49feea5ade8914e81a7a943020fcc4cc4094e12f6c56f6"}
Jul 30 15:52:40.310339 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:40.310334    2578 scope.go:117] "RemoveContainer" containerID="8ad27c946efb8946d7b12ebf6a9418557dbc6f0fd763a05d6edde2fe51baafb0"
Jul 30 15:52:40.311603 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:40.311578    2578 generic.go:356] "Generic (PLEG): container finished" podID="7ff6a146-1368-4741-8f31-9cb1f5814fcb" containerID="f947f27908eb1d8e653a2078544114d96d0bcb76d16097b5b2c8f263d89ae1cb" exitCode=0
Jul 30 15:52:40.311711 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:40.311618    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-2d48ec-69c575c68d-hhpls"
Jul 30 15:52:40.311770 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:40.311609    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-2d48ec-69c575c68d-hhpls" event={"ID":"7ff6a146-1368-4741-8f31-9cb1f5814fcb","Type":"ContainerDied","Data":"f947f27908eb1d8e653a2078544114d96d0bcb76d16097b5b2c8f263d89ae1cb"}
Jul 30 15:52:40.311841 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:40.311822    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-2d48ec-69c575c68d-hhpls" event={"ID":"7ff6a146-1368-4741-8f31-9cb1f5814fcb","Type":"ContainerDied","Data":"34aafa3e3e3ff9ebe904c286cb9cd872eb136300f44563ef15f21f6db2cb6df5"}
Jul 30 15:52:40.313071 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:40.313047    2578 generic.go:356] "Generic (PLEG): container finished" podID="5d9bfd52-6865-4f15-b84f-56748f8f4f58" containerID="8b3234cb44e5d038bd8ebb6e3c720d5a187f2d4cffd179a237830b8e2f3c3370" exitCode=0
Jul 30 15:52:40.313232 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:40.313100    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-2d48ec-58df6cff6d-slf95" event={"ID":"5d9bfd52-6865-4f15-b84f-56748f8f4f58","Type":"ContainerDied","Data":"8b3234cb44e5d038bd8ebb6e3c720d5a187f2d4cffd179a237830b8e2f3c3370"}
Jul 30 15:52:40.313232 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:40.313132    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-2d48ec-58df6cff6d-slf95" event={"ID":"5d9bfd52-6865-4f15-b84f-56748f8f4f58","Type":"ContainerDied","Data":"05530374be44725765a88a78bf568e22c478fd61eff51da5b3c1d22a8f0ce960"}
Jul 30 15:52:40.313232 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:40.313103    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-2d48ec-58df6cff6d-slf95"
Jul 30 15:52:40.322036 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:40.321944    2578 scope.go:117] "RemoveContainer" containerID="8ad27c946efb8946d7b12ebf6a9418557dbc6f0fd763a05d6edde2fe51baafb0"
Jul 30 15:52:40.322385 ip-10-0-133-117 kubenswrapper[2578]: E0730 15:52:40.322352    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ad27c946efb8946d7b12ebf6a9418557dbc6f0fd763a05d6edde2fe51baafb0\": container with ID starting with 8ad27c946efb8946d7b12ebf6a9418557dbc6f0fd763a05d6edde2fe51baafb0 not found: ID does not exist" containerID="8ad27c946efb8946d7b12ebf6a9418557dbc6f0fd763a05d6edde2fe51baafb0"
Jul 30 15:52:40.322488 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:40.322386    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ad27c946efb8946d7b12ebf6a9418557dbc6f0fd763a05d6edde2fe51baafb0"} err="failed to get container status \"8ad27c946efb8946d7b12ebf6a9418557dbc6f0fd763a05d6edde2fe51baafb0\": rpc error: code = NotFound desc = could not find container \"8ad27c946efb8946d7b12ebf6a9418557dbc6f0fd763a05d6edde2fe51baafb0\": container with ID starting with 8ad27c946efb8946d7b12ebf6a9418557dbc6f0fd763a05d6edde2fe51baafb0 not found: ID does not exist"
Jul 30 15:52:40.322488 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:40.322405    2578 scope.go:117] "RemoveContainer" containerID="f947f27908eb1d8e653a2078544114d96d0bcb76d16097b5b2c8f263d89ae1cb"
Jul 30 15:52:40.331208 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:40.331186    2578 scope.go:117] "RemoveContainer" containerID="f947f27908eb1d8e653a2078544114d96d0bcb76d16097b5b2c8f263d89ae1cb"
Jul 30 15:52:40.331435 ip-10-0-133-117 kubenswrapper[2578]: E0730 15:52:40.331418    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f947f27908eb1d8e653a2078544114d96d0bcb76d16097b5b2c8f263d89ae1cb\": container with ID starting with f947f27908eb1d8e653a2078544114d96d0bcb76d16097b5b2c8f263d89ae1cb not found: ID does not exist" containerID="f947f27908eb1d8e653a2078544114d96d0bcb76d16097b5b2c8f263d89ae1cb"
Jul 30 15:52:40.331493 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:40.331440    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f947f27908eb1d8e653a2078544114d96d0bcb76d16097b5b2c8f263d89ae1cb"} err="failed to get container status \"f947f27908eb1d8e653a2078544114d96d0bcb76d16097b5b2c8f263d89ae1cb\": rpc error: code = NotFound desc = could not find container \"f947f27908eb1d8e653a2078544114d96d0bcb76d16097b5b2c8f263d89ae1cb\": container with ID starting with f947f27908eb1d8e653a2078544114d96d0bcb76d16097b5b2c8f263d89ae1cb not found: ID does not exist"
Jul 30 15:52:40.331493 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:40.331456    2578 scope.go:117] "RemoveContainer" containerID="8b3234cb44e5d038bd8ebb6e3c720d5a187f2d4cffd179a237830b8e2f3c3370"
Jul 30 15:52:40.338007 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:40.337983    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2d48ec-7b59468cbd-qf4cn"]
Jul 30 15:52:40.339992 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:40.339973    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2d48ec-7b59468cbd-qf4cn"]
Jul 30 15:52:40.340341 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:40.340329    2578 scope.go:117] "RemoveContainer" containerID="8b3234cb44e5d038bd8ebb6e3c720d5a187f2d4cffd179a237830b8e2f3c3370"
Jul 30 15:52:40.340592 ip-10-0-133-117 kubenswrapper[2578]: E0730 15:52:40.340576    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b3234cb44e5d038bd8ebb6e3c720d5a187f2d4cffd179a237830b8e2f3c3370\": container with ID starting with 8b3234cb44e5d038bd8ebb6e3c720d5a187f2d4cffd179a237830b8e2f3c3370 not found: ID does not exist" containerID="8b3234cb44e5d038bd8ebb6e3c720d5a187f2d4cffd179a237830b8e2f3c3370"
Jul 30 15:52:40.340665 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:40.340595    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b3234cb44e5d038bd8ebb6e3c720d5a187f2d4cffd179a237830b8e2f3c3370"} err="failed to get container status \"8b3234cb44e5d038bd8ebb6e3c720d5a187f2d4cffd179a237830b8e2f3c3370\": rpc error: code = NotFound desc = could not find container \"8b3234cb44e5d038bd8ebb6e3c720d5a187f2d4cffd179a237830b8e2f3c3370\": container with ID starting with 8b3234cb44e5d038bd8ebb6e3c720d5a187f2d4cffd179a237830b8e2f3c3370 not found: ID does not exist"
Jul 30 15:52:40.349959 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:40.349935    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-2d48ec-58df6cff6d-slf95"]
Jul 30 15:52:40.353521 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:40.353503    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-2d48ec-58df6cff6d-slf95"]
Jul 30 15:52:40.362748 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:40.362698    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-2d48ec-69c575c68d-hhpls"]
Jul 30 15:52:40.364534 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:40.364512    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-2d48ec-69c575c68d-hhpls"]
Jul 30 15:52:40.369931 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:40.369912    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c9n7h\" (UniqueName: \"kubernetes.io/projected/5d9bfd52-6865-4f15-b84f-56748f8f4f58-kube-api-access-c9n7h\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:52:40.369996 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:40.369934    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d9bfd52-6865-4f15-b84f-56748f8f4f58-plugins-config-volume\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:52:40.985530 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:40.985497    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d9bfd52-6865-4f15-b84f-56748f8f4f58" path="/var/lib/kubelet/pods/5d9bfd52-6865-4f15-b84f-56748f8f4f58/volumes"
Jul 30 15:52:40.985901 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:40.985882    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ff6a146-1368-4741-8f31-9cb1f5814fcb" path="/var/lib/kubelet/pods/7ff6a146-1368-4741-8f31-9cb1f5814fcb/volumes"
Jul 30 15:52:40.986481 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:52:40.986278    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf4b2e33-34b0-4d2b-9e30-57d132f17cb3" path="/var/lib/kubelet/pods/bf4b2e33-34b0-4d2b-9e30-57d132f17cb3/volumes"
Jul 30 15:53:09.239017 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:53:09.238989    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4mjmp_de241220-5f27-4b0b-b7c6-6d4b350e24e1/console-operator/1.log"
Jul 30 15:53:09.264553 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:53:09.264529    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4mjmp_de241220-5f27-4b0b-b7c6-6d4b350e24e1/console-operator/1.log"
Jul 30 15:54:06.569067 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:06.568965    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-0d4073-5f86d56468-h49zr"]
Jul 30 15:54:06.569675 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:06.569249    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-0d4073-5f86d56468-h49zr" podUID="62142c95-4d70-4c88-8b5b-c479850ce466" containerName="payload-processing" containerID="cri-o://8ba2e138ae2a7fde051539d90baa6fff53df235c38663fbfbe450ddb40fb505a" gracePeriod=30
Jul 30 15:54:06.575919 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:06.575881    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-0d4073-7468ccbd48-wmkl7"]
Jul 30 15:54:06.576117 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:06.576067    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0d4073-7468ccbd48-wmkl7" podUID="4191ac92-c0bc-4c8d-9b52-22e11641fcd7" containerName="payload-pre-processing" containerID="cri-o://df4dec5601e86de764a39d8d530f59449848ac9a1b4009988414bec25f34d0e7" gracePeriod=30
Jul 30 15:54:06.745027 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:06.744997    2578 generic.go:356] "Generic (PLEG): container finished" podID="62142c95-4d70-4c88-8b5b-c479850ce466" containerID="8ba2e138ae2a7fde051539d90baa6fff53df235c38663fbfbe450ddb40fb505a" exitCode=0
Jul 30 15:54:06.745244 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:06.745073    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-0d4073-5f86d56468-h49zr" event={"ID":"62142c95-4d70-4c88-8b5b-c479850ce466","Type":"ContainerDied","Data":"8ba2e138ae2a7fde051539d90baa6fff53df235c38663fbfbe450ddb40fb505a"}
Jul 30 15:54:06.747743 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:06.746917    2578 generic.go:356] "Generic (PLEG): container finished" podID="4191ac92-c0bc-4c8d-9b52-22e11641fcd7" containerID="df4dec5601e86de764a39d8d530f59449848ac9a1b4009988414bec25f34d0e7" exitCode=0
Jul 30 15:54:06.747743 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:06.746981    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0d4073-7468ccbd48-wmkl7" event={"ID":"4191ac92-c0bc-4c8d-9b52-22e11641fcd7","Type":"ContainerDied","Data":"df4dec5601e86de764a39d8d530f59449848ac9a1b4009988414bec25f34d0e7"}
Jul 30 15:54:06.864813 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:06.864790    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-0d4073-5f86d56468-h49zr"
Jul 30 15:54:06.867705 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:06.867685    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0d4073-7468ccbd48-wmkl7"
Jul 30 15:54:07.000309 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:07.000281    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4191ac92-c0bc-4c8d-9b52-22e11641fcd7-plugins-config-volume\") pod \"4191ac92-c0bc-4c8d-9b52-22e11641fcd7\" (UID: \"4191ac92-c0bc-4c8d-9b52-22e11641fcd7\") "
Jul 30 15:54:07.000486 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:07.000357    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-chmqb\" (UniqueName: \"kubernetes.io/projected/4191ac92-c0bc-4c8d-9b52-22e11641fcd7-kube-api-access-chmqb\") pod \"4191ac92-c0bc-4c8d-9b52-22e11641fcd7\" (UID: \"4191ac92-c0bc-4c8d-9b52-22e11641fcd7\") "
Jul 30 15:54:07.000486 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:07.000408    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/62142c95-4d70-4c88-8b5b-c479850ce466-plugins-config-volume\") pod \"62142c95-4d70-4c88-8b5b-c479850ce466\" (UID: \"62142c95-4d70-4c88-8b5b-c479850ce466\") "
Jul 30 15:54:07.000608 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:07.000485    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4djz2\" (UniqueName: \"kubernetes.io/projected/62142c95-4d70-4c88-8b5b-c479850ce466-kube-api-access-4djz2\") pod \"62142c95-4d70-4c88-8b5b-c479850ce466\" (UID: \"62142c95-4d70-4c88-8b5b-c479850ce466\") "
Jul 30 15:54:07.000774 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:07.000747    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4191ac92-c0bc-4c8d-9b52-22e11641fcd7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4191ac92-c0bc-4c8d-9b52-22e11641fcd7" (UID: "4191ac92-c0bc-4c8d-9b52-22e11641fcd7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:54:07.000828 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:07.000794    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62142c95-4d70-4c88-8b5b-c479850ce466-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "62142c95-4d70-4c88-8b5b-c479850ce466" (UID: "62142c95-4d70-4c88-8b5b-c479850ce466"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:54:07.002514 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:07.002491    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4191ac92-c0bc-4c8d-9b52-22e11641fcd7-kube-api-access-chmqb" (OuterVolumeSpecName: "kube-api-access-chmqb") pod "4191ac92-c0bc-4c8d-9b52-22e11641fcd7" (UID: "4191ac92-c0bc-4c8d-9b52-22e11641fcd7"). InnerVolumeSpecName "kube-api-access-chmqb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:54:07.002585 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:07.002517    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62142c95-4d70-4c88-8b5b-c479850ce466-kube-api-access-4djz2" (OuterVolumeSpecName: "kube-api-access-4djz2") pod "62142c95-4d70-4c88-8b5b-c479850ce466" (UID: "62142c95-4d70-4c88-8b5b-c479850ce466"). InnerVolumeSpecName "kube-api-access-4djz2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:54:07.101453 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:07.101377    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4191ac92-c0bc-4c8d-9b52-22e11641fcd7-plugins-config-volume\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:54:07.101453 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:07.101405    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-chmqb\" (UniqueName: \"kubernetes.io/projected/4191ac92-c0bc-4c8d-9b52-22e11641fcd7-kube-api-access-chmqb\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:54:07.101453 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:07.101417    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/62142c95-4d70-4c88-8b5b-c479850ce466-plugins-config-volume\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:54:07.101453 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:07.101430    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4djz2\" (UniqueName: \"kubernetes.io/projected/62142c95-4d70-4c88-8b5b-c479850ce466-kube-api-access-4djz2\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:54:07.753556 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:07.753526    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-0d4073-5f86d56468-h49zr"
Jul 30 15:54:07.753556 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:07.753541    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-0d4073-5f86d56468-h49zr" event={"ID":"62142c95-4d70-4c88-8b5b-c479850ce466","Type":"ContainerDied","Data":"418ebdaf5ea3941194e28ff327cee95db77e005de180eb67cd1c8b4d3cfefe28"}
Jul 30 15:54:07.754055 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:07.753599    2578 scope.go:117] "RemoveContainer" containerID="8ba2e138ae2a7fde051539d90baa6fff53df235c38663fbfbe450ddb40fb505a"
Jul 30 15:54:07.754957 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:07.754932    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0d4073-7468ccbd48-wmkl7"
Jul 30 15:54:07.754957 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:07.754950    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0d4073-7468ccbd48-wmkl7" event={"ID":"4191ac92-c0bc-4c8d-9b52-22e11641fcd7","Type":"ContainerDied","Data":"1fdbae20589f804ec6ee6346a77d3bf39e20d172850aea533e092ae49c586291"}
Jul 30 15:54:07.763635 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:07.763618    2578 scope.go:117] "RemoveContainer" containerID="df4dec5601e86de764a39d8d530f59449848ac9a1b4009988414bec25f34d0e7"
Jul 30 15:54:07.776108 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:07.776086    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-0d4073-5f86d56468-h49zr"]
Jul 30 15:54:07.778816 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:07.778792    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-0d4073-5f86d56468-h49zr"]
Jul 30 15:54:07.787446 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:07.787384    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-0d4073-7468ccbd48-wmkl7"]
Jul 30 15:54:07.789247 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:07.789228    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-0d4073-7468ccbd48-wmkl7"]
Jul 30 15:54:08.978850 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:08.978814    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4191ac92-c0bc-4c8d-9b52-22e11641fcd7" path="/var/lib/kubelet/pods/4191ac92-c0bc-4c8d-9b52-22e11641fcd7/volumes"
Jul 30 15:54:08.979231 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:08.979213    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62142c95-4d70-4c88-8b5b-c479850ce466" path="/var/lib/kubelet/pods/62142c95-4d70-4c88-8b5b-c479850ce466/volumes"
Jul 30 15:54:24.588534 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:24.588497    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-845a26-7c7c6fbb89-6h6qp"]
Jul 30 15:54:24.589139 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:24.588732    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-845a26-7c7c6fbb89-6h6qp" podUID="7b059341-28e1-4546-b960-2dd668984f4b" containerName="maas-api" containerID="cri-o://71bd91b2962beaf6f74a2e20823a3bceb3ec50d6ea2515a1fa52d21d44a8afe3" gracePeriod=30
Jul 30 15:54:24.660789 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:24.660738    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-845a26-cdf79787d-k4thx"]
Jul 30 15:54:24.661342 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:24.661284    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-845a26-cdf79787d-k4thx" podUID="fff1cd8b-5cd0-447d-8f23-6f4a7f86aa57" containerName="payload-processing" containerID="cri-o://f8314eecface2ace481ee04a6ac0771fee9f281949743d91e05b9530cc44f1b0" gracePeriod=30
Jul 30 15:54:24.664543 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:24.664509    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-845a26-74cffd7c8d-gcnzh"]
Jul 30 15:54:24.664796 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:24.664747    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-845a26-74cffd7c8d-gcnzh" podUID="b9a2a6a2-e909-4d96-ac32-0f0bb8626d74" containerName="payload-pre-processing" containerID="cri-o://90d96689fc46a37e3fbb1af635ddc913f3a5e25d640a10afcd8b9fbcd8e23557" gracePeriod=30
Jul 30 15:54:24.850254 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:24.848858    2578 generic.go:356] "Generic (PLEG): container finished" podID="b9a2a6a2-e909-4d96-ac32-0f0bb8626d74" containerID="90d96689fc46a37e3fbb1af635ddc913f3a5e25d640a10afcd8b9fbcd8e23557" exitCode=0
Jul 30 15:54:24.850254 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:24.848995    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-845a26-74cffd7c8d-gcnzh" event={"ID":"b9a2a6a2-e909-4d96-ac32-0f0bb8626d74","Type":"ContainerDied","Data":"90d96689fc46a37e3fbb1af635ddc913f3a5e25d640a10afcd8b9fbcd8e23557"}
Jul 30 15:54:24.851066 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:24.851034    2578 generic.go:356] "Generic (PLEG): container finished" podID="7b059341-28e1-4546-b960-2dd668984f4b" containerID="71bd91b2962beaf6f74a2e20823a3bceb3ec50d6ea2515a1fa52d21d44a8afe3" exitCode=0
Jul 30 15:54:24.851230 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:24.851117    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-845a26-7c7c6fbb89-6h6qp" event={"ID":"7b059341-28e1-4546-b960-2dd668984f4b","Type":"ContainerDied","Data":"71bd91b2962beaf6f74a2e20823a3bceb3ec50d6ea2515a1fa52d21d44a8afe3"}
Jul 30 15:54:24.853396 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:24.853272    2578 generic.go:356] "Generic (PLEG): container finished" podID="fff1cd8b-5cd0-447d-8f23-6f4a7f86aa57" containerID="f8314eecface2ace481ee04a6ac0771fee9f281949743d91e05b9530cc44f1b0" exitCode=0
Jul 30 15:54:24.853396 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:24.853316    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-845a26-cdf79787d-k4thx" event={"ID":"fff1cd8b-5cd0-447d-8f23-6f4a7f86aa57","Type":"ContainerDied","Data":"f8314eecface2ace481ee04a6ac0771fee9f281949743d91e05b9530cc44f1b0"}
Jul 30 15:54:24.972000 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:24.971973    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-845a26-cdf79787d-k4thx"
Jul 30 15:54:24.975038 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:24.975011    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-845a26-7c7c6fbb89-6h6qp"
Jul 30 15:54:24.993259 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:24.993230    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-845a26-74cffd7c8d-gcnzh"
Jul 30 15:54:25.070878 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:25.070845    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b059341-28e1-4546-b960-2dd668984f4b-maas-api-tls\") pod \"7b059341-28e1-4546-b960-2dd668984f4b\" (UID: \"7b059341-28e1-4546-b960-2dd668984f4b\") "
Jul 30 15:54:25.071085 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:25.070884    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z89mb\" (UniqueName: \"kubernetes.io/projected/fff1cd8b-5cd0-447d-8f23-6f4a7f86aa57-kube-api-access-z89mb\") pod \"fff1cd8b-5cd0-447d-8f23-6f4a7f86aa57\" (UID: \"fff1cd8b-5cd0-447d-8f23-6f4a7f86aa57\") "
Jul 30 15:54:25.071085 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:25.070902    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9a2a6a2-e909-4d96-ac32-0f0bb8626d74-plugins-config-volume\") pod \"b9a2a6a2-e909-4d96-ac32-0f0bb8626d74\" (UID: \"b9a2a6a2-e909-4d96-ac32-0f0bb8626d74\") "
Jul 30 15:54:25.071085 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:25.070930    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b059341-28e1-4546-b960-2dd668984f4b-openshift-service-ca\") pod \"7b059341-28e1-4546-b960-2dd668984f4b\" (UID: \"7b059341-28e1-4546-b960-2dd668984f4b\") "
Jul 30 15:54:25.071085 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:25.070975    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hn9ks\" (UniqueName: \"kubernetes.io/projected/7b059341-28e1-4546-b960-2dd668984f4b-kube-api-access-hn9ks\") pod \"7b059341-28e1-4546-b960-2dd668984f4b\" (UID: \"7b059341-28e1-4546-b960-2dd668984f4b\") "
Jul 30 15:54:25.071085 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:25.071020    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5dgv6\" (UniqueName: \"kubernetes.io/projected/b9a2a6a2-e909-4d96-ac32-0f0bb8626d74-kube-api-access-5dgv6\") pod \"b9a2a6a2-e909-4d96-ac32-0f0bb8626d74\" (UID: \"b9a2a6a2-e909-4d96-ac32-0f0bb8626d74\") "
Jul 30 15:54:25.071426 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:25.071097    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fff1cd8b-5cd0-447d-8f23-6f4a7f86aa57-plugins-config-volume\") pod \"fff1cd8b-5cd0-447d-8f23-6f4a7f86aa57\" (UID: \"fff1cd8b-5cd0-447d-8f23-6f4a7f86aa57\") "
Jul 30 15:54:25.071426 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:25.071388    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b059341-28e1-4546-b960-2dd668984f4b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7b059341-28e1-4546-b960-2dd668984f4b" (UID: "7b059341-28e1-4546-b960-2dd668984f4b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:54:25.071538 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:25.071473    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b059341-28e1-4546-b960-2dd668984f4b-openshift-service-ca\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:54:25.071538 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:25.071385    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9a2a6a2-e909-4d96-ac32-0f0bb8626d74-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b9a2a6a2-e909-4d96-ac32-0f0bb8626d74" (UID: "b9a2a6a2-e909-4d96-ac32-0f0bb8626d74"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:54:25.071794 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:25.071768    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fff1cd8b-5cd0-447d-8f23-6f4a7f86aa57-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fff1cd8b-5cd0-447d-8f23-6f4a7f86aa57" (UID: "fff1cd8b-5cd0-447d-8f23-6f4a7f86aa57"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:54:25.073692 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:25.073657    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9a2a6a2-e909-4d96-ac32-0f0bb8626d74-kube-api-access-5dgv6" (OuterVolumeSpecName: "kube-api-access-5dgv6") pod "b9a2a6a2-e909-4d96-ac32-0f0bb8626d74" (UID: "b9a2a6a2-e909-4d96-ac32-0f0bb8626d74"). InnerVolumeSpecName "kube-api-access-5dgv6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:54:25.073787 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:25.073695    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b059341-28e1-4546-b960-2dd668984f4b-kube-api-access-hn9ks" (OuterVolumeSpecName: "kube-api-access-hn9ks") pod "7b059341-28e1-4546-b960-2dd668984f4b" (UID: "7b059341-28e1-4546-b960-2dd668984f4b"). InnerVolumeSpecName "kube-api-access-hn9ks". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:54:25.073787 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:25.073702    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7b059341-28e1-4546-b960-2dd668984f4b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7b059341-28e1-4546-b960-2dd668984f4b" (UID: "7b059341-28e1-4546-b960-2dd668984f4b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 15:54:25.073787 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:25.073721    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fff1cd8b-5cd0-447d-8f23-6f4a7f86aa57-kube-api-access-z89mb" (OuterVolumeSpecName: "kube-api-access-z89mb") pod "fff1cd8b-5cd0-447d-8f23-6f4a7f86aa57" (UID: "fff1cd8b-5cd0-447d-8f23-6f4a7f86aa57"). InnerVolumeSpecName "kube-api-access-z89mb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:54:25.172265 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:25.172199    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fff1cd8b-5cd0-447d-8f23-6f4a7f86aa57-plugins-config-volume\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:54:25.172265 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:25.172225    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b059341-28e1-4546-b960-2dd668984f4b-maas-api-tls\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:54:25.172265 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:25.172235    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z89mb\" (UniqueName: \"kubernetes.io/projected/fff1cd8b-5cd0-447d-8f23-6f4a7f86aa57-kube-api-access-z89mb\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:54:25.172265 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:25.172243    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9a2a6a2-e909-4d96-ac32-0f0bb8626d74-plugins-config-volume\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:54:25.172265 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:25.172252    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hn9ks\" (UniqueName: \"kubernetes.io/projected/7b059341-28e1-4546-b960-2dd668984f4b-kube-api-access-hn9ks\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:54:25.172265 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:25.172260    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5dgv6\" (UniqueName: \"kubernetes.io/projected/b9a2a6a2-e909-4d96-ac32-0f0bb8626d74-kube-api-access-5dgv6\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:54:25.859098 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:25.859019    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-845a26-74cffd7c8d-gcnzh"
Jul 30 15:54:25.859098 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:25.859045    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-845a26-74cffd7c8d-gcnzh" event={"ID":"b9a2a6a2-e909-4d96-ac32-0f0bb8626d74","Type":"ContainerDied","Data":"c5e84ddeba6630b0b0f88d8a545289677158f2f4c6c04adaaefa413e5f00f46d"}
Jul 30 15:54:25.859574 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:25.859111    2578 scope.go:117] "RemoveContainer" containerID="90d96689fc46a37e3fbb1af635ddc913f3a5e25d640a10afcd8b9fbcd8e23557"
Jul 30 15:54:25.860498 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:25.860480    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-845a26-7c7c6fbb89-6h6qp"
Jul 30 15:54:25.860584 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:25.860480    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-845a26-7c7c6fbb89-6h6qp" event={"ID":"7b059341-28e1-4546-b960-2dd668984f4b","Type":"ContainerDied","Data":"d6a89287ae4ee801588506a3dd032e6c80903d0fcb149326143946fb0ccbc010"}
Jul 30 15:54:25.861877 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:25.861850    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-845a26-cdf79787d-k4thx"
Jul 30 15:54:25.861975 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:25.861875    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-845a26-cdf79787d-k4thx" event={"ID":"fff1cd8b-5cd0-447d-8f23-6f4a7f86aa57","Type":"ContainerDied","Data":"97c97a48bf7d13006d64b9614ef4715e547a6bcef3ac6562aed97d5c49f0bc5e"}
Jul 30 15:54:25.869547 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:25.869525    2578 scope.go:117] "RemoveContainer" containerID="71bd91b2962beaf6f74a2e20823a3bceb3ec50d6ea2515a1fa52d21d44a8afe3"
Jul 30 15:54:25.878811 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:25.878793    2578 scope.go:117] "RemoveContainer" containerID="f8314eecface2ace481ee04a6ac0771fee9f281949743d91e05b9530cc44f1b0"
Jul 30 15:54:25.882274 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:25.882253    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-845a26-74cffd7c8d-gcnzh"]
Jul 30 15:54:25.885542 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:25.885520    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-845a26-74cffd7c8d-gcnzh"]
Jul 30 15:54:25.894843 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:25.894809    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-845a26-7c7c6fbb89-6h6qp"]
Jul 30 15:54:25.896942 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:25.896922    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-845a26-7c7c6fbb89-6h6qp"]
Jul 30 15:54:25.907478 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:25.907446    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-845a26-cdf79787d-k4thx"]
Jul 30 15:54:25.912128 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:25.912109    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-845a26-cdf79787d-k4thx"]
Jul 30 15:54:26.978659 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:26.978629    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b059341-28e1-4546-b960-2dd668984f4b" path="/var/lib/kubelet/pods/7b059341-28e1-4546-b960-2dd668984f4b/volumes"
Jul 30 15:54:26.979030 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:26.978997    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9a2a6a2-e909-4d96-ac32-0f0bb8626d74" path="/var/lib/kubelet/pods/b9a2a6a2-e909-4d96-ac32-0f0bb8626d74/volumes"
Jul 30 15:54:26.979338 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:26.979325    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fff1cd8b-5cd0-447d-8f23-6f4a7f86aa57" path="/var/lib/kubelet/pods/fff1cd8b-5cd0-447d-8f23-6f4a7f86aa57/volumes"
Jul 30 15:54:31.876173 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:31.876121    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl"]
Jul 30 15:54:31.876678 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:31.876404    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl" podUID="ea2aceba-fd54-43a6-8707-05f92f63ab75" containerName="istio-proxy" containerID="cri-o://5b09f9116a3ae3d6ea329a5ec98588248b8c11f8ddc1737108cc5d9fe883cb95" gracePeriod=30
Jul 30 15:54:31.935251 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:31.935208    2578 patch_prober.go:28] interesting pod/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 30 15:54:31.935404 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:31.935262    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl" podUID="ea2aceba-fd54-43a6-8707-05f92f63ab75" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 30 15:54:37.140673 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:37.140606    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl"
Jul 30 15:54:37.188392 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:37.188351    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea2aceba-fd54-43a6-8707-05f92f63ab75-istiod-ca-cert\") pod \"ea2aceba-fd54-43a6-8707-05f92f63ab75\" (UID: \"ea2aceba-fd54-43a6-8707-05f92f63ab75\") "
Jul 30 15:54:37.188585 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:37.188403    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea2aceba-fd54-43a6-8707-05f92f63ab75-istio-token\") pod \"ea2aceba-fd54-43a6-8707-05f92f63ab75\" (UID: \"ea2aceba-fd54-43a6-8707-05f92f63ab75\") "
Jul 30 15:54:37.188585 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:37.188455    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea2aceba-fd54-43a6-8707-05f92f63ab75-istio-podinfo\") pod \"ea2aceba-fd54-43a6-8707-05f92f63ab75\" (UID: \"ea2aceba-fd54-43a6-8707-05f92f63ab75\") "
Jul 30 15:54:37.188585 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:37.188484    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea2aceba-fd54-43a6-8707-05f92f63ab75-workload-certs\") pod \"ea2aceba-fd54-43a6-8707-05f92f63ab75\" (UID: \"ea2aceba-fd54-43a6-8707-05f92f63ab75\") "
Jul 30 15:54:37.188774 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:37.188626    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea2aceba-fd54-43a6-8707-05f92f63ab75-istio-envoy\") pod \"ea2aceba-fd54-43a6-8707-05f92f63ab75\" (UID: \"ea2aceba-fd54-43a6-8707-05f92f63ab75\") "
Jul 30 15:54:37.188774 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:37.188757    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5fn9q\" (UniqueName: \"kubernetes.io/projected/ea2aceba-fd54-43a6-8707-05f92f63ab75-kube-api-access-5fn9q\") pod \"ea2aceba-fd54-43a6-8707-05f92f63ab75\" (UID: \"ea2aceba-fd54-43a6-8707-05f92f63ab75\") "
Jul 30 15:54:37.188882 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:37.188788    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea2aceba-fd54-43a6-8707-05f92f63ab75-istio-data\") pod \"ea2aceba-fd54-43a6-8707-05f92f63ab75\" (UID: \"ea2aceba-fd54-43a6-8707-05f92f63ab75\") "
Jul 30 15:54:37.188882 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:37.188816    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea2aceba-fd54-43a6-8707-05f92f63ab75-workload-socket\") pod \"ea2aceba-fd54-43a6-8707-05f92f63ab75\" (UID: \"ea2aceba-fd54-43a6-8707-05f92f63ab75\") "
Jul 30 15:54:37.188882 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:37.188813    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea2aceba-fd54-43a6-8707-05f92f63ab75-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ea2aceba-fd54-43a6-8707-05f92f63ab75" (UID: "ea2aceba-fd54-43a6-8707-05f92f63ab75"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:54:37.188882 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:37.188852    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea2aceba-fd54-43a6-8707-05f92f63ab75-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ea2aceba-fd54-43a6-8707-05f92f63ab75" (UID: "ea2aceba-fd54-43a6-8707-05f92f63ab75"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:54:37.188882 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:37.188873    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea2aceba-fd54-43a6-8707-05f92f63ab75-credential-socket\") pod \"ea2aceba-fd54-43a6-8707-05f92f63ab75\" (UID: \"ea2aceba-fd54-43a6-8707-05f92f63ab75\") "
Jul 30 15:54:37.189171 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:37.189134    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea2aceba-fd54-43a6-8707-05f92f63ab75-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ea2aceba-fd54-43a6-8707-05f92f63ab75" (UID: "ea2aceba-fd54-43a6-8707-05f92f63ab75"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:54:37.189252 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:37.189235    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea2aceba-fd54-43a6-8707-05f92f63ab75-workload-socket\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:54:37.189308 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:37.189253    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea2aceba-fd54-43a6-8707-05f92f63ab75-istiod-ca-cert\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:54:37.189308 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:37.189268    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea2aceba-fd54-43a6-8707-05f92f63ab75-workload-certs\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:54:37.189412 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:37.189340    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea2aceba-fd54-43a6-8707-05f92f63ab75-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ea2aceba-fd54-43a6-8707-05f92f63ab75" (UID: "ea2aceba-fd54-43a6-8707-05f92f63ab75"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:54:37.189843 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:37.189814    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea2aceba-fd54-43a6-8707-05f92f63ab75-istio-data" (OuterVolumeSpecName: "istio-data") pod "ea2aceba-fd54-43a6-8707-05f92f63ab75" (UID: "ea2aceba-fd54-43a6-8707-05f92f63ab75"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:54:37.190980 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:37.190955    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea2aceba-fd54-43a6-8707-05f92f63ab75-kube-api-access-5fn9q" (OuterVolumeSpecName: "kube-api-access-5fn9q") pod "ea2aceba-fd54-43a6-8707-05f92f63ab75" (UID: "ea2aceba-fd54-43a6-8707-05f92f63ab75"). InnerVolumeSpecName "kube-api-access-5fn9q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:54:37.191096 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:37.191073    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea2aceba-fd54-43a6-8707-05f92f63ab75-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ea2aceba-fd54-43a6-8707-05f92f63ab75" (UID: "ea2aceba-fd54-43a6-8707-05f92f63ab75"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:54:37.191165 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:37.191117    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ea2aceba-fd54-43a6-8707-05f92f63ab75-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ea2aceba-fd54-43a6-8707-05f92f63ab75" (UID: "ea2aceba-fd54-43a6-8707-05f92f63ab75"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 15:54:37.191215 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:37.191170    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea2aceba-fd54-43a6-8707-05f92f63ab75-istio-token" (OuterVolumeSpecName: "istio-token") pod "ea2aceba-fd54-43a6-8707-05f92f63ab75" (UID: "ea2aceba-fd54-43a6-8707-05f92f63ab75"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:54:37.290747 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:37.290706    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea2aceba-fd54-43a6-8707-05f92f63ab75-credential-socket\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:54:37.290747 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:37.290741    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea2aceba-fd54-43a6-8707-05f92f63ab75-istio-token\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:54:37.290747 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:37.290750    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea2aceba-fd54-43a6-8707-05f92f63ab75-istio-podinfo\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:54:37.290747 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:37.290758    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea2aceba-fd54-43a6-8707-05f92f63ab75-istio-envoy\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:54:37.291016 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:37.290767    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5fn9q\" (UniqueName: \"kubernetes.io/projected/ea2aceba-fd54-43a6-8707-05f92f63ab75-kube-api-access-5fn9q\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:54:37.291016 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:37.290778    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea2aceba-fd54-43a6-8707-05f92f63ab75-istio-data\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:54:37.922499 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:37.922465    2578 generic.go:356] "Generic (PLEG): container finished" podID="ea2aceba-fd54-43a6-8707-05f92f63ab75" containerID="5b09f9116a3ae3d6ea329a5ec98588248b8c11f8ddc1737108cc5d9fe883cb95" exitCode=0
Jul 30 15:54:37.922694 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:37.922511    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl" event={"ID":"ea2aceba-fd54-43a6-8707-05f92f63ab75","Type":"ContainerDied","Data":"5b09f9116a3ae3d6ea329a5ec98588248b8c11f8ddc1737108cc5d9fe883cb95"}
Jul 30 15:54:37.922694 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:37.922552    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl" event={"ID":"ea2aceba-fd54-43a6-8707-05f92f63ab75","Type":"ContainerDied","Data":"2caaba0f33e13ec7be402601aad2f0bf0fb4bcaddce19b8d28efc8cd41cd49d8"}
Jul 30 15:54:37.922694 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:37.922574    2578 scope.go:117] "RemoveContainer" containerID="5b09f9116a3ae3d6ea329a5ec98588248b8c11f8ddc1737108cc5d9fe883cb95"
Jul 30 15:54:37.922694 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:37.922514    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl"
Jul 30 15:54:37.932092 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:37.932072    2578 scope.go:117] "RemoveContainer" containerID="6886cfb66beda6c4c58bf49990c29d12f7e45db6d1d4fa618f5a69044390abe5"
Jul 30 15:54:37.939840 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:37.939822    2578 scope.go:117] "RemoveContainer" containerID="5b09f9116a3ae3d6ea329a5ec98588248b8c11f8ddc1737108cc5d9fe883cb95"
Jul 30 15:54:37.940091 ip-10-0-133-117 kubenswrapper[2578]: E0730 15:54:37.940071    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b09f9116a3ae3d6ea329a5ec98588248b8c11f8ddc1737108cc5d9fe883cb95\": container with ID starting with 5b09f9116a3ae3d6ea329a5ec98588248b8c11f8ddc1737108cc5d9fe883cb95 not found: ID does not exist" containerID="5b09f9116a3ae3d6ea329a5ec98588248b8c11f8ddc1737108cc5d9fe883cb95"
Jul 30 15:54:37.940188 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:37.940104    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b09f9116a3ae3d6ea329a5ec98588248b8c11f8ddc1737108cc5d9fe883cb95"} err="failed to get container status \"5b09f9116a3ae3d6ea329a5ec98588248b8c11f8ddc1737108cc5d9fe883cb95\": rpc error: code = NotFound desc = could not find container \"5b09f9116a3ae3d6ea329a5ec98588248b8c11f8ddc1737108cc5d9fe883cb95\": container with ID starting with 5b09f9116a3ae3d6ea329a5ec98588248b8c11f8ddc1737108cc5d9fe883cb95 not found: ID does not exist"
Jul 30 15:54:37.940188 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:37.940128    2578 scope.go:117] "RemoveContainer" containerID="6886cfb66beda6c4c58bf49990c29d12f7e45db6d1d4fa618f5a69044390abe5"
Jul 30 15:54:37.940416 ip-10-0-133-117 kubenswrapper[2578]: E0730 15:54:37.940396    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6886cfb66beda6c4c58bf49990c29d12f7e45db6d1d4fa618f5a69044390abe5\": container with ID starting with 6886cfb66beda6c4c58bf49990c29d12f7e45db6d1d4fa618f5a69044390abe5 not found: ID does not exist" containerID="6886cfb66beda6c4c58bf49990c29d12f7e45db6d1d4fa618f5a69044390abe5"
Jul 30 15:54:37.940457 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:37.940422    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6886cfb66beda6c4c58bf49990c29d12f7e45db6d1d4fa618f5a69044390abe5"} err="failed to get container status \"6886cfb66beda6c4c58bf49990c29d12f7e45db6d1d4fa618f5a69044390abe5\": rpc error: code = NotFound desc = could not find container \"6886cfb66beda6c4c58bf49990c29d12f7e45db6d1d4fa618f5a69044390abe5\": container with ID starting with 6886cfb66beda6c4c58bf49990c29d12f7e45db6d1d4fa618f5a69044390abe5 not found: ID does not exist"
Jul 30 15:54:37.947029 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:37.947004    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl"]
Jul 30 15:54:37.949906 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:37.949884    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-845a26-openshift-default-9678fcb68-hzmsl"]
Jul 30 15:54:38.979095 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:38.979060    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea2aceba-fd54-43a6-8707-05f92f63ab75" path="/var/lib/kubelet/pods/ea2aceba-fd54-43a6-8707-05f92f63ab75/volumes"
Jul 30 15:54:42.794898 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:42.794859    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-2b1f1f-5c54dc7758-wfdx7"]
Jul 30 15:54:42.795406 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:42.795128    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-2b1f1f-5c54dc7758-wfdx7" podUID="2a2c0115-cc11-4717-b200-42d925bcd9ae" containerName="payload-processing" containerID="cri-o://f3990364922e8d96e9edd10d6a4facb089aae99a77d85493592fa2ae6bf7aae2" gracePeriod=30
Jul 30 15:54:42.948284 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:42.948250    2578 generic.go:356] "Generic (PLEG): container finished" podID="2a2c0115-cc11-4717-b200-42d925bcd9ae" containerID="f3990364922e8d96e9edd10d6a4facb089aae99a77d85493592fa2ae6bf7aae2" exitCode=0
Jul 30 15:54:42.948483 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:42.948325    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-2b1f1f-5c54dc7758-wfdx7" event={"ID":"2a2c0115-cc11-4717-b200-42d925bcd9ae","Type":"ContainerDied","Data":"f3990364922e8d96e9edd10d6a4facb089aae99a77d85493592fa2ae6bf7aae2"}
Jul 30 15:54:43.053478 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:43.053454    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-2b1f1f-5c54dc7758-wfdx7"
Jul 30 15:54:43.140232 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:43.140200    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a2c0115-cc11-4717-b200-42d925bcd9ae-plugins-config-volume\") pod \"2a2c0115-cc11-4717-b200-42d925bcd9ae\" (UID: \"2a2c0115-cc11-4717-b200-42d925bcd9ae\") "
Jul 30 15:54:43.140408 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:43.140371    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ppkm6\" (UniqueName: \"kubernetes.io/projected/2a2c0115-cc11-4717-b200-42d925bcd9ae-kube-api-access-ppkm6\") pod \"2a2c0115-cc11-4717-b200-42d925bcd9ae\" (UID: \"2a2c0115-cc11-4717-b200-42d925bcd9ae\") "
Jul 30 15:54:43.140580 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:43.140556    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a2c0115-cc11-4717-b200-42d925bcd9ae-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2a2c0115-cc11-4717-b200-42d925bcd9ae" (UID: "2a2c0115-cc11-4717-b200-42d925bcd9ae"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:54:43.142402 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:43.142382    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a2c0115-cc11-4717-b200-42d925bcd9ae-kube-api-access-ppkm6" (OuterVolumeSpecName: "kube-api-access-ppkm6") pod "2a2c0115-cc11-4717-b200-42d925bcd9ae" (UID: "2a2c0115-cc11-4717-b200-42d925bcd9ae"). InnerVolumeSpecName "kube-api-access-ppkm6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:54:43.241768 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:43.241735    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ppkm6\" (UniqueName: \"kubernetes.io/projected/2a2c0115-cc11-4717-b200-42d925bcd9ae-kube-api-access-ppkm6\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:54:43.241768 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:43.241763    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a2c0115-cc11-4717-b200-42d925bcd9ae-plugins-config-volume\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:54:43.954687 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:43.954644    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-2b1f1f-5c54dc7758-wfdx7" event={"ID":"2a2c0115-cc11-4717-b200-42d925bcd9ae","Type":"ContainerDied","Data":"fffdb8a82f2e5d1c05425aea3915d33c81fefb71620476ffb327a1e5f22bb29c"}
Jul 30 15:54:43.955125 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:43.954713    2578 scope.go:117] "RemoveContainer" containerID="f3990364922e8d96e9edd10d6a4facb089aae99a77d85493592fa2ae6bf7aae2"
Jul 30 15:54:43.955125 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:43.954659    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-2b1f1f-5c54dc7758-wfdx7"
Jul 30 15:54:43.977358 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:43.977334    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-2b1f1f-5c54dc7758-wfdx7"]
Jul 30 15:54:43.980801 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:43.980776    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-2b1f1f-5c54dc7758-wfdx7"]
Jul 30 15:54:44.978134 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:44.978100    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a2c0115-cc11-4717-b200-42d925bcd9ae" path="/var/lib/kubelet/pods/2a2c0115-cc11-4717-b200-42d925bcd9ae/volumes"
Jul 30 15:54:50.041850 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:50.041812    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s"]
Jul 30 15:54:50.042255 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:50.042074    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s" podUID="efe00480-68c7-4658-a34a-53ccefe2ba3e" containerName="istio-proxy" containerID="cri-o://5ee8c35aa05e2466d7e5efbefbc22062fd92eb3e837c644637ff2f1f83e2a4a5" gracePeriod=30
Jul 30 15:54:55.290400 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:55.290369    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s"
Jul 30 15:54:55.460777 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:55.460698    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efe00480-68c7-4658-a34a-53ccefe2ba3e-istio-podinfo\") pod \"efe00480-68c7-4658-a34a-53ccefe2ba3e\" (UID: \"efe00480-68c7-4658-a34a-53ccefe2ba3e\") "
Jul 30 15:54:55.460777 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:55.460758    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efe00480-68c7-4658-a34a-53ccefe2ba3e-workload-certs\") pod \"efe00480-68c7-4658-a34a-53ccefe2ba3e\" (UID: \"efe00480-68c7-4658-a34a-53ccefe2ba3e\") "
Jul 30 15:54:55.461024 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:55.460794    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efe00480-68c7-4658-a34a-53ccefe2ba3e-istio-envoy\") pod \"efe00480-68c7-4658-a34a-53ccefe2ba3e\" (UID: \"efe00480-68c7-4658-a34a-53ccefe2ba3e\") "
Jul 30 15:54:55.461024 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:55.460814    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efe00480-68c7-4658-a34a-53ccefe2ba3e-istio-data\") pod \"efe00480-68c7-4658-a34a-53ccefe2ba3e\" (UID: \"efe00480-68c7-4658-a34a-53ccefe2ba3e\") "
Jul 30 15:54:55.461024 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:55.460837    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/efe00480-68c7-4658-a34a-53ccefe2ba3e-istio-token\") pod \"efe00480-68c7-4658-a34a-53ccefe2ba3e\" (UID: \"efe00480-68c7-4658-a34a-53ccefe2ba3e\") "
Jul 30 15:54:55.461024 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:55.460865    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p7vhs\" (UniqueName: \"kubernetes.io/projected/efe00480-68c7-4658-a34a-53ccefe2ba3e-kube-api-access-p7vhs\") pod \"efe00480-68c7-4658-a34a-53ccefe2ba3e\" (UID: \"efe00480-68c7-4658-a34a-53ccefe2ba3e\") "
Jul 30 15:54:55.461024 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:55.460893    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efe00480-68c7-4658-a34a-53ccefe2ba3e-credential-socket\") pod \"efe00480-68c7-4658-a34a-53ccefe2ba3e\" (UID: \"efe00480-68c7-4658-a34a-53ccefe2ba3e\") "
Jul 30 15:54:55.461024 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:55.460934    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efe00480-68c7-4658-a34a-53ccefe2ba3e-workload-socket\") pod \"efe00480-68c7-4658-a34a-53ccefe2ba3e\" (UID: \"efe00480-68c7-4658-a34a-53ccefe2ba3e\") "
Jul 30 15:54:55.461024 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:55.460960    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efe00480-68c7-4658-a34a-53ccefe2ba3e-istiod-ca-cert\") pod \"efe00480-68c7-4658-a34a-53ccefe2ba3e\" (UID: \"efe00480-68c7-4658-a34a-53ccefe2ba3e\") "
Jul 30 15:54:55.461440 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:55.461123    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efe00480-68c7-4658-a34a-53ccefe2ba3e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "efe00480-68c7-4658-a34a-53ccefe2ba3e" (UID: "efe00480-68c7-4658-a34a-53ccefe2ba3e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:54:55.461440 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:55.461283    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efe00480-68c7-4658-a34a-53ccefe2ba3e-workload-certs\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:54:55.461440 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:55.461372    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efe00480-68c7-4658-a34a-53ccefe2ba3e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "efe00480-68c7-4658-a34a-53ccefe2ba3e" (UID: "efe00480-68c7-4658-a34a-53ccefe2ba3e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:54:55.461440 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:55.461430    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efe00480-68c7-4658-a34a-53ccefe2ba3e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "efe00480-68c7-4658-a34a-53ccefe2ba3e" (UID: "efe00480-68c7-4658-a34a-53ccefe2ba3e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:54:55.461658 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:55.461463    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/efe00480-68c7-4658-a34a-53ccefe2ba3e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "efe00480-68c7-4658-a34a-53ccefe2ba3e" (UID: "efe00480-68c7-4658-a34a-53ccefe2ba3e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:54:55.461776 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:55.461754    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efe00480-68c7-4658-a34a-53ccefe2ba3e-istio-data" (OuterVolumeSpecName: "istio-data") pod "efe00480-68c7-4658-a34a-53ccefe2ba3e" (UID: "efe00480-68c7-4658-a34a-53ccefe2ba3e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:54:55.463111 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:55.463080    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/efe00480-68c7-4658-a34a-53ccefe2ba3e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "efe00480-68c7-4658-a34a-53ccefe2ba3e" (UID: "efe00480-68c7-4658-a34a-53ccefe2ba3e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 15:54:55.463233 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:55.463174    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efe00480-68c7-4658-a34a-53ccefe2ba3e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "efe00480-68c7-4658-a34a-53ccefe2ba3e" (UID: "efe00480-68c7-4658-a34a-53ccefe2ba3e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:54:55.463282 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:55.463258    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efe00480-68c7-4658-a34a-53ccefe2ba3e-kube-api-access-p7vhs" (OuterVolumeSpecName: "kube-api-access-p7vhs") pod "efe00480-68c7-4658-a34a-53ccefe2ba3e" (UID: "efe00480-68c7-4658-a34a-53ccefe2ba3e"). InnerVolumeSpecName "kube-api-access-p7vhs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:54:55.463332 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:55.463309    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efe00480-68c7-4658-a34a-53ccefe2ba3e-istio-token" (OuterVolumeSpecName: "istio-token") pod "efe00480-68c7-4658-a34a-53ccefe2ba3e" (UID: "efe00480-68c7-4658-a34a-53ccefe2ba3e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:54:55.561999 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:55.561962    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efe00480-68c7-4658-a34a-53ccefe2ba3e-istio-envoy\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:54:55.561999 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:55.561993    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efe00480-68c7-4658-a34a-53ccefe2ba3e-istio-data\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:54:55.561999 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:55.562005    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/efe00480-68c7-4658-a34a-53ccefe2ba3e-istio-token\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:54:55.562277 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:55.562018    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7vhs\" (UniqueName: \"kubernetes.io/projected/efe00480-68c7-4658-a34a-53ccefe2ba3e-kube-api-access-p7vhs\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:54:55.562277 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:55.562030    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efe00480-68c7-4658-a34a-53ccefe2ba3e-credential-socket\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:54:55.562277 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:55.562041    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efe00480-68c7-4658-a34a-53ccefe2ba3e-workload-socket\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:54:55.562277 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:55.562052    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efe00480-68c7-4658-a34a-53ccefe2ba3e-istiod-ca-cert\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:54:55.562277 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:55.562063    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efe00480-68c7-4658-a34a-53ccefe2ba3e-istio-podinfo\") on node \"ip-10-0-133-117.ec2.internal\" DevicePath \"\""
Jul 30 15:54:56.012648 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:56.012612    2578 generic.go:356] "Generic (PLEG): container finished" podID="efe00480-68c7-4658-a34a-53ccefe2ba3e" containerID="5ee8c35aa05e2466d7e5efbefbc22062fd92eb3e837c644637ff2f1f83e2a4a5" exitCode=0
Jul 30 15:54:56.012825 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:56.012664    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s"
Jul 30 15:54:56.012825 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:56.012704    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s" event={"ID":"efe00480-68c7-4658-a34a-53ccefe2ba3e","Type":"ContainerDied","Data":"5ee8c35aa05e2466d7e5efbefbc22062fd92eb3e837c644637ff2f1f83e2a4a5"}
Jul 30 15:54:56.012825 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:56.012764    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s" event={"ID":"efe00480-68c7-4658-a34a-53ccefe2ba3e","Type":"ContainerDied","Data":"a98171d0aa7ff260e8e20654e6c9d95af139cbe72423c0ac74790bcdfda1dbf2"}
Jul 30 15:54:56.012825 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:56.012786    2578 scope.go:117] "RemoveContainer" containerID="5ee8c35aa05e2466d7e5efbefbc22062fd92eb3e837c644637ff2f1f83e2a4a5"
Jul 30 15:54:56.023370 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:56.023353    2578 scope.go:117] "RemoveContainer" containerID="0a1a0f8f7874632415fd76be600aaddba52cc5d4bf0e9c246f2c2f113f1925bc"
Jul 30 15:54:56.031235 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:56.031219    2578 scope.go:117] "RemoveContainer" containerID="5ee8c35aa05e2466d7e5efbefbc22062fd92eb3e837c644637ff2f1f83e2a4a5"
Jul 30 15:54:56.031461 ip-10-0-133-117 kubenswrapper[2578]: E0730 15:54:56.031441    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ee8c35aa05e2466d7e5efbefbc22062fd92eb3e837c644637ff2f1f83e2a4a5\": container with ID starting with 5ee8c35aa05e2466d7e5efbefbc22062fd92eb3e837c644637ff2f1f83e2a4a5 not found: ID does not exist" containerID="5ee8c35aa05e2466d7e5efbefbc22062fd92eb3e837c644637ff2f1f83e2a4a5"
Jul 30 15:54:56.031520 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:56.031469    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ee8c35aa05e2466d7e5efbefbc22062fd92eb3e837c644637ff2f1f83e2a4a5"} err="failed to get container status \"5ee8c35aa05e2466d7e5efbefbc22062fd92eb3e837c644637ff2f1f83e2a4a5\": rpc error: code = NotFound desc = could not find container \"5ee8c35aa05e2466d7e5efbefbc22062fd92eb3e837c644637ff2f1f83e2a4a5\": container with ID starting with 5ee8c35aa05e2466d7e5efbefbc22062fd92eb3e837c644637ff2f1f83e2a4a5 not found: ID does not exist"
Jul 30 15:54:56.031520 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:56.031485    2578 scope.go:117] "RemoveContainer" containerID="0a1a0f8f7874632415fd76be600aaddba52cc5d4bf0e9c246f2c2f113f1925bc"
Jul 30 15:54:56.031733 ip-10-0-133-117 kubenswrapper[2578]: E0730 15:54:56.031718    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a1a0f8f7874632415fd76be600aaddba52cc5d4bf0e9c246f2c2f113f1925bc\": container with ID starting with 0a1a0f8f7874632415fd76be600aaddba52cc5d4bf0e9c246f2c2f113f1925bc not found: ID does not exist" containerID="0a1a0f8f7874632415fd76be600aaddba52cc5d4bf0e9c246f2c2f113f1925bc"
Jul 30 15:54:56.031782 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:56.031736    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a1a0f8f7874632415fd76be600aaddba52cc5d4bf0e9c246f2c2f113f1925bc"} err="failed to get container status \"0a1a0f8f7874632415fd76be600aaddba52cc5d4bf0e9c246f2c2f113f1925bc\": rpc error: code = NotFound desc = could not find container \"0a1a0f8f7874632415fd76be600aaddba52cc5d4bf0e9c246f2c2f113f1925bc\": container with ID starting with 0a1a0f8f7874632415fd76be600aaddba52cc5d4bf0e9c246f2c2f113f1925bc not found: ID does not exist"
Jul 30 15:54:56.036612 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:56.036590    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s"]
Jul 30 15:54:56.039842 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:56.039818    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-2b1f1f-openshift-default-64ff5b5f5b-f4r2s"]
Jul 30 15:54:56.683343 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:56.683315    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6f8f467c55-jln6p_b17ea779-45c5-44d7-a8f7-f21d609224e0/manager/0.log"
Jul 30 15:54:56.978590 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:56.978511    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efe00480-68c7-4658-a34a-53ccefe2ba3e" path="/var/lib/kubelet/pods/efe00480-68c7-4658-a34a-53ccefe2ba3e/volumes"
Jul 30 15:54:57.320736 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:57.320711    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6bdff7d7c7-xqjhm_4086f535-988b-442d-b9cd-e2e7858f65e4/copy-manifests/0.log"
Jul 30 15:54:57.328031 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:57.328010    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6bdff7d7c7-xqjhm_4086f535-988b-442d-b9cd-e2e7858f65e4/manager/0.log"
Jul 30 15:54:57.690200 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:57.690111    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-ff4449fc7-cdnxs_54eeb596-5029-4848-997b-aeb298cda9b4/maas-api/0.log"
Jul 30 15:54:57.808095 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:57.808068    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29757075-pfxlq_093ed22f-2895-4cc9-ac0a-cfdcfbd240b9/cleanup/0.log"
Jul 30 15:54:58.185930 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:54:58.185903    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-wwfgt_3b517781-1975-48de-978c-c2148dc78f22/postgres/0.log"
Jul 30 15:55:01.215856 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:01.215806    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-59c4bbddff-cr7cb_bd797b96-255a-47e8-abb1-b2aa5620dd89/kube-auth-proxy/0.log"
Jul 30 15:55:01.592786 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:01.592753    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-bjkzn_2cb2ce43-84fb-480e-9b51-ea6c36a181b2/payload-processing/0.log"
Jul 30 15:55:02.241214 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:02.241186    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-g64tv_a874a8f6-a84a-493b-949e-6420d2278658/main/0.log"
Jul 30 15:55:02.483845 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:02.483813    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8vc55_d3664ed3-abea-4fa6-9437-1d6c941f85db/main/0.log"
Jul 30 15:55:08.326631 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:08.326598    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-ww6sr/must-gather-w8dbh"]
Jul 30 15:55:08.327067 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:08.326935    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9a2a6a2-e909-4d96-ac32-0f0bb8626d74" containerName="payload-pre-processing"
Jul 30 15:55:08.327067 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:08.326946    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9a2a6a2-e909-4d96-ac32-0f0bb8626d74" containerName="payload-pre-processing"
Jul 30 15:55:08.327067 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:08.326956    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea2aceba-fd54-43a6-8707-05f92f63ab75" containerName="istio-proxy"
Jul 30 15:55:08.327067 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:08.326962    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea2aceba-fd54-43a6-8707-05f92f63ab75" containerName="istio-proxy"
Jul 30 15:55:08.327067 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:08.326970    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ff6a146-1368-4741-8f31-9cb1f5814fcb" containerName="payload-processing"
Jul 30 15:55:08.327067 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:08.326976    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ff6a146-1368-4741-8f31-9cb1f5814fcb" containerName="payload-processing"
Jul 30 15:55:08.327067 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:08.326982    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea2aceba-fd54-43a6-8707-05f92f63ab75" containerName="istio-proxy"
Jul 30 15:55:08.327067 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:08.326987    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea2aceba-fd54-43a6-8707-05f92f63ab75" containerName="istio-proxy"
Jul 30 15:55:08.327067 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:08.326995    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf4b2e33-34b0-4d2b-9e30-57d132f17cb3" containerName="maas-api"
Jul 30 15:55:08.327067 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:08.327000    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf4b2e33-34b0-4d2b-9e30-57d132f17cb3" containerName="maas-api"
Jul 30 15:55:08.327067 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:08.327007    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efe00480-68c7-4658-a34a-53ccefe2ba3e" containerName="istio-proxy"
Jul 30 15:55:08.327067 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:08.327013    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="efe00480-68c7-4658-a34a-53ccefe2ba3e" containerName="istio-proxy"
Jul 30 15:55:08.327067 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:08.327024    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d9bfd52-6865-4f15-b84f-56748f8f4f58" containerName="payload-pre-processing"
Jul 30 15:55:08.327067 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:08.327029    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d9bfd52-6865-4f15-b84f-56748f8f4f58" containerName="payload-pre-processing"
Jul 30 15:55:08.327067 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:08.327035    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62142c95-4d70-4c88-8b5b-c479850ce466" containerName="payload-processing"
Jul 30 15:55:08.327067 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:08.327040    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="62142c95-4d70-4c88-8b5b-c479850ce466" containerName="payload-processing"
Jul 30 15:55:08.327067 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:08.327048    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b059341-28e1-4546-b960-2dd668984f4b" containerName="maas-api"
Jul 30 15:55:08.327067 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:08.327053    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b059341-28e1-4546-b960-2dd668984f4b" containerName="maas-api"
Jul 30 15:55:08.327067 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:08.327059    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4191ac92-c0bc-4c8d-9b52-22e11641fcd7" containerName="payload-pre-processing"
Jul 30 15:55:08.327067 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:08.327065    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4191ac92-c0bc-4c8d-9b52-22e11641fcd7" containerName="payload-pre-processing"
Jul 30 15:55:08.327067 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:08.327080    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fff1cd8b-5cd0-447d-8f23-6f4a7f86aa57" containerName="payload-processing"
Jul 30 15:55:08.327958 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:08.327085    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="fff1cd8b-5cd0-447d-8f23-6f4a7f86aa57" containerName="payload-processing"
Jul 30 15:55:08.327958 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:08.327093    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a2c0115-cc11-4717-b200-42d925bcd9ae" containerName="payload-processing"
Jul 30 15:55:08.327958 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:08.327098    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a2c0115-cc11-4717-b200-42d925bcd9ae" containerName="payload-processing"
Jul 30 15:55:08.327958 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:08.327104    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efe00480-68c7-4658-a34a-53ccefe2ba3e" containerName="istio-proxy"
Jul 30 15:55:08.327958 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:08.327109    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="efe00480-68c7-4658-a34a-53ccefe2ba3e" containerName="istio-proxy"
Jul 30 15:55:08.327958 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:08.327174    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="efe00480-68c7-4658-a34a-53ccefe2ba3e" containerName="istio-proxy"
Jul 30 15:55:08.327958 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:08.327182    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a2c0115-cc11-4717-b200-42d925bcd9ae" containerName="payload-processing"
Jul 30 15:55:08.327958 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:08.327189    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf4b2e33-34b0-4d2b-9e30-57d132f17cb3" containerName="maas-api"
Jul 30 15:55:08.327958 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:08.327195    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea2aceba-fd54-43a6-8707-05f92f63ab75" containerName="istio-proxy"
Jul 30 15:55:08.327958 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:08.327200    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="62142c95-4d70-4c88-8b5b-c479850ce466" containerName="payload-processing"
Jul 30 15:55:08.327958 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:08.327207    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d9bfd52-6865-4f15-b84f-56748f8f4f58" containerName="payload-pre-processing"
Jul 30 15:55:08.327958 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:08.327212    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b059341-28e1-4546-b960-2dd668984f4b" containerName="maas-api"
Jul 30 15:55:08.327958 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:08.327218    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="fff1cd8b-5cd0-447d-8f23-6f4a7f86aa57" containerName="payload-processing"
Jul 30 15:55:08.327958 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:08.327225    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ff6a146-1368-4741-8f31-9cb1f5814fcb" containerName="payload-processing"
Jul 30 15:55:08.327958 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:08.327231    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9a2a6a2-e909-4d96-ac32-0f0bb8626d74" containerName="payload-pre-processing"
Jul 30 15:55:08.327958 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:08.327236    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4191ac92-c0bc-4c8d-9b52-22e11641fcd7" containerName="payload-pre-processing"
Jul 30 15:55:08.327958 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:08.327339    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="efe00480-68c7-4658-a34a-53ccefe2ba3e" containerName="istio-proxy"
Jul 30 15:55:08.327958 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:08.327345    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea2aceba-fd54-43a6-8707-05f92f63ab75" containerName="istio-proxy"
Jul 30 15:55:08.330810 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:08.330789    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ww6sr/must-gather-w8dbh"
Jul 30 15:55:08.333046 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:08.333021    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ww6sr\"/\"openshift-service-ca.crt\""
Jul 30 15:55:08.333206 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:08.333183    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-ww6sr\"/\"default-dockercfg-sp7jx\""
Jul 30 15:55:08.333579 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:08.333564    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ww6sr\"/\"kube-root-ca.crt\""
Jul 30 15:55:08.341465 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:08.341443    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ww6sr/must-gather-w8dbh"]
Jul 30 15:55:08.377393 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:08.377363    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/3a20197b-a536-41a6-8d5e-54c1d6e0103c-must-gather-output\") pod \"must-gather-w8dbh\" (UID: \"3a20197b-a536-41a6-8d5e-54c1d6e0103c\") " pod="openshift-must-gather-ww6sr/must-gather-w8dbh"
Jul 30 15:55:08.377560 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:08.377411    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7s86\" (UniqueName: \"kubernetes.io/projected/3a20197b-a536-41a6-8d5e-54c1d6e0103c-kube-api-access-q7s86\") pod \"must-gather-w8dbh\" (UID: \"3a20197b-a536-41a6-8d5e-54c1d6e0103c\") " pod="openshift-must-gather-ww6sr/must-gather-w8dbh"
Jul 30 15:55:08.478201 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:08.478137    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/3a20197b-a536-41a6-8d5e-54c1d6e0103c-must-gather-output\") pod \"must-gather-w8dbh\" (UID: \"3a20197b-a536-41a6-8d5e-54c1d6e0103c\") " pod="openshift-must-gather-ww6sr/must-gather-w8dbh"
Jul 30 15:55:08.478372 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:08.478307    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7s86\" (UniqueName: \"kubernetes.io/projected/3a20197b-a536-41a6-8d5e-54c1d6e0103c-kube-api-access-q7s86\") pod \"must-gather-w8dbh\" (UID: \"3a20197b-a536-41a6-8d5e-54c1d6e0103c\") " pod="openshift-must-gather-ww6sr/must-gather-w8dbh"
Jul 30 15:55:08.478535 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:08.478517    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/3a20197b-a536-41a6-8d5e-54c1d6e0103c-must-gather-output\") pod \"must-gather-w8dbh\" (UID: \"3a20197b-a536-41a6-8d5e-54c1d6e0103c\") " pod="openshift-must-gather-ww6sr/must-gather-w8dbh"
Jul 30 15:55:08.487263 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:08.487238    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7s86\" (UniqueName: \"kubernetes.io/projected/3a20197b-a536-41a6-8d5e-54c1d6e0103c-kube-api-access-q7s86\") pod \"must-gather-w8dbh\" (UID: \"3a20197b-a536-41a6-8d5e-54c1d6e0103c\") " pod="openshift-must-gather-ww6sr/must-gather-w8dbh"
Jul 30 15:55:08.641898 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:08.641815    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ww6sr/must-gather-w8dbh"
Jul 30 15:55:08.772377 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:08.772352    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ww6sr/must-gather-w8dbh"]
Jul 30 15:55:08.774392 ip-10-0-133-117 kubenswrapper[2578]: W0730 15:55:08.774352    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3a20197b_a536_41a6_8d5e_54c1d6e0103c.slice/crio-931f4e089782eef5c550083ee2bea674cbdd78c4bd71e8ad673dc2ce99b4faee WatchSource:0}: Error finding container 931f4e089782eef5c550083ee2bea674cbdd78c4bd71e8ad673dc2ce99b4faee: Status 404 returned error can't find the container with id 931f4e089782eef5c550083ee2bea674cbdd78c4bd71e8ad673dc2ce99b4faee
Jul 30 15:55:09.074271 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:09.074235    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ww6sr/must-gather-w8dbh" event={"ID":"3a20197b-a536-41a6-8d5e-54c1d6e0103c","Type":"ContainerStarted","Data":"931f4e089782eef5c550083ee2bea674cbdd78c4bd71e8ad673dc2ce99b4faee"}
Jul 30 15:55:10.082957 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:10.082904    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ww6sr/must-gather-w8dbh" event={"ID":"3a20197b-a536-41a6-8d5e-54c1d6e0103c","Type":"ContainerStarted","Data":"f8020be42ad293645efc6ae78768de5e45d7aa3898cdc30d247c50316f2c6c39"}
Jul 30 15:55:10.083478 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:10.082976    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ww6sr/must-gather-w8dbh" event={"ID":"3a20197b-a536-41a6-8d5e-54c1d6e0103c","Type":"ContainerStarted","Data":"d598208a93ef3a092974634a11c052016666a7a2e7e217aa7d61d865a62abb53"}
Jul 30 15:55:10.100494 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:10.100432    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-ww6sr/must-gather-w8dbh" podStartSLOduration=1.085480378 podStartE2EDuration="2.100415408s" totalImagesPullingTime="1.01493503s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:55:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:55:10.098840751 +0000 UTC m=+4021.627779226" watchObservedRunningTime="2026-07-30 15:55:10.100415408 +0000 UTC m=+4021.629353895"
Jul 30 15:55:11.351472 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:11.351439    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-9lgq9_963eac2b-b8af-497c-a206-0a1c976a0e2c/global-pull-secret-syncer/0.log"
Jul 30 15:55:11.562403 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:11.562369    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-t65nt_b5b201dc-6dcd-44c8-9c70-9946315da836/konnectivity-agent/0.log"
Jul 30 15:55:11.615017 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:11.614940    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-117.ec2.internal_a5efaf84ea37e1b2796825887b9db661/haproxy/0.log"
Jul 30 15:55:19.442978 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:19.442937    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5c5df58559-ds468_00d9e19d-ff30-4139-984d-1c00ffbc85cd/metrics-server/0.log"
Jul 30 15:55:19.468872 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:19.468837    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-z4q4n_52b76eb8-59c8-4644-8dbf-aba3ace91eb2/monitoring-plugin/0.log"
Jul 30 15:55:19.503989 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:19.503945    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bf7cb_1991b76c-7420-4566-9e5e-9f17fd8d35de/node-exporter/0.log"
Jul 30 15:55:19.525725 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:19.525698    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bf7cb_1991b76c-7420-4566-9e5e-9f17fd8d35de/kube-rbac-proxy/0.log"
Jul 30 15:55:19.547791 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:19.547706    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bf7cb_1991b76c-7420-4566-9e5e-9f17fd8d35de/init-textfile/0.log"
Jul 30 15:55:19.747717 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:19.747669    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-4rg5f_9630d2b8-95d8-4c7b-a5a0-643b2746daf0/kube-rbac-proxy-main/0.log"
Jul 30 15:55:19.772374 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:19.772346    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-4rg5f_9630d2b8-95d8-4c7b-a5a0-643b2746daf0/kube-rbac-proxy-self/0.log"
Jul 30 15:55:19.796047 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:19.796019    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-4rg5f_9630d2b8-95d8-4c7b-a5a0-643b2746daf0/openshift-state-metrics/0.log"
Jul 30 15:55:19.843347 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:19.843305    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_13fc7a64-04f6-492b-bffd-c81059c7af28/prometheus/0.log"
Jul 30 15:55:19.863758 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:19.863731    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_13fc7a64-04f6-492b-bffd-c81059c7af28/config-reloader/0.log"
Jul 30 15:55:19.889031 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:19.889002    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_13fc7a64-04f6-492b-bffd-c81059c7af28/thanos-sidecar/0.log"
Jul 30 15:55:19.910996 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:19.910964    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_13fc7a64-04f6-492b-bffd-c81059c7af28/kube-rbac-proxy-web/0.log"
Jul 30 15:55:19.933321 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:19.933287    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_13fc7a64-04f6-492b-bffd-c81059c7af28/kube-rbac-proxy/0.log"
Jul 30 15:55:19.954879 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:19.954845    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_13fc7a64-04f6-492b-bffd-c81059c7af28/kube-rbac-proxy-thanos/0.log"
Jul 30 15:55:19.976116 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:19.976095    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_13fc7a64-04f6-492b-bffd-c81059c7af28/init-config-reloader/0.log"
Jul 30 15:55:21.764821 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:21.764785    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-ww6sr/perf-node-gather-daemonset-xmwls"]
Jul 30 15:55:21.773020 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:21.772993    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ww6sr/perf-node-gather-daemonset-xmwls"
Jul 30 15:55:21.777483 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:21.777073    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ww6sr/perf-node-gather-daemonset-xmwls"]
Jul 30 15:55:21.942063 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:21.942022    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ad700fc5-5e15-45c7-bde4-36736569465a-proc\") pod \"perf-node-gather-daemonset-xmwls\" (UID: \"ad700fc5-5e15-45c7-bde4-36736569465a\") " pod="openshift-must-gather-ww6sr/perf-node-gather-daemonset-xmwls"
Jul 30 15:55:21.942273 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:21.942086    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ad700fc5-5e15-45c7-bde4-36736569465a-lib-modules\") pod \"perf-node-gather-daemonset-xmwls\" (UID: \"ad700fc5-5e15-45c7-bde4-36736569465a\") " pod="openshift-must-gather-ww6sr/perf-node-gather-daemonset-xmwls"
Jul 30 15:55:21.942273 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:21.942130    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ad700fc5-5e15-45c7-bde4-36736569465a-sys\") pod \"perf-node-gather-daemonset-xmwls\" (UID: \"ad700fc5-5e15-45c7-bde4-36736569465a\") " pod="openshift-must-gather-ww6sr/perf-node-gather-daemonset-xmwls"
Jul 30 15:55:21.942273 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:21.942193    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ad700fc5-5e15-45c7-bde4-36736569465a-podres\") pod \"perf-node-gather-daemonset-xmwls\" (UID: \"ad700fc5-5e15-45c7-bde4-36736569465a\") " pod="openshift-must-gather-ww6sr/perf-node-gather-daemonset-xmwls"
Jul 30 15:55:21.942273 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:21.942252    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzdvc\" (UniqueName: \"kubernetes.io/projected/ad700fc5-5e15-45c7-bde4-36736569465a-kube-api-access-tzdvc\") pod \"perf-node-gather-daemonset-xmwls\" (UID: \"ad700fc5-5e15-45c7-bde4-36736569465a\") " pod="openshift-must-gather-ww6sr/perf-node-gather-daemonset-xmwls"
Jul 30 15:55:22.043622 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:22.043520    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzdvc\" (UniqueName: \"kubernetes.io/projected/ad700fc5-5e15-45c7-bde4-36736569465a-kube-api-access-tzdvc\") pod \"perf-node-gather-daemonset-xmwls\" (UID: \"ad700fc5-5e15-45c7-bde4-36736569465a\") " pod="openshift-must-gather-ww6sr/perf-node-gather-daemonset-xmwls"
Jul 30 15:55:22.043792 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:22.043624    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ad700fc5-5e15-45c7-bde4-36736569465a-proc\") pod \"perf-node-gather-daemonset-xmwls\" (UID: \"ad700fc5-5e15-45c7-bde4-36736569465a\") " pod="openshift-must-gather-ww6sr/perf-node-gather-daemonset-xmwls"
Jul 30 15:55:22.043792 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:22.043667    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ad700fc5-5e15-45c7-bde4-36736569465a-lib-modules\") pod \"perf-node-gather-daemonset-xmwls\" (UID: \"ad700fc5-5e15-45c7-bde4-36736569465a\") " pod="openshift-must-gather-ww6sr/perf-node-gather-daemonset-xmwls"
Jul 30 15:55:22.043792 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:22.043714    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ad700fc5-5e15-45c7-bde4-36736569465a-sys\") pod \"perf-node-gather-daemonset-xmwls\" (UID: \"ad700fc5-5e15-45c7-bde4-36736569465a\") " pod="openshift-must-gather-ww6sr/perf-node-gather-daemonset-xmwls"
Jul 30 15:55:22.043792 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:22.043746    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ad700fc5-5e15-45c7-bde4-36736569465a-proc\") pod \"perf-node-gather-daemonset-xmwls\" (UID: \"ad700fc5-5e15-45c7-bde4-36736569465a\") " pod="openshift-must-gather-ww6sr/perf-node-gather-daemonset-xmwls"
Jul 30 15:55:22.044033 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:22.043824    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ad700fc5-5e15-45c7-bde4-36736569465a-sys\") pod \"perf-node-gather-daemonset-xmwls\" (UID: \"ad700fc5-5e15-45c7-bde4-36736569465a\") " pod="openshift-must-gather-ww6sr/perf-node-gather-daemonset-xmwls"
Jul 30 15:55:22.044033 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:22.043764    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ad700fc5-5e15-45c7-bde4-36736569465a-podres\") pod \"perf-node-gather-daemonset-xmwls\" (UID: \"ad700fc5-5e15-45c7-bde4-36736569465a\") " pod="openshift-must-gather-ww6sr/perf-node-gather-daemonset-xmwls"
Jul 30 15:55:22.044033 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:22.043889    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ad700fc5-5e15-45c7-bde4-36736569465a-lib-modules\") pod \"perf-node-gather-daemonset-xmwls\" (UID: \"ad700fc5-5e15-45c7-bde4-36736569465a\") " pod="openshift-must-gather-ww6sr/perf-node-gather-daemonset-xmwls"
Jul 30 15:55:22.044033 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:22.043943    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ad700fc5-5e15-45c7-bde4-36736569465a-podres\") pod \"perf-node-gather-daemonset-xmwls\" (UID: \"ad700fc5-5e15-45c7-bde4-36736569465a\") " pod="openshift-must-gather-ww6sr/perf-node-gather-daemonset-xmwls"
Jul 30 15:55:22.053358 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:22.053326    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzdvc\" (UniqueName: \"kubernetes.io/projected/ad700fc5-5e15-45c7-bde4-36736569465a-kube-api-access-tzdvc\") pod \"perf-node-gather-daemonset-xmwls\" (UID: \"ad700fc5-5e15-45c7-bde4-36736569465a\") " pod="openshift-must-gather-ww6sr/perf-node-gather-daemonset-xmwls"
Jul 30 15:55:22.092337 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:22.092303    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ww6sr/perf-node-gather-daemonset-xmwls"
Jul 30 15:55:22.276712 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:22.276583    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ww6sr/perf-node-gather-daemonset-xmwls"]
Jul 30 15:55:22.279811 ip-10-0-133-117 kubenswrapper[2578]: W0730 15:55:22.279779    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podad700fc5_5e15_45c7_bde4_36736569465a.slice/crio-ae787b2976c0a615c8ee5ab9fca6b1e605037e79aacd88af5009062c7e370815 WatchSource:0}: Error finding container ae787b2976c0a615c8ee5ab9fca6b1e605037e79aacd88af5009062c7e370815: Status 404 returned error can't find the container with id ae787b2976c0a615c8ee5ab9fca6b1e605037e79aacd88af5009062c7e370815
Jul 30 15:55:22.282879 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:22.282855    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4mjmp_de241220-5f27-4b0b-b7c6-6d4b350e24e1/console-operator/1.log"
Jul 30 15:55:22.289261 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:22.289136    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4mjmp_de241220-5f27-4b0b-b7c6-6d4b350e24e1/console-operator/2.log"
Jul 30 15:55:22.830896 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:22.830858    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-cw96h_44c1d3a9-8aa4-4515-9a6c-2732f5eca386/download-server/0.log"
Jul 30 15:55:23.172544 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:23.172449    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ww6sr/perf-node-gather-daemonset-xmwls" event={"ID":"ad700fc5-5e15-45c7-bde4-36736569465a","Type":"ContainerStarted","Data":"58862ee648cbc3aa037361526734addcfe4b794064dbcdefe039364d2c836ee1"}
Jul 30 15:55:23.172544 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:23.172512    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ww6sr/perf-node-gather-daemonset-xmwls" event={"ID":"ad700fc5-5e15-45c7-bde4-36736569465a","Type":"ContainerStarted","Data":"ae787b2976c0a615c8ee5ab9fca6b1e605037e79aacd88af5009062c7e370815"}
Jul 30 15:55:23.172772 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:23.172580    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-ww6sr/perf-node-gather-daemonset-xmwls"
Jul 30 15:55:23.192965 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:23.192898    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-ww6sr/perf-node-gather-daemonset-xmwls" podStartSLOduration=2.192878378 podStartE2EDuration="2.192878378s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 15:55:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:55:23.190769717 +0000 UTC m=+4034.719708217" watchObservedRunningTime="2026-07-30 15:55:23.192878378 +0000 UTC m=+4034.721816853"
Jul 30 15:55:24.645878 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:24.645848    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nzzqf_64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1/dns/0.log"
Jul 30 15:55:24.667698 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:24.667670    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nzzqf_64ecf7a0-7c1e-4b43-b360-2a49baf8fdb1/kube-rbac-proxy/0.log"
Jul 30 15:55:24.695861 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:24.695825    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8jj48_fc042c9a-bc2a-4cf4-8606-e9e20e462cb3/dns-node-resolver/0.log"
Jul 30 15:55:25.326294 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:25.326254    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5b5697b754-9q6rd_a34002ea-a683-4b17-9e6a-419802c3e1c4/registry/0.log"
Jul 30 15:55:25.354942 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:25.354914    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9q72z_d9b6c536-3a43-49a8-853d-f9532668f149/node-ca/0.log"
Jul 30 15:55:26.650162 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:26.650118    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-59c4bbddff-cr7cb_bd797b96-255a-47e8-abb1-b2aa5620dd89/kube-auth-proxy/0.log"
Jul 30 15:55:26.784249 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:26.784219    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-bjkzn_2cb2ce43-84fb-480e-9b51-ea6c36a181b2/payload-processing/0.log"
Jul 30 15:55:27.698236 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:27.698195    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-frg92_8ce55aa3-7792-4ea0-83ed-64890b613632/serve-healthcheck-canary/0.log"
Jul 30 15:55:28.213514 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:28.213486    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-zqcwj_1f07334b-dd72-4771-b26f-bb23fc9a0ada/insights-operator/0.log"
Jul 30 15:55:28.214523 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:28.214505    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-zqcwj_1f07334b-dd72-4771-b26f-bb23fc9a0ada/insights-operator/1.log"
Jul 30 15:55:28.309314 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:28.309287    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9lk5c_623272af-807b-4e8c-b308-3c2737b1fe31/kube-rbac-proxy/0.log"
Jul 30 15:55:28.334002 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:28.333977    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9lk5c_623272af-807b-4e8c-b308-3c2737b1fe31/exporter/0.log"
Jul 30 15:55:28.356226 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:28.356190    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9lk5c_623272af-807b-4e8c-b308-3c2737b1fe31/extractor/0.log"
Jul 30 15:55:29.191332 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:29.191293    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-ww6sr/perf-node-gather-daemonset-xmwls"
Jul 30 15:55:31.305163 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:31.305116    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6f8f467c55-jln6p_b17ea779-45c5-44d7-a8f7-f21d609224e0/manager/0.log"
Jul 30 15:55:31.631472 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:31.631427    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6bdff7d7c7-xqjhm_4086f535-988b-442d-b9cd-e2e7858f65e4/manager/0.log"
Jul 30 15:55:31.663094 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:31.663054    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6bdff7d7c7-xqjhm_4086f535-988b-442d-b9cd-e2e7858f65e4/copy-manifests/0.log"
Jul 30 15:55:32.111093 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:32.111062    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-59c4bbddff-cr7cb_bd797b96-255a-47e8-abb1-b2aa5620dd89/kube-auth-proxy/0.log"
Jul 30 15:55:32.210318 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:32.210242    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-bjkzn_2cb2ce43-84fb-480e-9b51-ea6c36a181b2/payload-processing/0.log"
Jul 30 15:55:32.964132 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:32.964068    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7b696b7c-wdh8g_8179b7c2-1ef1-43a4-965f-7e44e6f58fb0/manager/0.log"
Jul 30 15:55:33.004770 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:33.004739    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-fhmcj_7dc9a4d7-863d-4f08-a805-304c39a22007/openshift-lws-operator/0.log"
Jul 30 15:55:33.903611 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:33.903575    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-59c4bbddff-cr7cb_bd797b96-255a-47e8-abb1-b2aa5620dd89/kube-auth-proxy/0.log"
Jul 30 15:55:34.000450 ip-10-0-133-117 kubenswrapper[2578]: I0730 15:55:34.000374    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-bjkzn_2cb2ce43-84fb-480e-9b51-ea6c36a181b2/payload-processing/0.log"