Jul 29 17:50:21.251627 ip-10-0-142-156 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 29 17:50:21.251639 ip-10-0-142-156 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 29 17:50:21.251649 ip-10-0-142-156 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 29 17:50:21.251998 ip-10-0-142-156 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 29 17:50:30.201434 ip-10-0-142-156 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 29 17:50:30.201450 ip-10-0-142-156 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 904491508c12485ca771ed4f2788275c --
Jul 29 17:52:00.001052 ip-10-0-142-156 systemd[1]: Starting Kubernetes Kubelet...
Jul 29 17:52:00.448521 ip-10-0-142-156 kubenswrapper[2557]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 17:52:00.448521 ip-10-0-142-156 kubenswrapper[2557]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 29 17:52:00.448521 ip-10-0-142-156 kubenswrapper[2557]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 17:52:00.448521 ip-10-0-142-156 kubenswrapper[2557]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 29 17:52:00.448521 ip-10-0-142-156 kubenswrapper[2557]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 17:52:00.450696 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.450588    2557 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 29 17:52:00.455546 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455527    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 17:52:00.455546 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455543    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 17:52:00.455546 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455549    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 17:52:00.455746 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455553    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 17:52:00.455746 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455558    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 17:52:00.455746 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455562    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 17:52:00.455746 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455567    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 17:52:00.455746 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455571    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 17:52:00.455746 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455575    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 17:52:00.455746 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455580    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 17:52:00.455746 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455584    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 17:52:00.455746 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455588    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 17:52:00.455746 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455591    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 17:52:00.455746 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455598    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 17:52:00.455746 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455603    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 17:52:00.455746 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455608    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 17:52:00.455746 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455617    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 17:52:00.455746 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455621    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 17:52:00.455746 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455627    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 17:52:00.455746 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455630    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 17:52:00.455746 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455635    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 17:52:00.455746 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455639    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 17:52:00.456574 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455643    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 17:52:00.456574 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455647    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 17:52:00.456574 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455651    2557 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 17:52:00.456574 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455655    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 17:52:00.456574 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455659    2557 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 17:52:00.456574 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455664    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 17:52:00.456574 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455669    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 17:52:00.456574 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455674    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 17:52:00.456574 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455679    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 17:52:00.456574 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455683    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 17:52:00.456574 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455687    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 17:52:00.456574 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455691    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 17:52:00.456574 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455695    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 17:52:00.456574 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455699    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 17:52:00.456574 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455704    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 17:52:00.456574 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455708    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 17:52:00.456574 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455713    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 17:52:00.456574 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455717    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 17:52:00.456574 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455741    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 17:52:00.456574 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455745    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 17:52:00.457418 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455750    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 17:52:00.457418 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455755    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 17:52:00.457418 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455760    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 17:52:00.457418 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455764    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 17:52:00.457418 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455768    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 17:52:00.457418 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455772    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 17:52:00.457418 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455776    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 17:52:00.457418 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455780    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 17:52:00.457418 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455784    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 17:52:00.457418 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455788    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 17:52:00.457418 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455793    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 17:52:00.457418 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455796    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 17:52:00.457418 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455801    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 17:52:00.457418 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455805    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 17:52:00.457418 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455809    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 17:52:00.457418 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455815    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 17:52:00.457418 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455819    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 17:52:00.457418 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455824    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 17:52:00.457418 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455828    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 17:52:00.457418 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455832    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 17:52:00.458076 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455839    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 17:52:00.458076 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455844    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 17:52:00.458076 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455848    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 17:52:00.458076 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455852    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 17:52:00.458076 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455856    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 17:52:00.458076 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455860    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 17:52:00.458076 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455866    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 17:52:00.458076 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455870    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 17:52:00.458076 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455875    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 17:52:00.458076 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455879    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 17:52:00.458076 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455884    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 17:52:00.458076 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455889    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 17:52:00.458076 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455893    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 17:52:00.458076 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455897    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 17:52:00.458076 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455903    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 17:52:00.458076 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455906    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 17:52:00.458076 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455911    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 17:52:00.458076 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455915    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 17:52:00.458076 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455920    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 17:52:00.458076 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455924    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 17:52:00.458585 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455928    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 17:52:00.458585 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455932    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 17:52:00.458585 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455939    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 17:52:00.458585 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455944    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 17:52:00.458585 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455950    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 17:52:00.458585 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455955    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 17:52:00.458585 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455960    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 17:52:00.458585 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455965    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 17:52:00.458585 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455969    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 17:52:00.458585 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455973    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 17:52:00.458585 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455977    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 17:52:00.458585 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455982    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 17:52:00.458585 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455986    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 17:52:00.458585 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455990    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 17:52:00.458585 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455994    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 17:52:00.458585 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.455999    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 17:52:00.458585 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.456003    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 17:52:00.458585 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457143    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 17:52:00.458585 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457153    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 17:52:00.459065 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457158    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 17:52:00.459065 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457162    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 17:52:00.459065 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457167    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 17:52:00.459065 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457171    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 17:52:00.459065 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457175    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 17:52:00.459065 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457179    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 17:52:00.459065 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457183    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 17:52:00.459065 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457188    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 17:52:00.459065 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457194    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 17:52:00.459065 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457198    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 17:52:00.459065 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457202    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 17:52:00.459065 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457208    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 17:52:00.459065 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457213    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 17:52:00.459065 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457217    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 17:52:00.459065 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457221    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 17:52:00.459065 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457225    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 17:52:00.459065 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457229    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 17:52:00.459065 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457234    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 17:52:00.459065 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457238    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 17:52:00.459065 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457243    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 17:52:00.459590 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457247    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 17:52:00.459590 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457252    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 17:52:00.459590 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457256    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 17:52:00.459590 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457261    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 17:52:00.459590 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457265    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 17:52:00.459590 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457270    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 17:52:00.459590 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457275    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 17:52:00.459590 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457279    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 17:52:00.459590 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457283    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 17:52:00.459590 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457287    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 17:52:00.459590 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457292    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 17:52:00.459590 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457297    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 17:52:00.459590 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457301    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 17:52:00.459590 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457305    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 17:52:00.459590 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457310    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 17:52:00.459590 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457315    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 17:52:00.459590 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457319    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 17:52:00.459590 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457323    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 17:52:00.459590 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457328    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 17:52:00.459590 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457332    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 17:52:00.460093 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457336    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 17:52:00.460093 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457341    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 17:52:00.460093 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457345    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 17:52:00.460093 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457350    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 17:52:00.460093 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457354    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 17:52:00.460093 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457358    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 17:52:00.460093 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457366    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 17:52:00.460093 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457372    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 17:52:00.460093 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457376    2557 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 17:52:00.460093 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457380    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 17:52:00.460093 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457384    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 17:52:00.460093 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457389    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 17:52:00.460093 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457393    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 17:52:00.460093 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457398    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 17:52:00.460093 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457402    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 17:52:00.460093 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457406    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 17:52:00.460093 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457410    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 17:52:00.460093 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457414    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 17:52:00.460093 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457418    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 17:52:00.460093 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457422    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 17:52:00.460577 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457426    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 17:52:00.460577 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457430    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 17:52:00.460577 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457434    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 17:52:00.460577 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457439    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 17:52:00.460577 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457443    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 17:52:00.460577 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457447    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 17:52:00.460577 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457452    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 17:52:00.460577 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457456    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 17:52:00.460577 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457461    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 17:52:00.460577 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457465    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 17:52:00.460577 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457472    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 17:52:00.460577 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457478    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 17:52:00.460577 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457482    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 17:52:00.460577 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457487    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 17:52:00.460577 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457492    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 17:52:00.460577 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457497    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 17:52:00.460577 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457502    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 17:52:00.460577 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457506    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 17:52:00.460577 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457511    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 17:52:00.461052 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457515    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 17:52:00.461052 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457520    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 17:52:00.461052 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457524    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 17:52:00.461052 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457530    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 17:52:00.461052 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457534    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 17:52:00.461052 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457538    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 17:52:00.461052 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457543    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 17:52:00.461052 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457550    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 17:52:00.461052 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457556    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 17:52:00.461052 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457561    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 17:52:00.461052 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457565    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 17:52:00.461052 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457570    2557 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 17:52:00.461052 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457574    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 17:52:00.461052 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457578    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 17:52:00.461052 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457582    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 17:52:00.461052 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457587    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 17:52:00.461052 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457591    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 17:52:00.461052 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.457596    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 17:52:00.461052 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458132    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 17:52:00.461520 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458145    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 17:52:00.461520 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458148    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 17:52:00.461520 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458151    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 17:52:00.461520 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458154    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 17:52:00.461520 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458157    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 17:52:00.461520 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458162    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 17:52:00.461520 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458166    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 17:52:00.461520 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458169    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 17:52:00.461520 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458172    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 17:52:00.461520 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458174    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 17:52:00.461520 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458177    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 17:52:00.461520 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458180    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 17:52:00.461520 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458182    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 17:52:00.461520 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458185    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 17:52:00.461520 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458188    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 17:52:00.461520 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458190    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 17:52:00.461520 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458193    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 17:52:00.461520 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458196    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 17:52:00.461520 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458198    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 17:52:00.462009 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458201    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 17:52:00.462009 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458212    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 17:52:00.462009 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458215    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 17:52:00.462009 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458218    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 17:52:00.462009 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458221    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 17:52:00.462009 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458223    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 17:52:00.462009 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458226    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 17:52:00.462009 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458229    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 17:52:00.462009 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458231    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 17:52:00.462009 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458234    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 17:52:00.462009 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458237    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 17:52:00.462009 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458239    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 17:52:00.462009 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458242    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 17:52:00.462009 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458246    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 17:52:00.462009 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458249    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 17:52:00.462009 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458251    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 17:52:00.462009 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458254    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 17:52:00.462009 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458260    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 17:52:00.462009 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458262    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 17:52:00.462009 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458265    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 17:52:00.462501 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458268    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 17:52:00.462501 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458270    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 17:52:00.462501 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458273    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 17:52:00.462501 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458276    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 17:52:00.462501 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458278    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 17:52:00.462501 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458280    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 17:52:00.462501 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458283    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 17:52:00.462501 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458287    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 17:52:00.462501 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458293    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 17:52:00.462501 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458297    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 17:52:00.462501 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458300    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 17:52:00.462501 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458304    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 17:52:00.462501 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458307    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 17:52:00.462501 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458309    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 17:52:00.462501 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458312    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 17:52:00.462501 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458315    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 17:52:00.462501 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458317    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 17:52:00.462501 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458320    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 17:52:00.462501 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458323    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 17:52:00.462979 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458326    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 17:52:00.462979 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458329    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 17:52:00.462979 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458331    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 17:52:00.462979 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458334    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 17:52:00.462979 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458336    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 17:52:00.462979 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458339    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 17:52:00.462979 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458344    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 17:52:00.462979 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458347    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 17:52:00.462979 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458349    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 17:52:00.462979 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458352    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 17:52:00.462979 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458356    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 17:52:00.462979 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458358    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 17:52:00.462979 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458361    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 17:52:00.462979 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458363    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 17:52:00.462979 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458366    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 17:52:00.462979 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458368    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 17:52:00.462979 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458371    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 17:52:00.462979 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458374    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 17:52:00.462979 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458376    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 17:52:00.462979 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458378    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 17:52:00.463471 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458381    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 17:52:00.463471 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458384    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 17:52:00.463471 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458386    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 17:52:00.463471 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458389    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 17:52:00.463471 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458392    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 17:52:00.463471 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458395    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 17:52:00.463471 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458397    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 17:52:00.463471 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458400    2557 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 17:52:00.463471 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458402    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 17:52:00.463471 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458405    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 17:52:00.463471 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458408    2557 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 17:52:00.463471 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458411    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 17:52:00.463471 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458413    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 17:52:00.463471 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458416    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 17:52:00.463471 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458418    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 17:52:00.463471 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458421    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 17:52:00.463471 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458423    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 17:52:00.463471 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458426    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 17:52:00.463471 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458430    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 17:52:00.463471 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.458433    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 17:52:00.463471 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458495    2557 flags.go:64] FLAG: --address="0.0.0.0"
Jul 29 17:52:00.464011 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458502    2557 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 29 17:52:00.464011 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458509    2557 flags.go:64] FLAG: --anonymous-auth="true"
Jul 29 17:52:00.464011 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458514    2557 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 29 17:52:00.464011 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458518    2557 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 29 17:52:00.464011 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458521    2557 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 29 17:52:00.464011 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458526    2557 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 29 17:52:00.464011 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458530    2557 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 29 17:52:00.464011 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458533    2557 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 29 17:52:00.464011 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458536    2557 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 29 17:52:00.464011 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458539    2557 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 29 17:52:00.464011 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458543    2557 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 29 17:52:00.464011 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458546    2557 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 29 17:52:00.464011 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458549    2557 flags.go:64] FLAG: --cgroup-root=""
Jul 29 17:52:00.464011 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458551    2557 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 29 17:52:00.464011 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458554    2557 flags.go:64] FLAG: --client-ca-file=""
Jul 29 17:52:00.464011 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458557    2557 flags.go:64] FLAG: --cloud-provider="external"
Jul 29 17:52:00.464011 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458560    2557 flags.go:64] FLAG: --cluster-dns="[]"
Jul 29 17:52:00.464011 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458564    2557 flags.go:64] FLAG: --cluster-domain=""
Jul 29 17:52:00.464011 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458567    2557 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 29 17:52:00.464011 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458570    2557 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 29 17:52:00.464011 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458573    2557 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 29 17:52:00.464011 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458577    2557 flags.go:64] FLAG: --container-log-max-files="5"
Jul 29 17:52:00.464011 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458580    2557 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 29 17:52:00.464011 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458583    2557 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 29 17:52:00.464624 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458592    2557 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 29 17:52:00.464624 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458595    2557 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 29 17:52:00.464624 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458598    2557 flags.go:64] FLAG: --contention-profiling="false"
Jul 29 17:52:00.464624 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458602    2557 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 29 17:52:00.464624 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458605    2557 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 29 17:52:00.464624 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458610    2557 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 29 17:52:00.464624 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458613    2557 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 29 17:52:00.464624 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458617    2557 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 29 17:52:00.464624 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458621    2557 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 29 17:52:00.464624 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458625    2557 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 29 17:52:00.464624 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458628    2557 flags.go:64] FLAG: --enable-load-reader="false"
Jul 29 17:52:00.464624 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458631    2557 flags.go:64] FLAG: --enable-server="true"
Jul 29 17:52:00.464624 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458634    2557 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 29 17:52:00.464624 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458638    2557 flags.go:64] FLAG: --event-burst="100"
Jul 29 17:52:00.464624 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458641    2557 flags.go:64] FLAG: --event-qps="50"
Jul 29 17:52:00.464624 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458644    2557 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 29 17:52:00.464624 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458647    2557 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 29 17:52:00.464624 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458650    2557 flags.go:64] FLAG: --eviction-hard=""
Jul 29 17:52:00.464624 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458653    2557 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 29 17:52:00.464624 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458656    2557 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 29 17:52:00.464624 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458659    2557 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 29 17:52:00.464624 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458662    2557 flags.go:64] FLAG: --eviction-soft=""
Jul 29 17:52:00.464624 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458665    2557 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 29 17:52:00.464624 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458667    2557 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 29 17:52:00.464624 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458671    2557 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 29 17:52:00.465346 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458673    2557 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 29 17:52:00.465346 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458676    2557 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 29 17:52:00.465346 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458679    2557 flags.go:64] FLAG: --fail-swap-on="true"
Jul 29 17:52:00.465346 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458682    2557 flags.go:64] FLAG: --feature-gates=""
Jul 29 17:52:00.465346 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458686    2557 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 29 17:52:00.465346 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458688    2557 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 29 17:52:00.465346 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458691    2557 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 29 17:52:00.465346 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458694    2557 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 29 17:52:00.465346 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458697    2557 flags.go:64] FLAG: --healthz-port="10248"
Jul 29 17:52:00.465346 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458700    2557 flags.go:64] FLAG: --help="false"
Jul 29 17:52:00.465346 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458703    2557 flags.go:64] FLAG: --hostname-override="ip-10-0-142-156.ec2.internal"
Jul 29 17:52:00.465346 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458706    2557 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 29 17:52:00.465346 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458710    2557 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 29 17:52:00.465346 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458713    2557 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 29 17:52:00.465346 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458716    2557 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 29 17:52:00.465346 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458737    2557 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 29 17:52:00.465346 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458742    2557 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 29 17:52:00.465346 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458745    2557 flags.go:64] FLAG: --image-service-endpoint=""
Jul 29 17:52:00.465346 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458748    2557 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 29 17:52:00.465346 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458751    2557 flags.go:64] FLAG: --kube-api-burst="100"
Jul 29 17:52:00.465346 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458754    2557 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 29 17:52:00.465346 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458758    2557 flags.go:64] FLAG: --kube-api-qps="50"
Jul 29 17:52:00.465346 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458760    2557 flags.go:64] FLAG: --kube-reserved=""
Jul 29 17:52:00.465346 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458763    2557 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 29 17:52:00.465976 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458766    2557 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 29 17:52:00.465976 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458769    2557 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 29 17:52:00.465976 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458772    2557 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 29 17:52:00.465976 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458775    2557 flags.go:64] FLAG: --lock-file=""
Jul 29 17:52:00.465976 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458778    2557 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 29 17:52:00.465976 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458781    2557 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 29 17:52:00.465976 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458784    2557 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 29 17:52:00.465976 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458789    2557 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 29 17:52:00.465976 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458792    2557 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 29 17:52:00.465976 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458795    2557 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 29 17:52:00.465976 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458798    2557 flags.go:64] FLAG: --logging-format="text"
Jul 29 17:52:00.465976 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458801    2557 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 29 17:52:00.465976 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458804    2557 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 29 17:52:00.465976 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458807    2557 flags.go:64] FLAG: --manifest-url=""
Jul 29 17:52:00.465976 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458810    2557 flags.go:64] FLAG: --manifest-url-header=""
Jul 29 17:52:00.465976 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458814    2557 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 29 17:52:00.465976 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458818    2557 flags.go:64] FLAG: --max-open-files="1000000"
Jul 29 17:52:00.465976 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458821    2557 flags.go:64] FLAG: --max-pods="110"
Jul 29 17:52:00.465976 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458824    2557 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 29 17:52:00.465976 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458827    2557 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 29 17:52:00.465976 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458831    2557 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 29 17:52:00.465976 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458834    2557 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 29 17:52:00.465976 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458837    2557 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 29 17:52:00.465976 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458840    2557 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 29 17:52:00.465976 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458844    2557 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 29 17:52:00.466863 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458851    2557 flags.go:64] FLAG: --node-status-max-images="50"
Jul 29 17:52:00.466863 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458854    2557 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 29 17:52:00.466863 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458858    2557 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 29 17:52:00.466863 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458860    2557 flags.go:64] FLAG: --pod-cidr=""
Jul 29 17:52:00.466863 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458863    2557 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 29 17:52:00.466863 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458868    2557 flags.go:64] FLAG: --pod-manifest-path=""
Jul 29 17:52:00.466863 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458872    2557 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 29 17:52:00.466863 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458875    2557 flags.go:64] FLAG: --pods-per-core="0"
Jul 29 17:52:00.466863 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458877    2557 flags.go:64] FLAG: --port="10250"
Jul 29 17:52:00.466863 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458880    2557 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 29 17:52:00.466863 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458883    2557 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0613189358fbdd344"
Jul 29 17:52:00.466863 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458887    2557 flags.go:64] FLAG: --qos-reserved=""
Jul 29 17:52:00.466863 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458889    2557 flags.go:64] FLAG: --read-only-port="10255"
Jul 29 17:52:00.466863 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458892    2557 flags.go:64] FLAG: --register-node="true"
Jul 29 17:52:00.466863 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458895    2557 flags.go:64] FLAG: --register-with-taints=""
Jul 29 17:52:00.466863 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458899    2557 flags.go:64] FLAG: --registry-burst="10"
Jul 29 17:52:00.466863 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458901    2557 flags.go:64] FLAG: --registry-qps="5"
Jul 29 17:52:00.466863 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458904    2557 flags.go:64] FLAG: --reserved-cpus=""
Jul 29 17:52:00.466863 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458907    2557 flags.go:64] FLAG: --reserved-memory=""
Jul 29 17:52:00.466863 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458911    2557 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 29 17:52:00.466863 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458914    2557 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 29 17:52:00.466863 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458916    2557 flags.go:64] FLAG: --rotate-certificates="false"
Jul 29 17:52:00.466863 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458919    2557 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 29 17:52:00.466863 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458922    2557 flags.go:64] FLAG: --runonce="false"
Jul 29 17:52:00.466863 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458924    2557 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 29 17:52:00.467599 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458927    2557 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 29 17:52:00.467599 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458930    2557 flags.go:64] FLAG: --seccomp-default="false"
Jul 29 17:52:00.467599 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458933    2557 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 29 17:52:00.467599 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458938    2557 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 29 17:52:00.467599 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458941    2557 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 29 17:52:00.467599 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458944    2557 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 29 17:52:00.467599 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458948    2557 flags.go:64] FLAG: --storage-driver-password="root"
Jul 29 17:52:00.467599 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458951    2557 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 29 17:52:00.467599 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458955    2557 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 29 17:52:00.467599 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458958    2557 flags.go:64] FLAG: --storage-driver-user="root"
Jul 29 17:52:00.467599 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458960    2557 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 29 17:52:00.467599 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458963    2557 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 29 17:52:00.467599 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458966    2557 flags.go:64] FLAG: --system-cgroups=""
Jul 29 17:52:00.467599 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458969    2557 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 29 17:52:00.467599 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458974    2557 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 29 17:52:00.467599 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458977    2557 flags.go:64] FLAG: --tls-cert-file=""
Jul 29 17:52:00.467599 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458980    2557 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 29 17:52:00.467599 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458984    2557 flags.go:64] FLAG: --tls-min-version=""
Jul 29 17:52:00.467599 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458987    2557 flags.go:64] FLAG: --tls-private-key-file=""
Jul 29 17:52:00.467599 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458989    2557 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 29 17:52:00.467599 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458992    2557 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 29 17:52:00.467599 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458995    2557 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 29 17:52:00.467599 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.458998    2557 flags.go:64] FLAG: --v="2"
Jul 29 17:52:00.467599 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.459001    2557 flags.go:64] FLAG: --version="false"
Jul 29 17:52:00.467599 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.459005    2557 flags.go:64] FLAG: --vmodule=""
Jul 29 17:52:00.468232 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.459009    2557 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 29 17:52:00.468232 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.459012    2557 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 29 17:52:00.468232 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459117    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 17:52:00.468232 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459121    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 17:52:00.468232 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459124    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 17:52:00.468232 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459127    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 17:52:00.468232 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459130    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 17:52:00.468232 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459133    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 17:52:00.468232 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459135    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 17:52:00.468232 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459138    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 17:52:00.468232 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459142    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 17:52:00.468232 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459144    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 17:52:00.468232 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459147    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 17:52:00.468232 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459150    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 17:52:00.468232 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459153    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 17:52:00.468232 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459156    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 17:52:00.468232 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459160    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 17:52:00.468232 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459164    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 17:52:00.468232 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459167    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 17:52:00.468738 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459170    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 17:52:00.468738 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459173    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 17:52:00.468738 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459176    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 17:52:00.468738 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459178    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 17:52:00.468738 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459181    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 17:52:00.468738 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459184    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 17:52:00.468738 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459186    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 17:52:00.468738 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459190    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 17:52:00.468738 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459193    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 17:52:00.468738 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459196    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 17:52:00.468738 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459199    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 17:52:00.468738 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459202    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 17:52:00.468738 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459204    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 17:52:00.468738 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459207    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 17:52:00.468738 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459209    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 17:52:00.468738 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459212    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 17:52:00.468738 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459214    2557 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 17:52:00.468738 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459217    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 17:52:00.468738 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459219    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 17:52:00.469212 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459222    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 17:52:00.469212 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459225    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 17:52:00.469212 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459227    2557 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 17:52:00.469212 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459229    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 17:52:00.469212 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459232    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 17:52:00.469212 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459235    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 17:52:00.469212 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459237    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 17:52:00.469212 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459241    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 17:52:00.469212 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459243    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 17:52:00.469212 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459246    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 17:52:00.469212 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459248    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 17:52:00.469212 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459251    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 17:52:00.469212 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459254    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 17:52:00.469212 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459256    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 17:52:00.469212 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459259    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 17:52:00.469212 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459261    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 17:52:00.469212 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459264    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 17:52:00.469212 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459266    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 17:52:00.469212 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459269    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 17:52:00.469212 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459272    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 17:52:00.469703 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459274    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 17:52:00.469703 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459277    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 17:52:00.469703 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459279    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 17:52:00.469703 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459281    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 17:52:00.469703 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459284    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 17:52:00.469703 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459286    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 17:52:00.469703 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459289    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 17:52:00.469703 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459291    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 17:52:00.469703 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459294    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 17:52:00.469703 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459297    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 17:52:00.469703 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459299    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 17:52:00.469703 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459302    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 17:52:00.469703 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459304    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 17:52:00.469703 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459307    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 17:52:00.469703 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459310    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 17:52:00.469703 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459312    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 17:52:00.469703 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459315    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 17:52:00.469703 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459318    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 17:52:00.469703 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459322    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 17:52:00.469703 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459326    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 17:52:00.470199 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459328    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 17:52:00.470199 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459331    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 17:52:00.470199 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459333    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 17:52:00.470199 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459336    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 17:52:00.470199 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459339    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 17:52:00.470199 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459342    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 17:52:00.470199 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459344    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 17:52:00.470199 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459347    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 17:52:00.470199 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459349    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 17:52:00.470199 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459352    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 17:52:00.470199 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459354    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 17:52:00.470199 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459357    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 17:52:00.470199 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459360    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 17:52:00.470199 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459362    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 17:52:00.470199 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459365    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 17:52:00.470199 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459367    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 17:52:00.470199 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459370    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 17:52:00.470199 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459372    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 17:52:00.470199 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459375    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 17:52:00.470199 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459378    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 17:52:00.470702 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459380    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 17:52:00.470702 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459383    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 17:52:00.470702 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.459385    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 17:52:00.470702 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.460137    2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 17:52:00.470702 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.468712    2557 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 29 17:52:00.470702 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.468747    2557 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 29 17:52:00.470702 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.468768    2557 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 29 17:52:00.470702 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.468773    2557 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 29 17:52:00.470702 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468828    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 17:52:00.470702 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468833    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 17:52:00.470702 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468836    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 17:52:00.470702 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468840    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 17:52:00.470702 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468843    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 17:52:00.470702 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468846    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 17:52:00.471065 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468849    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 17:52:00.471065 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468852    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 17:52:00.471065 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468854    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 17:52:00.471065 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468857    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 17:52:00.471065 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468860    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 17:52:00.471065 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468863    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 17:52:00.471065 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468865    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 17:52:00.471065 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468867    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 17:52:00.471065 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468870    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 17:52:00.471065 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468872    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 17:52:00.471065 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468875    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 17:52:00.471065 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468877    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 17:52:00.471065 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468880    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 17:52:00.471065 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468883    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 17:52:00.471065 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468886    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 17:52:00.471065 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468889    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 17:52:00.471065 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468892    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 17:52:00.471065 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468895    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 17:52:00.471065 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468897    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 17:52:00.471065 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468900    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 17:52:00.471545 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468903    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 17:52:00.471545 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468905    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 17:52:00.471545 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468908    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 17:52:00.471545 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468910    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 17:52:00.471545 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468913    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 17:52:00.471545 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468915    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 17:52:00.471545 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468918    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 17:52:00.471545 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468920    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 17:52:00.471545 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468923    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 17:52:00.471545 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468925    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 17:52:00.471545 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468930    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 17:52:00.471545 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468933    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 17:52:00.471545 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468936    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 17:52:00.471545 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468939    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 17:52:00.471545 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468942    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 17:52:00.471545 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468944    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 17:52:00.471545 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468947    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 17:52:00.471545 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468949    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 17:52:00.471545 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468952    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 17:52:00.471545 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468954    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 17:52:00.472094 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468957    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 17:52:00.472094 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468960    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 17:52:00.472094 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468962    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 17:52:00.472094 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468965    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 17:52:00.472094 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468967    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 17:52:00.472094 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468970    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 17:52:00.472094 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468973    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 17:52:00.472094 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468975    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 17:52:00.472094 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468978    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 17:52:00.472094 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468980    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 17:52:00.472094 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468983    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 17:52:00.472094 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468985    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 17:52:00.472094 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468988    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 17:52:00.472094 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468991    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 17:52:00.472094 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468993    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 17:52:00.472094 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468995    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 17:52:00.472094 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.468998    2557 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 17:52:00.472094 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469001    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 17:52:00.472094 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469004    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 17:52:00.472094 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469006    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 17:52:00.472573 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469010    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 17:52:00.472573 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469014    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 17:52:00.472573 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469017    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 17:52:00.472573 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469020    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 17:52:00.472573 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469022    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 17:52:00.472573 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469025    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 17:52:00.472573 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469028    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 17:52:00.472573 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469030    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 17:52:00.472573 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469032    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 17:52:00.472573 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469035    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 17:52:00.472573 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469038    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 17:52:00.472573 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469041    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 17:52:00.472573 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469043    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 17:52:00.472573 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469046    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 17:52:00.472573 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469049    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 17:52:00.472573 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469051    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 17:52:00.472573 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469054    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 17:52:00.472573 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469056    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 17:52:00.472573 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469059    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 17:52:00.473049 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469061    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 17:52:00.473049 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469064    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 17:52:00.473049 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469066    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 17:52:00.473049 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469069    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 17:52:00.473049 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469071    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 17:52:00.473049 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469074    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 17:52:00.473049 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469078    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 17:52:00.473049 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469081    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 17:52:00.473049 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469084    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 17:52:00.473049 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469087    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 17:52:00.473049 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469089    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 17:52:00.473049 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469092    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 17:52:00.473049 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469094    2557 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 17:52:00.473049 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469097    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 17:52:00.473387 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.469102    2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 17:52:00.473387 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469210    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 17:52:00.473387 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469214    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 17:52:00.473387 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469217    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 17:52:00.473387 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469220    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 17:52:00.473387 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469223    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 17:52:00.473387 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469226    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 17:52:00.473387 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469229    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 17:52:00.473387 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469233    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 17:52:00.473387 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469236    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 17:52:00.473387 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469239    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 17:52:00.473387 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469242    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 17:52:00.473387 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469244    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 17:52:00.473387 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469247    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 17:52:00.473387 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469250    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 17:52:00.473773 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469253    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 17:52:00.473773 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469255    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 17:52:00.473773 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469258    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 17:52:00.473773 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469260    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 17:52:00.473773 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469263    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 17:52:00.473773 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469265    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 17:52:00.473773 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469268    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 17:52:00.473773 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469270    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 17:52:00.473773 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469273    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 17:52:00.473773 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469275    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 17:52:00.473773 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469278    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 17:52:00.473773 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469280    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 17:52:00.473773 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469283    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 17:52:00.473773 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469287    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 17:52:00.473773 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469290    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 17:52:00.473773 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469293    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 17:52:00.473773 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469296    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 17:52:00.473773 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469299    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 17:52:00.473773 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469302    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 17:52:00.473773 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469304    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 17:52:00.474254 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469307    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 17:52:00.474254 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469310    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 17:52:00.474254 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469312    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 17:52:00.474254 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469315    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 17:52:00.474254 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469319    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 17:52:00.474254 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469322    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 17:52:00.474254 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469325    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 17:52:00.474254 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469327    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 17:52:00.474254 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469330    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 17:52:00.474254 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469332    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 17:52:00.474254 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469335    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 17:52:00.474254 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469338    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 17:52:00.474254 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469341    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 17:52:00.474254 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469343    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 17:52:00.474254 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469346    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 17:52:00.474254 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469348    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 17:52:00.474254 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469351    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 17:52:00.474254 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469353    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 17:52:00.474254 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469356    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 17:52:00.474713 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469358    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 17:52:00.474713 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469361    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 17:52:00.474713 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469363    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 17:52:00.474713 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469366    2557 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 17:52:00.474713 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469369    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 17:52:00.474713 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469371    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 17:52:00.474713 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469373    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 17:52:00.474713 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469376    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 17:52:00.474713 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469378    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 17:52:00.474713 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469381    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 17:52:00.474713 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469383    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 17:52:00.474713 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469386    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 17:52:00.474713 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469388    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 17:52:00.474713 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469391    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 17:52:00.474713 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469393    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 17:52:00.474713 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469395    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 17:52:00.474713 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469398    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 17:52:00.474713 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469400    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 17:52:00.474713 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469403    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 17:52:00.474713 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469405    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 17:52:00.475271 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469407    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 17:52:00.475271 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469410    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 17:52:00.475271 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469412    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 17:52:00.475271 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469415    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 17:52:00.475271 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469418    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 17:52:00.475271 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469421    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 17:52:00.475271 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469426    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 17:52:00.475271 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469429    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 17:52:00.475271 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469432    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 17:52:00.475271 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469435    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 17:52:00.475271 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469437    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 17:52:00.475271 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469440    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 17:52:00.475271 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469442    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 17:52:00.475271 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469445    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 17:52:00.475271 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469447    2557 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 17:52:00.475271 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469450    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 17:52:00.475271 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469453    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 17:52:00.475271 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469455    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 17:52:00.475271 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469458    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 17:52:00.475718 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469460    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 17:52:00.475718 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469463    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 17:52:00.475718 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469465    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 17:52:00.475718 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469468    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 17:52:00.475718 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469470    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 17:52:00.475718 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469473    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 17:52:00.475718 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:00.469475    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 17:52:00.475718 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.469480    2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 17:52:00.475718 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.469572    2557 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 29 17:52:00.475718 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.471932    2557 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 29 17:52:00.475718 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.473600    2557 server.go:1019] "Starting client certificate rotation"
Jul 29 17:52:00.475718 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.473702    2557 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 29 17:52:00.475718 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.473747    2557 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 29 17:52:00.476522 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.476503    2557 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 29 17:52:00.477768 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.477752    2557 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 29 17:52:00.485849 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.485834    2557 log.go:25] "Validated CRI v1 runtime API"
Jul 29 17:52:00.488308 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.488295    2557 log.go:25] "Validated CRI v1 image API"
Jul 29 17:52:00.489246 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.489229    2557 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 29 17:52:00.492687 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.492668    2557 fs.go:135] Filesystem UUIDs: map[3ba78cf3-48f9-42dc-82c8-5e8e80039505:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 88ab0942-6e58-4cb5-a491-36e5f6408fff:/dev/nvme0n1p4]
Jul 29 17:52:00.492769 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.492686    2557 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 29 17:52:00.498962 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.498852    2557 manager.go:216] Machine: {Timestamp:2026-07-29 17:52:00.498363582 +0000 UTC m=+0.367586381 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3085313 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec212c735384c53449cd3b42321d347f SystemUUID:ec212c73-5384-c534-49cd-3b42321d347f BootID:90449150-8c12-485c-a771-ed4f2788275c Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:5a:99:bf:6e:61 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:5a:99:bf:6e:61 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:9a:42:42:66:0b:30 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 29 17:52:00.498962 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.498952    2557 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 29 17:52:00.499086 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.499052    2557 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 29 17:52:00.502540 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.502517    2557 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 29 17:52:00.502667 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.502543    2557 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-156.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 29 17:52:00.503409 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.503397    2557 topology_manager.go:138] "Creating topology manager with none policy"
Jul 29 17:52:00.503409 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.503410    2557 container_manager_linux.go:309] "Creating device plugin manager"
Jul 29 17:52:00.503520 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.503418    2557 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 29 17:52:00.504269 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.504257    2557 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 29 17:52:00.506188 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.506178    2557 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 29 17:52:00.508946 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.508931    2557 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 29 17:52:00.511495 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.511482    2557 state_mem.go:36] "Initialized new in-memory state store"
Jul 29 17:52:00.511610 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.511601    2557 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 29 17:52:00.511643 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.511631    2557 kubelet.go:486] "Attempting to sync node with API server"
Jul 29 17:52:00.511677 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.511643    2557 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 29 17:52:00.511677 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.511659    2557 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 29 17:52:00.511677 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.511671    2557 kubelet.go:398] "Adding apiserver pod source"
Jul 29 17:52:00.511809 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.511683    2557 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 29 17:52:00.512029 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.512017    2557 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 29 17:52:00.512067 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.512036    2557 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 29 17:52:00.514869 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.514851    2557 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 29 17:52:00.516567 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.516546    2557 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 29 17:52:00.516614 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.516585    2557 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 29 17:52:00.517986 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.517972    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 29 17:52:00.518049 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.517992    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 29 17:52:00.518049 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.518000    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 29 17:52:00.518049 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.518006    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 29 17:52:00.518049 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.518020    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 29 17:52:00.518049 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.518026    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 29 17:52:00.518049 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.518034    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 29 17:52:00.518049 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.518040    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 29 17:52:00.518049 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.518046    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 29 17:52:00.518049 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.518051    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 29 17:52:00.518360 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.518062    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 29 17:52:00.518360 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.518069    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 29 17:52:00.518969 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.518957    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 29 17:52:00.519005 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.518971    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 29 17:52:00.522276 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.522262    2557 server.go:1268] "Started kubelet"
Jul 29 17:52:00.522457 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.522346    2557 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 29 17:52:00.522556 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.522441    2557 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 29 17:52:00.522634 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.522477    2557 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 29 17:52:00.523159 ip-10-0-142-156 systemd[1]: Started Kubernetes Kubelet.
Jul 29 17:52:00.524293 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.524270    2557 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-156.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 29 17:52:00.524352 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:00.524286    2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-156.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 29 17:52:00.524397 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:00.524376    2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 29 17:52:00.528997 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.528843    2557 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 29 17:52:00.529605 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.529580    2557 server.go:310] "Adding debug handlers to kubelet server"
Jul 29 17:52:00.529955 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:00.529916    2557 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 29 17:52:00.536126 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:00.534992    2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-156.ec2.internal.18c6d41c595a376a  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-156.ec2.internal,UID:ip-10-0-142-156.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-156.ec2.internal,},FirstTimestamp:2026-07-29 17:52:00.522237802 +0000 UTC m=+0.391460600,LastTimestamp:2026-07-29 17:52:00.522237802 +0000 UTC m=+0.391460600,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-156.ec2.internal,}"
Jul 29 17:52:00.538385 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.538370    2557 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 29 17:52:00.538458 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.538383    2557 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 29 17:52:00.538754 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:00.538554    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-156.ec2.internal\" not found"
Jul 29 17:52:00.539388 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.539162    2557 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 29 17:52:00.539388 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.539172    2557 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 29 17:52:00.539388 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.539189    2557 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 29 17:52:00.539388 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.539293    2557 reconstruct.go:97] "Volume reconstruction finished"
Jul 29 17:52:00.539388 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.539299    2557 factory.go:55] Registering systemd factory
Jul 29 17:52:00.539388 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.539315    2557 factory.go:223] Registration of the systemd container factory successfully
Jul 29 17:52:00.539388 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.539302    2557 reconciler.go:29] "Reconciler: start to sync state"
Jul 29 17:52:00.539740 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.539706    2557 factory.go:153] Registering CRI-O factory
Jul 29 17:52:00.539801 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.539743    2557 factory.go:223] Registration of the crio container factory successfully
Jul 29 17:52:00.539848 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.539795    2557 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 29 17:52:00.539897 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.539869    2557 factory.go:103] Registering Raw factory
Jul 29 17:52:00.539897 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.539894    2557 manager.go:1196] Started watching for new ooms in manager
Jul 29 17:52:00.540841 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:00.540489    2557 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-156.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 29 17:52:00.540841 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.540511    2557 manager.go:349] Starting recovery of all containers
Jul 29 17:52:00.540841 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:00.540601    2557 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 29 17:52:00.551057 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.551043    2557 manager.go:354] Recovery completed
Jul 29 17:52:00.554937 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.554924    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 17:52:00.557136 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.557115    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-156.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 17:52:00.557197 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.557151    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-156.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 17:52:00.557197 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.557166    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-156.ec2.internal" event="NodeHasSufficientPID"
Jul 29 17:52:00.557390 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.557376    2557 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xhdxv"
Jul 29 17:52:00.557687 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.557670    2557 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 29 17:52:00.557687 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.557683    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 17:52:00.557687 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.557686    2557 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 29 17:52:00.557879 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.557704    2557 state_mem.go:36] "Initialized new in-memory state store"
Jul 29 17:52:00.558535 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.558523    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-156.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 17:52:00.558580 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.558549    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-156.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 17:52:00.558580 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.558559    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-156.ec2.internal" event="NodeHasSufficientPID"
Jul 29 17:52:00.561422 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.561406    2557 policy_none.go:49] "None policy: Start"
Jul 29 17:52:00.561507 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.561425    2557 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 29 17:52:00.561507 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.561439    2557 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 29 17:52:00.563514 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.563502    2557 policy_none.go:47] "Start"
Jul 29 17:52:00.565062 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.565042    2557 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xhdxv"
Jul 29 17:52:00.568801 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:00.568626    2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-156.ec2.internal.18c6d41c5b6eb271  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-156.ec2.internal,UID:ip-10-0-142-156.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-156.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-156.ec2.internal,},FirstTimestamp:2026-07-29 17:52:00.557134449 +0000 UTC m=+0.426357252,LastTimestamp:2026-07-29 17:52:00.557134449 +0000 UTC m=+0.426357252,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-156.ec2.internal,}"
Jul 29 17:52:00.602349 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.602325    2557 manager.go:337] "Starting Device Plugin manager"
Jul 29 17:52:00.622962 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:00.602354    2557 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 29 17:52:00.622962 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.602364    2557 server.go:87] "Starting device plugin registration server"
Jul 29 17:52:00.622962 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.602656    2557 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 29 17:52:00.622962 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.602673    2557 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 29 17:52:00.622962 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.602793    2557 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 29 17:52:00.622962 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.602880    2557 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 29 17:52:00.622962 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.602896    2557 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 29 17:52:00.622962 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:00.603382    2557 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 29 17:52:00.622962 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:00.603438    2557 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-156.ec2.internal\" not found"
Jul 29 17:52:00.644449 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.644424    2557 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 29 17:52:00.645781 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.645767    2557 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 29 17:52:00.645870 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.645807    2557 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 29 17:52:00.645870 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.645830    2557 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 29 17:52:00.645957 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:00.645870    2557 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 29 17:52:00.647860 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.647838    2557 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 17:52:00.703051 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.702968    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 17:52:00.703977 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.703961    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-156.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 17:52:00.704080 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.703997    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-156.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 17:52:00.704080 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.704012    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-156.ec2.internal" event="NodeHasSufficientPID"
Jul 29 17:52:00.704080 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.704042    2557 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-156.ec2.internal"
Jul 29 17:52:00.716114 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.716097    2557 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-156.ec2.internal"
Jul 29 17:52:00.716177 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:00.716122    2557 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-156.ec2.internal\": node \"ip-10-0-142-156.ec2.internal\" not found"
Jul 29 17:52:00.737448 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:00.737418    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-156.ec2.internal\" not found"
Jul 29 17:52:00.746162 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.746136    2557 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-156.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-156.ec2.internal"]
Jul 29 17:52:00.746258 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.746196    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 17:52:00.747736 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.747704    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-156.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 17:52:00.747816 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.747753    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-156.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 17:52:00.747816 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.747766    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-156.ec2.internal" event="NodeHasSufficientPID"
Jul 29 17:52:00.750070 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.750058    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 17:52:00.750182 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.750168    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-156.ec2.internal"
Jul 29 17:52:00.750231 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.750213    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 17:52:00.750846 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.750830    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-156.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 17:52:00.750925 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.750861    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-156.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 17:52:00.750925 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.750871    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-156.ec2.internal" event="NodeHasSufficientPID"
Jul 29 17:52:00.750925 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.750833    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-156.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 17:52:00.750925 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.750925    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-156.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 17:52:00.751067 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.750938    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-156.ec2.internal" event="NodeHasSufficientPID"
Jul 29 17:52:00.753263 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.753240    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-156.ec2.internal"
Jul 29 17:52:00.753351 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.753280    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 17:52:00.753926 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.753911    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-156.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 17:52:00.753992 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.753941    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-156.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 17:52:00.753992 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.753957    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-156.ec2.internal" event="NodeHasSufficientPID"
Jul 29 17:52:00.777477 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:00.777452    2557 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-156.ec2.internal\" not found" node="ip-10-0-142-156.ec2.internal"
Jul 29 17:52:00.781831 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:00.781812    2557 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-156.ec2.internal\" not found" node="ip-10-0-142-156.ec2.internal"
Jul 29 17:52:00.838128 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:00.838098    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-156.ec2.internal\" not found"
Jul 29 17:52:00.841418 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.841391    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2e82424c7e271bc2aa44f8eb2dc7dc92-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-156.ec2.internal\" (UID: \"2e82424c7e271bc2aa44f8eb2dc7dc92\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-156.ec2.internal"
Jul 29 17:52:00.841548 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.841422    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2e82424c7e271bc2aa44f8eb2dc7dc92-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-156.ec2.internal\" (UID: \"2e82424c7e271bc2aa44f8eb2dc7dc92\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-156.ec2.internal"
Jul 29 17:52:00.841548 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.841445    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/591bc9043de6ffb60a0ece0a3a196263-config\") pod \"kube-apiserver-proxy-ip-10-0-142-156.ec2.internal\" (UID: \"591bc9043de6ffb60a0ece0a3a196263\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-156.ec2.internal"
Jul 29 17:52:00.938642 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:00.938608    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-156.ec2.internal\" not found"
Jul 29 17:52:00.941881 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.941865    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2e82424c7e271bc2aa44f8eb2dc7dc92-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-156.ec2.internal\" (UID: \"2e82424c7e271bc2aa44f8eb2dc7dc92\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-156.ec2.internal"
Jul 29 17:52:00.941950 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.941889    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2e82424c7e271bc2aa44f8eb2dc7dc92-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-156.ec2.internal\" (UID: \"2e82424c7e271bc2aa44f8eb2dc7dc92\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-156.ec2.internal"
Jul 29 17:52:00.941950 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.941906    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/591bc9043de6ffb60a0ece0a3a196263-config\") pod \"kube-apiserver-proxy-ip-10-0-142-156.ec2.internal\" (UID: \"591bc9043de6ffb60a0ece0a3a196263\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-156.ec2.internal"
Jul 29 17:52:00.942040 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.941946    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/591bc9043de6ffb60a0ece0a3a196263-config\") pod \"kube-apiserver-proxy-ip-10-0-142-156.ec2.internal\" (UID: \"591bc9043de6ffb60a0ece0a3a196263\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-156.ec2.internal"
Jul 29 17:52:00.942040 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.941992    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2e82424c7e271bc2aa44f8eb2dc7dc92-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-156.ec2.internal\" (UID: \"2e82424c7e271bc2aa44f8eb2dc7dc92\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-156.ec2.internal"
Jul 29 17:52:00.942040 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:00.941994    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2e82424c7e271bc2aa44f8eb2dc7dc92-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-156.ec2.internal\" (UID: \"2e82424c7e271bc2aa44f8eb2dc7dc92\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-156.ec2.internal"
Jul 29 17:52:01.039392 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:01.039361    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-156.ec2.internal\" not found"
Jul 29 17:52:01.079761 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:01.079707    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-156.ec2.internal"
Jul 29 17:52:01.085354 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:01.085135    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-156.ec2.internal"
Jul 29 17:52:01.140461 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:01.140420    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-156.ec2.internal\" not found"
Jul 29 17:52:01.240959 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:01.240925    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-156.ec2.internal\" not found"
Jul 29 17:52:01.341496 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:01.341420    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-156.ec2.internal\" not found"
Jul 29 17:52:01.442163 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:01.442130    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-156.ec2.internal\" not found"
Jul 29 17:52:01.472449 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:01.472415    2557 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 29 17:52:01.472856 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:01.472549    2557 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 29 17:52:01.513385 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:01.513352    2557 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 17:52:01.532934 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:01.532913    2557 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 17:52:01.542716 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:01.542689    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-156.ec2.internal\" not found"
Jul 29 17:52:01.571020 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:01.571000    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 17:52:01.571020 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:01.571001    2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-28 17:47:00 +0000 UTC" deadline="2028-04-22 15:20:33.762997742 +0000 UTC"
Jul 29 17:52:01.571159 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:01.571026    2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15189h28m32.19197428s"
Jul 29 17:52:01.572675 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:01.572654    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-156.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 17:52:01.572817 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:01.572687    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-156.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 17:52:01.572817 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:01.572700    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-156.ec2.internal" event="NodeHasSufficientPID"
Jul 29 17:52:01.643352 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:01.643292    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-156.ec2.internal\" not found"
Jul 29 17:52:01.656830 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:01.656800    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod591bc9043de6ffb60a0ece0a3a196263.slice/crio-c620795526d948ef5ce40d2d2b14305e472254c83adfb02feeb6283c27578ae7 WatchSource:0}: Error finding container c620795526d948ef5ce40d2d2b14305e472254c83adfb02feeb6283c27578ae7: Status 404 returned error can't find the container with id c620795526d948ef5ce40d2d2b14305e472254c83adfb02feeb6283c27578ae7
Jul 29 17:52:01.657344 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:01.657331    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e82424c7e271bc2aa44f8eb2dc7dc92.slice/crio-168120096f924b8f4d7ba0023647ce11ce368d74a19a5613b61895155f34f289 WatchSource:0}: Error finding container 168120096f924b8f4d7ba0023647ce11ce368d74a19a5613b61895155f34f289: Status 404 returned error can't find the container with id 168120096f924b8f4d7ba0023647ce11ce368d74a19a5613b61895155f34f289
Jul 29 17:52:01.660898 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:01.660883    2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 17:52:01.744033 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:01.744005    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-156.ec2.internal\" not found"
Jul 29 17:52:01.785882 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:01.785855    2557 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 17:52:01.839057 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:01.839022    2557 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-156.ec2.internal"
Jul 29 17:52:01.850072 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:01.850047    2557 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 29 17:52:01.851132 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:01.851119    2557 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-156.ec2.internal"
Jul 29 17:52:01.862842 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:01.862817    2557 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 29 17:52:02.512977 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.512942    2557 apiserver.go:52] "Watching apiserver"
Jul 29 17:52:02.521411 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.521382    2557 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 29 17:52:02.523421 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.523394    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-nx4kv","openshift-dns/node-resolver-5wxd2","openshift-image-registry/node-ca-vmhq9","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-156.ec2.internal","openshift-multus/multus-58bv4","kube-system/global-pull-secret-syncer-tdktb","openshift-multus/multus-additional-cni-plugins-rxw8d","openshift-multus/network-metrics-daemon-kbx7n","openshift-network-diagnostics/network-check-target-svfrm","openshift-network-operator/iptables-alerter-9s48j","openshift-ovn-kubernetes/ovnkube-node-t9s7z","kube-system/konnectivity-agent-mb6fv","kube-system/kube-apiserver-proxy-ip-10-0-142-156.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drvsm"]
Jul 29 17:52:02.525963 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.525942    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9s48j"
Jul 29 17:52:02.528427 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.528408    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.528988 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.528969    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 29 17:52:02.529075 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.528994    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 29 17:52:02.529127 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.529117    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 29 17:52:02.529178 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.529121    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-plb7c\""
Jul 29 17:52:02.530848 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.530827    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vmhq9"
Jul 29 17:52:02.530977 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.530959    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 29 17:52:02.532199 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.532181    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 29 17:52:02.532280 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.532201    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 29 17:52:02.532336 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.532293    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8svpb\""
Jul 29 17:52:02.532403 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.532385    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 29 17:52:02.532451 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.532423    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 29 17:52:02.532657 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.532633    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drvsm"
Jul 29 17:52:02.533311 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.533293    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 29 17:52:02.533880 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.533661    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 29 17:52:02.533880 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.533695    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 29 17:52:02.533880 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.533749    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-wjjxx\""
Jul 29 17:52:02.533880 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.533780    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 29 17:52:02.535480 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.535465    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-58bv4"
Jul 29 17:52:02.536452 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.536428    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 29 17:52:02.536542 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.536458    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 29 17:52:02.536542 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.536463    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-7lc9b\""
Jul 29 17:52:02.536542 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.536428    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 29 17:52:02.537511 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.537267    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tdktb"
Jul 29 17:52:02.537511 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:02.537338    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tdktb" podUID="905f9420-cd73-4e49-a85f-2163a04b8fcd"
Jul 29 17:52:02.539928 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.537886    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 29 17:52:02.539928 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.538173    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-djbq2\""
Jul 29 17:52:02.539928 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.538537    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 29 17:52:02.539928 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.538791    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 29 17:52:02.539928 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.538962    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 29 17:52:02.539928 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.539386    2557 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 29 17:52:02.540921 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.540902    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-svfrm"
Jul 29 17:52:02.541008 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:02.540980    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-svfrm" podUID="5540549f-0d35-480f-9135-2e51348f86ea"
Jul 29 17:52:02.543530 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.543506    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kbx7n"
Jul 29 17:52:02.543616 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:02.543593    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kbx7n" podUID="57afeaa9-b72d-4389-940b-f56a30b67995"
Jul 29 17:52:02.546143 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.545898    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5wxd2"
Jul 29 17:52:02.548783 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.548152    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mb6fv"
Jul 29 17:52:02.548783 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.548435    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 29 17:52:02.548783 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.548659    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 29 17:52:02.548918 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.548858    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-tbtcr\""
Jul 29 17:52:02.550399 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.550380    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 29 17:52:02.550672 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.550649    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-nhrjs\""
Jul 29 17:52:02.550989 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.550972    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rxw8d"
Jul 29 17:52:02.551096 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.550973    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 29 17:52:02.551970 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.551931    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/00b49d9f-e6ef-4506-89ea-ecb8a3c3d0e8-sys-fs\") pod \"aws-ebs-csi-driver-node-drvsm\" (UID: \"00b49d9f-e6ef-4506-89ea-ecb8a3c3d0e8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drvsm"
Jul 29 17:52:02.552056 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.551985    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9smkn\" (UniqueName: \"kubernetes.io/projected/3acade3e-0ce3-498e-99e4-c8a8af56ab59-kube-api-access-9smkn\") pod \"multus-58bv4\" (UID: \"3acade3e-0ce3-498e-99e4-c8a8af56ab59\") " pod="openshift-multus/multus-58bv4"
Jul 29 17:52:02.552056 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.552038    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7d94bd69-70db-430b-b18c-3bd4767d48da-etc-openvswitch\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.552140 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.552061    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7d94bd69-70db-430b-b18c-3bd4767d48da-ovnkube-config\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.552140 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.552097    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/00b49d9f-e6ef-4506-89ea-ecb8a3c3d0e8-device-dir\") pod \"aws-ebs-csi-driver-node-drvsm\" (UID: \"00b49d9f-e6ef-4506-89ea-ecb8a3c3d0e8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drvsm"
Jul 29 17:52:02.552140 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.552118    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7d94bd69-70db-430b-b18c-3bd4767d48da-ovnkube-script-lib\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.552279 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.552140    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpbkr\" (UniqueName: \"kubernetes.io/projected/8ef211d7-e236-4476-b5df-48aadf6e6195-kube-api-access-hpbkr\") pod \"node-ca-vmhq9\" (UID: \"8ef211d7-e236-4476-b5df-48aadf6e6195\") " pod="openshift-image-registry/node-ca-vmhq9"
Jul 29 17:52:02.552279 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.552162    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3acade3e-0ce3-498e-99e4-c8a8af56ab59-os-release\") pod \"multus-58bv4\" (UID: \"3acade3e-0ce3-498e-99e4-c8a8af56ab59\") " pod="openshift-multus/multus-58bv4"
Jul 29 17:52:02.552279 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.552184    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/905f9420-cd73-4e49-a85f-2163a04b8fcd-kubelet-config\") pod \"global-pull-secret-syncer-tdktb\" (UID: \"905f9420-cd73-4e49-a85f-2163a04b8fcd\") " pod="kube-system/global-pull-secret-syncer-tdktb"
Jul 29 17:52:02.552279 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.552212    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7d94bd69-70db-430b-b18c-3bd4767d48da-node-log\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.552279 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.552260    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/00b49d9f-e6ef-4506-89ea-ecb8a3c3d0e8-registration-dir\") pod \"aws-ebs-csi-driver-node-drvsm\" (UID: \"00b49d9f-e6ef-4506-89ea-ecb8a3c3d0e8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drvsm"
Jul 29 17:52:02.552461 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.552296    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7d94bd69-70db-430b-b18c-3bd4767d48da-host-slash\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.552461 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.552322    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3acade3e-0ce3-498e-99e4-c8a8af56ab59-host-run-multus-certs\") pod \"multus-58bv4\" (UID: \"3acade3e-0ce3-498e-99e4-c8a8af56ab59\") " pod="openshift-multus/multus-58bv4"
Jul 29 17:52:02.552461 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.552353    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7d94bd69-70db-430b-b18c-3bd4767d48da-host-run-netns\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.552461 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.552380    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7d94bd69-70db-430b-b18c-3bd4767d48da-var-lib-openvswitch\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.552461 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.552415    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7d94bd69-70db-430b-b18c-3bd4767d48da-host-run-ovn-kubernetes\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.552461 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.552436    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7d94bd69-70db-430b-b18c-3bd4767d48da-host-cni-bin\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.552461 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.552457    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8ef211d7-e236-4476-b5df-48aadf6e6195-serviceca\") pod \"node-ca-vmhq9\" (UID: \"8ef211d7-e236-4476-b5df-48aadf6e6195\") " pod="openshift-image-registry/node-ca-vmhq9"
Jul 29 17:52:02.552839 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.552494    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gmqt\" (UniqueName: \"kubernetes.io/projected/a238ef80-08e3-4247-b5c9-6d9dacaf76db-kube-api-access-8gmqt\") pod \"iptables-alerter-9s48j\" (UID: \"a238ef80-08e3-4247-b5c9-6d9dacaf76db\") " pod="openshift-network-operator/iptables-alerter-9s48j"
Jul 29 17:52:02.552839 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.552567    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7d94bd69-70db-430b-b18c-3bd4767d48da-run-openvswitch\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.552839 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.552583    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhzqd\" (UniqueName: \"kubernetes.io/projected/7d94bd69-70db-430b-b18c-3bd4767d48da-kube-api-access-bhzqd\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.552839 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.552611    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3acade3e-0ce3-498e-99e4-c8a8af56ab59-multus-daemon-config\") pod \"multus-58bv4\" (UID: \"3acade3e-0ce3-498e-99e4-c8a8af56ab59\") " pod="openshift-multus/multus-58bv4"
Jul 29 17:52:02.552839 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.552631    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7d94bd69-70db-430b-b18c-3bd4767d48da-host-kubelet\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.552839 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.552652    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7d94bd69-70db-430b-b18c-3bd4767d48da-host-cni-netd\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.552839 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.552688    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmh5k\" (UniqueName: \"kubernetes.io/projected/00b49d9f-e6ef-4506-89ea-ecb8a3c3d0e8-kube-api-access-jmh5k\") pod \"aws-ebs-csi-driver-node-drvsm\" (UID: \"00b49d9f-e6ef-4506-89ea-ecb8a3c3d0e8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drvsm"
Jul 29 17:52:02.552839 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.552709    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3acade3e-0ce3-498e-99e4-c8a8af56ab59-host-var-lib-cni-bin\") pod \"multus-58bv4\" (UID: \"3acade3e-0ce3-498e-99e4-c8a8af56ab59\") " pod="openshift-multus/multus-58bv4"
Jul 29 17:52:02.552839 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.552765    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/905f9420-cd73-4e49-a85f-2163a04b8fcd-original-pull-secret\") pod \"global-pull-secret-syncer-tdktb\" (UID: \"905f9420-cd73-4e49-a85f-2163a04b8fcd\") " pod="kube-system/global-pull-secret-syncer-tdktb"
Jul 29 17:52:02.552839 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.552784    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3acade3e-0ce3-498e-99e4-c8a8af56ab59-cnibin\") pod \"multus-58bv4\" (UID: \"3acade3e-0ce3-498e-99e4-c8a8af56ab59\") " pod="openshift-multus/multus-58bv4"
Jul 29 17:52:02.552839 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.552806    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3acade3e-0ce3-498e-99e4-c8a8af56ab59-host-var-lib-cni-multus\") pod \"multus-58bv4\" (UID: \"3acade3e-0ce3-498e-99e4-c8a8af56ab59\") " pod="openshift-multus/multus-58bv4"
Jul 29 17:52:02.553322 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.552848    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nx4kv"
Jul 29 17:52:02.553322 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.552847    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3acade3e-0ce3-498e-99e4-c8a8af56ab59-multus-conf-dir\") pod \"multus-58bv4\" (UID: \"3acade3e-0ce3-498e-99e4-c8a8af56ab59\") " pod="openshift-multus/multus-58bv4"
Jul 29 17:52:02.553322 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.553018    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a238ef80-08e3-4247-b5c9-6d9dacaf76db-host-slash\") pod \"iptables-alerter-9s48j\" (UID: \"a238ef80-08e3-4247-b5c9-6d9dacaf76db\") " pod="openshift-network-operator/iptables-alerter-9s48j"
Jul 29 17:52:02.553322 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.553044    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7d94bd69-70db-430b-b18c-3bd4767d48da-run-ovn\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.553322 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.553068    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7d94bd69-70db-430b-b18c-3bd4767d48da-ovn-node-metrics-cert\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.553322 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.553092    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8ef211d7-e236-4476-b5df-48aadf6e6195-host\") pod \"node-ca-vmhq9\" (UID: \"8ef211d7-e236-4476-b5df-48aadf6e6195\") " pod="openshift-image-registry/node-ca-vmhq9"
Jul 29 17:52:02.553322 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.553132    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3acade3e-0ce3-498e-99e4-c8a8af56ab59-cni-binary-copy\") pod \"multus-58bv4\" (UID: \"3acade3e-0ce3-498e-99e4-c8a8af56ab59\") " pod="openshift-multus/multus-58bv4"
Jul 29 17:52:02.553322 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.553163    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7d94bd69-70db-430b-b18c-3bd4767d48da-systemd-units\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.553322 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.553188    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znrl2\" (UniqueName: \"kubernetes.io/projected/5540549f-0d35-480f-9135-2e51348f86ea-kube-api-access-znrl2\") pod \"network-check-target-svfrm\" (UID: \"5540549f-0d35-480f-9135-2e51348f86ea\") " pod="openshift-network-diagnostics/network-check-target-svfrm"
Jul 29 17:52:02.553322 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.553211    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3acade3e-0ce3-498e-99e4-c8a8af56ab59-hostroot\") pod \"multus-58bv4\" (UID: \"3acade3e-0ce3-498e-99e4-c8a8af56ab59\") " pod="openshift-multus/multus-58bv4"
Jul 29 17:52:02.553322 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.553232    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3acade3e-0ce3-498e-99e4-c8a8af56ab59-etc-kubernetes\") pod \"multus-58bv4\" (UID: \"3acade3e-0ce3-498e-99e4-c8a8af56ab59\") " pod="openshift-multus/multus-58bv4"
Jul 29 17:52:02.553322 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.553252    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7d94bd69-70db-430b-b18c-3bd4767d48da-run-systemd\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.553322 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.553273    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7d94bd69-70db-430b-b18c-3bd4767d48da-log-socket\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.553322 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.553300    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/00b49d9f-e6ef-4506-89ea-ecb8a3c3d0e8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-drvsm\" (UID: \"00b49d9f-e6ef-4506-89ea-ecb8a3c3d0e8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drvsm"
Jul 29 17:52:02.553322 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.553328    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3acade3e-0ce3-498e-99e4-c8a8af56ab59-multus-cni-dir\") pod \"multus-58bv4\" (UID: \"3acade3e-0ce3-498e-99e4-c8a8af56ab59\") " pod="openshift-multus/multus-58bv4"
Jul 29 17:52:02.554014 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.553350    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3acade3e-0ce3-498e-99e4-c8a8af56ab59-host-run-k8s-cni-cncf-io\") pod \"multus-58bv4\" (UID: \"3acade3e-0ce3-498e-99e4-c8a8af56ab59\") " pod="openshift-multus/multus-58bv4"
Jul 29 17:52:02.554014 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.553373    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/905f9420-cd73-4e49-a85f-2163a04b8fcd-dbus\") pod \"global-pull-secret-syncer-tdktb\" (UID: \"905f9420-cd73-4e49-a85f-2163a04b8fcd\") " pod="kube-system/global-pull-secret-syncer-tdktb"
Jul 29 17:52:02.554014 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.553396    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a238ef80-08e3-4247-b5c9-6d9dacaf76db-iptables-alerter-script\") pod \"iptables-alerter-9s48j\" (UID: \"a238ef80-08e3-4247-b5c9-6d9dacaf76db\") " pod="openshift-network-operator/iptables-alerter-9s48j"
Jul 29 17:52:02.554014 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.553420    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7d94bd69-70db-430b-b18c-3bd4767d48da-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.554014 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.553435    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-qgz2n\""
Jul 29 17:52:02.554014 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.553446    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7d94bd69-70db-430b-b18c-3bd4767d48da-env-overrides\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.554014 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.553470    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/00b49d9f-e6ef-4506-89ea-ecb8a3c3d0e8-etc-selinux\") pod \"aws-ebs-csi-driver-node-drvsm\" (UID: \"00b49d9f-e6ef-4506-89ea-ecb8a3c3d0e8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drvsm"
Jul 29 17:52:02.554014 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.553493    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3acade3e-0ce3-498e-99e4-c8a8af56ab59-host-var-lib-kubelet\") pod \"multus-58bv4\" (UID: \"3acade3e-0ce3-498e-99e4-c8a8af56ab59\") " pod="openshift-multus/multus-58bv4"
Jul 29 17:52:02.554014 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.553514    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/00b49d9f-e6ef-4506-89ea-ecb8a3c3d0e8-socket-dir\") pod \"aws-ebs-csi-driver-node-drvsm\" (UID: \"00b49d9f-e6ef-4506-89ea-ecb8a3c3d0e8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drvsm"
Jul 29 17:52:02.554014 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.553535    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3acade3e-0ce3-498e-99e4-c8a8af56ab59-system-cni-dir\") pod \"multus-58bv4\" (UID: \"3acade3e-0ce3-498e-99e4-c8a8af56ab59\") " pod="openshift-multus/multus-58bv4"
Jul 29 17:52:02.554014 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.553544    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 29 17:52:02.554014 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.553556    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3acade3e-0ce3-498e-99e4-c8a8af56ab59-multus-socket-dir-parent\") pod \"multus-58bv4\" (UID: \"3acade3e-0ce3-498e-99e4-c8a8af56ab59\") " pod="openshift-multus/multus-58bv4"
Jul 29 17:52:02.554014 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.553577    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3acade3e-0ce3-498e-99e4-c8a8af56ab59-host-run-netns\") pod \"multus-58bv4\" (UID: \"3acade3e-0ce3-498e-99e4-c8a8af56ab59\") " pod="openshift-multus/multus-58bv4"
Jul 29 17:52:02.554014 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.554002    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 29 17:52:02.554574 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.554481    2557 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 29 17:52:02.555846 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.555821    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 29 17:52:02.555932 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.555824    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 29 17:52:02.555993 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.555940    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-9w5bh\""
Jul 29 17:52:02.572659 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.572627    2557 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-gnzgm"
Jul 29 17:52:02.583134 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.583111    2557 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-gnzgm"
Jul 29 17:52:02.639980 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.639957    2557 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 29 17:52:02.652353 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.652307    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-156.ec2.internal" event={"ID":"2e82424c7e271bc2aa44f8eb2dc7dc92","Type":"ContainerStarted","Data":"168120096f924b8f4d7ba0023647ce11ce368d74a19a5613b61895155f34f289"}
Jul 29 17:52:02.653950 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.653926    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-156.ec2.internal" event={"ID":"591bc9043de6ffb60a0ece0a3a196263","Type":"ContainerStarted","Data":"c620795526d948ef5ce40d2d2b14305e472254c83adfb02feeb6283c27578ae7"}
Jul 29 17:52:02.653950 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.653943    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8ed81c5a-bf81-4c94-aafa-1d54d038d8df-run\") pod \"tuned-nx4kv\" (UID: \"8ed81c5a-bf81-4c94-aafa-1d54d038d8df\") " pod="openshift-cluster-node-tuning-operator/tuned-nx4kv"
Jul 29 17:52:02.654134 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.653981    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7d94bd69-70db-430b-b18c-3bd4767d48da-node-log\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.654134 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.654016    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/00b49d9f-e6ef-4506-89ea-ecb8a3c3d0e8-registration-dir\") pod \"aws-ebs-csi-driver-node-drvsm\" (UID: \"00b49d9f-e6ef-4506-89ea-ecb8a3c3d0e8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drvsm"
Jul 29 17:52:02.654134 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.654050    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/85585baa-2c99-4feb-91ec-4a34a98df21b-tmp-dir\") pod \"node-resolver-5wxd2\" (UID: \"85585baa-2c99-4feb-91ec-4a34a98df21b\") " pod="openshift-dns/node-resolver-5wxd2"
Jul 29 17:52:02.654134 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.654065    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7d94bd69-70db-430b-b18c-3bd4767d48da-node-log\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.654134 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.654081    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f54sw\" (UniqueName: \"kubernetes.io/projected/85585baa-2c99-4feb-91ec-4a34a98df21b-kube-api-access-f54sw\") pod \"node-resolver-5wxd2\" (UID: \"85585baa-2c99-4feb-91ec-4a34a98df21b\") " pod="openshift-dns/node-resolver-5wxd2"
Jul 29 17:52:02.654134 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.654123    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8ed81c5a-bf81-4c94-aafa-1d54d038d8df-etc-kubernetes\") pod \"tuned-nx4kv\" (UID: \"8ed81c5a-bf81-4c94-aafa-1d54d038d8df\") " pod="openshift-cluster-node-tuning-operator/tuned-nx4kv"
Jul 29 17:52:02.654356 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.654181    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/00b49d9f-e6ef-4506-89ea-ecb8a3c3d0e8-registration-dir\") pod \"aws-ebs-csi-driver-node-drvsm\" (UID: \"00b49d9f-e6ef-4506-89ea-ecb8a3c3d0e8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drvsm"
Jul 29 17:52:02.654356 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.654237    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7d94bd69-70db-430b-b18c-3bd4767d48da-host-slash\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.654356 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.654295    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3acade3e-0ce3-498e-99e4-c8a8af56ab59-host-run-multus-certs\") pod \"multus-58bv4\" (UID: \"3acade3e-0ce3-498e-99e4-c8a8af56ab59\") " pod="openshift-multus/multus-58bv4"
Jul 29 17:52:02.654356 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.654308    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7d94bd69-70db-430b-b18c-3bd4767d48da-host-slash\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.654520 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.654357    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/277437f5-ac5a-4dd2-8f8e-c2155788e605-cnibin\") pod \"multus-additional-cni-plugins-rxw8d\" (UID: \"277437f5-ac5a-4dd2-8f8e-c2155788e605\") " pod="openshift-multus/multus-additional-cni-plugins-rxw8d"
Jul 29 17:52:02.656709 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.656685    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8ed81c5a-bf81-4c94-aafa-1d54d038d8df-etc-sysconfig\") pod \"tuned-nx4kv\" (UID: \"8ed81c5a-bf81-4c94-aafa-1d54d038d8df\") " pod="openshift-cluster-node-tuning-operator/tuned-nx4kv"
Jul 29 17:52:02.656828 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.656711    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3acade3e-0ce3-498e-99e4-c8a8af56ab59-host-run-multus-certs\") pod \"multus-58bv4\" (UID: \"3acade3e-0ce3-498e-99e4-c8a8af56ab59\") " pod="openshift-multus/multus-58bv4"
Jul 29 17:52:02.656894 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.656807    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8ed81c5a-bf81-4c94-aafa-1d54d038d8df-etc-systemd\") pod \"tuned-nx4kv\" (UID: \"8ed81c5a-bf81-4c94-aafa-1d54d038d8df\") " pod="openshift-cluster-node-tuning-operator/tuned-nx4kv"
Jul 29 17:52:02.656894 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.656865    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7d94bd69-70db-430b-b18c-3bd4767d48da-host-run-netns\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.657316 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.656897    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7d94bd69-70db-430b-b18c-3bd4767d48da-var-lib-openvswitch\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.657316 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.657306    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7d94bd69-70db-430b-b18c-3bd4767d48da-host-run-netns\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.657431 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.657343    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7d94bd69-70db-430b-b18c-3bd4767d48da-host-run-ovn-kubernetes\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.657431 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.657349    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7d94bd69-70db-430b-b18c-3bd4767d48da-var-lib-openvswitch\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.657431 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.657373    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7d94bd69-70db-430b-b18c-3bd4767d48da-host-cni-bin\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.657431 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.657396    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8ef211d7-e236-4476-b5df-48aadf6e6195-serviceca\") pod \"node-ca-vmhq9\" (UID: \"8ef211d7-e236-4476-b5df-48aadf6e6195\") " pod="openshift-image-registry/node-ca-vmhq9"
Jul 29 17:52:02.657431 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.657402    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7d94bd69-70db-430b-b18c-3bd4767d48da-host-run-ovn-kubernetes\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.657431 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.657419    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gmqt\" (UniqueName: \"kubernetes.io/projected/a238ef80-08e3-4247-b5c9-6d9dacaf76db-kube-api-access-8gmqt\") pod \"iptables-alerter-9s48j\" (UID: \"a238ef80-08e3-4247-b5c9-6d9dacaf76db\") " pod="openshift-network-operator/iptables-alerter-9s48j"
Jul 29 17:52:02.657431 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.657424    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7d94bd69-70db-430b-b18c-3bd4767d48da-host-cni-bin\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.657773 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.657440    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7d94bd69-70db-430b-b18c-3bd4767d48da-run-openvswitch\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.657773 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.657462    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhzqd\" (UniqueName: \"kubernetes.io/projected/7d94bd69-70db-430b-b18c-3bd4767d48da-kube-api-access-bhzqd\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.657773 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.657482    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3acade3e-0ce3-498e-99e4-c8a8af56ab59-multus-daemon-config\") pod \"multus-58bv4\" (UID: \"3acade3e-0ce3-498e-99e4-c8a8af56ab59\") " pod="openshift-multus/multus-58bv4"
Jul 29 17:52:02.657773 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.657506    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8ed81c5a-bf81-4c94-aafa-1d54d038d8df-lib-modules\") pod \"tuned-nx4kv\" (UID: \"8ed81c5a-bf81-4c94-aafa-1d54d038d8df\") " pod="openshift-cluster-node-tuning-operator/tuned-nx4kv"
Jul 29 17:52:02.657773 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.657514    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7d94bd69-70db-430b-b18c-3bd4767d48da-run-openvswitch\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.657773 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.657529    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7d94bd69-70db-430b-b18c-3bd4767d48da-host-kubelet\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.657773 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.657550    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7d94bd69-70db-430b-b18c-3bd4767d48da-host-cni-netd\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.657773 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.657572    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmh5k\" (UniqueName: \"kubernetes.io/projected/00b49d9f-e6ef-4506-89ea-ecb8a3c3d0e8-kube-api-access-jmh5k\") pod \"aws-ebs-csi-driver-node-drvsm\" (UID: \"00b49d9f-e6ef-4506-89ea-ecb8a3c3d0e8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drvsm"
Jul 29 17:52:02.657773 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.657602    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3acade3e-0ce3-498e-99e4-c8a8af56ab59-host-var-lib-cni-bin\") pod \"multus-58bv4\" (UID: \"3acade3e-0ce3-498e-99e4-c8a8af56ab59\") " pod="openshift-multus/multus-58bv4"
Jul 29 17:52:02.657773 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.657628    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/905f9420-cd73-4e49-a85f-2163a04b8fcd-original-pull-secret\") pod \"global-pull-secret-syncer-tdktb\" (UID: \"905f9420-cd73-4e49-a85f-2163a04b8fcd\") " pod="kube-system/global-pull-secret-syncer-tdktb"
Jul 29 17:52:02.657773 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.657650    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/277437f5-ac5a-4dd2-8f8e-c2155788e605-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rxw8d\" (UID: \"277437f5-ac5a-4dd2-8f8e-c2155788e605\") " pod="openshift-multus/multus-additional-cni-plugins-rxw8d"
Jul 29 17:52:02.657773 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.657678    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/277437f5-ac5a-4dd2-8f8e-c2155788e605-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rxw8d\" (UID: \"277437f5-ac5a-4dd2-8f8e-c2155788e605\") " pod="openshift-multus/multus-additional-cni-plugins-rxw8d"
Jul 29 17:52:02.657773 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.657709    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8ed81c5a-bf81-4c94-aafa-1d54d038d8df-host\") pod \"tuned-nx4kv\" (UID: \"8ed81c5a-bf81-4c94-aafa-1d54d038d8df\") " pod="openshift-cluster-node-tuning-operator/tuned-nx4kv"
Jul 29 17:52:02.657773 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.657749    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8ed81c5a-bf81-4c94-aafa-1d54d038d8df-tmp\") pod \"tuned-nx4kv\" (UID: \"8ed81c5a-bf81-4c94-aafa-1d54d038d8df\") " pod="openshift-cluster-node-tuning-operator/tuned-nx4kv"
Jul 29 17:52:02.657773 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.657771    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4v84d\" (UniqueName: \"kubernetes.io/projected/8ed81c5a-bf81-4c94-aafa-1d54d038d8df-kube-api-access-4v84d\") pod \"tuned-nx4kv\" (UID: \"8ed81c5a-bf81-4c94-aafa-1d54d038d8df\") " pod="openshift-cluster-node-tuning-operator/tuned-nx4kv"
Jul 29 17:52:02.658450 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.657794    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3acade3e-0ce3-498e-99e4-c8a8af56ab59-cnibin\") pod \"multus-58bv4\" (UID: \"3acade3e-0ce3-498e-99e4-c8a8af56ab59\") " pod="openshift-multus/multus-58bv4"
Jul 29 17:52:02.658450 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.657816    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3acade3e-0ce3-498e-99e4-c8a8af56ab59-host-var-lib-cni-multus\") pod \"multus-58bv4\" (UID: \"3acade3e-0ce3-498e-99e4-c8a8af56ab59\") " pod="openshift-multus/multus-58bv4"
Jul 29 17:52:02.658450 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.657857    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3acade3e-0ce3-498e-99e4-c8a8af56ab59-multus-conf-dir\") pod \"multus-58bv4\" (UID: \"3acade3e-0ce3-498e-99e4-c8a8af56ab59\") " pod="openshift-multus/multus-58bv4"
Jul 29 17:52:02.658450 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.657881    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thbz9\" (UniqueName: \"kubernetes.io/projected/277437f5-ac5a-4dd2-8f8e-c2155788e605-kube-api-access-thbz9\") pod \"multus-additional-cni-plugins-rxw8d\" (UID: \"277437f5-ac5a-4dd2-8f8e-c2155788e605\") " pod="openshift-multus/multus-additional-cni-plugins-rxw8d"
Jul 29 17:52:02.658450 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.657898    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7d94bd69-70db-430b-b18c-3bd4767d48da-host-kubelet\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.658450 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.657904    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a238ef80-08e3-4247-b5c9-6d9dacaf76db-host-slash\") pod \"iptables-alerter-9s48j\" (UID: \"a238ef80-08e3-4247-b5c9-6d9dacaf76db\") " pod="openshift-network-operator/iptables-alerter-9s48j"
Jul 29 17:52:02.658450 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.657937    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7d94bd69-70db-430b-b18c-3bd4767d48da-run-ovn\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.658450 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.657940    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8ef211d7-e236-4476-b5df-48aadf6e6195-serviceca\") pod \"node-ca-vmhq9\" (UID: \"8ef211d7-e236-4476-b5df-48aadf6e6195\") " pod="openshift-image-registry/node-ca-vmhq9"
Jul 29 17:52:02.658450 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.657947    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a238ef80-08e3-4247-b5c9-6d9dacaf76db-host-slash\") pod \"iptables-alerter-9s48j\" (UID: \"a238ef80-08e3-4247-b5c9-6d9dacaf76db\") " pod="openshift-network-operator/iptables-alerter-9s48j"
Jul 29 17:52:02.658450 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.657962    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7d94bd69-70db-430b-b18c-3bd4767d48da-ovn-node-metrics-cert\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.658450 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.657992    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8ef211d7-e236-4476-b5df-48aadf6e6195-host\") pod \"node-ca-vmhq9\" (UID: \"8ef211d7-e236-4476-b5df-48aadf6e6195\") " pod="openshift-image-registry/node-ca-vmhq9"
Jul 29 17:52:02.658450 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.658042    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3acade3e-0ce3-498e-99e4-c8a8af56ab59-multus-conf-dir\") pod \"multus-58bv4\" (UID: \"3acade3e-0ce3-498e-99e4-c8a8af56ab59\") " pod="openshift-multus/multus-58bv4"
Jul 29 17:52:02.658450 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.658048    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3acade3e-0ce3-498e-99e4-c8a8af56ab59-host-var-lib-cni-bin\") pod \"multus-58bv4\" (UID: \"3acade3e-0ce3-498e-99e4-c8a8af56ab59\") " pod="openshift-multus/multus-58bv4"
Jul 29 17:52:02.658450 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.658052    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3acade3e-0ce3-498e-99e4-c8a8af56ab59-cnibin\") pod \"multus-58bv4\" (UID: \"3acade3e-0ce3-498e-99e4-c8a8af56ab59\") " pod="openshift-multus/multus-58bv4"
Jul 29 17:52:02.658450 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.658083    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3acade3e-0ce3-498e-99e4-c8a8af56ab59-cni-binary-copy\") pod \"multus-58bv4\" (UID: \"3acade3e-0ce3-498e-99e4-c8a8af56ab59\") " pod="openshift-multus/multus-58bv4"
Jul 29 17:52:02.658450 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.658107    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1eecf48c-b25b-4449-a782-c3b702cc0368-agent-certs\") pod \"konnectivity-agent-mb6fv\" (UID: \"1eecf48c-b25b-4449-a782-c3b702cc0368\") " pod="kube-system/konnectivity-agent-mb6fv"
Jul 29 17:52:02.658450 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.658086    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8ef211d7-e236-4476-b5df-48aadf6e6195-host\") pod \"node-ca-vmhq9\" (UID: \"8ef211d7-e236-4476-b5df-48aadf6e6195\") " pod="openshift-image-registry/node-ca-vmhq9"
Jul 29 17:52:02.658450 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:02.658147    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 17:52:02.659300 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.658157    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7d94bd69-70db-430b-b18c-3bd4767d48da-systemd-units\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.659300 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.658131    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7d94bd69-70db-430b-b18c-3bd4767d48da-systemd-units\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.659300 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:02.658212    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/905f9420-cd73-4e49-a85f-2163a04b8fcd-original-pull-secret podName:905f9420-cd73-4e49-a85f-2163a04b8fcd nodeName:}" failed. No retries permitted until 2026-07-29 17:52:03.158196245 +0000 UTC m=+3.027419050 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/905f9420-cd73-4e49-a85f-2163a04b8fcd-original-pull-secret") pod "global-pull-secret-syncer-tdktb" (UID: "905f9420-cd73-4e49-a85f-2163a04b8fcd") : object "kube-system"/"original-pull-secret" not registered
Jul 29 17:52:02.659300 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.658229    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3acade3e-0ce3-498e-99e4-c8a8af56ab59-multus-daemon-config\") pod \"multus-58bv4\" (UID: \"3acade3e-0ce3-498e-99e4-c8a8af56ab59\") " pod="openshift-multus/multus-58bv4"
Jul 29 17:52:02.659300 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.658278    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znrl2\" (UniqueName: \"kubernetes.io/projected/5540549f-0d35-480f-9135-2e51348f86ea-kube-api-access-znrl2\") pod \"network-check-target-svfrm\" (UID: \"5540549f-0d35-480f-9135-2e51348f86ea\") " pod="openshift-network-diagnostics/network-check-target-svfrm"
Jul 29 17:52:02.659300 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.658284    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3acade3e-0ce3-498e-99e4-c8a8af56ab59-host-var-lib-cni-multus\") pod \"multus-58bv4\" (UID: \"3acade3e-0ce3-498e-99e4-c8a8af56ab59\") " pod="openshift-multus/multus-58bv4"
Jul 29 17:52:02.659300 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.658312    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3acade3e-0ce3-498e-99e4-c8a8af56ab59-hostroot\") pod \"multus-58bv4\" (UID: \"3acade3e-0ce3-498e-99e4-c8a8af56ab59\") " pod="openshift-multus/multus-58bv4"
Jul 29 17:52:02.659300 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.658326    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7d94bd69-70db-430b-b18c-3bd4767d48da-run-ovn\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.659300 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.658343    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3acade3e-0ce3-498e-99e4-c8a8af56ab59-etc-kubernetes\") pod \"multus-58bv4\" (UID: \"3acade3e-0ce3-498e-99e4-c8a8af56ab59\") " pod="openshift-multus/multus-58bv4"
Jul 29 17:52:02.659300 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.658363    2557 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 29 17:52:02.659300 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.658398    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3acade3e-0ce3-498e-99e4-c8a8af56ab59-hostroot\") pod \"multus-58bv4\" (UID: \"3acade3e-0ce3-498e-99e4-c8a8af56ab59\") " pod="openshift-multus/multus-58bv4"
Jul 29 17:52:02.659300 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.658438    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3acade3e-0ce3-498e-99e4-c8a8af56ab59-etc-kubernetes\") pod \"multus-58bv4\" (UID: \"3acade3e-0ce3-498e-99e4-c8a8af56ab59\") " pod="openshift-multus/multus-58bv4"
Jul 29 17:52:02.659300 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.658448    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/277437f5-ac5a-4dd2-8f8e-c2155788e605-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rxw8d\" (UID: \"277437f5-ac5a-4dd2-8f8e-c2155788e605\") " pod="openshift-multus/multus-additional-cni-plugins-rxw8d"
Jul 29 17:52:02.659300 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.658482    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57afeaa9-b72d-4389-940b-f56a30b67995-metrics-certs\") pod \"network-metrics-daemon-kbx7n\" (UID: \"57afeaa9-b72d-4389-940b-f56a30b67995\") " pod="openshift-multus/network-metrics-daemon-kbx7n"
Jul 29 17:52:02.659300 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.658506    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8ed81c5a-bf81-4c94-aafa-1d54d038d8df-etc-sysctl-d\") pod \"tuned-nx4kv\" (UID: \"8ed81c5a-bf81-4c94-aafa-1d54d038d8df\") " pod="openshift-cluster-node-tuning-operator/tuned-nx4kv"
Jul 29 17:52:02.659300 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.658515    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7d94bd69-70db-430b-b18c-3bd4767d48da-host-cni-netd\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.659300 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.658530    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7d94bd69-70db-430b-b18c-3bd4767d48da-run-systemd\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.660059 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.658556    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7d94bd69-70db-430b-b18c-3bd4767d48da-log-socket\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.660059 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.658570    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/00b49d9f-e6ef-4506-89ea-ecb8a3c3d0e8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-drvsm\" (UID: \"00b49d9f-e6ef-4506-89ea-ecb8a3c3d0e8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drvsm"
Jul 29 17:52:02.660059 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.658596    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7d94bd69-70db-430b-b18c-3bd4767d48da-run-systemd\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.660059 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.658609    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7d94bd69-70db-430b-b18c-3bd4767d48da-log-socket\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.660059 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.658627    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/00b49d9f-e6ef-4506-89ea-ecb8a3c3d0e8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-drvsm\" (UID: \"00b49d9f-e6ef-4506-89ea-ecb8a3c3d0e8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drvsm"
Jul 29 17:52:02.660059 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.658646    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3acade3e-0ce3-498e-99e4-c8a8af56ab59-multus-cni-dir\") pod \"multus-58bv4\" (UID: \"3acade3e-0ce3-498e-99e4-c8a8af56ab59\") " pod="openshift-multus/multus-58bv4"
Jul 29 17:52:02.660059 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.658676    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3acade3e-0ce3-498e-99e4-c8a8af56ab59-host-run-k8s-cni-cncf-io\") pod \"multus-58bv4\" (UID: \"3acade3e-0ce3-498e-99e4-c8a8af56ab59\") " pod="openshift-multus/multus-58bv4"
Jul 29 17:52:02.660059 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.658697    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/905f9420-cd73-4e49-a85f-2163a04b8fcd-dbus\") pod \"global-pull-secret-syncer-tdktb\" (UID: \"905f9420-cd73-4e49-a85f-2163a04b8fcd\") " pod="kube-system/global-pull-secret-syncer-tdktb"
Jul 29 17:52:02.660059 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.658740    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a238ef80-08e3-4247-b5c9-6d9dacaf76db-iptables-alerter-script\") pod \"iptables-alerter-9s48j\" (UID: \"a238ef80-08e3-4247-b5c9-6d9dacaf76db\") " pod="openshift-network-operator/iptables-alerter-9s48j"
Jul 29 17:52:02.660059 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.658744    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3acade3e-0ce3-498e-99e4-c8a8af56ab59-host-run-k8s-cni-cncf-io\") pod \"multus-58bv4\" (UID: \"3acade3e-0ce3-498e-99e4-c8a8af56ab59\") " pod="openshift-multus/multus-58bv4"
Jul 29 17:52:02.660059 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.658761    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7d94bd69-70db-430b-b18c-3bd4767d48da-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.660059 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.658783    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7d94bd69-70db-430b-b18c-3bd4767d48da-env-overrides\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.660059 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.658801    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/00b49d9f-e6ef-4506-89ea-ecb8a3c3d0e8-etc-selinux\") pod \"aws-ebs-csi-driver-node-drvsm\" (UID: \"00b49d9f-e6ef-4506-89ea-ecb8a3c3d0e8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drvsm"
Jul 29 17:52:02.660059 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.658763    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3acade3e-0ce3-498e-99e4-c8a8af56ab59-cni-binary-copy\") pod \"multus-58bv4\" (UID: \"3acade3e-0ce3-498e-99e4-c8a8af56ab59\") " pod="openshift-multus/multus-58bv4"
Jul 29 17:52:02.660059 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.658820    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3acade3e-0ce3-498e-99e4-c8a8af56ab59-host-var-lib-kubelet\") pod \"multus-58bv4\" (UID: \"3acade3e-0ce3-498e-99e4-c8a8af56ab59\") " pod="openshift-multus/multus-58bv4"
Jul 29 17:52:02.660059 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.658842    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/277437f5-ac5a-4dd2-8f8e-c2155788e605-os-release\") pod \"multus-additional-cni-plugins-rxw8d\" (UID: \"277437f5-ac5a-4dd2-8f8e-c2155788e605\") " pod="openshift-multus/multus-additional-cni-plugins-rxw8d"
Jul 29 17:52:02.660059 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.658855    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7d94bd69-70db-430b-b18c-3bd4767d48da-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.660904 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.658864    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/905f9420-cd73-4e49-a85f-2163a04b8fcd-dbus\") pod \"global-pull-secret-syncer-tdktb\" (UID: \"905f9420-cd73-4e49-a85f-2163a04b8fcd\") " pod="kube-system/global-pull-secret-syncer-tdktb"
Jul 29 17:52:02.660904 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.658876    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3acade3e-0ce3-498e-99e4-c8a8af56ab59-multus-cni-dir\") pod \"multus-58bv4\" (UID: \"3acade3e-0ce3-498e-99e4-c8a8af56ab59\") " pod="openshift-multus/multus-58bv4"
Jul 29 17:52:02.660904 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.658867    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mqj7\" (UniqueName: \"kubernetes.io/projected/57afeaa9-b72d-4389-940b-f56a30b67995-kube-api-access-5mqj7\") pod \"network-metrics-daemon-kbx7n\" (UID: \"57afeaa9-b72d-4389-940b-f56a30b67995\") " pod="openshift-multus/network-metrics-daemon-kbx7n"
Jul 29 17:52:02.660904 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.658903    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/00b49d9f-e6ef-4506-89ea-ecb8a3c3d0e8-etc-selinux\") pod \"aws-ebs-csi-driver-node-drvsm\" (UID: \"00b49d9f-e6ef-4506-89ea-ecb8a3c3d0e8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drvsm"
Jul 29 17:52:02.660904 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.658912    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3acade3e-0ce3-498e-99e4-c8a8af56ab59-host-var-lib-kubelet\") pod \"multus-58bv4\" (UID: \"3acade3e-0ce3-498e-99e4-c8a8af56ab59\") " pod="openshift-multus/multus-58bv4"
Jul 29 17:52:02.660904 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.658942    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8ed81c5a-bf81-4c94-aafa-1d54d038d8df-etc-modprobe-d\") pod \"tuned-nx4kv\" (UID: \"8ed81c5a-bf81-4c94-aafa-1d54d038d8df\") " pod="openshift-cluster-node-tuning-operator/tuned-nx4kv"
Jul 29 17:52:02.660904 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.658977    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/00b49d9f-e6ef-4506-89ea-ecb8a3c3d0e8-socket-dir\") pod \"aws-ebs-csi-driver-node-drvsm\" (UID: \"00b49d9f-e6ef-4506-89ea-ecb8a3c3d0e8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drvsm"
Jul 29 17:52:02.660904 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.659001    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3acade3e-0ce3-498e-99e4-c8a8af56ab59-system-cni-dir\") pod \"multus-58bv4\" (UID: \"3acade3e-0ce3-498e-99e4-c8a8af56ab59\") " pod="openshift-multus/multus-58bv4"
Jul 29 17:52:02.660904 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.659035    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1eecf48c-b25b-4449-a782-c3b702cc0368-konnectivity-ca\") pod \"konnectivity-agent-mb6fv\" (UID: \"1eecf48c-b25b-4449-a782-c3b702cc0368\") " pod="kube-system/konnectivity-agent-mb6fv"
Jul 29 17:52:02.660904 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.659067    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3acade3e-0ce3-498e-99e4-c8a8af56ab59-system-cni-dir\") pod \"multus-58bv4\" (UID: \"3acade3e-0ce3-498e-99e4-c8a8af56ab59\") " pod="openshift-multus/multus-58bv4"
Jul 29 17:52:02.660904 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.659071    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/00b49d9f-e6ef-4506-89ea-ecb8a3c3d0e8-socket-dir\") pod \"aws-ebs-csi-driver-node-drvsm\" (UID: \"00b49d9f-e6ef-4506-89ea-ecb8a3c3d0e8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drvsm"
Jul 29 17:52:02.660904 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.659105    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8ed81c5a-bf81-4c94-aafa-1d54d038d8df-sys\") pod \"tuned-nx4kv\" (UID: \"8ed81c5a-bf81-4c94-aafa-1d54d038d8df\") " pod="openshift-cluster-node-tuning-operator/tuned-nx4kv"
Jul 29 17:52:02.660904 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.659138    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8ed81c5a-bf81-4c94-aafa-1d54d038d8df-var-lib-kubelet\") pod \"tuned-nx4kv\" (UID: \"8ed81c5a-bf81-4c94-aafa-1d54d038d8df\") " pod="openshift-cluster-node-tuning-operator/tuned-nx4kv"
Jul 29 17:52:02.660904 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.659174    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3acade3e-0ce3-498e-99e4-c8a8af56ab59-multus-socket-dir-parent\") pod \"multus-58bv4\" (UID: \"3acade3e-0ce3-498e-99e4-c8a8af56ab59\") " pod="openshift-multus/multus-58bv4"
Jul 29 17:52:02.660904 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.659201    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3acade3e-0ce3-498e-99e4-c8a8af56ab59-host-run-netns\") pod \"multus-58bv4\" (UID: \"3acade3e-0ce3-498e-99e4-c8a8af56ab59\") " pod="openshift-multus/multus-58bv4"
Jul 29 17:52:02.660904 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.659222    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8ed81c5a-bf81-4c94-aafa-1d54d038d8df-etc-sysctl-conf\") pod \"tuned-nx4kv\" (UID: \"8ed81c5a-bf81-4c94-aafa-1d54d038d8df\") " pod="openshift-cluster-node-tuning-operator/tuned-nx4kv"
Jul 29 17:52:02.660904 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.659229    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3acade3e-0ce3-498e-99e4-c8a8af56ab59-multus-socket-dir-parent\") pod \"multus-58bv4\" (UID: \"3acade3e-0ce3-498e-99e4-c8a8af56ab59\") " pod="openshift-multus/multus-58bv4"
Jul 29 17:52:02.661645 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.659240    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3acade3e-0ce3-498e-99e4-c8a8af56ab59-host-run-netns\") pod \"multus-58bv4\" (UID: \"3acade3e-0ce3-498e-99e4-c8a8af56ab59\") " pod="openshift-multus/multus-58bv4"
Jul 29 17:52:02.661645 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.659244    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/00b49d9f-e6ef-4506-89ea-ecb8a3c3d0e8-sys-fs\") pod \"aws-ebs-csi-driver-node-drvsm\" (UID: \"00b49d9f-e6ef-4506-89ea-ecb8a3c3d0e8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drvsm"
Jul 29 17:52:02.661645 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.659270    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9smkn\" (UniqueName: \"kubernetes.io/projected/3acade3e-0ce3-498e-99e4-c8a8af56ab59-kube-api-access-9smkn\") pod \"multus-58bv4\" (UID: \"3acade3e-0ce3-498e-99e4-c8a8af56ab59\") " pod="openshift-multus/multus-58bv4"
Jul 29 17:52:02.661645 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.659306    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a238ef80-08e3-4247-b5c9-6d9dacaf76db-iptables-alerter-script\") pod \"iptables-alerter-9s48j\" (UID: \"a238ef80-08e3-4247-b5c9-6d9dacaf76db\") " pod="openshift-network-operator/iptables-alerter-9s48j"
Jul 29 17:52:02.661645 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.659310    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/00b49d9f-e6ef-4506-89ea-ecb8a3c3d0e8-sys-fs\") pod \"aws-ebs-csi-driver-node-drvsm\" (UID: \"00b49d9f-e6ef-4506-89ea-ecb8a3c3d0e8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drvsm"
Jul 29 17:52:02.661645 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.659328    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7d94bd69-70db-430b-b18c-3bd4767d48da-env-overrides\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.661645 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.659347    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/277437f5-ac5a-4dd2-8f8e-c2155788e605-system-cni-dir\") pod \"multus-additional-cni-plugins-rxw8d\" (UID: \"277437f5-ac5a-4dd2-8f8e-c2155788e605\") " pod="openshift-multus/multus-additional-cni-plugins-rxw8d"
Jul 29 17:52:02.661645 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.659366    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/277437f5-ac5a-4dd2-8f8e-c2155788e605-cni-binary-copy\") pod \"multus-additional-cni-plugins-rxw8d\" (UID: \"277437f5-ac5a-4dd2-8f8e-c2155788e605\") " pod="openshift-multus/multus-additional-cni-plugins-rxw8d"
Jul 29 17:52:02.661645 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.659384    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/85585baa-2c99-4feb-91ec-4a34a98df21b-hosts-file\") pod \"node-resolver-5wxd2\" (UID: \"85585baa-2c99-4feb-91ec-4a34a98df21b\") " pod="openshift-dns/node-resolver-5wxd2"
Jul 29 17:52:02.661645 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.659406    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8ed81c5a-bf81-4c94-aafa-1d54d038d8df-etc-tuned\") pod \"tuned-nx4kv\" (UID: \"8ed81c5a-bf81-4c94-aafa-1d54d038d8df\") " pod="openshift-cluster-node-tuning-operator/tuned-nx4kv"
Jul 29 17:52:02.661645 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.659430    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7d94bd69-70db-430b-b18c-3bd4767d48da-etc-openvswitch\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.661645 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.659445    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7d94bd69-70db-430b-b18c-3bd4767d48da-ovnkube-config\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.661645 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.659458    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/00b49d9f-e6ef-4506-89ea-ecb8a3c3d0e8-device-dir\") pod \"aws-ebs-csi-driver-node-drvsm\" (UID: \"00b49d9f-e6ef-4506-89ea-ecb8a3c3d0e8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drvsm"
Jul 29 17:52:02.661645 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.659471    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7d94bd69-70db-430b-b18c-3bd4767d48da-etc-openvswitch\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.661645 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.659472    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7d94bd69-70db-430b-b18c-3bd4767d48da-ovnkube-script-lib\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.661645 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.659512    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpbkr\" (UniqueName: \"kubernetes.io/projected/8ef211d7-e236-4476-b5df-48aadf6e6195-kube-api-access-hpbkr\") pod \"node-ca-vmhq9\" (UID: \"8ef211d7-e236-4476-b5df-48aadf6e6195\") " pod="openshift-image-registry/node-ca-vmhq9"
Jul 29 17:52:02.661645 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.659536    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3acade3e-0ce3-498e-99e4-c8a8af56ab59-os-release\") pod \"multus-58bv4\" (UID: \"3acade3e-0ce3-498e-99e4-c8a8af56ab59\") " pod="openshift-multus/multus-58bv4"
Jul 29 17:52:02.662437 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.659558    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/905f9420-cd73-4e49-a85f-2163a04b8fcd-kubelet-config\") pod \"global-pull-secret-syncer-tdktb\" (UID: \"905f9420-cd73-4e49-a85f-2163a04b8fcd\") " pod="kube-system/global-pull-secret-syncer-tdktb"
Jul 29 17:52:02.662437 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.659638    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/905f9420-cd73-4e49-a85f-2163a04b8fcd-kubelet-config\") pod \"global-pull-secret-syncer-tdktb\" (UID: \"905f9420-cd73-4e49-a85f-2163a04b8fcd\") " pod="kube-system/global-pull-secret-syncer-tdktb"
Jul 29 17:52:02.662437 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.659535    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/00b49d9f-e6ef-4506-89ea-ecb8a3c3d0e8-device-dir\") pod \"aws-ebs-csi-driver-node-drvsm\" (UID: \"00b49d9f-e6ef-4506-89ea-ecb8a3c3d0e8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drvsm"
Jul 29 17:52:02.662437 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.659741    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3acade3e-0ce3-498e-99e4-c8a8af56ab59-os-release\") pod \"multus-58bv4\" (UID: \"3acade3e-0ce3-498e-99e4-c8a8af56ab59\") " pod="openshift-multus/multus-58bv4"
Jul 29 17:52:02.662437 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.659946    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7d94bd69-70db-430b-b18c-3bd4767d48da-ovnkube-config\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.662437 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.659957    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7d94bd69-70db-430b-b18c-3bd4767d48da-ovnkube-script-lib\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.662437 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.661679    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7d94bd69-70db-430b-b18c-3bd4767d48da-ovn-node-metrics-cert\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.664168 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:02.664030    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 17:52:02.664168 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:02.664049    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 17:52:02.664168 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:02.664062    2557 projected.go:196] Error preparing data for projected volume kube-api-access-znrl2 for pod openshift-network-diagnostics/network-check-target-svfrm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 17:52:02.664168 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:02.664119    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5540549f-0d35-480f-9135-2e51348f86ea-kube-api-access-znrl2 podName:5540549f-0d35-480f-9135-2e51348f86ea nodeName:}" failed. No retries permitted until 2026-07-29 17:52:03.164103727 +0000 UTC m=+3.033326527 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-znrl2" (UniqueName: "kubernetes.io/projected/5540549f-0d35-480f-9135-2e51348f86ea-kube-api-access-znrl2") pod "network-check-target-svfrm" (UID: "5540549f-0d35-480f-9135-2e51348f86ea") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 17:52:02.674098 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.665625    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhzqd\" (UniqueName: \"kubernetes.io/projected/7d94bd69-70db-430b-b18c-3bd4767d48da-kube-api-access-bhzqd\") pod \"ovnkube-node-t9s7z\" (UID: \"7d94bd69-70db-430b-b18c-3bd4767d48da\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.674098 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.666370    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmh5k\" (UniqueName: \"kubernetes.io/projected/00b49d9f-e6ef-4506-89ea-ecb8a3c3d0e8-kube-api-access-jmh5k\") pod \"aws-ebs-csi-driver-node-drvsm\" (UID: \"00b49d9f-e6ef-4506-89ea-ecb8a3c3d0e8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drvsm"
Jul 29 17:52:02.674098 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.667649    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9smkn\" (UniqueName: \"kubernetes.io/projected/3acade3e-0ce3-498e-99e4-c8a8af56ab59-kube-api-access-9smkn\") pod \"multus-58bv4\" (UID: \"3acade3e-0ce3-498e-99e4-c8a8af56ab59\") " pod="openshift-multus/multus-58bv4"
Jul 29 17:52:02.674098 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.668078    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gmqt\" (UniqueName: \"kubernetes.io/projected/a238ef80-08e3-4247-b5c9-6d9dacaf76db-kube-api-access-8gmqt\") pod \"iptables-alerter-9s48j\" (UID: \"a238ef80-08e3-4247-b5c9-6d9dacaf76db\") " pod="openshift-network-operator/iptables-alerter-9s48j"
Jul 29 17:52:02.674098 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.668123    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpbkr\" (UniqueName: \"kubernetes.io/projected/8ef211d7-e236-4476-b5df-48aadf6e6195-kube-api-access-hpbkr\") pod \"node-ca-vmhq9\" (UID: \"8ef211d7-e236-4476-b5df-48aadf6e6195\") " pod="openshift-image-registry/node-ca-vmhq9"
Jul 29 17:52:02.760770 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.760705    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8ed81c5a-bf81-4c94-aafa-1d54d038d8df-run\") pod \"tuned-nx4kv\" (UID: \"8ed81c5a-bf81-4c94-aafa-1d54d038d8df\") " pod="openshift-cluster-node-tuning-operator/tuned-nx4kv"
Jul 29 17:52:02.760770 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.760777    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/85585baa-2c99-4feb-91ec-4a34a98df21b-tmp-dir\") pod \"node-resolver-5wxd2\" (UID: \"85585baa-2c99-4feb-91ec-4a34a98df21b\") " pod="openshift-dns/node-resolver-5wxd2"
Jul 29 17:52:02.760998 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.760800    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f54sw\" (UniqueName: \"kubernetes.io/projected/85585baa-2c99-4feb-91ec-4a34a98df21b-kube-api-access-f54sw\") pod \"node-resolver-5wxd2\" (UID: \"85585baa-2c99-4feb-91ec-4a34a98df21b\") " pod="openshift-dns/node-resolver-5wxd2"
Jul 29 17:52:02.760998 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.760825    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8ed81c5a-bf81-4c94-aafa-1d54d038d8df-etc-kubernetes\") pod \"tuned-nx4kv\" (UID: \"8ed81c5a-bf81-4c94-aafa-1d54d038d8df\") " pod="openshift-cluster-node-tuning-operator/tuned-nx4kv"
Jul 29 17:52:02.760998 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.760847    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8ed81c5a-bf81-4c94-aafa-1d54d038d8df-run\") pod \"tuned-nx4kv\" (UID: \"8ed81c5a-bf81-4c94-aafa-1d54d038d8df\") " pod="openshift-cluster-node-tuning-operator/tuned-nx4kv"
Jul 29 17:52:02.760998 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.760906    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8ed81c5a-bf81-4c94-aafa-1d54d038d8df-etc-kubernetes\") pod \"tuned-nx4kv\" (UID: \"8ed81c5a-bf81-4c94-aafa-1d54d038d8df\") " pod="openshift-cluster-node-tuning-operator/tuned-nx4kv"
Jul 29 17:52:02.760998 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.760908    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/277437f5-ac5a-4dd2-8f8e-c2155788e605-cnibin\") pod \"multus-additional-cni-plugins-rxw8d\" (UID: \"277437f5-ac5a-4dd2-8f8e-c2155788e605\") " pod="openshift-multus/multus-additional-cni-plugins-rxw8d"
Jul 29 17:52:02.760998 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.760851    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/277437f5-ac5a-4dd2-8f8e-c2155788e605-cnibin\") pod \"multus-additional-cni-plugins-rxw8d\" (UID: \"277437f5-ac5a-4dd2-8f8e-c2155788e605\") " pod="openshift-multus/multus-additional-cni-plugins-rxw8d"
Jul 29 17:52:02.760998 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.760952    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8ed81c5a-bf81-4c94-aafa-1d54d038d8df-etc-sysconfig\") pod \"tuned-nx4kv\" (UID: \"8ed81c5a-bf81-4c94-aafa-1d54d038d8df\") " pod="openshift-cluster-node-tuning-operator/tuned-nx4kv"
Jul 29 17:52:02.760998 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.760976    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8ed81c5a-bf81-4c94-aafa-1d54d038d8df-etc-systemd\") pod \"tuned-nx4kv\" (UID: \"8ed81c5a-bf81-4c94-aafa-1d54d038d8df\") " pod="openshift-cluster-node-tuning-operator/tuned-nx4kv"
Jul 29 17:52:02.760998 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.761002    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8ed81c5a-bf81-4c94-aafa-1d54d038d8df-lib-modules\") pod \"tuned-nx4kv\" (UID: \"8ed81c5a-bf81-4c94-aafa-1d54d038d8df\") " pod="openshift-cluster-node-tuning-operator/tuned-nx4kv"
Jul 29 17:52:02.761406 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.761037    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/277437f5-ac5a-4dd2-8f8e-c2155788e605-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rxw8d\" (UID: \"277437f5-ac5a-4dd2-8f8e-c2155788e605\") " pod="openshift-multus/multus-additional-cni-plugins-rxw8d"
Jul 29 17:52:02.761406 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.761073    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/277437f5-ac5a-4dd2-8f8e-c2155788e605-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rxw8d\" (UID: \"277437f5-ac5a-4dd2-8f8e-c2155788e605\") " pod="openshift-multus/multus-additional-cni-plugins-rxw8d"
Jul 29 17:52:02.761406 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.761099    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8ed81c5a-bf81-4c94-aafa-1d54d038d8df-host\") pod \"tuned-nx4kv\" (UID: \"8ed81c5a-bf81-4c94-aafa-1d54d038d8df\") " pod="openshift-cluster-node-tuning-operator/tuned-nx4kv"
Jul 29 17:52:02.761406 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.761120    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8ed81c5a-bf81-4c94-aafa-1d54d038d8df-tmp\") pod \"tuned-nx4kv\" (UID: \"8ed81c5a-bf81-4c94-aafa-1d54d038d8df\") " pod="openshift-cluster-node-tuning-operator/tuned-nx4kv"
Jul 29 17:52:02.761406 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.761131    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/85585baa-2c99-4feb-91ec-4a34a98df21b-tmp-dir\") pod \"node-resolver-5wxd2\" (UID: \"85585baa-2c99-4feb-91ec-4a34a98df21b\") " pod="openshift-dns/node-resolver-5wxd2"
Jul 29 17:52:02.761406 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.761150    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4v84d\" (UniqueName: \"kubernetes.io/projected/8ed81c5a-bf81-4c94-aafa-1d54d038d8df-kube-api-access-4v84d\") pod \"tuned-nx4kv\" (UID: \"8ed81c5a-bf81-4c94-aafa-1d54d038d8df\") " pod="openshift-cluster-node-tuning-operator/tuned-nx4kv"
Jul 29 17:52:02.761406 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.761179    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thbz9\" (UniqueName: \"kubernetes.io/projected/277437f5-ac5a-4dd2-8f8e-c2155788e605-kube-api-access-thbz9\") pod \"multus-additional-cni-plugins-rxw8d\" (UID: \"277437f5-ac5a-4dd2-8f8e-c2155788e605\") " pod="openshift-multus/multus-additional-cni-plugins-rxw8d"
Jul 29 17:52:02.761406 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.761134    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8ed81c5a-bf81-4c94-aafa-1d54d038d8df-etc-sysconfig\") pod \"tuned-nx4kv\" (UID: \"8ed81c5a-bf81-4c94-aafa-1d54d038d8df\") " pod="openshift-cluster-node-tuning-operator/tuned-nx4kv"
Jul 29 17:52:02.761406 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.761209    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1eecf48c-b25b-4449-a782-c3b702cc0368-agent-certs\") pod \"konnectivity-agent-mb6fv\" (UID: \"1eecf48c-b25b-4449-a782-c3b702cc0368\") " pod="kube-system/konnectivity-agent-mb6fv"
Jul 29 17:52:02.761406 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.761246    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/277437f5-ac5a-4dd2-8f8e-c2155788e605-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rxw8d\" (UID: \"277437f5-ac5a-4dd2-8f8e-c2155788e605\") " pod="openshift-multus/multus-additional-cni-plugins-rxw8d"
Jul 29 17:52:02.761406 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.761277    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57afeaa9-b72d-4389-940b-f56a30b67995-metrics-certs\") pod \"network-metrics-daemon-kbx7n\" (UID: \"57afeaa9-b72d-4389-940b-f56a30b67995\") " pod="openshift-multus/network-metrics-daemon-kbx7n"
Jul 29 17:52:02.761406 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.761320    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8ed81c5a-bf81-4c94-aafa-1d54d038d8df-etc-sysctl-d\") pod \"tuned-nx4kv\" (UID: \"8ed81c5a-bf81-4c94-aafa-1d54d038d8df\") " pod="openshift-cluster-node-tuning-operator/tuned-nx4kv"
Jul 29 17:52:02.761406 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.761359    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/277437f5-ac5a-4dd2-8f8e-c2155788e605-os-release\") pod \"multus-additional-cni-plugins-rxw8d\" (UID: \"277437f5-ac5a-4dd2-8f8e-c2155788e605\") " pod="openshift-multus/multus-additional-cni-plugins-rxw8d"
Jul 29 17:52:02.761406 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.761381    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mqj7\" (UniqueName: \"kubernetes.io/projected/57afeaa9-b72d-4389-940b-f56a30b67995-kube-api-access-5mqj7\") pod \"network-metrics-daemon-kbx7n\" (UID: \"57afeaa9-b72d-4389-940b-f56a30b67995\") " pod="openshift-multus/network-metrics-daemon-kbx7n"
Jul 29 17:52:02.761406 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.761405    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8ed81c5a-bf81-4c94-aafa-1d54d038d8df-etc-modprobe-d\") pod \"tuned-nx4kv\" (UID: \"8ed81c5a-bf81-4c94-aafa-1d54d038d8df\") " pod="openshift-cluster-node-tuning-operator/tuned-nx4kv"
Jul 29 17:52:02.762259 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.761430    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1eecf48c-b25b-4449-a782-c3b702cc0368-konnectivity-ca\") pod \"konnectivity-agent-mb6fv\" (UID: \"1eecf48c-b25b-4449-a782-c3b702cc0368\") " pod="kube-system/konnectivity-agent-mb6fv"
Jul 29 17:52:02.762259 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.761450    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8ed81c5a-bf81-4c94-aafa-1d54d038d8df-sys\") pod \"tuned-nx4kv\" (UID: \"8ed81c5a-bf81-4c94-aafa-1d54d038d8df\") " pod="openshift-cluster-node-tuning-operator/tuned-nx4kv"
Jul 29 17:52:02.762259 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.761470    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8ed81c5a-bf81-4c94-aafa-1d54d038d8df-var-lib-kubelet\") pod \"tuned-nx4kv\" (UID: \"8ed81c5a-bf81-4c94-aafa-1d54d038d8df\") " pod="openshift-cluster-node-tuning-operator/tuned-nx4kv"
Jul 29 17:52:02.762259 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.761495    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8ed81c5a-bf81-4c94-aafa-1d54d038d8df-etc-sysctl-conf\") pod \"tuned-nx4kv\" (UID: \"8ed81c5a-bf81-4c94-aafa-1d54d038d8df\") " pod="openshift-cluster-node-tuning-operator/tuned-nx4kv"
Jul 29 17:52:02.762259 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:02.761529    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 17:52:02.762259 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.761563    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/277437f5-ac5a-4dd2-8f8e-c2155788e605-system-cni-dir\") pod \"multus-additional-cni-plugins-rxw8d\" (UID: \"277437f5-ac5a-4dd2-8f8e-c2155788e605\") " pod="openshift-multus/multus-additional-cni-plugins-rxw8d"
Jul 29 17:52:02.762259 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.761592    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/277437f5-ac5a-4dd2-8f8e-c2155788e605-cni-binary-copy\") pod \"multus-additional-cni-plugins-rxw8d\" (UID: \"277437f5-ac5a-4dd2-8f8e-c2155788e605\") " pod="openshift-multus/multus-additional-cni-plugins-rxw8d"
Jul 29 17:52:02.762259 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:02.761607    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57afeaa9-b72d-4389-940b-f56a30b67995-metrics-certs podName:57afeaa9-b72d-4389-940b-f56a30b67995 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:03.261587859 +0000 UTC m=+3.130810644 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57afeaa9-b72d-4389-940b-f56a30b67995-metrics-certs") pod "network-metrics-daemon-kbx7n" (UID: "57afeaa9-b72d-4389-940b-f56a30b67995") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 17:52:02.762259 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.761643    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/85585baa-2c99-4feb-91ec-4a34a98df21b-hosts-file\") pod \"node-resolver-5wxd2\" (UID: \"85585baa-2c99-4feb-91ec-4a34a98df21b\") " pod="openshift-dns/node-resolver-5wxd2"
Jul 29 17:52:02.762259 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.761650    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/277437f5-ac5a-4dd2-8f8e-c2155788e605-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rxw8d\" (UID: \"277437f5-ac5a-4dd2-8f8e-c2155788e605\") " pod="openshift-multus/multus-additional-cni-plugins-rxw8d"
Jul 29 17:52:02.762259 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.761666    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8ed81c5a-bf81-4c94-aafa-1d54d038d8df-etc-tuned\") pod \"tuned-nx4kv\" (UID: \"8ed81c5a-bf81-4c94-aafa-1d54d038d8df\") " pod="openshift-cluster-node-tuning-operator/tuned-nx4kv"
Jul 29 17:52:02.762259 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.761791    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8ed81c5a-bf81-4c94-aafa-1d54d038d8df-host\") pod \"tuned-nx4kv\" (UID: \"8ed81c5a-bf81-4c94-aafa-1d54d038d8df\") " pod="openshift-cluster-node-tuning-operator/tuned-nx4kv"
Jul 29 17:52:02.762259 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.761100    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8ed81c5a-bf81-4c94-aafa-1d54d038d8df-etc-systemd\") pod \"tuned-nx4kv\" (UID: \"8ed81c5a-bf81-4c94-aafa-1d54d038d8df\") " pod="openshift-cluster-node-tuning-operator/tuned-nx4kv"
Jul 29 17:52:02.762259 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.761815    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8ed81c5a-bf81-4c94-aafa-1d54d038d8df-lib-modules\") pod \"tuned-nx4kv\" (UID: \"8ed81c5a-bf81-4c94-aafa-1d54d038d8df\") " pod="openshift-cluster-node-tuning-operator/tuned-nx4kv"
Jul 29 17:52:02.762259 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.761940    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/277437f5-ac5a-4dd2-8f8e-c2155788e605-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rxw8d\" (UID: \"277437f5-ac5a-4dd2-8f8e-c2155788e605\") " pod="openshift-multus/multus-additional-cni-plugins-rxw8d"
Jul 29 17:52:02.762259 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.762021    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/277437f5-ac5a-4dd2-8f8e-c2155788e605-cni-binary-copy\") pod \"multus-additional-cni-plugins-rxw8d\" (UID: \"277437f5-ac5a-4dd2-8f8e-c2155788e605\") " pod="openshift-multus/multus-additional-cni-plugins-rxw8d"
Jul 29 17:52:02.762259 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.762151    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8ed81c5a-bf81-4c94-aafa-1d54d038d8df-etc-sysctl-d\") pod \"tuned-nx4kv\" (UID: \"8ed81c5a-bf81-4c94-aafa-1d54d038d8df\") " pod="openshift-cluster-node-tuning-operator/tuned-nx4kv"
Jul 29 17:52:02.763072 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.762220    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/277437f5-ac5a-4dd2-8f8e-c2155788e605-os-release\") pod \"multus-additional-cni-plugins-rxw8d\" (UID: \"277437f5-ac5a-4dd2-8f8e-c2155788e605\") " pod="openshift-multus/multus-additional-cni-plugins-rxw8d"
Jul 29 17:52:02.763072 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.762285    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/85585baa-2c99-4feb-91ec-4a34a98df21b-hosts-file\") pod \"node-resolver-5wxd2\" (UID: \"85585baa-2c99-4feb-91ec-4a34a98df21b\") " pod="openshift-dns/node-resolver-5wxd2"
Jul 29 17:52:02.763072 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.762286    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8ed81c5a-bf81-4c94-aafa-1d54d038d8df-sys\") pod \"tuned-nx4kv\" (UID: \"8ed81c5a-bf81-4c94-aafa-1d54d038d8df\") " pod="openshift-cluster-node-tuning-operator/tuned-nx4kv"
Jul 29 17:52:02.763072 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.762359    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8ed81c5a-bf81-4c94-aafa-1d54d038d8df-etc-modprobe-d\") pod \"tuned-nx4kv\" (UID: \"8ed81c5a-bf81-4c94-aafa-1d54d038d8df\") " pod="openshift-cluster-node-tuning-operator/tuned-nx4kv"
Jul 29 17:52:02.763072 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.762442    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8ed81c5a-bf81-4c94-aafa-1d54d038d8df-etc-sysctl-conf\") pod \"tuned-nx4kv\" (UID: \"8ed81c5a-bf81-4c94-aafa-1d54d038d8df\") " pod="openshift-cluster-node-tuning-operator/tuned-nx4kv"
Jul 29 17:52:02.763072 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.762442    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/277437f5-ac5a-4dd2-8f8e-c2155788e605-system-cni-dir\") pod \"multus-additional-cni-plugins-rxw8d\" (UID: \"277437f5-ac5a-4dd2-8f8e-c2155788e605\") " pod="openshift-multus/multus-additional-cni-plugins-rxw8d"
Jul 29 17:52:02.763072 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.762491    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8ed81c5a-bf81-4c94-aafa-1d54d038d8df-var-lib-kubelet\") pod \"tuned-nx4kv\" (UID: \"8ed81c5a-bf81-4c94-aafa-1d54d038d8df\") " pod="openshift-cluster-node-tuning-operator/tuned-nx4kv"
Jul 29 17:52:02.763072 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.762621    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/277437f5-ac5a-4dd2-8f8e-c2155788e605-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rxw8d\" (UID: \"277437f5-ac5a-4dd2-8f8e-c2155788e605\") " pod="openshift-multus/multus-additional-cni-plugins-rxw8d"
Jul 29 17:52:02.763072 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.762787    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1eecf48c-b25b-4449-a782-c3b702cc0368-konnectivity-ca\") pod \"konnectivity-agent-mb6fv\" (UID: \"1eecf48c-b25b-4449-a782-c3b702cc0368\") " pod="kube-system/konnectivity-agent-mb6fv"
Jul 29 17:52:02.763822 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.763796    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8ed81c5a-bf81-4c94-aafa-1d54d038d8df-tmp\") pod \"tuned-nx4kv\" (UID: \"8ed81c5a-bf81-4c94-aafa-1d54d038d8df\") " pod="openshift-cluster-node-tuning-operator/tuned-nx4kv"
Jul 29 17:52:02.764280 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.764262    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8ed81c5a-bf81-4c94-aafa-1d54d038d8df-etc-tuned\") pod \"tuned-nx4kv\" (UID: \"8ed81c5a-bf81-4c94-aafa-1d54d038d8df\") " pod="openshift-cluster-node-tuning-operator/tuned-nx4kv"
Jul 29 17:52:02.764391 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.764377    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1eecf48c-b25b-4449-a782-c3b702cc0368-agent-certs\") pod \"konnectivity-agent-mb6fv\" (UID: \"1eecf48c-b25b-4449-a782-c3b702cc0368\") " pod="kube-system/konnectivity-agent-mb6fv"
Jul 29 17:52:02.770616 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.770592    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mqj7\" (UniqueName: \"kubernetes.io/projected/57afeaa9-b72d-4389-940b-f56a30b67995-kube-api-access-5mqj7\") pod \"network-metrics-daemon-kbx7n\" (UID: \"57afeaa9-b72d-4389-940b-f56a30b67995\") " pod="openshift-multus/network-metrics-daemon-kbx7n"
Jul 29 17:52:02.770926 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.770901    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f54sw\" (UniqueName: \"kubernetes.io/projected/85585baa-2c99-4feb-91ec-4a34a98df21b-kube-api-access-f54sw\") pod \"node-resolver-5wxd2\" (UID: \"85585baa-2c99-4feb-91ec-4a34a98df21b\") " pod="openshift-dns/node-resolver-5wxd2"
Jul 29 17:52:02.771048 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.771027    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4v84d\" (UniqueName: \"kubernetes.io/projected/8ed81c5a-bf81-4c94-aafa-1d54d038d8df-kube-api-access-4v84d\") pod \"tuned-nx4kv\" (UID: \"8ed81c5a-bf81-4c94-aafa-1d54d038d8df\") " pod="openshift-cluster-node-tuning-operator/tuned-nx4kv"
Jul 29 17:52:02.771309 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.771295    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-thbz9\" (UniqueName: \"kubernetes.io/projected/277437f5-ac5a-4dd2-8f8e-c2155788e605-kube-api-access-thbz9\") pod \"multus-additional-cni-plugins-rxw8d\" (UID: \"277437f5-ac5a-4dd2-8f8e-c2155788e605\") " pod="openshift-multus/multus-additional-cni-plugins-rxw8d"
Jul 29 17:52:02.787637 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.787615    2557 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 17:52:02.839620 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.839587    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9s48j"
Jul 29 17:52:02.847982 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.847963    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vmhq9"
Jul 29 17:52:02.859473 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.859448    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:02.864066 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.864046    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drvsm"
Jul 29 17:52:02.871193 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.871175    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-58bv4"
Jul 29 17:52:02.877614 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.877590    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5wxd2"
Jul 29 17:52:02.883089 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.883071    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mb6fv"
Jul 29 17:52:02.890407 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.890389    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rxw8d"
Jul 29 17:52:02.896305 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:02.896291    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nx4kv"
Jul 29 17:52:03.164656 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:03.164617    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/905f9420-cd73-4e49-a85f-2163a04b8fcd-original-pull-secret\") pod \"global-pull-secret-syncer-tdktb\" (UID: \"905f9420-cd73-4e49-a85f-2163a04b8fcd\") " pod="kube-system/global-pull-secret-syncer-tdktb"
Jul 29 17:52:03.164868 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:03.164669    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znrl2\" (UniqueName: \"kubernetes.io/projected/5540549f-0d35-480f-9135-2e51348f86ea-kube-api-access-znrl2\") pod \"network-check-target-svfrm\" (UID: \"5540549f-0d35-480f-9135-2e51348f86ea\") " pod="openshift-network-diagnostics/network-check-target-svfrm"
Jul 29 17:52:03.164868 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:03.164803    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 17:52:03.164868 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:03.164831    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 17:52:03.164868 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:03.164847    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 17:52:03.164868 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:03.164859    2557 projected.go:196] Error preparing data for projected volume kube-api-access-znrl2 for pod openshift-network-diagnostics/network-check-target-svfrm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 17:52:03.165072 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:03.164876    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/905f9420-cd73-4e49-a85f-2163a04b8fcd-original-pull-secret podName:905f9420-cd73-4e49-a85f-2163a04b8fcd nodeName:}" failed. No retries permitted until 2026-07-29 17:52:04.164859462 +0000 UTC m=+4.034082247 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/905f9420-cd73-4e49-a85f-2163a04b8fcd-original-pull-secret") pod "global-pull-secret-syncer-tdktb" (UID: "905f9420-cd73-4e49-a85f-2163a04b8fcd") : object "kube-system"/"original-pull-secret" not registered
Jul 29 17:52:03.165072 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:03.164899    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5540549f-0d35-480f-9135-2e51348f86ea-kube-api-access-znrl2 podName:5540549f-0d35-480f-9135-2e51348f86ea nodeName:}" failed. No retries permitted until 2026-07-29 17:52:04.164886992 +0000 UTC m=+4.034109777 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-znrl2" (UniqueName: "kubernetes.io/projected/5540549f-0d35-480f-9135-2e51348f86ea-kube-api-access-znrl2") pod "network-check-target-svfrm" (UID: "5540549f-0d35-480f-9135-2e51348f86ea") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 17:52:03.265780 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:03.265719    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57afeaa9-b72d-4389-940b-f56a30b67995-metrics-certs\") pod \"network-metrics-daemon-kbx7n\" (UID: \"57afeaa9-b72d-4389-940b-f56a30b67995\") " pod="openshift-multus/network-metrics-daemon-kbx7n"
Jul 29 17:52:03.265922 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:03.265902    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 17:52:03.266004 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:03.265984    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57afeaa9-b72d-4389-940b-f56a30b67995-metrics-certs podName:57afeaa9-b72d-4389-940b-f56a30b67995 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:04.265965197 +0000 UTC m=+4.135188001 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57afeaa9-b72d-4389-940b-f56a30b67995-metrics-certs") pod "network-metrics-daemon-kbx7n" (UID: "57afeaa9-b72d-4389-940b-f56a30b67995") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 17:52:03.587662 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:03.587625    2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 17:47:02 +0000 UTC" deadline="2028-01-31 21:15:37.514180872 +0000 UTC"
Jul 29 17:52:03.588071 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:03.587663    2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13227h23m33.926522528s"
Jul 29 17:52:03.601207 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:03.601180    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d94bd69_70db_430b_b18c_3bd4767d48da.slice/crio-0fd0c637d97340317034367c3bd5db384fab4f9c6feb3e0946038e3241bf7ba9 WatchSource:0}: Error finding container 0fd0c637d97340317034367c3bd5db384fab4f9c6feb3e0946038e3241bf7ba9: Status 404 returned error can't find the container with id 0fd0c637d97340317034367c3bd5db384fab4f9c6feb3e0946038e3241bf7ba9
Jul 29 17:52:03.603808 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:03.603673    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod277437f5_ac5a_4dd2_8f8e_c2155788e605.slice/crio-7f090cd00581b22bccb1fb14e9295a4d0ca459438dca5a277cc0c4d1094ad234 WatchSource:0}: Error finding container 7f090cd00581b22bccb1fb14e9295a4d0ca459438dca5a277cc0c4d1094ad234: Status 404 returned error can't find the container with id 7f090cd00581b22bccb1fb14e9295a4d0ca459438dca5a277cc0c4d1094ad234
Jul 29 17:52:03.606429 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:03.606404    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00b49d9f_e6ef_4506_89ea_ecb8a3c3d0e8.slice/crio-db839e5e30af591c57c89fd957c875573bfbd2dafc1e6c7f1f7eb4d909bc5ac6 WatchSource:0}: Error finding container db839e5e30af591c57c89fd957c875573bfbd2dafc1e6c7f1f7eb4d909bc5ac6: Status 404 returned error can't find the container with id db839e5e30af591c57c89fd957c875573bfbd2dafc1e6c7f1f7eb4d909bc5ac6
Jul 29 17:52:03.607931 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:03.607914    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3acade3e_0ce3_498e_99e4_c8a8af56ab59.slice/crio-d03cb3de94bcf20228097cdd5013dfda8b045d1f3771ca3a867bb6e8fbcd5c10 WatchSource:0}: Error finding container d03cb3de94bcf20228097cdd5013dfda8b045d1f3771ca3a867bb6e8fbcd5c10: Status 404 returned error can't find the container with id d03cb3de94bcf20228097cdd5013dfda8b045d1f3771ca3a867bb6e8fbcd5c10
Jul 29 17:52:03.608821 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:03.608798    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ed81c5a_bf81_4c94_aafa_1d54d038d8df.slice/crio-91f77077290e41db62df5a607fe5f9f577b1e8194f46a91c237255020696dbd9 WatchSource:0}: Error finding container 91f77077290e41db62df5a607fe5f9f577b1e8194f46a91c237255020696dbd9: Status 404 returned error can't find the container with id 91f77077290e41db62df5a607fe5f9f577b1e8194f46a91c237255020696dbd9
Jul 29 17:52:03.610667 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:03.610384    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85585baa_2c99_4feb_91ec_4a34a98df21b.slice/crio-86141a45feaa185e9a8641cebf6a501ae98005cd3ca63908c2fe6f46df6038d3 WatchSource:0}: Error finding container 86141a45feaa185e9a8641cebf6a501ae98005cd3ca63908c2fe6f46df6038d3: Status 404 returned error can't find the container with id 86141a45feaa185e9a8641cebf6a501ae98005cd3ca63908c2fe6f46df6038d3
Jul 29 17:52:03.611164 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:03.611143    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ef211d7_e236_4476_b5df_48aadf6e6195.slice/crio-2b397cd215845b07a46f4392cc53e8b4fe21aaa2f4f4e68e2b6eef3a0af72dfa WatchSource:0}: Error finding container 2b397cd215845b07a46f4392cc53e8b4fe21aaa2f4f4e68e2b6eef3a0af72dfa: Status 404 returned error can't find the container with id 2b397cd215845b07a46f4392cc53e8b4fe21aaa2f4f4e68e2b6eef3a0af72dfa
Jul 29 17:52:03.611914 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:03.611764    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda238ef80_08e3_4247_b5c9_6d9dacaf76db.slice/crio-8fb7163a6cb1d2ba433a8779fe701a7a8da385ae6241bae5bcc7cc0c7b78d5b6 WatchSource:0}: Error finding container 8fb7163a6cb1d2ba433a8779fe701a7a8da385ae6241bae5bcc7cc0c7b78d5b6: Status 404 returned error can't find the container with id 8fb7163a6cb1d2ba433a8779fe701a7a8da385ae6241bae5bcc7cc0c7b78d5b6
Jul 29 17:52:03.614266 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:03.613901    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1eecf48c_b25b_4449_a782_c3b702cc0368.slice/crio-620ed04f4ddd365b765dd90220a3fac6ecb2aa9d39c8c0f9ebc574fc0ce469de WatchSource:0}: Error finding container 620ed04f4ddd365b765dd90220a3fac6ecb2aa9d39c8c0f9ebc574fc0ce469de: Status 404 returned error can't find the container with id 620ed04f4ddd365b765dd90220a3fac6ecb2aa9d39c8c0f9ebc574fc0ce469de
Jul 29 17:52:03.658204 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:03.658170    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-58bv4" event={"ID":"3acade3e-0ce3-498e-99e4-c8a8af56ab59","Type":"ContainerStarted","Data":"d03cb3de94bcf20228097cdd5013dfda8b045d1f3771ca3a867bb6e8fbcd5c10"}
Jul 29 17:52:03.659775 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:03.659699    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rxw8d" event={"ID":"277437f5-ac5a-4dd2-8f8e-c2155788e605","Type":"ContainerStarted","Data":"7f090cd00581b22bccb1fb14e9295a4d0ca459438dca5a277cc0c4d1094ad234"}
Jul 29 17:52:03.660609 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:03.660593    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vmhq9" event={"ID":"8ef211d7-e236-4476-b5df-48aadf6e6195","Type":"ContainerStarted","Data":"2b397cd215845b07a46f4392cc53e8b4fe21aaa2f4f4e68e2b6eef3a0af72dfa"}
Jul 29 17:52:03.661435 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:03.661414    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nx4kv" event={"ID":"8ed81c5a-bf81-4c94-aafa-1d54d038d8df","Type":"ContainerStarted","Data":"91f77077290e41db62df5a607fe5f9f577b1e8194f46a91c237255020696dbd9"}
Jul 29 17:52:03.662271 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:03.662251    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drvsm" event={"ID":"00b49d9f-e6ef-4506-89ea-ecb8a3c3d0e8","Type":"ContainerStarted","Data":"db839e5e30af591c57c89fd957c875573bfbd2dafc1e6c7f1f7eb4d909bc5ac6"}
Jul 29 17:52:03.663351 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:03.663172    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z" event={"ID":"7d94bd69-70db-430b-b18c-3bd4767d48da","Type":"ContainerStarted","Data":"0fd0c637d97340317034367c3bd5db384fab4f9c6feb3e0946038e3241bf7ba9"}
Jul 29 17:52:03.667824 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:03.667803    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mb6fv" event={"ID":"1eecf48c-b25b-4449-a782-c3b702cc0368","Type":"ContainerStarted","Data":"620ed04f4ddd365b765dd90220a3fac6ecb2aa9d39c8c0f9ebc574fc0ce469de"}
Jul 29 17:52:03.675259 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:03.675227    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9s48j" event={"ID":"a238ef80-08e3-4247-b5c9-6d9dacaf76db","Type":"ContainerStarted","Data":"8fb7163a6cb1d2ba433a8779fe701a7a8da385ae6241bae5bcc7cc0c7b78d5b6"}
Jul 29 17:52:03.676321 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:03.676303    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5wxd2" event={"ID":"85585baa-2c99-4feb-91ec-4a34a98df21b","Type":"ContainerStarted","Data":"86141a45feaa185e9a8641cebf6a501ae98005cd3ca63908c2fe6f46df6038d3"}
Jul 29 17:52:04.171942 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:04.171624    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/905f9420-cd73-4e49-a85f-2163a04b8fcd-original-pull-secret\") pod \"global-pull-secret-syncer-tdktb\" (UID: \"905f9420-cd73-4e49-a85f-2163a04b8fcd\") " pod="kube-system/global-pull-secret-syncer-tdktb"
Jul 29 17:52:04.171942 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:04.171933    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znrl2\" (UniqueName: \"kubernetes.io/projected/5540549f-0d35-480f-9135-2e51348f86ea-kube-api-access-znrl2\") pod \"network-check-target-svfrm\" (UID: \"5540549f-0d35-480f-9135-2e51348f86ea\") " pod="openshift-network-diagnostics/network-check-target-svfrm"
Jul 29 17:52:04.172171 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:04.172131    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 17:52:04.172171 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:04.172149    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 17:52:04.172171 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:04.172163    2557 projected.go:196] Error preparing data for projected volume kube-api-access-znrl2 for pod openshift-network-diagnostics/network-check-target-svfrm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 17:52:04.172329 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:04.172222    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5540549f-0d35-480f-9135-2e51348f86ea-kube-api-access-znrl2 podName:5540549f-0d35-480f-9135-2e51348f86ea nodeName:}" failed. No retries permitted until 2026-07-29 17:52:06.172203648 +0000 UTC m=+6.041426440 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-znrl2" (UniqueName: "kubernetes.io/projected/5540549f-0d35-480f-9135-2e51348f86ea-kube-api-access-znrl2") pod "network-check-target-svfrm" (UID: "5540549f-0d35-480f-9135-2e51348f86ea") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 17:52:04.172639 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:04.172620    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 17:52:04.172764 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:04.172672    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/905f9420-cd73-4e49-a85f-2163a04b8fcd-original-pull-secret podName:905f9420-cd73-4e49-a85f-2163a04b8fcd nodeName:}" failed. No retries permitted until 2026-07-29 17:52:06.172657805 +0000 UTC m=+6.041880595 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/905f9420-cd73-4e49-a85f-2163a04b8fcd-original-pull-secret") pod "global-pull-secret-syncer-tdktb" (UID: "905f9420-cd73-4e49-a85f-2163a04b8fcd") : object "kube-system"/"original-pull-secret" not registered
Jul 29 17:52:04.273385 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:04.273342    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57afeaa9-b72d-4389-940b-f56a30b67995-metrics-certs\") pod \"network-metrics-daemon-kbx7n\" (UID: \"57afeaa9-b72d-4389-940b-f56a30b67995\") " pod="openshift-multus/network-metrics-daemon-kbx7n"
Jul 29 17:52:04.273571 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:04.273492    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 17:52:04.273571 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:04.273559    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57afeaa9-b72d-4389-940b-f56a30b67995-metrics-certs podName:57afeaa9-b72d-4389-940b-f56a30b67995 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:06.273540018 +0000 UTC m=+6.142762807 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57afeaa9-b72d-4389-940b-f56a30b67995-metrics-certs") pod "network-metrics-daemon-kbx7n" (UID: "57afeaa9-b72d-4389-940b-f56a30b67995") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 17:52:04.588069 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:04.587978    2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 17:47:02 +0000 UTC" deadline="2028-03-18 19:08:22.12393013 +0000 UTC"
Jul 29 17:52:04.588069 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:04.588038    2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14353h16m17.535896272s"
Jul 29 17:52:04.649074 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:04.649042    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-svfrm"
Jul 29 17:52:04.649230 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:04.649173    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-svfrm" podUID="5540549f-0d35-480f-9135-2e51348f86ea"
Jul 29 17:52:04.649651 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:04.649569    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tdktb"
Jul 29 17:52:04.649769 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:04.649662    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tdktb" podUID="905f9420-cd73-4e49-a85f-2163a04b8fcd"
Jul 29 17:52:04.649769 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:04.649747    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kbx7n"
Jul 29 17:52:04.649875 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:04.649831    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kbx7n" podUID="57afeaa9-b72d-4389-940b-f56a30b67995"
Jul 29 17:52:04.689716 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:04.689422    2557 generic.go:356] "Generic (PLEG): container finished" podID="2e82424c7e271bc2aa44f8eb2dc7dc92" containerID="8d1820ea8ed725d258bc9e8fa6252094b585fd0adc8cb24b522895e5557cd711" exitCode=0
Jul 29 17:52:04.689716 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:04.689513    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-156.ec2.internal" event={"ID":"2e82424c7e271bc2aa44f8eb2dc7dc92","Type":"ContainerDied","Data":"8d1820ea8ed725d258bc9e8fa6252094b585fd0adc8cb24b522895e5557cd711"}
Jul 29 17:52:04.693983 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:04.693956    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-156.ec2.internal" event={"ID":"591bc9043de6ffb60a0ece0a3a196263","Type":"ContainerStarted","Data":"c550c607cffd1a83572450518efc8dfdc93f51931f8bceb158d3bc05c42791b9"}
Jul 29 17:52:05.728894 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:05.728805    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-156.ec2.internal" event={"ID":"2e82424c7e271bc2aa44f8eb2dc7dc92","Type":"ContainerStarted","Data":"196a39d23b5aacbe96ab8d93ac80b355aab64028760612d09dbbb04f1edee1e1"}
Jul 29 17:52:05.746152 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:05.745713    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-156.ec2.internal" podStartSLOduration=4.745690027 podStartE2EDuration="4.745690027s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 17:52:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:52:04.717523628 +0000 UTC m=+4.586746436" watchObservedRunningTime="2026-07-29 17:52:05.745690027 +0000 UTC m=+5.614912837"
Jul 29 17:52:06.195992 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:06.195953    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/905f9420-cd73-4e49-a85f-2163a04b8fcd-original-pull-secret\") pod \"global-pull-secret-syncer-tdktb\" (UID: \"905f9420-cd73-4e49-a85f-2163a04b8fcd\") " pod="kube-system/global-pull-secret-syncer-tdktb"
Jul 29 17:52:06.196185 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:06.196015    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znrl2\" (UniqueName: \"kubernetes.io/projected/5540549f-0d35-480f-9135-2e51348f86ea-kube-api-access-znrl2\") pod \"network-check-target-svfrm\" (UID: \"5540549f-0d35-480f-9135-2e51348f86ea\") " pod="openshift-network-diagnostics/network-check-target-svfrm"
Jul 29 17:52:06.196258 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:06.196189    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 17:52:06.196258 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:06.196212    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 17:52:06.196258 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:06.196225    2557 projected.go:196] Error preparing data for projected volume kube-api-access-znrl2 for pod openshift-network-diagnostics/network-check-target-svfrm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 17:52:06.196394 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:06.196282    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5540549f-0d35-480f-9135-2e51348f86ea-kube-api-access-znrl2 podName:5540549f-0d35-480f-9135-2e51348f86ea nodeName:}" failed. No retries permitted until 2026-07-29 17:52:10.196264067 +0000 UTC m=+10.065486854 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-znrl2" (UniqueName: "kubernetes.io/projected/5540549f-0d35-480f-9135-2e51348f86ea-kube-api-access-znrl2") pod "network-check-target-svfrm" (UID: "5540549f-0d35-480f-9135-2e51348f86ea") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 17:52:06.196394 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:06.196350    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 17:52:06.196394 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:06.196387    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/905f9420-cd73-4e49-a85f-2163a04b8fcd-original-pull-secret podName:905f9420-cd73-4e49-a85f-2163a04b8fcd nodeName:}" failed. No retries permitted until 2026-07-29 17:52:10.196376175 +0000 UTC m=+10.065598961 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/905f9420-cd73-4e49-a85f-2163a04b8fcd-original-pull-secret") pod "global-pull-secret-syncer-tdktb" (UID: "905f9420-cd73-4e49-a85f-2163a04b8fcd") : object "kube-system"/"original-pull-secret" not registered
Jul 29 17:52:06.297374 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:06.296829    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57afeaa9-b72d-4389-940b-f56a30b67995-metrics-certs\") pod \"network-metrics-daemon-kbx7n\" (UID: \"57afeaa9-b72d-4389-940b-f56a30b67995\") " pod="openshift-multus/network-metrics-daemon-kbx7n"
Jul 29 17:52:06.297374 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:06.296978    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 17:52:06.297374 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:06.297042    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57afeaa9-b72d-4389-940b-f56a30b67995-metrics-certs podName:57afeaa9-b72d-4389-940b-f56a30b67995 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:10.297023458 +0000 UTC m=+10.166246248 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57afeaa9-b72d-4389-940b-f56a30b67995-metrics-certs") pod "network-metrics-daemon-kbx7n" (UID: "57afeaa9-b72d-4389-940b-f56a30b67995") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 17:52:06.649307 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:06.649275    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tdktb"
Jul 29 17:52:06.649492 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:06.649400    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tdktb" podUID="905f9420-cd73-4e49-a85f-2163a04b8fcd"
Jul 29 17:52:06.649933 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:06.649755    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-svfrm"
Jul 29 17:52:06.649933 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:06.649843    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-svfrm" podUID="5540549f-0d35-480f-9135-2e51348f86ea"
Jul 29 17:52:06.650435 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:06.650232    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kbx7n"
Jul 29 17:52:06.650435 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:06.650343    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kbx7n" podUID="57afeaa9-b72d-4389-940b-f56a30b67995"
Jul 29 17:52:08.646876 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:08.646841    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tdktb"
Jul 29 17:52:08.647252 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:08.646974    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tdktb" podUID="905f9420-cd73-4e49-a85f-2163a04b8fcd"
Jul 29 17:52:08.647497 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:08.647455    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-svfrm"
Jul 29 17:52:08.647593 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:08.647574    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-svfrm" podUID="5540549f-0d35-480f-9135-2e51348f86ea"
Jul 29 17:52:08.647654 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:08.647644    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kbx7n"
Jul 29 17:52:08.647753 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:08.647718    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kbx7n" podUID="57afeaa9-b72d-4389-940b-f56a30b67995"
Jul 29 17:52:10.230688 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:10.230024    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/905f9420-cd73-4e49-a85f-2163a04b8fcd-original-pull-secret\") pod \"global-pull-secret-syncer-tdktb\" (UID: \"905f9420-cd73-4e49-a85f-2163a04b8fcd\") " pod="kube-system/global-pull-secret-syncer-tdktb"
Jul 29 17:52:10.230688 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:10.230081    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znrl2\" (UniqueName: \"kubernetes.io/projected/5540549f-0d35-480f-9135-2e51348f86ea-kube-api-access-znrl2\") pod \"network-check-target-svfrm\" (UID: \"5540549f-0d35-480f-9135-2e51348f86ea\") " pod="openshift-network-diagnostics/network-check-target-svfrm"
Jul 29 17:52:10.230688 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:10.230252    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 17:52:10.230688 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:10.230270    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 17:52:10.230688 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:10.230283    2557 projected.go:196] Error preparing data for projected volume kube-api-access-znrl2 for pod openshift-network-diagnostics/network-check-target-svfrm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 17:52:10.230688 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:10.230340    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5540549f-0d35-480f-9135-2e51348f86ea-kube-api-access-znrl2 podName:5540549f-0d35-480f-9135-2e51348f86ea nodeName:}" failed. No retries permitted until 2026-07-29 17:52:18.230321621 +0000 UTC m=+18.099544412 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-znrl2" (UniqueName: "kubernetes.io/projected/5540549f-0d35-480f-9135-2e51348f86ea-kube-api-access-znrl2") pod "network-check-target-svfrm" (UID: "5540549f-0d35-480f-9135-2e51348f86ea") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 17:52:10.230688 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:10.230462    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 17:52:10.230688 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:10.230514    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/905f9420-cd73-4e49-a85f-2163a04b8fcd-original-pull-secret podName:905f9420-cd73-4e49-a85f-2163a04b8fcd nodeName:}" failed. No retries permitted until 2026-07-29 17:52:18.230502016 +0000 UTC m=+18.099724802 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/905f9420-cd73-4e49-a85f-2163a04b8fcd-original-pull-secret") pod "global-pull-secret-syncer-tdktb" (UID: "905f9420-cd73-4e49-a85f-2163a04b8fcd") : object "kube-system"/"original-pull-secret" not registered
Jul 29 17:52:10.331045 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:10.331011    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57afeaa9-b72d-4389-940b-f56a30b67995-metrics-certs\") pod \"network-metrics-daemon-kbx7n\" (UID: \"57afeaa9-b72d-4389-940b-f56a30b67995\") " pod="openshift-multus/network-metrics-daemon-kbx7n"
Jul 29 17:52:10.331191 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:10.331169    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 17:52:10.331246 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:10.331234    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57afeaa9-b72d-4389-940b-f56a30b67995-metrics-certs podName:57afeaa9-b72d-4389-940b-f56a30b67995 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:18.331216265 +0000 UTC m=+18.200439056 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57afeaa9-b72d-4389-940b-f56a30b67995-metrics-certs") pod "network-metrics-daemon-kbx7n" (UID: "57afeaa9-b72d-4389-940b-f56a30b67995") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 17:52:10.646778 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:10.646215    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-svfrm"
Jul 29 17:52:10.646778 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:10.646335    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-svfrm" podUID="5540549f-0d35-480f-9135-2e51348f86ea"
Jul 29 17:52:10.646778 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:10.646690    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tdktb"
Jul 29 17:52:10.647063 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:10.646808    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tdktb" podUID="905f9420-cd73-4e49-a85f-2163a04b8fcd"
Jul 29 17:52:10.647063 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:10.646880    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kbx7n"
Jul 29 17:52:10.647985 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:10.647958    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kbx7n" podUID="57afeaa9-b72d-4389-940b-f56a30b67995"
Jul 29 17:52:12.646568 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:12.646528    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tdktb"
Jul 29 17:52:12.646568 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:12.646549    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kbx7n"
Jul 29 17:52:12.647031 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:12.646662    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tdktb" podUID="905f9420-cd73-4e49-a85f-2163a04b8fcd"
Jul 29 17:52:12.647031 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:12.646718    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-svfrm"
Jul 29 17:52:12.647031 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:12.646853    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kbx7n" podUID="57afeaa9-b72d-4389-940b-f56a30b67995"
Jul 29 17:52:12.647031 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:12.646923    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-svfrm" podUID="5540549f-0d35-480f-9135-2e51348f86ea"
Jul 29 17:52:14.646381 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:14.646353    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tdktb"
Jul 29 17:52:14.646381 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:14.646379    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kbx7n"
Jul 29 17:52:14.646856 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:14.646481    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tdktb" podUID="905f9420-cd73-4e49-a85f-2163a04b8fcd"
Jul 29 17:52:14.646856 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:14.646555    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kbx7n" podUID="57afeaa9-b72d-4389-940b-f56a30b67995"
Jul 29 17:52:14.646856 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:14.646583    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-svfrm"
Jul 29 17:52:14.646856 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:14.646679    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-svfrm" podUID="5540549f-0d35-480f-9135-2e51348f86ea"
Jul 29 17:52:16.646118 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:16.646079    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-svfrm"
Jul 29 17:52:16.646118 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:16.646110    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kbx7n"
Jul 29 17:52:16.646592 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:16.646084    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tdktb"
Jul 29 17:52:16.646592 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:16.646203    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-svfrm" podUID="5540549f-0d35-480f-9135-2e51348f86ea"
Jul 29 17:52:16.646592 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:16.646278    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tdktb" podUID="905f9420-cd73-4e49-a85f-2163a04b8fcd"
Jul 29 17:52:16.646592 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:16.646386    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kbx7n" podUID="57afeaa9-b72d-4389-940b-f56a30b67995"
Jul 29 17:52:18.291840 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:18.291806    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znrl2\" (UniqueName: \"kubernetes.io/projected/5540549f-0d35-480f-9135-2e51348f86ea-kube-api-access-znrl2\") pod \"network-check-target-svfrm\" (UID: \"5540549f-0d35-480f-9135-2e51348f86ea\") " pod="openshift-network-diagnostics/network-check-target-svfrm"
Jul 29 17:52:18.292303 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:18.291866    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/905f9420-cd73-4e49-a85f-2163a04b8fcd-original-pull-secret\") pod \"global-pull-secret-syncer-tdktb\" (UID: \"905f9420-cd73-4e49-a85f-2163a04b8fcd\") " pod="kube-system/global-pull-secret-syncer-tdktb"
Jul 29 17:52:18.292303 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:18.291949    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 17:52:18.292303 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:18.291982    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 17:52:18.292303 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:18.291999    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/905f9420-cd73-4e49-a85f-2163a04b8fcd-original-pull-secret podName:905f9420-cd73-4e49-a85f-2163a04b8fcd nodeName:}" failed. No retries permitted until 2026-07-29 17:52:34.291986181 +0000 UTC m=+34.161208966 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/905f9420-cd73-4e49-a85f-2163a04b8fcd-original-pull-secret") pod "global-pull-secret-syncer-tdktb" (UID: "905f9420-cd73-4e49-a85f-2163a04b8fcd") : object "kube-system"/"original-pull-secret" not registered
Jul 29 17:52:18.292303 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:18.292002    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 17:52:18.292303 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:18.292016    2557 projected.go:196] Error preparing data for projected volume kube-api-access-znrl2 for pod openshift-network-diagnostics/network-check-target-svfrm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 17:52:18.292303 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:18.292077    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5540549f-0d35-480f-9135-2e51348f86ea-kube-api-access-znrl2 podName:5540549f-0d35-480f-9135-2e51348f86ea nodeName:}" failed. No retries permitted until 2026-07-29 17:52:34.292065454 +0000 UTC m=+34.161288244 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-znrl2" (UniqueName: "kubernetes.io/projected/5540549f-0d35-480f-9135-2e51348f86ea-kube-api-access-znrl2") pod "network-check-target-svfrm" (UID: "5540549f-0d35-480f-9135-2e51348f86ea") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 17:52:18.392761 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:18.392715    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57afeaa9-b72d-4389-940b-f56a30b67995-metrics-certs\") pod \"network-metrics-daemon-kbx7n\" (UID: \"57afeaa9-b72d-4389-940b-f56a30b67995\") " pod="openshift-multus/network-metrics-daemon-kbx7n"
Jul 29 17:52:18.392904 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:18.392823    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 17:52:18.392904 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:18.392889    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57afeaa9-b72d-4389-940b-f56a30b67995-metrics-certs podName:57afeaa9-b72d-4389-940b-f56a30b67995 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:34.392872748 +0000 UTC m=+34.262095545 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57afeaa9-b72d-4389-940b-f56a30b67995-metrics-certs") pod "network-metrics-daemon-kbx7n" (UID: "57afeaa9-b72d-4389-940b-f56a30b67995") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 17:52:18.647786 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:18.647698    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-svfrm"
Jul 29 17:52:18.647936 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:18.647843    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-svfrm" podUID="5540549f-0d35-480f-9135-2e51348f86ea"
Jul 29 17:52:18.648191 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:18.648172    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kbx7n"
Jul 29 17:52:18.648286 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:18.648231    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tdktb"
Jul 29 17:52:18.648346 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:18.648320    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tdktb" podUID="905f9420-cd73-4e49-a85f-2163a04b8fcd"
Jul 29 17:52:18.648423 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:18.648403    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kbx7n" podUID="57afeaa9-b72d-4389-940b-f56a30b67995"
Jul 29 17:52:20.647442 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:20.647415    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-svfrm"
Jul 29 17:52:20.647929 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:20.647518    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-svfrm" podUID="5540549f-0d35-480f-9135-2e51348f86ea"
Jul 29 17:52:20.647929 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:20.647608    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tdktb"
Jul 29 17:52:20.647929 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:20.647689    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kbx7n"
Jul 29 17:52:20.647929 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:20.647767    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kbx7n" podUID="57afeaa9-b72d-4389-940b-f56a30b67995"
Jul 29 17:52:20.647929 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:20.647798    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tdktb" podUID="905f9420-cd73-4e49-a85f-2163a04b8fcd"
Jul 29 17:52:22.648282 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:22.646397    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-svfrm"
Jul 29 17:52:22.648282 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:22.646745    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-svfrm" podUID="5540549f-0d35-480f-9135-2e51348f86ea"
Jul 29 17:52:22.648282 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:22.646556    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kbx7n"
Jul 29 17:52:22.648282 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:22.646866    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kbx7n" podUID="57afeaa9-b72d-4389-940b-f56a30b67995"
Jul 29 17:52:22.648282 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:22.646409    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tdktb"
Jul 29 17:52:22.648282 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:22.646949    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tdktb" podUID="905f9420-cd73-4e49-a85f-2163a04b8fcd"
Jul 29 17:52:22.769034 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:22.768773    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-58bv4" event={"ID":"3acade3e-0ce3-498e-99e4-c8a8af56ab59","Type":"ContainerStarted","Data":"6a42d50e679d872db0c7f5fda924ee338af91e38e3779a97279b874e0f784676"}
Jul 29 17:52:22.774258 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:22.774214    2557 generic.go:356] "Generic (PLEG): container finished" podID="277437f5-ac5a-4dd2-8f8e-c2155788e605" containerID="feca77252bf546843ec5391d77cce031d23f86f792ebcaa6e6db777e71fab7af" exitCode=0
Jul 29 17:52:22.774399 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:22.774305    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rxw8d" event={"ID":"277437f5-ac5a-4dd2-8f8e-c2155788e605","Type":"ContainerDied","Data":"feca77252bf546843ec5391d77cce031d23f86f792ebcaa6e6db777e71fab7af"}
Jul 29 17:52:22.776633 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:22.776609    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vmhq9" event={"ID":"8ef211d7-e236-4476-b5df-48aadf6e6195","Type":"ContainerStarted","Data":"a11a46a30eb6de326cbe643e69a8339ff62d007e43092f3a3df1d548cf7ba8a5"}
Jul 29 17:52:22.778657 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:22.778629    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nx4kv" event={"ID":"8ed81c5a-bf81-4c94-aafa-1d54d038d8df","Type":"ContainerStarted","Data":"f1f12263456b0a51a937b4d98eabddc036169328eaa518b818f2898286c5122e"}
Jul 29 17:52:22.780551 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:22.780523    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drvsm" event={"ID":"00b49d9f-e6ef-4506-89ea-ecb8a3c3d0e8","Type":"ContainerStarted","Data":"ca6fcf6bbf8e2214d065a7fcfea4ef12bfc0d5805b75138301c72cdadc6e6c47"}
Jul 29 17:52:22.783191 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:22.783171    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z" event={"ID":"7d94bd69-70db-430b-b18c-3bd4767d48da","Type":"ContainerStarted","Data":"28018810e05136d6fe1cb5dd913ca5b3d5ef7a0049a254d074308f545fdd312c"}
Jul 29 17:52:22.783296 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:22.783229    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z" event={"ID":"7d94bd69-70db-430b-b18c-3bd4767d48da","Type":"ContainerStarted","Data":"6f2fa61c6b2447e1782e1752ac32c69362b877c5b7800a2c4c70d95628adcf1a"}
Jul 29 17:52:22.784811 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:22.784784    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mb6fv" event={"ID":"1eecf48c-b25b-4449-a782-c3b702cc0368","Type":"ContainerStarted","Data":"2122d301a41237e6bbcb5132615fe19aec7a669541773411598afcd1a95e1365"}
Jul 29 17:52:22.786348 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:22.786323    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5wxd2" event={"ID":"85585baa-2c99-4feb-91ec-4a34a98df21b","Type":"ContainerStarted","Data":"e3efdfe4e2a8660c7e6964636edd7cd55dd5208935a9188b3f6c0a77109aba6a"}
Jul 29 17:52:22.790252 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:22.790216    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-156.ec2.internal" podStartSLOduration=21.790204517 podStartE2EDuration="21.790204517s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 17:52:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:52:05.745125267 +0000 UTC m=+5.614348075" watchObservedRunningTime="2026-07-29 17:52:22.790204517 +0000 UTC m=+22.659427325"
Jul 29 17:52:22.790351 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:22.790290    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-58bv4" podStartSLOduration=3.923231513 podStartE2EDuration="22.790283868s" totalImagesPullingTime="18.867052355s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 17:52:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:52:22.78838681 +0000 UTC m=+22.657609616" watchObservedRunningTime="2026-07-29 17:52:22.790283868 +0000 UTC m=+22.659506676"
Jul 29 17:52:22.805089 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:22.805049    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-mb6fv" podStartSLOduration=11.897560111 podStartE2EDuration="22.805034024s" totalImagesPullingTime="10.907473913s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:52:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:52:22.804574251 +0000 UTC m=+22.673797058" watchObservedRunningTime="2026-07-29 17:52:22.805034024 +0000 UTC m=+22.674256831"
Jul 29 17:52:22.839434 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:22.839402    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-nx4kv" podStartSLOduration=3.022210769 podStartE2EDuration="21.839389836s" totalImagesPullingTime="18.817179067s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 17:52:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:52:22.821472845 +0000 UTC m=+22.690695653" watchObservedRunningTime="2026-07-29 17:52:22.839389836 +0000 UTC m=+22.708612646"
Jul 29 17:52:22.839533 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:22.839491    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-5wxd2" podStartSLOduration=4.025718678 podStartE2EDuration="22.839487117s" totalImagesPullingTime="18.813768439s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 17:52:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:52:22.839437537 +0000 UTC m=+22.708660346" watchObservedRunningTime="2026-07-29 17:52:22.839487117 +0000 UTC m=+22.708709903"
Jul 29 17:52:22.886956 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:22.886920    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-vmhq9" podStartSLOduration=4.073918185 podStartE2EDuration="22.886906155s" totalImagesPullingTime="18.81298797s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 17:52:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:52:22.886793866 +0000 UTC m=+22.756016677" watchObservedRunningTime="2026-07-29 17:52:22.886906155 +0000 UTC m=+22.756128963"
Jul 29 17:52:23.791124 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:23.791098    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z" event={"ID":"7d94bd69-70db-430b-b18c-3bd4767d48da","Type":"ContainerStarted","Data":"026c8cae7c3e6c011365e0d455618e6dbdaecf8d5fe7dbc03c32c20be6279378"}
Jul 29 17:52:23.791567 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:23.791175    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z" event={"ID":"7d94bd69-70db-430b-b18c-3bd4767d48da","Type":"ContainerStarted","Data":"1ddeb26a170bcb624cce29ab22a1c87dce394541b92dcd719eaeac445aeb1b7a"}
Jul 29 17:52:23.791567 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:23.791224    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z" event={"ID":"7d94bd69-70db-430b-b18c-3bd4767d48da","Type":"ContainerStarted","Data":"d984e476a638431fc46955f1e3ba74ad0ef05862def5d4c7643393bfc02874aa"}
Jul 29 17:52:23.791567 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:23.791261    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z" event={"ID":"7d94bd69-70db-430b-b18c-3bd4767d48da","Type":"ContainerStarted","Data":"4b8d4b0df330726629d2c27312877199b12bdf787cff830c4915b01804ba6610"}
Jul 29 17:52:23.792507 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:23.792486    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9s48j" event={"ID":"a238ef80-08e3-4247-b5c9-6d9dacaf76db","Type":"ContainerStarted","Data":"5b1773a88699894f151dabd2972828a4e8a3bef355a1109a3106f5de01a9ed5b"}
Jul 29 17:52:23.816582 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:23.816030    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-9s48j" podStartSLOduration=5.004259988 podStartE2EDuration="23.816011508s" totalImagesPullingTime="18.81175152s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 17:52:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:52:23.815749013 +0000 UTC m=+23.684971821" watchObservedRunningTime="2026-07-29 17:52:23.816011508 +0000 UTC m=+23.685234316"
Jul 29 17:52:23.919559 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:23.919529    2557 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 29 17:52:24.616971 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:24.616864    2557 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-29T17:52:23.919548747Z","UUID":"98fa28d7-46bb-42ba-bce7-72bf8e68f8bd","Handler":null,"Name":"","Endpoint":""}
Jul 29 17:52:24.618412 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:24.618393    2557 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 29 17:52:24.618543 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:24.618430    2557 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 29 17:52:24.646875 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:24.646854    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-svfrm"
Jul 29 17:52:24.647016 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:24.646861    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tdktb"
Jul 29 17:52:24.647016 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:24.646971    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-svfrm" podUID="5540549f-0d35-480f-9135-2e51348f86ea"
Jul 29 17:52:24.647131 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:24.647023    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tdktb" podUID="905f9420-cd73-4e49-a85f-2163a04b8fcd"
Jul 29 17:52:24.647131 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:24.647064    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kbx7n"
Jul 29 17:52:24.647237 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:24.647140    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kbx7n" podUID="57afeaa9-b72d-4389-940b-f56a30b67995"
Jul 29 17:52:24.796814 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:24.796745    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drvsm" event={"ID":"00b49d9f-e6ef-4506-89ea-ecb8a3c3d0e8","Type":"ContainerStarted","Data":"0e36e997943b45be610a7fe947034a2f6750ecc411e13f47329f7ccb622a550f"}
Jul 29 17:52:25.801958 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:25.801921    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drvsm" event={"ID":"00b49d9f-e6ef-4506-89ea-ecb8a3c3d0e8","Type":"ContainerStarted","Data":"dd39194ba652a2734762059d45206891e12baa9d821ec511a0fef2a7a510ac3f"}
Jul 29 17:52:25.805461 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:25.805434    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z" event={"ID":"7d94bd69-70db-430b-b18c-3bd4767d48da","Type":"ContainerStarted","Data":"59d3529f4e23a608e28d92018797a8817de1a8f1ee058e9f7d9ff0f3a1985d03"}
Jul 29 17:52:25.826664 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:25.826619    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drvsm" podStartSLOduration=4.740063664 podStartE2EDuration="25.826607051s" totalImagesPullingTime="21.086543387s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 17:52:00 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:52:25.826066047 +0000 UTC m=+25.695288855" watchObservedRunningTime="2026-07-29 17:52:25.826607051 +0000 UTC m=+25.695829858"
Jul 29 17:52:26.104153 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:26.104075    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-mb6fv"
Jul 29 17:52:26.104702 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:26.104682    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-mb6fv"
Jul 29 17:52:26.650245 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:26.650214    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-svfrm"
Jul 29 17:52:26.650428 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:26.650217    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kbx7n"
Jul 29 17:52:26.650428 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:26.650325    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-svfrm" podUID="5540549f-0d35-480f-9135-2e51348f86ea"
Jul 29 17:52:26.650428 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:26.650217    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tdktb"
Jul 29 17:52:26.650595 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:26.650395    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kbx7n" podUID="57afeaa9-b72d-4389-940b-f56a30b67995"
Jul 29 17:52:26.650595 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:26.650494    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tdktb" podUID="905f9420-cd73-4e49-a85f-2163a04b8fcd"
Jul 29 17:52:26.808002 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:26.807966    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-mb6fv"
Jul 29 17:52:26.808442 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:26.808423    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-mb6fv"
Jul 29 17:52:27.814835 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:27.814569    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z" event={"ID":"7d94bd69-70db-430b-b18c-3bd4767d48da","Type":"ContainerStarted","Data":"8659eb22999f43780d141c7b7faf33b3f2b7f853182c0993d89d5f3a7fd3acb3"}
Jul 29 17:52:27.815543 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:27.814904    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:27.816893 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:27.816867    2557 generic.go:356] "Generic (PLEG): container finished" podID="277437f5-ac5a-4dd2-8f8e-c2155788e605" containerID="8d11e48d76d61bd5b5775773183745be370e76dfc92f71b23ca5eebf93da19ff" exitCode=0
Jul 29 17:52:27.817005 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:27.816938    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rxw8d" event={"ID":"277437f5-ac5a-4dd2-8f8e-c2155788e605","Type":"ContainerDied","Data":"8d11e48d76d61bd5b5775773183745be370e76dfc92f71b23ca5eebf93da19ff"}
Jul 29 17:52:27.830335 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:27.830315    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:27.843967 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:27.843897    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z" podStartSLOduration=8.963455517 podStartE2EDuration="27.843882719s" totalImagesPullingTime="18.880427202s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 17:52:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:52:27.843585802 +0000 UTC m=+27.712808614" watchObservedRunningTime="2026-07-29 17:52:27.843882719 +0000 UTC m=+27.713105525"
Jul 29 17:52:28.646715 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:28.646684    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-svfrm"
Jul 29 17:52:28.647001 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:28.646683    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tdktb"
Jul 29 17:52:28.647001 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:28.646799    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-svfrm" podUID="5540549f-0d35-480f-9135-2e51348f86ea"
Jul 29 17:52:28.647001 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:28.646884    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tdktb" podUID="905f9420-cd73-4e49-a85f-2163a04b8fcd"
Jul 29 17:52:28.647001 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:28.646954    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kbx7n"
Jul 29 17:52:28.647215 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:28.647039    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kbx7n" podUID="57afeaa9-b72d-4389-940b-f56a30b67995"
Jul 29 17:52:28.820959 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:28.820927    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:28.821545 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:28.821451    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:28.852083 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:28.852051    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:52:29.753750 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:29.753554    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-svfrm"]
Jul 29 17:52:29.754025 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:29.753844    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-svfrm"
Jul 29 17:52:29.754025 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:29.753941    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-svfrm" podUID="5540549f-0d35-480f-9135-2e51348f86ea"
Jul 29 17:52:29.757222 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:29.757186    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kbx7n"]
Jul 29 17:52:29.757369 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:29.757297    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kbx7n"
Jul 29 17:52:29.757429 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:29.757392    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kbx7n" podUID="57afeaa9-b72d-4389-940b-f56a30b67995"
Jul 29 17:52:29.757996 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:29.757970    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tdktb"]
Jul 29 17:52:29.758097 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:29.758042    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tdktb"
Jul 29 17:52:29.758160 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:29.758114    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tdktb" podUID="905f9420-cd73-4e49-a85f-2163a04b8fcd"
Jul 29 17:52:29.824101 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:29.824065    2557 generic.go:356] "Generic (PLEG): container finished" podID="277437f5-ac5a-4dd2-8f8e-c2155788e605" containerID="5235c2576516b48a9b486a5331d428db44f907a48d72a4aad637af9a4b2d00f2" exitCode=0
Jul 29 17:52:29.824560 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:29.824103    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rxw8d" event={"ID":"277437f5-ac5a-4dd2-8f8e-c2155788e605","Type":"ContainerDied","Data":"5235c2576516b48a9b486a5331d428db44f907a48d72a4aad637af9a4b2d00f2"}
Jul 29 17:52:31.646438 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:31.646402    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-svfrm"
Jul 29 17:52:31.646860 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:31.646456    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tdktb"
Jul 29 17:52:31.646860 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:31.646529    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kbx7n"
Jul 29 17:52:31.646860 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:31.646541    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-svfrm" podUID="5540549f-0d35-480f-9135-2e51348f86ea"
Jul 29 17:52:31.646860 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:31.646615    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kbx7n" podUID="57afeaa9-b72d-4389-940b-f56a30b67995"
Jul 29 17:52:31.646860 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:31.646703    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tdktb" podUID="905f9420-cd73-4e49-a85f-2163a04b8fcd"
Jul 29 17:52:31.831494 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:31.831455    2557 generic.go:356] "Generic (PLEG): container finished" podID="277437f5-ac5a-4dd2-8f8e-c2155788e605" containerID="4e28d7a04fc732e936fc1523491cc8bc45cba70cfdf5f83bf06f333332956bbb" exitCode=0
Jul 29 17:52:31.831634 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:31.831521    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rxw8d" event={"ID":"277437f5-ac5a-4dd2-8f8e-c2155788e605","Type":"ContainerDied","Data":"4e28d7a04fc732e936fc1523491cc8bc45cba70cfdf5f83bf06f333332956bbb"}
Jul 29 17:52:33.647098 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:33.647063    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kbx7n"
Jul 29 17:52:33.647506 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:33.647064    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tdktb"
Jul 29 17:52:33.647506 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:33.647191    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kbx7n" podUID="57afeaa9-b72d-4389-940b-f56a30b67995"
Jul 29 17:52:33.647506 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:33.647064    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-svfrm"
Jul 29 17:52:33.647506 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:33.647249    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tdktb" podUID="905f9420-cd73-4e49-a85f-2163a04b8fcd"
Jul 29 17:52:33.647506 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:33.647368    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-svfrm" podUID="5540549f-0d35-480f-9135-2e51348f86ea"
Jul 29 17:52:34.313849 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:34.313811    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/905f9420-cd73-4e49-a85f-2163a04b8fcd-original-pull-secret\") pod \"global-pull-secret-syncer-tdktb\" (UID: \"905f9420-cd73-4e49-a85f-2163a04b8fcd\") " pod="kube-system/global-pull-secret-syncer-tdktb"
Jul 29 17:52:34.313849 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:34.313864    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znrl2\" (UniqueName: \"kubernetes.io/projected/5540549f-0d35-480f-9135-2e51348f86ea-kube-api-access-znrl2\") pod \"network-check-target-svfrm\" (UID: \"5540549f-0d35-480f-9135-2e51348f86ea\") " pod="openshift-network-diagnostics/network-check-target-svfrm"
Jul 29 17:52:34.314101 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:34.313979    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 17:52:34.314101 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:34.313996    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 17:52:34.314101 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:34.314016    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 17:52:34.314101 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:34.314027    2557 projected.go:196] Error preparing data for projected volume kube-api-access-znrl2 for pod openshift-network-diagnostics/network-check-target-svfrm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 17:52:34.314101 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:34.314054    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/905f9420-cd73-4e49-a85f-2163a04b8fcd-original-pull-secret podName:905f9420-cd73-4e49-a85f-2163a04b8fcd nodeName:}" failed. No retries permitted until 2026-07-29 17:53:06.314033384 +0000 UTC m=+66.183256193 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/905f9420-cd73-4e49-a85f-2163a04b8fcd-original-pull-secret") pod "global-pull-secret-syncer-tdktb" (UID: "905f9420-cd73-4e49-a85f-2163a04b8fcd") : object "kube-system"/"original-pull-secret" not registered
Jul 29 17:52:34.314101 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:34.314073    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5540549f-0d35-480f-9135-2e51348f86ea-kube-api-access-znrl2 podName:5540549f-0d35-480f-9135-2e51348f86ea nodeName:}" failed. No retries permitted until 2026-07-29 17:53:06.31406293 +0000 UTC m=+66.183285715 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-znrl2" (UniqueName: "kubernetes.io/projected/5540549f-0d35-480f-9135-2e51348f86ea-kube-api-access-znrl2") pod "network-check-target-svfrm" (UID: "5540549f-0d35-480f-9135-2e51348f86ea") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 17:52:34.415005 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:34.414970    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57afeaa9-b72d-4389-940b-f56a30b67995-metrics-certs\") pod \"network-metrics-daemon-kbx7n\" (UID: \"57afeaa9-b72d-4389-940b-f56a30b67995\") " pod="openshift-multus/network-metrics-daemon-kbx7n"
Jul 29 17:52:34.415158 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:34.415075    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 17:52:34.415158 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:34.415126    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57afeaa9-b72d-4389-940b-f56a30b67995-metrics-certs podName:57afeaa9-b72d-4389-940b-f56a30b67995 nodeName:}" failed. No retries permitted until 2026-07-29 17:53:06.415113282 +0000 UTC m=+66.284336067 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57afeaa9-b72d-4389-940b-f56a30b67995-metrics-certs") pod "network-metrics-daemon-kbx7n" (UID: "57afeaa9-b72d-4389-940b-f56a30b67995") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 17:52:35.483574 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.483541    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-156.ec2.internal" event="NodeReady"
Jul 29 17:52:35.483992 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.483674    2557 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 29 17:52:35.518254 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.518222    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6774b48fd9-hrjxr"]
Jul 29 17:52:35.520996 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.520975    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6774b48fd9-hrjxr"
Jul 29 17:52:35.523691 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.523666    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 29 17:52:35.524025 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.523998    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 29 17:52:35.524132 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.524044    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-w8nx9\""
Jul 29 17:52:35.525357 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.525202    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 29 17:52:35.529803 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.529782    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 29 17:52:35.545165 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.545095    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6774b48fd9-hrjxr"]
Jul 29 17:52:35.546242 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.545765    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-6kfpl"]
Jul 29 17:52:35.548191 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.548168    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-bwjgj"]
Jul 29 17:52:35.548339 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.548309    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6kfpl"
Jul 29 17:52:35.551135 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.551117    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bwjgj"
Jul 29 17:52:35.552859 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.551493    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 29 17:52:35.552859 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.551857    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 29 17:52:35.552859 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.551863    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 29 17:52:35.552859 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.551975    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-nmb94\""
Jul 29 17:52:35.553772 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.553753    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 29 17:52:35.554147 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.554127    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-tr8hk\""
Jul 29 17:52:35.554307 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.554283    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 29 17:52:35.557458 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.557364    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6kfpl"]
Jul 29 17:52:35.569640 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.569617    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bwjgj"]
Jul 29 17:52:35.623643 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.623615    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/09c3a81a-22ca-4700-af82-9c7e86f77f07-bound-sa-token\") pod \"image-registry-6774b48fd9-hrjxr\" (UID: \"09c3a81a-22ca-4700-af82-9c7e86f77f07\") " pod="openshift-image-registry/image-registry-6774b48fd9-hrjxr"
Jul 29 17:52:35.623796 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.623658    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cldh\" (UniqueName: \"kubernetes.io/projected/09c3a81a-22ca-4700-af82-9c7e86f77f07-kube-api-access-7cldh\") pod \"image-registry-6774b48fd9-hrjxr\" (UID: \"09c3a81a-22ca-4700-af82-9c7e86f77f07\") " pod="openshift-image-registry/image-registry-6774b48fd9-hrjxr"
Jul 29 17:52:35.623796 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.623684    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/09c3a81a-22ca-4700-af82-9c7e86f77f07-registry-certificates\") pod \"image-registry-6774b48fd9-hrjxr\" (UID: \"09c3a81a-22ca-4700-af82-9c7e86f77f07\") " pod="openshift-image-registry/image-registry-6774b48fd9-hrjxr"
Jul 29 17:52:35.623796 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.623745    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/09c3a81a-22ca-4700-af82-9c7e86f77f07-trusted-ca\") pod \"image-registry-6774b48fd9-hrjxr\" (UID: \"09c3a81a-22ca-4700-af82-9c7e86f77f07\") " pod="openshift-image-registry/image-registry-6774b48fd9-hrjxr"
Jul 29 17:52:35.623923 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.623791    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/09c3a81a-22ca-4700-af82-9c7e86f77f07-installation-pull-secrets\") pod \"image-registry-6774b48fd9-hrjxr\" (UID: \"09c3a81a-22ca-4700-af82-9c7e86f77f07\") " pod="openshift-image-registry/image-registry-6774b48fd9-hrjxr"
Jul 29 17:52:35.623923 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.623877    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/09c3a81a-22ca-4700-af82-9c7e86f77f07-registry-tls\") pod \"image-registry-6774b48fd9-hrjxr\" (UID: \"09c3a81a-22ca-4700-af82-9c7e86f77f07\") " pod="openshift-image-registry/image-registry-6774b48fd9-hrjxr"
Jul 29 17:52:35.623923 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.623900    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/09c3a81a-22ca-4700-af82-9c7e86f77f07-ca-trust-extracted\") pod \"image-registry-6774b48fd9-hrjxr\" (UID: \"09c3a81a-22ca-4700-af82-9c7e86f77f07\") " pod="openshift-image-registry/image-registry-6774b48fd9-hrjxr"
Jul 29 17:52:35.624044 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.623968    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/09c3a81a-22ca-4700-af82-9c7e86f77f07-image-registry-private-configuration\") pod \"image-registry-6774b48fd9-hrjxr\" (UID: \"09c3a81a-22ca-4700-af82-9c7e86f77f07\") " pod="openshift-image-registry/image-registry-6774b48fd9-hrjxr"
Jul 29 17:52:35.646860 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.646809    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kbx7n"
Jul 29 17:52:35.647008 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.646934    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tdktb"
Jul 29 17:52:35.647141 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.647124    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-svfrm"
Jul 29 17:52:35.650123 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.650106    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 29 17:52:35.650123 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.650121    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 29 17:52:35.650287 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.650153    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 29 17:52:35.650287 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.650197    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-llfsb\""
Jul 29 17:52:35.650287 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.650275    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 29 17:52:35.650436 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.650346    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-w7s6p\""
Jul 29 17:52:35.724927 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.724903    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/dedf5ed1-ad33-4658-b79c-ae80134b18c7-config-volume\") pod \"dns-default-bwjgj\" (UID: \"dedf5ed1-ad33-4658-b79c-ae80134b18c7\") " pod="openshift-dns/dns-default-bwjgj"
Jul 29 17:52:35.725066 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.724941    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cldh\" (UniqueName: \"kubernetes.io/projected/09c3a81a-22ca-4700-af82-9c7e86f77f07-kube-api-access-7cldh\") pod \"image-registry-6774b48fd9-hrjxr\" (UID: \"09c3a81a-22ca-4700-af82-9c7e86f77f07\") " pod="openshift-image-registry/image-registry-6774b48fd9-hrjxr"
Jul 29 17:52:35.725066 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.724970    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxqn7\" (UniqueName: \"kubernetes.io/projected/dedf5ed1-ad33-4658-b79c-ae80134b18c7-kube-api-access-zxqn7\") pod \"dns-default-bwjgj\" (UID: \"dedf5ed1-ad33-4658-b79c-ae80134b18c7\") " pod="openshift-dns/dns-default-bwjgj"
Jul 29 17:52:35.725066 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.725002    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/09c3a81a-22ca-4700-af82-9c7e86f77f07-registry-certificates\") pod \"image-registry-6774b48fd9-hrjxr\" (UID: \"09c3a81a-22ca-4700-af82-9c7e86f77f07\") " pod="openshift-image-registry/image-registry-6774b48fd9-hrjxr"
Jul 29 17:52:35.725066 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.725023    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f9eb2a9b-3f29-455c-9cc2-0d3924a4bef2-cert\") pod \"ingress-canary-6kfpl\" (UID: \"f9eb2a9b-3f29-455c-9cc2-0d3924a4bef2\") " pod="openshift-ingress-canary/ingress-canary-6kfpl"
Jul 29 17:52:35.725066 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.725064    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/09c3a81a-22ca-4700-af82-9c7e86f77f07-trusted-ca\") pod \"image-registry-6774b48fd9-hrjxr\" (UID: \"09c3a81a-22ca-4700-af82-9c7e86f77f07\") " pod="openshift-image-registry/image-registry-6774b48fd9-hrjxr"
Jul 29 17:52:35.725284 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.725080    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/09c3a81a-22ca-4700-af82-9c7e86f77f07-installation-pull-secrets\") pod \"image-registry-6774b48fd9-hrjxr\" (UID: \"09c3a81a-22ca-4700-af82-9c7e86f77f07\") " pod="openshift-image-registry/image-registry-6774b48fd9-hrjxr"
Jul 29 17:52:35.725284 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.725111    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/09c3a81a-22ca-4700-af82-9c7e86f77f07-registry-tls\") pod \"image-registry-6774b48fd9-hrjxr\" (UID: \"09c3a81a-22ca-4700-af82-9c7e86f77f07\") " pod="openshift-image-registry/image-registry-6774b48fd9-hrjxr"
Jul 29 17:52:35.725284 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.725138    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/09c3a81a-22ca-4700-af82-9c7e86f77f07-ca-trust-extracted\") pod \"image-registry-6774b48fd9-hrjxr\" (UID: \"09c3a81a-22ca-4700-af82-9c7e86f77f07\") " pod="openshift-image-registry/image-registry-6774b48fd9-hrjxr"
Jul 29 17:52:35.725284 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.725191    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvvsf\" (UniqueName: \"kubernetes.io/projected/f9eb2a9b-3f29-455c-9cc2-0d3924a4bef2-kube-api-access-vvvsf\") pod \"ingress-canary-6kfpl\" (UID: \"f9eb2a9b-3f29-455c-9cc2-0d3924a4bef2\") " pod="openshift-ingress-canary/ingress-canary-6kfpl"
Jul 29 17:52:35.725284 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.725225    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/09c3a81a-22ca-4700-af82-9c7e86f77f07-image-registry-private-configuration\") pod \"image-registry-6774b48fd9-hrjxr\" (UID: \"09c3a81a-22ca-4700-af82-9c7e86f77f07\") " pod="openshift-image-registry/image-registry-6774b48fd9-hrjxr"
Jul 29 17:52:35.725284 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.725260    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dedf5ed1-ad33-4658-b79c-ae80134b18c7-tmp-dir\") pod \"dns-default-bwjgj\" (UID: \"dedf5ed1-ad33-4658-b79c-ae80134b18c7\") " pod="openshift-dns/dns-default-bwjgj"
Jul 29 17:52:35.725528 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.725290    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/09c3a81a-22ca-4700-af82-9c7e86f77f07-bound-sa-token\") pod \"image-registry-6774b48fd9-hrjxr\" (UID: \"09c3a81a-22ca-4700-af82-9c7e86f77f07\") " pod="openshift-image-registry/image-registry-6774b48fd9-hrjxr"
Jul 29 17:52:35.725528 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.725319    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dedf5ed1-ad33-4658-b79c-ae80134b18c7-metrics-tls\") pod \"dns-default-bwjgj\" (UID: \"dedf5ed1-ad33-4658-b79c-ae80134b18c7\") " pod="openshift-dns/dns-default-bwjgj"
Jul 29 17:52:35.725528 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:35.725418    2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 17:52:35.725528 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:35.725432    2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6774b48fd9-hrjxr: secret "image-registry-tls" not found
Jul 29 17:52:35.725528 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:35.725492    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/09c3a81a-22ca-4700-af82-9c7e86f77f07-registry-tls podName:09c3a81a-22ca-4700-af82-9c7e86f77f07 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:36.22547529 +0000 UTC m=+36.094698076 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/09c3a81a-22ca-4700-af82-9c7e86f77f07-registry-tls") pod "image-registry-6774b48fd9-hrjxr" (UID: "09c3a81a-22ca-4700-af82-9c7e86f77f07") : secret "image-registry-tls" not found
Jul 29 17:52:35.726388 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.726209    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/09c3a81a-22ca-4700-af82-9c7e86f77f07-ca-trust-extracted\") pod \"image-registry-6774b48fd9-hrjxr\" (UID: \"09c3a81a-22ca-4700-af82-9c7e86f77f07\") " pod="openshift-image-registry/image-registry-6774b48fd9-hrjxr"
Jul 29 17:52:35.726491 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.726358    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/09c3a81a-22ca-4700-af82-9c7e86f77f07-trusted-ca\") pod \"image-registry-6774b48fd9-hrjxr\" (UID: \"09c3a81a-22ca-4700-af82-9c7e86f77f07\") " pod="openshift-image-registry/image-registry-6774b48fd9-hrjxr"
Jul 29 17:52:35.726545 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.726490    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/09c3a81a-22ca-4700-af82-9c7e86f77f07-registry-certificates\") pod \"image-registry-6774b48fd9-hrjxr\" (UID: \"09c3a81a-22ca-4700-af82-9c7e86f77f07\") " pod="openshift-image-registry/image-registry-6774b48fd9-hrjxr"
Jul 29 17:52:35.730046 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.730027    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/09c3a81a-22ca-4700-af82-9c7e86f77f07-installation-pull-secrets\") pod \"image-registry-6774b48fd9-hrjxr\" (UID: \"09c3a81a-22ca-4700-af82-9c7e86f77f07\") " pod="openshift-image-registry/image-registry-6774b48fd9-hrjxr"
Jul 29 17:52:35.730046 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.730041    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/09c3a81a-22ca-4700-af82-9c7e86f77f07-image-registry-private-configuration\") pod \"image-registry-6774b48fd9-hrjxr\" (UID: \"09c3a81a-22ca-4700-af82-9c7e86f77f07\") " pod="openshift-image-registry/image-registry-6774b48fd9-hrjxr"
Jul 29 17:52:35.734075 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.734018    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/09c3a81a-22ca-4700-af82-9c7e86f77f07-bound-sa-token\") pod \"image-registry-6774b48fd9-hrjxr\" (UID: \"09c3a81a-22ca-4700-af82-9c7e86f77f07\") " pod="openshift-image-registry/image-registry-6774b48fd9-hrjxr"
Jul 29 17:52:35.734429 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.734414    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cldh\" (UniqueName: \"kubernetes.io/projected/09c3a81a-22ca-4700-af82-9c7e86f77f07-kube-api-access-7cldh\") pod \"image-registry-6774b48fd9-hrjxr\" (UID: \"09c3a81a-22ca-4700-af82-9c7e86f77f07\") " pod="openshift-image-registry/image-registry-6774b48fd9-hrjxr"
Jul 29 17:52:35.826441 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.826410    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxqn7\" (UniqueName: \"kubernetes.io/projected/dedf5ed1-ad33-4658-b79c-ae80134b18c7-kube-api-access-zxqn7\") pod \"dns-default-bwjgj\" (UID: \"dedf5ed1-ad33-4658-b79c-ae80134b18c7\") " pod="openshift-dns/dns-default-bwjgj"
Jul 29 17:52:35.826622 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.826455    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f9eb2a9b-3f29-455c-9cc2-0d3924a4bef2-cert\") pod \"ingress-canary-6kfpl\" (UID: \"f9eb2a9b-3f29-455c-9cc2-0d3924a4bef2\") " pod="openshift-ingress-canary/ingress-canary-6kfpl"
Jul 29 17:52:35.826622 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.826538    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvvsf\" (UniqueName: \"kubernetes.io/projected/f9eb2a9b-3f29-455c-9cc2-0d3924a4bef2-kube-api-access-vvvsf\") pod \"ingress-canary-6kfpl\" (UID: \"f9eb2a9b-3f29-455c-9cc2-0d3924a4bef2\") " pod="openshift-ingress-canary/ingress-canary-6kfpl"
Jul 29 17:52:35.826622 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.826582    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dedf5ed1-ad33-4658-b79c-ae80134b18c7-tmp-dir\") pod \"dns-default-bwjgj\" (UID: \"dedf5ed1-ad33-4658-b79c-ae80134b18c7\") " pod="openshift-dns/dns-default-bwjgj"
Jul 29 17:52:35.826622 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.826611    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dedf5ed1-ad33-4658-b79c-ae80134b18c7-metrics-tls\") pod \"dns-default-bwjgj\" (UID: \"dedf5ed1-ad33-4658-b79c-ae80134b18c7\") " pod="openshift-dns/dns-default-bwjgj"
Jul 29 17:52:35.826849 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.826639    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/dedf5ed1-ad33-4658-b79c-ae80134b18c7-config-volume\") pod \"dns-default-bwjgj\" (UID: \"dedf5ed1-ad33-4658-b79c-ae80134b18c7\") " pod="openshift-dns/dns-default-bwjgj"
Jul 29 17:52:35.826970 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:35.826948    2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 17:52:35.827052 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:35.827039    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dedf5ed1-ad33-4658-b79c-ae80134b18c7-metrics-tls podName:dedf5ed1-ad33-4658-b79c-ae80134b18c7 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:36.327014922 +0000 UTC m=+36.196237711 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dedf5ed1-ad33-4658-b79c-ae80134b18c7-metrics-tls") pod "dns-default-bwjgj" (UID: "dedf5ed1-ad33-4658-b79c-ae80134b18c7") : secret "dns-default-metrics-tls" not found
Jul 29 17:52:35.827800 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.827772    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dedf5ed1-ad33-4658-b79c-ae80134b18c7-tmp-dir\") pod \"dns-default-bwjgj\" (UID: \"dedf5ed1-ad33-4658-b79c-ae80134b18c7\") " pod="openshift-dns/dns-default-bwjgj"
Jul 29 17:52:35.827977 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.827951    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/dedf5ed1-ad33-4658-b79c-ae80134b18c7-config-volume\") pod \"dns-default-bwjgj\" (UID: \"dedf5ed1-ad33-4658-b79c-ae80134b18c7\") " pod="openshift-dns/dns-default-bwjgj"
Jul 29 17:52:35.829419 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.829395    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f9eb2a9b-3f29-455c-9cc2-0d3924a4bef2-cert\") pod \"ingress-canary-6kfpl\" (UID: \"f9eb2a9b-3f29-455c-9cc2-0d3924a4bef2\") " pod="openshift-ingress-canary/ingress-canary-6kfpl"
Jul 29 17:52:35.837507 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.837487    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxqn7\" (UniqueName: \"kubernetes.io/projected/dedf5ed1-ad33-4658-b79c-ae80134b18c7-kube-api-access-zxqn7\") pod \"dns-default-bwjgj\" (UID: \"dedf5ed1-ad33-4658-b79c-ae80134b18c7\") " pod="openshift-dns/dns-default-bwjgj"
Jul 29 17:52:35.837744 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.837705    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvvsf\" (UniqueName: \"kubernetes.io/projected/f9eb2a9b-3f29-455c-9cc2-0d3924a4bef2-kube-api-access-vvvsf\") pod \"ingress-canary-6kfpl\" (UID: \"f9eb2a9b-3f29-455c-9cc2-0d3924a4bef2\") " pod="openshift-ingress-canary/ingress-canary-6kfpl"
Jul 29 17:52:35.862709 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:35.862684    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6kfpl"
Jul 29 17:52:36.229121 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:36.229079    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/09c3a81a-22ca-4700-af82-9c7e86f77f07-registry-tls\") pod \"image-registry-6774b48fd9-hrjxr\" (UID: \"09c3a81a-22ca-4700-af82-9c7e86f77f07\") " pod="openshift-image-registry/image-registry-6774b48fd9-hrjxr"
Jul 29 17:52:36.229304 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:36.229244    2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 17:52:36.229304 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:36.229263    2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6774b48fd9-hrjxr: secret "image-registry-tls" not found
Jul 29 17:52:36.229375 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:36.229322    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/09c3a81a-22ca-4700-af82-9c7e86f77f07-registry-tls podName:09c3a81a-22ca-4700-af82-9c7e86f77f07 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:37.229304993 +0000 UTC m=+37.098527778 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/09c3a81a-22ca-4700-af82-9c7e86f77f07-registry-tls") pod "image-registry-6774b48fd9-hrjxr" (UID: "09c3a81a-22ca-4700-af82-9c7e86f77f07") : secret "image-registry-tls" not found
Jul 29 17:52:36.330283 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:36.330007    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dedf5ed1-ad33-4658-b79c-ae80134b18c7-metrics-tls\") pod \"dns-default-bwjgj\" (UID: \"dedf5ed1-ad33-4658-b79c-ae80134b18c7\") " pod="openshift-dns/dns-default-bwjgj"
Jul 29 17:52:36.330283 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:36.330230    2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 17:52:36.330404 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:36.330333    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dedf5ed1-ad33-4658-b79c-ae80134b18c7-metrics-tls podName:dedf5ed1-ad33-4658-b79c-ae80134b18c7 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:37.330272073 +0000 UTC m=+37.199494875 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dedf5ed1-ad33-4658-b79c-ae80134b18c7-metrics-tls") pod "dns-default-bwjgj" (UID: "dedf5ed1-ad33-4658-b79c-ae80134b18c7") : secret "dns-default-metrics-tls" not found
Jul 29 17:52:36.379247 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:36.379226    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6kfpl"]
Jul 29 17:52:36.381451 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:36.381425    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9eb2a9b_3f29_455c_9cc2_0d3924a4bef2.slice/crio-edc5745cc20066b14c78459799337b85edf1f8553946a4e12752549578304db2 WatchSource:0}: Error finding container edc5745cc20066b14c78459799337b85edf1f8553946a4e12752549578304db2: Status 404 returned error can't find the container with id edc5745cc20066b14c78459799337b85edf1f8553946a4e12752549578304db2
Jul 29 17:52:36.846213 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:36.846180    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6kfpl" event={"ID":"f9eb2a9b-3f29-455c-9cc2-0d3924a4bef2","Type":"ContainerStarted","Data":"edc5745cc20066b14c78459799337b85edf1f8553946a4e12752549578304db2"}
Jul 29 17:52:36.848619 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:36.848595    2557 generic.go:356] "Generic (PLEG): container finished" podID="277437f5-ac5a-4dd2-8f8e-c2155788e605" containerID="f42bb80531c7de05228571d854efefb4771ae6cb92f0e2943c7288d9c467a421" exitCode=0
Jul 29 17:52:36.848742 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:36.848634    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rxw8d" event={"ID":"277437f5-ac5a-4dd2-8f8e-c2155788e605","Type":"ContainerDied","Data":"f42bb80531c7de05228571d854efefb4771ae6cb92f0e2943c7288d9c467a421"}
Jul 29 17:52:37.236621 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:37.236534    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/09c3a81a-22ca-4700-af82-9c7e86f77f07-registry-tls\") pod \"image-registry-6774b48fd9-hrjxr\" (UID: \"09c3a81a-22ca-4700-af82-9c7e86f77f07\") " pod="openshift-image-registry/image-registry-6774b48fd9-hrjxr"
Jul 29 17:52:37.236786 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:37.236707    2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 17:52:37.236786 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:37.236747    2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6774b48fd9-hrjxr: secret "image-registry-tls" not found
Jul 29 17:52:37.236878 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:37.236815    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/09c3a81a-22ca-4700-af82-9c7e86f77f07-registry-tls podName:09c3a81a-22ca-4700-af82-9c7e86f77f07 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:39.236794267 +0000 UTC m=+39.106017069 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/09c3a81a-22ca-4700-af82-9c7e86f77f07-registry-tls") pod "image-registry-6774b48fd9-hrjxr" (UID: "09c3a81a-22ca-4700-af82-9c7e86f77f07") : secret "image-registry-tls" not found
Jul 29 17:52:37.337553 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:37.337523    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dedf5ed1-ad33-4658-b79c-ae80134b18c7-metrics-tls\") pod \"dns-default-bwjgj\" (UID: \"dedf5ed1-ad33-4658-b79c-ae80134b18c7\") " pod="openshift-dns/dns-default-bwjgj"
Jul 29 17:52:37.337719 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:37.337630    2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 17:52:37.337719 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:37.337680    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dedf5ed1-ad33-4658-b79c-ae80134b18c7-metrics-tls podName:dedf5ed1-ad33-4658-b79c-ae80134b18c7 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:39.337667083 +0000 UTC m=+39.206889868 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dedf5ed1-ad33-4658-b79c-ae80134b18c7-metrics-tls") pod "dns-default-bwjgj" (UID: "dedf5ed1-ad33-4658-b79c-ae80134b18c7") : secret "dns-default-metrics-tls" not found
Jul 29 17:52:37.854269 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:37.854233    2557 generic.go:356] "Generic (PLEG): container finished" podID="277437f5-ac5a-4dd2-8f8e-c2155788e605" containerID="1c980f6450e609f530d55de8c12de15ccc9f9718a55c955cfed3e6463d3eb899" exitCode=0
Jul 29 17:52:37.854622 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:37.854273    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rxw8d" event={"ID":"277437f5-ac5a-4dd2-8f8e-c2155788e605","Type":"ContainerDied","Data":"1c980f6450e609f530d55de8c12de15ccc9f9718a55c955cfed3e6463d3eb899"}
Jul 29 17:52:38.860793 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:38.860572    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rxw8d" event={"ID":"277437f5-ac5a-4dd2-8f8e-c2155788e605","Type":"ContainerStarted","Data":"bd1228ec3636f41c122f97bb17e235bdcbf480c119d847b5ad2409b16d30640f"}
Jul 29 17:52:38.885502 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:38.885451    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-rxw8d" podStartSLOduration=8.55393385 podStartE2EDuration="38.885431812s" totalImagesPullingTime="30.331497962s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 17:52:00 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:52:38.885262522 +0000 UTC m=+38.754485447" watchObservedRunningTime="2026-07-29 17:52:38.885431812 +0000 UTC m=+38.754654619"
Jul 29 17:52:39.252914 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:39.252871    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/09c3a81a-22ca-4700-af82-9c7e86f77f07-registry-tls\") pod \"image-registry-6774b48fd9-hrjxr\" (UID: \"09c3a81a-22ca-4700-af82-9c7e86f77f07\") " pod="openshift-image-registry/image-registry-6774b48fd9-hrjxr"
Jul 29 17:52:39.253086 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:39.253059    2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 17:52:39.253086 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:39.253081    2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6774b48fd9-hrjxr: secret "image-registry-tls" not found
Jul 29 17:52:39.253205 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:39.253143    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/09c3a81a-22ca-4700-af82-9c7e86f77f07-registry-tls podName:09c3a81a-22ca-4700-af82-9c7e86f77f07 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:43.253128788 +0000 UTC m=+43.122351573 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/09c3a81a-22ca-4700-af82-9c7e86f77f07-registry-tls") pod "image-registry-6774b48fd9-hrjxr" (UID: "09c3a81a-22ca-4700-af82-9c7e86f77f07") : secret "image-registry-tls" not found
Jul 29 17:52:39.353269 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:39.353235    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dedf5ed1-ad33-4658-b79c-ae80134b18c7-metrics-tls\") pod \"dns-default-bwjgj\" (UID: \"dedf5ed1-ad33-4658-b79c-ae80134b18c7\") " pod="openshift-dns/dns-default-bwjgj"
Jul 29 17:52:39.353406 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:39.353385    2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 17:52:39.353488 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:39.353478    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dedf5ed1-ad33-4658-b79c-ae80134b18c7-metrics-tls podName:dedf5ed1-ad33-4658-b79c-ae80134b18c7 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:43.35346342 +0000 UTC m=+43.222686205 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dedf5ed1-ad33-4658-b79c-ae80134b18c7-metrics-tls") pod "dns-default-bwjgj" (UID: "dedf5ed1-ad33-4658-b79c-ae80134b18c7") : secret "dns-default-metrics-tls" not found
Jul 29 17:52:39.864649 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:39.864615    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6kfpl" event={"ID":"f9eb2a9b-3f29-455c-9cc2-0d3924a4bef2","Type":"ContainerStarted","Data":"3d17b715399c13e18ca032b15e6b0cf328b560660c23b92f7f20af3b863551e2"}
Jul 29 17:52:39.883309 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:39.883267    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-6kfpl" podStartSLOduration=1.9051787359999999 podStartE2EDuration="4.883253439s" totalImagesPullingTime="2.978074703s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:52:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:52:39.882580475 +0000 UTC m=+39.751803283" watchObservedRunningTime="2026-07-29 17:52:39.883253439 +0000 UTC m=+39.752476224"
Jul 29 17:52:41.240183 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.240153    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-5v6zw"]
Jul 29 17:52:41.276905 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.276869    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-2fks5"]
Jul 29 17:52:41.277038 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.276985    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-5v6zw"
Jul 29 17:52:41.279738 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.279691    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 29 17:52:41.279738 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.279706    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 29 17:52:41.279903 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.279746    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-q6lr4\""
Jul 29 17:52:41.279903 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.279749    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 29 17:52:41.290888 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.290865    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-5v6zw"]
Jul 29 17:52:41.290985 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.290903    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-2fks5"]
Jul 29 17:52:41.290985 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.290970    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-2fks5"
Jul 29 17:52:41.293458 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.293436    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 29 17:52:41.293575 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.293472    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 29 17:52:41.293575 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.293490    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 29 17:52:41.293575 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.293472    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-b4ggt\""
Jul 29 17:52:41.293818 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.293801    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 29 17:52:41.339686 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.339655    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-hhwbp"]
Jul 29 17:52:41.358922 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.358891    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-4s8jb"]
Jul 29 17:52:41.359199 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.359021    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-hhwbp"
Jul 29 17:52:41.362536 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.362138    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-q7lc8\""
Jul 29 17:52:41.362536 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.362251    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 29 17:52:41.362536 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.362395    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 29 17:52:41.362536 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.362480    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 29 17:52:41.362809 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.362794    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 29 17:52:41.381056 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.381029    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-4j8zv"]
Jul 29 17:52:41.381207 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.381189    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-4s8jb"
Jul 29 17:52:41.383848 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.383830    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-ssdj7\""
Jul 29 17:52:41.405208 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.405189    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-m2x44"]
Jul 29 17:52:41.405334 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.405318    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-4j8zv"
Jul 29 17:52:41.408403 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.408383    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-xmk6h\""
Jul 29 17:52:41.408559 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.408525    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 29 17:52:41.408707 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.408684    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 29 17:52:41.408893 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.408869    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 29 17:52:41.410512 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.409414    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 29 17:52:41.414957 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.414940    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 29 17:52:41.421214 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.421190    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-gg2wj"]
Jul 29 17:52:41.421305 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.421294    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-m2x44"
Jul 29 17:52:41.423735 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.423700    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-mkmh9\""
Jul 29 17:52:41.423846 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.423741    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 29 17:52:41.423846 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.423776    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 29 17:52:41.440217 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.440200    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-hhwbp"]
Jul 29 17:52:41.440284 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.440226    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-4s8jb"]
Jul 29 17:52:41.440284 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.440240    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-m2x44"]
Jul 29 17:52:41.440284 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.440252    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-4j8zv"]
Jul 29 17:52:41.440284 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.440267    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-gg2wj"]
Jul 29 17:52:41.440407 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.440306    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-gg2wj"
Jul 29 17:52:41.441829 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.441813    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-grlrl"]
Jul 29 17:52:41.442977 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.442954    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-f4kc7\""
Jul 29 17:52:41.443103 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.443024    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 29 17:52:41.443179 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.443129    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 29 17:52:41.443226 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.443215    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 29 17:52:41.443289 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.443278    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 29 17:52:41.448304 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.448283    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 29 17:52:41.461270 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.461217    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-rgj9m"]
Jul 29 17:52:41.461378 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.461331    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-grlrl"
Jul 29 17:52:41.463740 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.463699    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-flh7w\""
Jul 29 17:52:41.463956 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.463941    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 29 17:52:41.464262 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.464244    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 29 17:52:41.468877 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.468854    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a91229a9-7326-45f0-9bed-57c0f1868780-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-hhwbp\" (UID: \"a91229a9-7326-45f0-9bed-57c0f1868780\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-hhwbp"
Jul 29 17:52:41.469026 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.469004    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-844jv\" (UniqueName: \"kubernetes.io/projected/a91229a9-7326-45f0-9bed-57c0f1868780-kube-api-access-844jv\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-hhwbp\" (UID: \"a91229a9-7326-45f0-9bed-57c0f1868780\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-hhwbp"
Jul 29 17:52:41.469116 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.469061    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a91229a9-7326-45f0-9bed-57c0f1868780-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-hhwbp\" (UID: \"a91229a9-7326-45f0-9bed-57c0f1868780\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-hhwbp"
Jul 29 17:52:41.469116 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.469092    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-269fs\" (UniqueName: \"kubernetes.io/projected/fcb6432a-cde0-4f40-9e00-2523c43ca2c8-kube-api-access-269fs\") pod \"service-ca-operator-5d6c75f84f-2fks5\" (UID: \"fcb6432a-cde0-4f40-9e00-2523c43ca2c8\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-2fks5"
Jul 29 17:52:41.469218 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.469116    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a1be716c-91ad-4ab5-a476-32486b3c0506-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-5v6zw\" (UID: \"a1be716c-91ad-4ab5-a476-32486b3c0506\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-5v6zw"
Jul 29 17:52:41.469218 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.469141    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbthb\" (UniqueName: \"kubernetes.io/projected/a1be716c-91ad-4ab5-a476-32486b3c0506-kube-api-access-dbthb\") pod \"cluster-samples-operator-6748489656-5v6zw\" (UID: \"a1be716c-91ad-4ab5-a476-32486b3c0506\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-5v6zw"
Jul 29 17:52:41.469218 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.469186    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fcb6432a-cde0-4f40-9e00-2523c43ca2c8-config\") pod \"service-ca-operator-5d6c75f84f-2fks5\" (UID: \"fcb6432a-cde0-4f40-9e00-2523c43ca2c8\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-2fks5"
Jul 29 17:52:41.469218 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.469213    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fcb6432a-cde0-4f40-9e00-2523c43ca2c8-serving-cert\") pod \"service-ca-operator-5d6c75f84f-2fks5\" (UID: \"fcb6432a-cde0-4f40-9e00-2523c43ca2c8\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-2fks5"
Jul 29 17:52:41.487458 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.487423    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-grlrl"]
Jul 29 17:52:41.487458 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.487455    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-rgj9m"]
Jul 29 17:52:41.487636 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.487474    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-676c59fd4f-tvcgb"]
Jul 29 17:52:41.487636 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.487547    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rgj9m"
Jul 29 17:52:41.490273 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.490210    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 29 17:52:41.490273 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.490261    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 29 17:52:41.490397 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.490298    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-jd426\""
Jul 29 17:52:41.491442 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.491428    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 29 17:52:41.491442 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.491441    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 29 17:52:41.511627 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.511597    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-676c59fd4f-tvcgb"]
Jul 29 17:52:41.511755 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.511687    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-676c59fd4f-tvcgb"
Jul 29 17:52:41.514426 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.514399    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 29 17:52:41.514426 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.514418    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 29 17:52:41.514599 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.514451    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 29 17:52:41.514599 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.514483    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 29 17:52:41.514599 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.514518    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 29 17:52:41.514752 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.514740    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-thk5g\""
Jul 29 17:52:41.515155 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.515141    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 29 17:52:41.570470 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.570434    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4kl4\" (UniqueName: \"kubernetes.io/projected/6554ff13-9d69-4062-be7c-a8d3ff442945-kube-api-access-r4kl4\") pod \"volume-data-source-validator-549c6d54cf-m2x44\" (UID: \"6554ff13-9d69-4062-be7c-a8d3ff442945\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-m2x44"
Jul 29 17:52:41.570615 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.570500    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2bf0bb71-4047-45a7-bd8e-5d6efabd26bd-trusted-ca\") pod \"console-operator-584dcb6b54-gg2wj\" (UID: \"2bf0bb71-4047-45a7-bd8e-5d6efabd26bd\") " pod="openshift-console-operator/console-operator-584dcb6b54-gg2wj"
Jul 29 17:52:41.570615 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.570570    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8pg8\" (UniqueName: \"kubernetes.io/projected/7843c951-e574-4cca-bbc8-ddc52d5d2b55-kube-api-access-z8pg8\") pod \"network-check-source-7d88cb55d8-4s8jb\" (UID: \"7843c951-e574-4cca-bbc8-ddc52d5d2b55\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-4s8jb"
Jul 29 17:52:41.570615 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.570598    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/a9042be4-9aa4-4bc5-b9b8-364a7f782353-snapshots\") pod \"insights-operator-5885fdb759-4j8zv\" (UID: \"a9042be4-9aa4-4bc5-b9b8-364a7f782353\") " pod="openshift-insights/insights-operator-5885fdb759-4j8zv"
Jul 29 17:52:41.570766 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.570620    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5kqs\" (UniqueName: \"kubernetes.io/projected/a9042be4-9aa4-4bc5-b9b8-364a7f782353-kube-api-access-f5kqs\") pod \"insights-operator-5885fdb759-4j8zv\" (UID: \"a9042be4-9aa4-4bc5-b9b8-364a7f782353\") " pod="openshift-insights/insights-operator-5885fdb759-4j8zv"
Jul 29 17:52:41.570766 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.570636    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a9042be4-9aa4-4bc5-b9b8-364a7f782353-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-4j8zv\" (UID: \"a9042be4-9aa4-4bc5-b9b8-364a7f782353\") " pod="openshift-insights/insights-operator-5885fdb759-4j8zv"
Jul 29 17:52:41.570766 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.570652    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a9042be4-9aa4-4bc5-b9b8-364a7f782353-service-ca-bundle\") pod \"insights-operator-5885fdb759-4j8zv\" (UID: \"a9042be4-9aa4-4bc5-b9b8-364a7f782353\") " pod="openshift-insights/insights-operator-5885fdb759-4j8zv"
Jul 29 17:52:41.570766 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.570737    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a9042be4-9aa4-4bc5-b9b8-364a7f782353-serving-cert\") pod \"insights-operator-5885fdb759-4j8zv\" (UID: \"a9042be4-9aa4-4bc5-b9b8-364a7f782353\") " pod="openshift-insights/insights-operator-5885fdb759-4j8zv"
Jul 29 17:52:41.570884 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.570802    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2bf0bb71-4047-45a7-bd8e-5d6efabd26bd-serving-cert\") pod \"console-operator-584dcb6b54-gg2wj\" (UID: \"2bf0bb71-4047-45a7-bd8e-5d6efabd26bd\") " pod="openshift-console-operator/console-operator-584dcb6b54-gg2wj"
Jul 29 17:52:41.570884 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.570828    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3aad4b60-ee91-42d3-bbf1-865219602575-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-grlrl\" (UID: \"3aad4b60-ee91-42d3-bbf1-865219602575\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-grlrl"
Jul 29 17:52:41.570884 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.570861    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fcb6432a-cde0-4f40-9e00-2523c43ca2c8-config\") pod \"service-ca-operator-5d6c75f84f-2fks5\" (UID: \"fcb6432a-cde0-4f40-9e00-2523c43ca2c8\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-2fks5"
Jul 29 17:52:41.570976 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.570890    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fcb6432a-cde0-4f40-9e00-2523c43ca2c8-serving-cert\") pod \"service-ca-operator-5d6c75f84f-2fks5\" (UID: \"fcb6432a-cde0-4f40-9e00-2523c43ca2c8\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-2fks5"
Jul 29 17:52:41.570976 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.570917    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdxqs\" (UniqueName: \"kubernetes.io/projected/2bf0bb71-4047-45a7-bd8e-5d6efabd26bd-kube-api-access-wdxqs\") pod \"console-operator-584dcb6b54-gg2wj\" (UID: \"2bf0bb71-4047-45a7-bd8e-5d6efabd26bd\") " pod="openshift-console-operator/console-operator-584dcb6b54-gg2wj"
Jul 29 17:52:41.570976 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.570944    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2bf0bb71-4047-45a7-bd8e-5d6efabd26bd-config\") pod \"console-operator-584dcb6b54-gg2wj\" (UID: \"2bf0bb71-4047-45a7-bd8e-5d6efabd26bd\") " pod="openshift-console-operator/console-operator-584dcb6b54-gg2wj"
Jul 29 17:52:41.570976 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.570968    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3aad4b60-ee91-42d3-bbf1-865219602575-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-grlrl\" (UID: \"3aad4b60-ee91-42d3-bbf1-865219602575\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-grlrl"
Jul 29 17:52:41.571114 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.570991    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a91229a9-7326-45f0-9bed-57c0f1868780-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-hhwbp\" (UID: \"a91229a9-7326-45f0-9bed-57c0f1868780\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-hhwbp"
Jul 29 17:52:41.571114 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.571018    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-844jv\" (UniqueName: \"kubernetes.io/projected/a91229a9-7326-45f0-9bed-57c0f1868780-kube-api-access-844jv\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-hhwbp\" (UID: \"a91229a9-7326-45f0-9bed-57c0f1868780\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-hhwbp"
Jul 29 17:52:41.571114 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.571057    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a91229a9-7326-45f0-9bed-57c0f1868780-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-hhwbp\" (UID: \"a91229a9-7326-45f0-9bed-57c0f1868780\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-hhwbp"
Jul 29 17:52:41.571114 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.571083    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-269fs\" (UniqueName: \"kubernetes.io/projected/fcb6432a-cde0-4f40-9e00-2523c43ca2c8-kube-api-access-269fs\") pod \"service-ca-operator-5d6c75f84f-2fks5\" (UID: \"fcb6432a-cde0-4f40-9e00-2523c43ca2c8\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-2fks5"
Jul 29 17:52:41.571114 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.571106    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a1be716c-91ad-4ab5-a476-32486b3c0506-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-5v6zw\" (UID: \"a1be716c-91ad-4ab5-a476-32486b3c0506\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-5v6zw"
Jul 29 17:52:41.571340 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.571128    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a9042be4-9aa4-4bc5-b9b8-364a7f782353-tmp\") pod \"insights-operator-5885fdb759-4j8zv\" (UID: \"a9042be4-9aa4-4bc5-b9b8-364a7f782353\") " pod="openshift-insights/insights-operator-5885fdb759-4j8zv"
Jul 29 17:52:41.571340 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.571153    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbthb\" (UniqueName: \"kubernetes.io/projected/a1be716c-91ad-4ab5-a476-32486b3c0506-kube-api-access-dbthb\") pod \"cluster-samples-operator-6748489656-5v6zw\" (UID: \"a1be716c-91ad-4ab5-a476-32486b3c0506\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-5v6zw"
Jul 29 17:52:41.571340 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:41.571239    2557 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 29 17:52:41.571340 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:41.571309    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1be716c-91ad-4ab5-a476-32486b3c0506-samples-operator-tls podName:a1be716c-91ad-4ab5-a476-32486b3c0506 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:42.071290296 +0000 UTC m=+41.940513081 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a1be716c-91ad-4ab5-a476-32486b3c0506-samples-operator-tls") pod "cluster-samples-operator-6748489656-5v6zw" (UID: "a1be716c-91ad-4ab5-a476-32486b3c0506") : secret "samples-operator-tls" not found
Jul 29 17:52:41.571555 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.571518    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fcb6432a-cde0-4f40-9e00-2523c43ca2c8-config\") pod \"service-ca-operator-5d6c75f84f-2fks5\" (UID: \"fcb6432a-cde0-4f40-9e00-2523c43ca2c8\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-2fks5"
Jul 29 17:52:41.571652 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.571632    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a91229a9-7326-45f0-9bed-57c0f1868780-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-hhwbp\" (UID: \"a91229a9-7326-45f0-9bed-57c0f1868780\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-hhwbp"
Jul 29 17:52:41.574427 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.574403    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a91229a9-7326-45f0-9bed-57c0f1868780-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-hhwbp\" (UID: \"a91229a9-7326-45f0-9bed-57c0f1868780\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-hhwbp"
Jul 29 17:52:41.574427 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.574416    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fcb6432a-cde0-4f40-9e00-2523c43ca2c8-serving-cert\") pod \"service-ca-operator-5d6c75f84f-2fks5\" (UID: \"fcb6432a-cde0-4f40-9e00-2523c43ca2c8\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-2fks5"
Jul 29 17:52:41.580375 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.580338    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-269fs\" (UniqueName: \"kubernetes.io/projected/fcb6432a-cde0-4f40-9e00-2523c43ca2c8-kube-api-access-269fs\") pod \"service-ca-operator-5d6c75f84f-2fks5\" (UID: \"fcb6432a-cde0-4f40-9e00-2523c43ca2c8\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-2fks5"
Jul 29 17:52:41.580375 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.580359    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-844jv\" (UniqueName: \"kubernetes.io/projected/a91229a9-7326-45f0-9bed-57c0f1868780-kube-api-access-844jv\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-hhwbp\" (UID: \"a91229a9-7326-45f0-9bed-57c0f1868780\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-hhwbp"
Jul 29 17:52:41.580696 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.580679    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbthb\" (UniqueName: \"kubernetes.io/projected/a1be716c-91ad-4ab5-a476-32486b3c0506-kube-api-access-dbthb\") pod \"cluster-samples-operator-6748489656-5v6zw\" (UID: \"a1be716c-91ad-4ab5-a476-32486b3c0506\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-5v6zw"
Jul 29 17:52:41.599543 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.599516    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-2fks5"
Jul 29 17:52:41.669247 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.669215    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-hhwbp"
Jul 29 17:52:41.671650 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.671618    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2bf0bb71-4047-45a7-bd8e-5d6efabd26bd-config\") pod \"console-operator-584dcb6b54-gg2wj\" (UID: \"2bf0bb71-4047-45a7-bd8e-5d6efabd26bd\") " pod="openshift-console-operator/console-operator-584dcb6b54-gg2wj"
Jul 29 17:52:41.672040 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.671665    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3aad4b60-ee91-42d3-bbf1-865219602575-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-grlrl\" (UID: \"3aad4b60-ee91-42d3-bbf1-865219602575\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-grlrl"
Jul 29 17:52:41.672040 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.671747    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a9042be4-9aa4-4bc5-b9b8-364a7f782353-tmp\") pod \"insights-operator-5885fdb759-4j8zv\" (UID: \"a9042be4-9aa4-4bc5-b9b8-364a7f782353\") " pod="openshift-insights/insights-operator-5885fdb759-4j8zv"
Jul 29 17:52:41.672040 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.671783    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4kl4\" (UniqueName: \"kubernetes.io/projected/6554ff13-9d69-4062-be7c-a8d3ff442945-kube-api-access-r4kl4\") pod \"volume-data-source-validator-549c6d54cf-m2x44\" (UID: \"6554ff13-9d69-4062-be7c-a8d3ff442945\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-m2x44"
Jul 29 17:52:41.672040 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.671831    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2bf0bb71-4047-45a7-bd8e-5d6efabd26bd-trusted-ca\") pod \"console-operator-584dcb6b54-gg2wj\" (UID: \"2bf0bb71-4047-45a7-bd8e-5d6efabd26bd\") " pod="openshift-console-operator/console-operator-584dcb6b54-gg2wj"
Jul 29 17:52:41.672040 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.671854    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0da2e7ec-9cb4-42dc-a537-a5b46687ee32-metrics-certs\") pod \"router-default-676c59fd4f-tvcgb\" (UID: \"0da2e7ec-9cb4-42dc-a537-a5b46687ee32\") " pod="openshift-ingress/router-default-676c59fd4f-tvcgb"
Jul 29 17:52:41.672040 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.671895    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/427a5e33-ee40-4ef0-a223-32a6ba9e52cb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-rgj9m\" (UID: \"427a5e33-ee40-4ef0-a223-32a6ba9e52cb\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rgj9m"
Jul 29 17:52:41.672040 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.671920    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8pg8\" (UniqueName: \"kubernetes.io/projected/7843c951-e574-4cca-bbc8-ddc52d5d2b55-kube-api-access-z8pg8\") pod \"network-check-source-7d88cb55d8-4s8jb\" (UID: \"7843c951-e574-4cca-bbc8-ddc52d5d2b55\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-4s8jb"
Jul 29 17:52:41.672040 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.671941    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/a9042be4-9aa4-4bc5-b9b8-364a7f782353-snapshots\") pod \"insights-operator-5885fdb759-4j8zv\" (UID: \"a9042be4-9aa4-4bc5-b9b8-364a7f782353\") " pod="openshift-insights/insights-operator-5885fdb759-4j8zv"
Jul 29 17:52:41.672040 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.671973    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5kqs\" (UniqueName: \"kubernetes.io/projected/a9042be4-9aa4-4bc5-b9b8-364a7f782353-kube-api-access-f5kqs\") pod \"insights-operator-5885fdb759-4j8zv\" (UID: \"a9042be4-9aa4-4bc5-b9b8-364a7f782353\") " pod="openshift-insights/insights-operator-5885fdb759-4j8zv"
Jul 29 17:52:41.672040 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.671998    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a9042be4-9aa4-4bc5-b9b8-364a7f782353-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-4j8zv\" (UID: \"a9042be4-9aa4-4bc5-b9b8-364a7f782353\") " pod="openshift-insights/insights-operator-5885fdb759-4j8zv"
Jul 29 17:52:41.672040 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.672046    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a9042be4-9aa4-4bc5-b9b8-364a7f782353-service-ca-bundle\") pod \"insights-operator-5885fdb759-4j8zv\" (UID: \"a9042be4-9aa4-4bc5-b9b8-364a7f782353\") " pod="openshift-insights/insights-operator-5885fdb759-4j8zv"
Jul 29 17:52:41.672754 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.672070    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/0da2e7ec-9cb4-42dc-a537-a5b46687ee32-default-certificate\") pod \"router-default-676c59fd4f-tvcgb\" (UID: \"0da2e7ec-9cb4-42dc-a537-a5b46687ee32\") " pod="openshift-ingress/router-default-676c59fd4f-tvcgb"
Jul 29 17:52:41.672754 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.672097    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8shk\" (UniqueName: \"kubernetes.io/projected/0da2e7ec-9cb4-42dc-a537-a5b46687ee32-kube-api-access-n8shk\") pod \"router-default-676c59fd4f-tvcgb\" (UID: \"0da2e7ec-9cb4-42dc-a537-a5b46687ee32\") " pod="openshift-ingress/router-default-676c59fd4f-tvcgb"
Jul 29 17:52:41.672754 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.672127    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a9042be4-9aa4-4bc5-b9b8-364a7f782353-serving-cert\") pod \"insights-operator-5885fdb759-4j8zv\" (UID: \"a9042be4-9aa4-4bc5-b9b8-364a7f782353\") " pod="openshift-insights/insights-operator-5885fdb759-4j8zv"
Jul 29 17:52:41.672754 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.672162    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2bf0bb71-4047-45a7-bd8e-5d6efabd26bd-serving-cert\") pod \"console-operator-584dcb6b54-gg2wj\" (UID: \"2bf0bb71-4047-45a7-bd8e-5d6efabd26bd\") " pod="openshift-console-operator/console-operator-584dcb6b54-gg2wj"
Jul 29 17:52:41.672754 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.672194    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3aad4b60-ee91-42d3-bbf1-865219602575-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-grlrl\" (UID: \"3aad4b60-ee91-42d3-bbf1-865219602575\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-grlrl"
Jul 29 17:52:41.672754 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.672228    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2w99\" (UniqueName: \"kubernetes.io/projected/427a5e33-ee40-4ef0-a223-32a6ba9e52cb-kube-api-access-w2w99\") pod \"cluster-monitoring-operator-548b489cb8-rgj9m\" (UID: \"427a5e33-ee40-4ef0-a223-32a6ba9e52cb\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rgj9m"
Jul 29 17:52:41.672754 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.672274    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0da2e7ec-9cb4-42dc-a537-a5b46687ee32-service-ca-bundle\") pod \"router-default-676c59fd4f-tvcgb\" (UID: \"0da2e7ec-9cb4-42dc-a537-a5b46687ee32\") " pod="openshift-ingress/router-default-676c59fd4f-tvcgb"
Jul 29 17:52:41.672754 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.672301    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/0da2e7ec-9cb4-42dc-a537-a5b46687ee32-stats-auth\") pod \"router-default-676c59fd4f-tvcgb\" (UID: \"0da2e7ec-9cb4-42dc-a537-a5b46687ee32\") " pod="openshift-ingress/router-default-676c59fd4f-tvcgb"
Jul 29 17:52:41.672754 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.672329    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/427a5e33-ee40-4ef0-a223-32a6ba9e52cb-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-rgj9m\" (UID: \"427a5e33-ee40-4ef0-a223-32a6ba9e52cb\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rgj9m"
Jul 29 17:52:41.672754 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.672362    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdxqs\" (UniqueName: \"kubernetes.io/projected/2bf0bb71-4047-45a7-bd8e-5d6efabd26bd-kube-api-access-wdxqs\") pod \"console-operator-584dcb6b54-gg2wj\" (UID: \"2bf0bb71-4047-45a7-bd8e-5d6efabd26bd\") " pod="openshift-console-operator/console-operator-584dcb6b54-gg2wj"
Jul 29 17:52:41.672754 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.672368    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2bf0bb71-4047-45a7-bd8e-5d6efabd26bd-config\") pod \"console-operator-584dcb6b54-gg2wj\" (UID: \"2bf0bb71-4047-45a7-bd8e-5d6efabd26bd\") " pod="openshift-console-operator/console-operator-584dcb6b54-gg2wj"
Jul 29 17:52:41.673471 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.673021    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a9042be4-9aa4-4bc5-b9b8-364a7f782353-service-ca-bundle\") pod \"insights-operator-5885fdb759-4j8zv\" (UID: \"a9042be4-9aa4-4bc5-b9b8-364a7f782353\") " pod="openshift-insights/insights-operator-5885fdb759-4j8zv"
Jul 29 17:52:41.673770 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.673602    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a9042be4-9aa4-4bc5-b9b8-364a7f782353-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-4j8zv\" (UID: \"a9042be4-9aa4-4bc5-b9b8-364a7f782353\") " pod="openshift-insights/insights-operator-5885fdb759-4j8zv"
Jul 29 17:52:41.673770 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:41.673747    2557 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 29 17:52:41.673953 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:41.673820    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3aad4b60-ee91-42d3-bbf1-865219602575-networking-console-plugin-cert podName:3aad4b60-ee91-42d3-bbf1-865219602575 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:42.173800743 +0000 UTC m=+42.043023531 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3aad4b60-ee91-42d3-bbf1-865219602575-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-grlrl" (UID: "3aad4b60-ee91-42d3-bbf1-865219602575") : secret "networking-console-plugin-cert" not found
Jul 29 17:52:41.674615 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.674592    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3aad4b60-ee91-42d3-bbf1-865219602575-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-grlrl\" (UID: \"3aad4b60-ee91-42d3-bbf1-865219602575\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-grlrl"
Jul 29 17:52:41.674866 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.674843    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a9042be4-9aa4-4bc5-b9b8-364a7f782353-tmp\") pod \"insights-operator-5885fdb759-4j8zv\" (UID: \"a9042be4-9aa4-4bc5-b9b8-364a7f782353\") " pod="openshift-insights/insights-operator-5885fdb759-4j8zv"
Jul 29 17:52:41.675267 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.675239    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2bf0bb71-4047-45a7-bd8e-5d6efabd26bd-trusted-ca\") pod \"console-operator-584dcb6b54-gg2wj\" (UID: \"2bf0bb71-4047-45a7-bd8e-5d6efabd26bd\") " pod="openshift-console-operator/console-operator-584dcb6b54-gg2wj"
Jul 29 17:52:41.675359 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.675274    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/a9042be4-9aa4-4bc5-b9b8-364a7f782353-snapshots\") pod \"insights-operator-5885fdb759-4j8zv\" (UID: \"a9042be4-9aa4-4bc5-b9b8-364a7f782353\") " pod="openshift-insights/insights-operator-5885fdb759-4j8zv"
Jul 29 17:52:41.676169 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.676145    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a9042be4-9aa4-4bc5-b9b8-364a7f782353-serving-cert\") pod \"insights-operator-5885fdb759-4j8zv\" (UID: \"a9042be4-9aa4-4bc5-b9b8-364a7f782353\") " pod="openshift-insights/insights-operator-5885fdb759-4j8zv"
Jul 29 17:52:41.678060 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.678036    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2bf0bb71-4047-45a7-bd8e-5d6efabd26bd-serving-cert\") pod \"console-operator-584dcb6b54-gg2wj\" (UID: \"2bf0bb71-4047-45a7-bd8e-5d6efabd26bd\") " pod="openshift-console-operator/console-operator-584dcb6b54-gg2wj"
Jul 29 17:52:41.684668 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.684637    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8pg8\" (UniqueName: \"kubernetes.io/projected/7843c951-e574-4cca-bbc8-ddc52d5d2b55-kube-api-access-z8pg8\") pod \"network-check-source-7d88cb55d8-4s8jb\" (UID: \"7843c951-e574-4cca-bbc8-ddc52d5d2b55\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-4s8jb"
Jul 29 17:52:41.684668 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.684658    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5kqs\" (UniqueName: \"kubernetes.io/projected/a9042be4-9aa4-4bc5-b9b8-364a7f782353-kube-api-access-f5kqs\") pod \"insights-operator-5885fdb759-4j8zv\" (UID: \"a9042be4-9aa4-4bc5-b9b8-364a7f782353\") " pod="openshift-insights/insights-operator-5885fdb759-4j8zv"
Jul 29 17:52:41.684845 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.684826    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdxqs\" (UniqueName: \"kubernetes.io/projected/2bf0bb71-4047-45a7-bd8e-5d6efabd26bd-kube-api-access-wdxqs\") pod \"console-operator-584dcb6b54-gg2wj\" (UID: \"2bf0bb71-4047-45a7-bd8e-5d6efabd26bd\") " pod="openshift-console-operator/console-operator-584dcb6b54-gg2wj"
Jul 29 17:52:41.685411 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.685370    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4kl4\" (UniqueName: \"kubernetes.io/projected/6554ff13-9d69-4062-be7c-a8d3ff442945-kube-api-access-r4kl4\") pod \"volume-data-source-validator-549c6d54cf-m2x44\" (UID: \"6554ff13-9d69-4062-be7c-a8d3ff442945\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-m2x44"
Jul 29 17:52:41.689120 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.689103    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-4s8jb"
Jul 29 17:52:41.716088 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.716063    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-4j8zv"
Jul 29 17:52:41.729294 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.729253    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-m2x44"
Jul 29 17:52:41.747900 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.747873    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-2fks5"]
Jul 29 17:52:41.749635 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.749618    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-gg2wj"
Jul 29 17:52:41.751345 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:41.751327    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcb6432a_cde0_4f40_9e00_2523c43ca2c8.slice/crio-467205a6ccc7704c2327b7105a7009f3bdad2a873c98d946c2e648a0e9d7bfb0 WatchSource:0}: Error finding container 467205a6ccc7704c2327b7105a7009f3bdad2a873c98d946c2e648a0e9d7bfb0: Status 404 returned error can't find the container with id 467205a6ccc7704c2327b7105a7009f3bdad2a873c98d946c2e648a0e9d7bfb0
Jul 29 17:52:41.775986 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.774684    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0da2e7ec-9cb4-42dc-a537-a5b46687ee32-metrics-certs\") pod \"router-default-676c59fd4f-tvcgb\" (UID: \"0da2e7ec-9cb4-42dc-a537-a5b46687ee32\") " pod="openshift-ingress/router-default-676c59fd4f-tvcgb"
Jul 29 17:52:41.775986 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.774773    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/427a5e33-ee40-4ef0-a223-32a6ba9e52cb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-rgj9m\" (UID: \"427a5e33-ee40-4ef0-a223-32a6ba9e52cb\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rgj9m"
Jul 29 17:52:41.775986 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.774818    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/0da2e7ec-9cb4-42dc-a537-a5b46687ee32-default-certificate\") pod \"router-default-676c59fd4f-tvcgb\" (UID: \"0da2e7ec-9cb4-42dc-a537-a5b46687ee32\") " pod="openshift-ingress/router-default-676c59fd4f-tvcgb"
Jul 29 17:52:41.775986 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.774840    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8shk\" (UniqueName: \"kubernetes.io/projected/0da2e7ec-9cb4-42dc-a537-a5b46687ee32-kube-api-access-n8shk\") pod \"router-default-676c59fd4f-tvcgb\" (UID: \"0da2e7ec-9cb4-42dc-a537-a5b46687ee32\") " pod="openshift-ingress/router-default-676c59fd4f-tvcgb"
Jul 29 17:52:41.775986 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.774893    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2w99\" (UniqueName: \"kubernetes.io/projected/427a5e33-ee40-4ef0-a223-32a6ba9e52cb-kube-api-access-w2w99\") pod \"cluster-monitoring-operator-548b489cb8-rgj9m\" (UID: \"427a5e33-ee40-4ef0-a223-32a6ba9e52cb\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rgj9m"
Jul 29 17:52:41.775986 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.774937    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0da2e7ec-9cb4-42dc-a537-a5b46687ee32-service-ca-bundle\") pod \"router-default-676c59fd4f-tvcgb\" (UID: \"0da2e7ec-9cb4-42dc-a537-a5b46687ee32\") " pod="openshift-ingress/router-default-676c59fd4f-tvcgb"
Jul 29 17:52:41.775986 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.774960    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/0da2e7ec-9cb4-42dc-a537-a5b46687ee32-stats-auth\") pod \"router-default-676c59fd4f-tvcgb\" (UID: \"0da2e7ec-9cb4-42dc-a537-a5b46687ee32\") " pod="openshift-ingress/router-default-676c59fd4f-tvcgb"
Jul 29 17:52:41.775986 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.774984    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/427a5e33-ee40-4ef0-a223-32a6ba9e52cb-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-rgj9m\" (UID: \"427a5e33-ee40-4ef0-a223-32a6ba9e52cb\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rgj9m"
Jul 29 17:52:41.775986 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.775689    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/427a5e33-ee40-4ef0-a223-32a6ba9e52cb-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-rgj9m\" (UID: \"427a5e33-ee40-4ef0-a223-32a6ba9e52cb\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rgj9m"
Jul 29 17:52:41.778244 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:41.777129    2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 29 17:52:41.778244 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:41.777187    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0da2e7ec-9cb4-42dc-a537-a5b46687ee32-metrics-certs podName:0da2e7ec-9cb4-42dc-a537-a5b46687ee32 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:42.277168749 +0000 UTC m=+42.146391537 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0da2e7ec-9cb4-42dc-a537-a5b46687ee32-metrics-certs") pod "router-default-676c59fd4f-tvcgb" (UID: "0da2e7ec-9cb4-42dc-a537-a5b46687ee32") : secret "router-metrics-certs-default" not found
Jul 29 17:52:41.778244 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:41.777374    2557 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 29 17:52:41.778244 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:41.777412    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/427a5e33-ee40-4ef0-a223-32a6ba9e52cb-cluster-monitoring-operator-tls podName:427a5e33-ee40-4ef0-a223-32a6ba9e52cb nodeName:}" failed. No retries permitted until 2026-07-29 17:52:42.277400313 +0000 UTC m=+42.146623104 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/427a5e33-ee40-4ef0-a223-32a6ba9e52cb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-rgj9m" (UID: "427a5e33-ee40-4ef0-a223-32a6ba9e52cb") : secret "cluster-monitoring-operator-tls" not found
Jul 29 17:52:41.778244 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:41.777980    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0da2e7ec-9cb4-42dc-a537-a5b46687ee32-service-ca-bundle podName:0da2e7ec-9cb4-42dc-a537-a5b46687ee32 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:42.277965741 +0000 UTC m=+42.147188543 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0da2e7ec-9cb4-42dc-a537-a5b46687ee32-service-ca-bundle") pod "router-default-676c59fd4f-tvcgb" (UID: "0da2e7ec-9cb4-42dc-a537-a5b46687ee32") : configmap references non-existent config key: service-ca.crt
Jul 29 17:52:41.782647 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.782170    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/0da2e7ec-9cb4-42dc-a537-a5b46687ee32-default-certificate\") pod \"router-default-676c59fd4f-tvcgb\" (UID: \"0da2e7ec-9cb4-42dc-a537-a5b46687ee32\") " pod="openshift-ingress/router-default-676c59fd4f-tvcgb"
Jul 29 17:52:41.782647 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.782238    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/0da2e7ec-9cb4-42dc-a537-a5b46687ee32-stats-auth\") pod \"router-default-676c59fd4f-tvcgb\" (UID: \"0da2e7ec-9cb4-42dc-a537-a5b46687ee32\") " pod="openshift-ingress/router-default-676c59fd4f-tvcgb"
Jul 29 17:52:41.790715 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.790671    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2w99\" (UniqueName: \"kubernetes.io/projected/427a5e33-ee40-4ef0-a223-32a6ba9e52cb-kube-api-access-w2w99\") pod \"cluster-monitoring-operator-548b489cb8-rgj9m\" (UID: \"427a5e33-ee40-4ef0-a223-32a6ba9e52cb\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rgj9m"
Jul 29 17:52:41.794220 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.793926    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8shk\" (UniqueName: \"kubernetes.io/projected/0da2e7ec-9cb4-42dc-a537-a5b46687ee32-kube-api-access-n8shk\") pod \"router-default-676c59fd4f-tvcgb\" (UID: \"0da2e7ec-9cb4-42dc-a537-a5b46687ee32\") " pod="openshift-ingress/router-default-676c59fd4f-tvcgb"
Jul 29 17:52:41.871409 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.871346    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-2fks5" event={"ID":"fcb6432a-cde0-4f40-9e00-2523c43ca2c8","Type":"ContainerStarted","Data":"467205a6ccc7704c2327b7105a7009f3bdad2a873c98d946c2e648a0e9d7bfb0"}
Jul 29 17:52:41.874251 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.874139    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-hhwbp"]
Jul 29 17:52:41.881839 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:41.881810    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda91229a9_7326_45f0_9bed_57c0f1868780.slice/crio-51f14720fe079867a67730f86ca8c771ecf080407009953dc41dff7ca2010116 WatchSource:0}: Error finding container 51f14720fe079867a67730f86ca8c771ecf080407009953dc41dff7ca2010116: Status 404 returned error can't find the container with id 51f14720fe079867a67730f86ca8c771ecf080407009953dc41dff7ca2010116
Jul 29 17:52:41.895214 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.895030    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-4s8jb"]
Jul 29 17:52:41.898850 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:41.898667    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7843c951_e574_4cca_bbc8_ddc52d5d2b55.slice/crio-e35f29ba6a8a1444f4e74adb459c9a4b63b8be2947d2c9ef8230cc0d4922333e WatchSource:0}: Error finding container e35f29ba6a8a1444f4e74adb459c9a4b63b8be2947d2c9ef8230cc0d4922333e: Status 404 returned error can't find the container with id e35f29ba6a8a1444f4e74adb459c9a4b63b8be2947d2c9ef8230cc0d4922333e
Jul 29 17:52:41.958000 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.957972    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-4j8zv"]
Jul 29 17:52:41.961869 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:41.961840    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9042be4_9aa4_4bc5_b9b8_364a7f782353.slice/crio-58ab786d6041d6db7733c466382f43d6d9e9aaf33700f29c67aecf986b05b291 WatchSource:0}: Error finding container 58ab786d6041d6db7733c466382f43d6d9e9aaf33700f29c67aecf986b05b291: Status 404 returned error can't find the container with id 58ab786d6041d6db7733c466382f43d6d9e9aaf33700f29c67aecf986b05b291
Jul 29 17:52:41.971479 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:41.971391    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-gg2wj"]
Jul 29 17:52:41.974490 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:41.974468    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bf0bb71_4047_45a7_bd8e_5d6efabd26bd.slice/crio-e144ad934c2c4a776b48632bc2368229ded7ee2352abb34a224db3696739cc4b WatchSource:0}: Error finding container e144ad934c2c4a776b48632bc2368229ded7ee2352abb34a224db3696739cc4b: Status 404 returned error can't find the container with id e144ad934c2c4a776b48632bc2368229ded7ee2352abb34a224db3696739cc4b
Jul 29 17:52:42.012142 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:42.012088    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-m2x44"]
Jul 29 17:52:42.014697 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:52:42.014676    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6554ff13_9d69_4062_be7c_a8d3ff442945.slice/crio-164a4840c12376d7f16b7e3e531beaecacd8e97b4a0b7590f3532e0bb20fd066 WatchSource:0}: Error finding container 164a4840c12376d7f16b7e3e531beaecacd8e97b4a0b7590f3532e0bb20fd066: Status 404 returned error can't find the container with id 164a4840c12376d7f16b7e3e531beaecacd8e97b4a0b7590f3532e0bb20fd066
Jul 29 17:52:42.077397 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:42.077362    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a1be716c-91ad-4ab5-a476-32486b3c0506-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-5v6zw\" (UID: \"a1be716c-91ad-4ab5-a476-32486b3c0506\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-5v6zw"
Jul 29 17:52:42.077546 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:42.077485    2557 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 29 17:52:42.077613 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:42.077554    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1be716c-91ad-4ab5-a476-32486b3c0506-samples-operator-tls podName:a1be716c-91ad-4ab5-a476-32486b3c0506 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:43.077534364 +0000 UTC m=+42.946757163 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a1be716c-91ad-4ab5-a476-32486b3c0506-samples-operator-tls") pod "cluster-samples-operator-6748489656-5v6zw" (UID: "a1be716c-91ad-4ab5-a476-32486b3c0506") : secret "samples-operator-tls" not found
Jul 29 17:52:42.177960 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:42.177923    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3aad4b60-ee91-42d3-bbf1-865219602575-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-grlrl\" (UID: \"3aad4b60-ee91-42d3-bbf1-865219602575\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-grlrl"
Jul 29 17:52:42.178105 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:42.178068    2557 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 29 17:52:42.178165 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:42.178128    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3aad4b60-ee91-42d3-bbf1-865219602575-networking-console-plugin-cert podName:3aad4b60-ee91-42d3-bbf1-865219602575 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:43.178112637 +0000 UTC m=+43.047335426 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3aad4b60-ee91-42d3-bbf1-865219602575-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-grlrl" (UID: "3aad4b60-ee91-42d3-bbf1-865219602575") : secret "networking-console-plugin-cert" not found
Jul 29 17:52:42.278988 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:42.278913    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0da2e7ec-9cb4-42dc-a537-a5b46687ee32-service-ca-bundle\") pod \"router-default-676c59fd4f-tvcgb\" (UID: \"0da2e7ec-9cb4-42dc-a537-a5b46687ee32\") " pod="openshift-ingress/router-default-676c59fd4f-tvcgb"
Jul 29 17:52:42.279408 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:42.278993    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0da2e7ec-9cb4-42dc-a537-a5b46687ee32-metrics-certs\") pod \"router-default-676c59fd4f-tvcgb\" (UID: \"0da2e7ec-9cb4-42dc-a537-a5b46687ee32\") " pod="openshift-ingress/router-default-676c59fd4f-tvcgb"
Jul 29 17:52:42.279408 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:42.279027    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/427a5e33-ee40-4ef0-a223-32a6ba9e52cb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-rgj9m\" (UID: \"427a5e33-ee40-4ef0-a223-32a6ba9e52cb\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rgj9m"
Jul 29 17:52:42.279408 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:42.279090    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0da2e7ec-9cb4-42dc-a537-a5b46687ee32-service-ca-bundle podName:0da2e7ec-9cb4-42dc-a537-a5b46687ee32 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:43.279067614 +0000 UTC m=+43.148290399 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0da2e7ec-9cb4-42dc-a537-a5b46687ee32-service-ca-bundle") pod "router-default-676c59fd4f-tvcgb" (UID: "0da2e7ec-9cb4-42dc-a537-a5b46687ee32") : configmap references non-existent config key: service-ca.crt
Jul 29 17:52:42.279408 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:42.279129    2557 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 29 17:52:42.279408 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:42.279126    2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 29 17:52:42.279408 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:42.279175    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0da2e7ec-9cb4-42dc-a537-a5b46687ee32-metrics-certs podName:0da2e7ec-9cb4-42dc-a537-a5b46687ee32 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:43.279163021 +0000 UTC m=+43.148385809 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0da2e7ec-9cb4-42dc-a537-a5b46687ee32-metrics-certs") pod "router-default-676c59fd4f-tvcgb" (UID: "0da2e7ec-9cb4-42dc-a537-a5b46687ee32") : secret "router-metrics-certs-default" not found
Jul 29 17:52:42.279408 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:42.279188    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/427a5e33-ee40-4ef0-a223-32a6ba9e52cb-cluster-monitoring-operator-tls podName:427a5e33-ee40-4ef0-a223-32a6ba9e52cb nodeName:}" failed. No retries permitted until 2026-07-29 17:52:43.279182074 +0000 UTC m=+43.148404859 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/427a5e33-ee40-4ef0-a223-32a6ba9e52cb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-rgj9m" (UID: "427a5e33-ee40-4ef0-a223-32a6ba9e52cb") : secret "cluster-monitoring-operator-tls" not found
Jul 29 17:52:42.876575 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:42.876537    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-4s8jb" event={"ID":"7843c951-e574-4cca-bbc8-ddc52d5d2b55","Type":"ContainerStarted","Data":"e35f29ba6a8a1444f4e74adb459c9a4b63b8be2947d2c9ef8230cc0d4922333e"}
Jul 29 17:52:42.878472 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:42.878441    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-hhwbp" event={"ID":"a91229a9-7326-45f0-9bed-57c0f1868780","Type":"ContainerStarted","Data":"51f14720fe079867a67730f86ca8c771ecf080407009953dc41dff7ca2010116"}
Jul 29 17:52:42.881579 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:42.881537    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-4j8zv" event={"ID":"a9042be4-9aa4-4bc5-b9b8-364a7f782353","Type":"ContainerStarted","Data":"58ab786d6041d6db7733c466382f43d6d9e9aaf33700f29c67aecf986b05b291"}
Jul 29 17:52:42.883360 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:42.883333    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-m2x44" event={"ID":"6554ff13-9d69-4062-be7c-a8d3ff442945","Type":"ContainerStarted","Data":"164a4840c12376d7f16b7e3e531beaecacd8e97b4a0b7590f3532e0bb20fd066"}
Jul 29 17:52:42.884901 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:42.884857    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-gg2wj" event={"ID":"2bf0bb71-4047-45a7-bd8e-5d6efabd26bd","Type":"ContainerStarted","Data":"e144ad934c2c4a776b48632bc2368229ded7ee2352abb34a224db3696739cc4b"}
Jul 29 17:52:43.087187 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:43.086478    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a1be716c-91ad-4ab5-a476-32486b3c0506-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-5v6zw\" (UID: \"a1be716c-91ad-4ab5-a476-32486b3c0506\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-5v6zw"
Jul 29 17:52:43.087187 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:43.086789    2557 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 29 17:52:43.087187 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:43.086855    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1be716c-91ad-4ab5-a476-32486b3c0506-samples-operator-tls podName:a1be716c-91ad-4ab5-a476-32486b3c0506 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:45.086837582 +0000 UTC m=+44.956060372 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a1be716c-91ad-4ab5-a476-32486b3c0506-samples-operator-tls") pod "cluster-samples-operator-6748489656-5v6zw" (UID: "a1be716c-91ad-4ab5-a476-32486b3c0506") : secret "samples-operator-tls" not found
Jul 29 17:52:43.187178 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:43.187102    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3aad4b60-ee91-42d3-bbf1-865219602575-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-grlrl\" (UID: \"3aad4b60-ee91-42d3-bbf1-865219602575\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-grlrl"
Jul 29 17:52:43.187326 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:43.187251    2557 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 29 17:52:43.187326 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:43.187312    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3aad4b60-ee91-42d3-bbf1-865219602575-networking-console-plugin-cert podName:3aad4b60-ee91-42d3-bbf1-865219602575 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:45.187296798 +0000 UTC m=+45.056519587 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3aad4b60-ee91-42d3-bbf1-865219602575-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-grlrl" (UID: "3aad4b60-ee91-42d3-bbf1-865219602575") : secret "networking-console-plugin-cert" not found
Jul 29 17:52:43.289545 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:43.288561    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0da2e7ec-9cb4-42dc-a537-a5b46687ee32-service-ca-bundle\") pod \"router-default-676c59fd4f-tvcgb\" (UID: \"0da2e7ec-9cb4-42dc-a537-a5b46687ee32\") " pod="openshift-ingress/router-default-676c59fd4f-tvcgb"
Jul 29 17:52:43.289545 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:43.288658    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/09c3a81a-22ca-4700-af82-9c7e86f77f07-registry-tls\") pod \"image-registry-6774b48fd9-hrjxr\" (UID: \"09c3a81a-22ca-4700-af82-9c7e86f77f07\") " pod="openshift-image-registry/image-registry-6774b48fd9-hrjxr"
Jul 29 17:52:43.289545 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:43.288694    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0da2e7ec-9cb4-42dc-a537-a5b46687ee32-metrics-certs\") pod \"router-default-676c59fd4f-tvcgb\" (UID: \"0da2e7ec-9cb4-42dc-a537-a5b46687ee32\") " pod="openshift-ingress/router-default-676c59fd4f-tvcgb"
Jul 29 17:52:43.289545 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:43.288763    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/427a5e33-ee40-4ef0-a223-32a6ba9e52cb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-rgj9m\" (UID: \"427a5e33-ee40-4ef0-a223-32a6ba9e52cb\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rgj9m"
Jul 29 17:52:43.289545 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:43.288901    2557 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 29 17:52:43.289545 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:43.288960    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/427a5e33-ee40-4ef0-a223-32a6ba9e52cb-cluster-monitoring-operator-tls podName:427a5e33-ee40-4ef0-a223-32a6ba9e52cb nodeName:}" failed. No retries permitted until 2026-07-29 17:52:45.288942749 +0000 UTC m=+45.158165548 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/427a5e33-ee40-4ef0-a223-32a6ba9e52cb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-rgj9m" (UID: "427a5e33-ee40-4ef0-a223-32a6ba9e52cb") : secret "cluster-monitoring-operator-tls" not found
Jul 29 17:52:43.289545 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:43.289031    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0da2e7ec-9cb4-42dc-a537-a5b46687ee32-service-ca-bundle podName:0da2e7ec-9cb4-42dc-a537-a5b46687ee32 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:45.289020401 +0000 UTC m=+45.158243190 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0da2e7ec-9cb4-42dc-a537-a5b46687ee32-service-ca-bundle") pod "router-default-676c59fd4f-tvcgb" (UID: "0da2e7ec-9cb4-42dc-a537-a5b46687ee32") : configmap references non-existent config key: service-ca.crt
Jul 29 17:52:43.289545 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:43.289098    2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 17:52:43.289545 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:43.289110    2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6774b48fd9-hrjxr: secret "image-registry-tls" not found
Jul 29 17:52:43.289545 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:43.289139    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/09c3a81a-22ca-4700-af82-9c7e86f77f07-registry-tls podName:09c3a81a-22ca-4700-af82-9c7e86f77f07 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:51.289129192 +0000 UTC m=+51.158351980 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/09c3a81a-22ca-4700-af82-9c7e86f77f07-registry-tls") pod "image-registry-6774b48fd9-hrjxr" (UID: "09c3a81a-22ca-4700-af82-9c7e86f77f07") : secret "image-registry-tls" not found
Jul 29 17:52:43.289545 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:43.289190    2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 29 17:52:43.289545 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:43.289218    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0da2e7ec-9cb4-42dc-a537-a5b46687ee32-metrics-certs podName:0da2e7ec-9cb4-42dc-a537-a5b46687ee32 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:45.289208055 +0000 UTC m=+45.158430842 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0da2e7ec-9cb4-42dc-a537-a5b46687ee32-metrics-certs") pod "router-default-676c59fd4f-tvcgb" (UID: "0da2e7ec-9cb4-42dc-a537-a5b46687ee32") : secret "router-metrics-certs-default" not found
Jul 29 17:52:43.389961 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:43.389332    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dedf5ed1-ad33-4658-b79c-ae80134b18c7-metrics-tls\") pod \"dns-default-bwjgj\" (UID: \"dedf5ed1-ad33-4658-b79c-ae80134b18c7\") " pod="openshift-dns/dns-default-bwjgj"
Jul 29 17:52:43.389961 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:43.389548    2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 17:52:43.389961 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:43.389610    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dedf5ed1-ad33-4658-b79c-ae80134b18c7-metrics-tls podName:dedf5ed1-ad33-4658-b79c-ae80134b18c7 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:51.389590774 +0000 UTC m=+51.258813562 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dedf5ed1-ad33-4658-b79c-ae80134b18c7-metrics-tls") pod "dns-default-bwjgj" (UID: "dedf5ed1-ad33-4658-b79c-ae80134b18c7") : secret "dns-default-metrics-tls" not found
Jul 29 17:52:45.104938 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:45.104893    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a1be716c-91ad-4ab5-a476-32486b3c0506-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-5v6zw\" (UID: \"a1be716c-91ad-4ab5-a476-32486b3c0506\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-5v6zw"
Jul 29 17:52:45.105376 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:45.105066    2557 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 29 17:52:45.105376 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:45.105154    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1be716c-91ad-4ab5-a476-32486b3c0506-samples-operator-tls podName:a1be716c-91ad-4ab5-a476-32486b3c0506 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:49.105132195 +0000 UTC m=+48.974354981 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a1be716c-91ad-4ab5-a476-32486b3c0506-samples-operator-tls") pod "cluster-samples-operator-6748489656-5v6zw" (UID: "a1be716c-91ad-4ab5-a476-32486b3c0506") : secret "samples-operator-tls" not found
Jul 29 17:52:45.205609 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:45.205574    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3aad4b60-ee91-42d3-bbf1-865219602575-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-grlrl\" (UID: \"3aad4b60-ee91-42d3-bbf1-865219602575\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-grlrl"
Jul 29 17:52:45.205798 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:45.205772    2557 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 29 17:52:45.205871 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:45.205858    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3aad4b60-ee91-42d3-bbf1-865219602575-networking-console-plugin-cert podName:3aad4b60-ee91-42d3-bbf1-865219602575 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:49.205838238 +0000 UTC m=+49.075061026 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3aad4b60-ee91-42d3-bbf1-865219602575-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-grlrl" (UID: "3aad4b60-ee91-42d3-bbf1-865219602575") : secret "networking-console-plugin-cert" not found
Jul 29 17:52:45.306668 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:45.306622    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0da2e7ec-9cb4-42dc-a537-a5b46687ee32-metrics-certs\") pod \"router-default-676c59fd4f-tvcgb\" (UID: \"0da2e7ec-9cb4-42dc-a537-a5b46687ee32\") " pod="openshift-ingress/router-default-676c59fd4f-tvcgb"
Jul 29 17:52:45.306862 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:45.306684    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/427a5e33-ee40-4ef0-a223-32a6ba9e52cb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-rgj9m\" (UID: \"427a5e33-ee40-4ef0-a223-32a6ba9e52cb\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rgj9m"
Jul 29 17:52:45.306862 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:45.306785    2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 29 17:52:45.306862 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:45.306823    2557 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 29 17:52:45.306862 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:45.306852    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0da2e7ec-9cb4-42dc-a537-a5b46687ee32-metrics-certs podName:0da2e7ec-9cb4-42dc-a537-a5b46687ee32 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:49.30683327 +0000 UTC m=+49.176056055 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0da2e7ec-9cb4-42dc-a537-a5b46687ee32-metrics-certs") pod "router-default-676c59fd4f-tvcgb" (UID: "0da2e7ec-9cb4-42dc-a537-a5b46687ee32") : secret "router-metrics-certs-default" not found
Jul 29 17:52:45.306862 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:45.306787    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0da2e7ec-9cb4-42dc-a537-a5b46687ee32-service-ca-bundle\") pod \"router-default-676c59fd4f-tvcgb\" (UID: \"0da2e7ec-9cb4-42dc-a537-a5b46687ee32\") " pod="openshift-ingress/router-default-676c59fd4f-tvcgb"
Jul 29 17:52:45.307078 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:45.306874    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0da2e7ec-9cb4-42dc-a537-a5b46687ee32-service-ca-bundle podName:0da2e7ec-9cb4-42dc-a537-a5b46687ee32 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:49.306860066 +0000 UTC m=+49.176082854 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0da2e7ec-9cb4-42dc-a537-a5b46687ee32-service-ca-bundle") pod "router-default-676c59fd4f-tvcgb" (UID: "0da2e7ec-9cb4-42dc-a537-a5b46687ee32") : configmap references non-existent config key: service-ca.crt
Jul 29 17:52:45.307078 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:45.306901    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/427a5e33-ee40-4ef0-a223-32a6ba9e52cb-cluster-monitoring-operator-tls podName:427a5e33-ee40-4ef0-a223-32a6ba9e52cb nodeName:}" failed. No retries permitted until 2026-07-29 17:52:49.306891645 +0000 UTC m=+49.176114434 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/427a5e33-ee40-4ef0-a223-32a6ba9e52cb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-rgj9m" (UID: "427a5e33-ee40-4ef0-a223-32a6ba9e52cb") : secret "cluster-monitoring-operator-tls" not found
Jul 29 17:52:48.908666 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:48.908619    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-4j8zv" event={"ID":"a9042be4-9aa4-4bc5-b9b8-364a7f782353","Type":"ContainerStarted","Data":"49e400a66020ca8c5f2c93ab06e319b3b36fd28b35ee109017ddc41c13c0c480"}
Jul 29 17:52:48.910648 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:48.910623    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-2fks5" event={"ID":"fcb6432a-cde0-4f40-9e00-2523c43ca2c8","Type":"ContainerStarted","Data":"544d4adf675f1719be6927feebcafa2607a757fb323f000d524a6550b41317c7"}
Jul 29 17:52:48.912833 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:48.912796    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-m2x44" event={"ID":"6554ff13-9d69-4062-be7c-a8d3ff442945","Type":"ContainerStarted","Data":"e3f8245b78f30fc3036b480e864b0dbef66fe5d692aa2bb3ccf4f72b2830e84d"}
Jul 29 17:52:48.914578 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:48.914557    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gg2wj_2bf0bb71-4047-45a7-bd8e-5d6efabd26bd/console-operator/0.log"
Jul 29 17:52:48.914673 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:48.914598    2557 generic.go:356] "Generic (PLEG): container finished" podID="2bf0bb71-4047-45a7-bd8e-5d6efabd26bd" containerID="b741bb8638422d6a683c598a28e604a6f886006b5ddb4d49b620c85295a97510" exitCode=255
Jul 29 17:52:48.914673 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:48.914651    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-gg2wj" event={"ID":"2bf0bb71-4047-45a7-bd8e-5d6efabd26bd","Type":"ContainerDied","Data":"b741bb8638422d6a683c598a28e604a6f886006b5ddb4d49b620c85295a97510"}
Jul 29 17:52:48.914851 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:48.914832    2557 scope.go:117] "RemoveContainer" containerID="b741bb8638422d6a683c598a28e604a6f886006b5ddb4d49b620c85295a97510"
Jul 29 17:52:48.916603 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:48.916583    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-4s8jb" event={"ID":"7843c951-e574-4cca-bbc8-ddc52d5d2b55","Type":"ContainerStarted","Data":"492bdd4459a67d133ad913a1f483eaafd7e2e13d279e5bdee75c2c51fa6bba2a"}
Jul 29 17:52:48.918097 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:48.918059    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-hhwbp" event={"ID":"a91229a9-7326-45f0-9bed-57c0f1868780","Type":"ContainerStarted","Data":"e9526e567f8bd964658ac572513d265bd31493c64fc1d6cc6d9dc37847b647c7"}
Jul 29 17:52:48.927005 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:48.926967    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-4j8zv" podStartSLOduration=1.348557928 podStartE2EDuration="7.926954641s" totalImagesPullingTime="6.578396713s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:52:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:52:48.925355671 +0000 UTC m=+48.794578478" watchObservedRunningTime="2026-07-29 17:52:48.926954641 +0000 UTC m=+48.796177444"
Jul 29 17:52:48.940850 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:48.940806    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-2fks5" podStartSLOduration=1.152494526 podStartE2EDuration="7.9407917s" totalImagesPullingTime="6.788297174s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:52:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:52:48.939855127 +0000 UTC m=+48.809077935" watchObservedRunningTime="2026-07-29 17:52:48.9407917 +0000 UTC m=+48.810014508"
Jul 29 17:52:48.956431 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:48.956387    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-m2x44" podStartSLOduration=1.555371907 podStartE2EDuration="7.956372503s" totalImagesPullingTime="6.401000596s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:52:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:52:48.955715345 +0000 UTC m=+48.824938154" watchObservedRunningTime="2026-07-29 17:52:48.956372503 +0000 UTC m=+48.825595313"
Jul 29 17:52:49.011935 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:49.011836    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-hhwbp" podStartSLOduration=1.354428891 podStartE2EDuration="8.011813247s" totalImagesPullingTime="6.657384356s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:52:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:52:49.010566315 +0000 UTC m=+48.879789122" watchObservedRunningTime="2026-07-29 17:52:49.011813247 +0000 UTC m=+48.881036056"
Jul 29 17:52:49.012195 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:49.012166    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-4s8jb" podStartSLOduration=1.362934637 podStartE2EDuration="8.012158129s" totalImagesPullingTime="6.649223492s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:52:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:52:48.990371599 +0000 UTC m=+48.859594406" watchObservedRunningTime="2026-07-29 17:52:49.012158129 +0000 UTC m=+48.881380938"
Jul 29 17:52:49.144943 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:49.144908    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a1be716c-91ad-4ab5-a476-32486b3c0506-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-5v6zw\" (UID: \"a1be716c-91ad-4ab5-a476-32486b3c0506\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-5v6zw"
Jul 29 17:52:49.145099 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:49.145063    2557 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 29 17:52:49.145146 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:49.145136    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1be716c-91ad-4ab5-a476-32486b3c0506-samples-operator-tls podName:a1be716c-91ad-4ab5-a476-32486b3c0506 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:57.145115883 +0000 UTC m=+57.014338668 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a1be716c-91ad-4ab5-a476-32486b3c0506-samples-operator-tls") pod "cluster-samples-operator-6748489656-5v6zw" (UID: "a1be716c-91ad-4ab5-a476-32486b3c0506") : secret "samples-operator-tls" not found
Jul 29 17:52:49.246076 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:49.246034    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3aad4b60-ee91-42d3-bbf1-865219602575-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-grlrl\" (UID: \"3aad4b60-ee91-42d3-bbf1-865219602575\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-grlrl"
Jul 29 17:52:49.246275 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:49.246195    2557 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 29 17:52:49.246344 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:49.246277    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3aad4b60-ee91-42d3-bbf1-865219602575-networking-console-plugin-cert podName:3aad4b60-ee91-42d3-bbf1-865219602575 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:57.24625706 +0000 UTC m=+57.115479852 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3aad4b60-ee91-42d3-bbf1-865219602575-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-grlrl" (UID: "3aad4b60-ee91-42d3-bbf1-865219602575") : secret "networking-console-plugin-cert" not found
Jul 29 17:52:49.347162 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:49.347121    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0da2e7ec-9cb4-42dc-a537-a5b46687ee32-metrics-certs\") pod \"router-default-676c59fd4f-tvcgb\" (UID: \"0da2e7ec-9cb4-42dc-a537-a5b46687ee32\") " pod="openshift-ingress/router-default-676c59fd4f-tvcgb"
Jul 29 17:52:49.347342 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:49.347204    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/427a5e33-ee40-4ef0-a223-32a6ba9e52cb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-rgj9m\" (UID: \"427a5e33-ee40-4ef0-a223-32a6ba9e52cb\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rgj9m"
Jul 29 17:52:49.347342 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:49.347281    2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 29 17:52:49.347451 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:49.347355    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0da2e7ec-9cb4-42dc-a537-a5b46687ee32-metrics-certs podName:0da2e7ec-9cb4-42dc-a537-a5b46687ee32 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:57.347334905 +0000 UTC m=+57.216557695 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0da2e7ec-9cb4-42dc-a537-a5b46687ee32-metrics-certs") pod "router-default-676c59fd4f-tvcgb" (UID: "0da2e7ec-9cb4-42dc-a537-a5b46687ee32") : secret "router-metrics-certs-default" not found
Jul 29 17:52:49.347451 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:49.347353    2557 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 29 17:52:49.347451 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:49.347378    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0da2e7ec-9cb4-42dc-a537-a5b46687ee32-service-ca-bundle podName:0da2e7ec-9cb4-42dc-a537-a5b46687ee32 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:57.347362503 +0000 UTC m=+57.216585293 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0da2e7ec-9cb4-42dc-a537-a5b46687ee32-service-ca-bundle") pod "router-default-676c59fd4f-tvcgb" (UID: "0da2e7ec-9cb4-42dc-a537-a5b46687ee32") : configmap references non-existent config key: service-ca.crt
Jul 29 17:52:49.347451 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:49.347286    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0da2e7ec-9cb4-42dc-a537-a5b46687ee32-service-ca-bundle\") pod \"router-default-676c59fd4f-tvcgb\" (UID: \"0da2e7ec-9cb4-42dc-a537-a5b46687ee32\") " pod="openshift-ingress/router-default-676c59fd4f-tvcgb"
Jul 29 17:52:49.347451 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:49.347411    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/427a5e33-ee40-4ef0-a223-32a6ba9e52cb-cluster-monitoring-operator-tls podName:427a5e33-ee40-4ef0-a223-32a6ba9e52cb nodeName:}" failed. No retries permitted until 2026-07-29 17:52:57.34739672 +0000 UTC m=+57.216619511 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/427a5e33-ee40-4ef0-a223-32a6ba9e52cb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-rgj9m" (UID: "427a5e33-ee40-4ef0-a223-32a6ba9e52cb") : secret "cluster-monitoring-operator-tls" not found
Jul 29 17:52:49.923481 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:49.923447    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gg2wj_2bf0bb71-4047-45a7-bd8e-5d6efabd26bd/console-operator/1.log"
Jul 29 17:52:49.923977 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:49.923932    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gg2wj_2bf0bb71-4047-45a7-bd8e-5d6efabd26bd/console-operator/0.log"
Jul 29 17:52:49.924035 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:49.923971    2557 generic.go:356] "Generic (PLEG): container finished" podID="2bf0bb71-4047-45a7-bd8e-5d6efabd26bd" containerID="f53188d3eed2ed73dca8f879cd8ca3b0f1b702659324c4ea0728d494e81b6212" exitCode=255
Jul 29 17:52:49.924035 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:49.924018    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-gg2wj" event={"ID":"2bf0bb71-4047-45a7-bd8e-5d6efabd26bd","Type":"ContainerDied","Data":"f53188d3eed2ed73dca8f879cd8ca3b0f1b702659324c4ea0728d494e81b6212"}
Jul 29 17:52:49.924132 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:49.924100    2557 scope.go:117] "RemoveContainer" containerID="b741bb8638422d6a683c598a28e604a6f886006b5ddb4d49b620c85295a97510"
Jul 29 17:52:49.924741 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:49.924303    2557 scope.go:117] "RemoveContainer" containerID="f53188d3eed2ed73dca8f879cd8ca3b0f1b702659324c4ea0728d494e81b6212"
Jul 29 17:52:49.924741 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:49.924499    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-gg2wj_openshift-console-operator(2bf0bb71-4047-45a7-bd8e-5d6efabd26bd)\"" pod="openshift-console-operator/console-operator-584dcb6b54-gg2wj" podUID="2bf0bb71-4047-45a7-bd8e-5d6efabd26bd"
Jul 29 17:52:50.225954 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:50.225873    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-wtc5s"]
Jul 29 17:52:50.246843 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:50.246806    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-wtc5s"]
Jul 29 17:52:50.247019 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:50.246940    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-wtc5s"
Jul 29 17:52:50.249588 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:50.249563    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 29 17:52:50.249747 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:50.249563    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 29 17:52:50.249747 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:50.249570    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-fvz84\""
Jul 29 17:52:50.357334 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:50.357296    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ee11bff0-fc98-47a5-9f2b-8a1f2675c98c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wtc5s\" (UID: \"ee11bff0-fc98-47a5-9f2b-8a1f2675c98c\") " pod="openshift-insights/insights-runtime-extractor-wtc5s"
Jul 29 17:52:50.357517 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:50.357355    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ee11bff0-fc98-47a5-9f2b-8a1f2675c98c-data-volume\") pod \"insights-runtime-extractor-wtc5s\" (UID: \"ee11bff0-fc98-47a5-9f2b-8a1f2675c98c\") " pod="openshift-insights/insights-runtime-extractor-wtc5s"
Jul 29 17:52:50.357517 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:50.357498    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ee11bff0-fc98-47a5-9f2b-8a1f2675c98c-crio-socket\") pod \"insights-runtime-extractor-wtc5s\" (UID: \"ee11bff0-fc98-47a5-9f2b-8a1f2675c98c\") " pod="openshift-insights/insights-runtime-extractor-wtc5s"
Jul 29 17:52:50.357626 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:50.357536    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ee11bff0-fc98-47a5-9f2b-8a1f2675c98c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wtc5s\" (UID: \"ee11bff0-fc98-47a5-9f2b-8a1f2675c98c\") " pod="openshift-insights/insights-runtime-extractor-wtc5s"
Jul 29 17:52:50.357680 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:50.357646    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfqhd\" (UniqueName: \"kubernetes.io/projected/ee11bff0-fc98-47a5-9f2b-8a1f2675c98c-kube-api-access-jfqhd\") pod \"insights-runtime-extractor-wtc5s\" (UID: \"ee11bff0-fc98-47a5-9f2b-8a1f2675c98c\") " pod="openshift-insights/insights-runtime-extractor-wtc5s"
Jul 29 17:52:50.458163 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:50.458119    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ee11bff0-fc98-47a5-9f2b-8a1f2675c98c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wtc5s\" (UID: \"ee11bff0-fc98-47a5-9f2b-8a1f2675c98c\") " pod="openshift-insights/insights-runtime-extractor-wtc5s"
Jul 29 17:52:50.458375 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:50.458187    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ee11bff0-fc98-47a5-9f2b-8a1f2675c98c-data-volume\") pod \"insights-runtime-extractor-wtc5s\" (UID: \"ee11bff0-fc98-47a5-9f2b-8a1f2675c98c\") " pod="openshift-insights/insights-runtime-extractor-wtc5s"
Jul 29 17:52:50.458375 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:50.458278    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ee11bff0-fc98-47a5-9f2b-8a1f2675c98c-crio-socket\") pod \"insights-runtime-extractor-wtc5s\" (UID: \"ee11bff0-fc98-47a5-9f2b-8a1f2675c98c\") " pod="openshift-insights/insights-runtime-extractor-wtc5s"
Jul 29 17:52:50.458375 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:50.458283    2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 17:52:50.458375 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:50.458315    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ee11bff0-fc98-47a5-9f2b-8a1f2675c98c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wtc5s\" (UID: \"ee11bff0-fc98-47a5-9f2b-8a1f2675c98c\") " pod="openshift-insights/insights-runtime-extractor-wtc5s"
Jul 29 17:52:50.458375 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:50.458367    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ee11bff0-fc98-47a5-9f2b-8a1f2675c98c-kube-rbac-proxy-cm podName:ee11bff0-fc98-47a5-9f2b-8a1f2675c98c nodeName:}" failed. No retries permitted until 2026-07-29 17:52:50.958345787 +0000 UTC m=+50.827568572 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ee11bff0-fc98-47a5-9f2b-8a1f2675c98c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wtc5s" (UID: "ee11bff0-fc98-47a5-9f2b-8a1f2675c98c") : configmap "kube-rbac-proxy" not found
Jul 29 17:52:50.458622 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:50.458440    2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 17:52:50.458622 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:50.458439    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfqhd\" (UniqueName: \"kubernetes.io/projected/ee11bff0-fc98-47a5-9f2b-8a1f2675c98c-kube-api-access-jfqhd\") pod \"insights-runtime-extractor-wtc5s\" (UID: \"ee11bff0-fc98-47a5-9f2b-8a1f2675c98c\") " pod="openshift-insights/insights-runtime-extractor-wtc5s"
Jul 29 17:52:50.458622 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:50.458445    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ee11bff0-fc98-47a5-9f2b-8a1f2675c98c-crio-socket\") pod \"insights-runtime-extractor-wtc5s\" (UID: \"ee11bff0-fc98-47a5-9f2b-8a1f2675c98c\") " pod="openshift-insights/insights-runtime-extractor-wtc5s"
Jul 29 17:52:50.458622 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:50.458492    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee11bff0-fc98-47a5-9f2b-8a1f2675c98c-insights-runtime-extractor-tls podName:ee11bff0-fc98-47a5-9f2b-8a1f2675c98c nodeName:}" failed. No retries permitted until 2026-07-29 17:52:50.958475504 +0000 UTC m=+50.827698293 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ee11bff0-fc98-47a5-9f2b-8a1f2675c98c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wtc5s" (UID: "ee11bff0-fc98-47a5-9f2b-8a1f2675c98c") : secret "insights-runtime-extractor-tls" not found
Jul 29 17:52:50.458622 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:50.458610    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ee11bff0-fc98-47a5-9f2b-8a1f2675c98c-data-volume\") pod \"insights-runtime-extractor-wtc5s\" (UID: \"ee11bff0-fc98-47a5-9f2b-8a1f2675c98c\") " pod="openshift-insights/insights-runtime-extractor-wtc5s"
Jul 29 17:52:50.467061 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:50.467035    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfqhd\" (UniqueName: \"kubernetes.io/projected/ee11bff0-fc98-47a5-9f2b-8a1f2675c98c-kube-api-access-jfqhd\") pod \"insights-runtime-extractor-wtc5s\" (UID: \"ee11bff0-fc98-47a5-9f2b-8a1f2675c98c\") " pod="openshift-insights/insights-runtime-extractor-wtc5s"
Jul 29 17:52:50.929139 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:50.929110    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gg2wj_2bf0bb71-4047-45a7-bd8e-5d6efabd26bd/console-operator/1.log"
Jul 29 17:52:50.929501 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:50.929455    2557 scope.go:117] "RemoveContainer" containerID="f53188d3eed2ed73dca8f879cd8ca3b0f1b702659324c4ea0728d494e81b6212"
Jul 29 17:52:50.929630 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:50.929609    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-gg2wj_openshift-console-operator(2bf0bb71-4047-45a7-bd8e-5d6efabd26bd)\"" pod="openshift-console-operator/console-operator-584dcb6b54-gg2wj" podUID="2bf0bb71-4047-45a7-bd8e-5d6efabd26bd"
Jul 29 17:52:50.963133 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:50.963101    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ee11bff0-fc98-47a5-9f2b-8a1f2675c98c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wtc5s\" (UID: \"ee11bff0-fc98-47a5-9f2b-8a1f2675c98c\") " pod="openshift-insights/insights-runtime-extractor-wtc5s"
Jul 29 17:52:50.963284 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:50.963197    2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 17:52:50.963284 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:50.963253    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ee11bff0-fc98-47a5-9f2b-8a1f2675c98c-kube-rbac-proxy-cm podName:ee11bff0-fc98-47a5-9f2b-8a1f2675c98c nodeName:}" failed. No retries permitted until 2026-07-29 17:52:51.963235818 +0000 UTC m=+51.832458606 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ee11bff0-fc98-47a5-9f2b-8a1f2675c98c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wtc5s" (UID: "ee11bff0-fc98-47a5-9f2b-8a1f2675c98c") : configmap "kube-rbac-proxy" not found
Jul 29 17:52:50.963446 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:50.963424    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ee11bff0-fc98-47a5-9f2b-8a1f2675c98c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wtc5s\" (UID: \"ee11bff0-fc98-47a5-9f2b-8a1f2675c98c\") " pod="openshift-insights/insights-runtime-extractor-wtc5s"
Jul 29 17:52:50.963562 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:50.963546    2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 17:52:50.963600 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:50.963594    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee11bff0-fc98-47a5-9f2b-8a1f2675c98c-insights-runtime-extractor-tls podName:ee11bff0-fc98-47a5-9f2b-8a1f2675c98c nodeName:}" failed. No retries permitted until 2026-07-29 17:52:51.963582039 +0000 UTC m=+51.832804825 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ee11bff0-fc98-47a5-9f2b-8a1f2675c98c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wtc5s" (UID: "ee11bff0-fc98-47a5-9f2b-8a1f2675c98c") : secret "insights-runtime-extractor-tls" not found
Jul 29 17:52:51.366894 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:51.366864    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/09c3a81a-22ca-4700-af82-9c7e86f77f07-registry-tls\") pod \"image-registry-6774b48fd9-hrjxr\" (UID: \"09c3a81a-22ca-4700-af82-9c7e86f77f07\") " pod="openshift-image-registry/image-registry-6774b48fd9-hrjxr"
Jul 29 17:52:51.367054 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:51.367017    2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 17:52:51.367054 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:51.367035    2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6774b48fd9-hrjxr: secret "image-registry-tls" not found
Jul 29 17:52:51.367124 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:51.367095    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/09c3a81a-22ca-4700-af82-9c7e86f77f07-registry-tls podName:09c3a81a-22ca-4700-af82-9c7e86f77f07 nodeName:}" failed. No retries permitted until 2026-07-29 17:53:07.367075674 +0000 UTC m=+67.236298463 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/09c3a81a-22ca-4700-af82-9c7e86f77f07-registry-tls") pod "image-registry-6774b48fd9-hrjxr" (UID: "09c3a81a-22ca-4700-af82-9c7e86f77f07") : secret "image-registry-tls" not found
Jul 29 17:52:51.467393 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:51.467359    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dedf5ed1-ad33-4658-b79c-ae80134b18c7-metrics-tls\") pod \"dns-default-bwjgj\" (UID: \"dedf5ed1-ad33-4658-b79c-ae80134b18c7\") " pod="openshift-dns/dns-default-bwjgj"
Jul 29 17:52:51.467562 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:51.467528    2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 17:52:51.467616 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:51.467606    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dedf5ed1-ad33-4658-b79c-ae80134b18c7-metrics-tls podName:dedf5ed1-ad33-4658-b79c-ae80134b18c7 nodeName:}" failed. No retries permitted until 2026-07-29 17:53:07.467588454 +0000 UTC m=+67.336811244 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dedf5ed1-ad33-4658-b79c-ae80134b18c7-metrics-tls") pod "dns-default-bwjgj" (UID: "dedf5ed1-ad33-4658-b79c-ae80134b18c7") : secret "dns-default-metrics-tls" not found
Jul 29 17:52:51.615948 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:51.615922    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5wxd2_85585baa-2c99-4feb-91ec-4a34a98df21b/dns-node-resolver/0.log"
Jul 29 17:52:51.750817 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:51.750718    2557 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-gg2wj"
Jul 29 17:52:51.750817 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:51.750795    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-gg2wj"
Jul 29 17:52:51.932920 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:51.932889    2557 scope.go:117] "RemoveContainer" containerID="f53188d3eed2ed73dca8f879cd8ca3b0f1b702659324c4ea0728d494e81b6212"
Jul 29 17:52:51.933299 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:51.933042    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-gg2wj_openshift-console-operator(2bf0bb71-4047-45a7-bd8e-5d6efabd26bd)\"" pod="openshift-console-operator/console-operator-584dcb6b54-gg2wj" podUID="2bf0bb71-4047-45a7-bd8e-5d6efabd26bd"
Jul 29 17:52:51.970631 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:51.970590    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ee11bff0-fc98-47a5-9f2b-8a1f2675c98c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wtc5s\" (UID: \"ee11bff0-fc98-47a5-9f2b-8a1f2675c98c\") " pod="openshift-insights/insights-runtime-extractor-wtc5s"
Jul 29 17:52:51.970880 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:51.970860    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ee11bff0-fc98-47a5-9f2b-8a1f2675c98c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wtc5s\" (UID: \"ee11bff0-fc98-47a5-9f2b-8a1f2675c98c\") " pod="openshift-insights/insights-runtime-extractor-wtc5s"
Jul 29 17:52:51.970964 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:51.970948    2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 17:52:51.971024 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:51.970996    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee11bff0-fc98-47a5-9f2b-8a1f2675c98c-insights-runtime-extractor-tls podName:ee11bff0-fc98-47a5-9f2b-8a1f2675c98c nodeName:}" failed. No retries permitted until 2026-07-29 17:52:53.970984803 +0000 UTC m=+53.840207588 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ee11bff0-fc98-47a5-9f2b-8a1f2675c98c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wtc5s" (UID: "ee11bff0-fc98-47a5-9f2b-8a1f2675c98c") : secret "insights-runtime-extractor-tls" not found
Jul 29 17:52:51.971162 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:51.971144    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ee11bff0-fc98-47a5-9f2b-8a1f2675c98c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wtc5s\" (UID: \"ee11bff0-fc98-47a5-9f2b-8a1f2675c98c\") " pod="openshift-insights/insights-runtime-extractor-wtc5s"
Jul 29 17:52:52.825406 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:52.825380    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vmhq9_8ef211d7-e236-4476-b5df-48aadf6e6195/node-ca/0.log"
Jul 29 17:52:53.422901 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:53.422873    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6kfpl_f9eb2a9b-3f29-455c-9cc2-0d3924a4bef2/serve-healthcheck-canary/0.log"
Jul 29 17:52:53.988030 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:53.987998    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ee11bff0-fc98-47a5-9f2b-8a1f2675c98c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wtc5s\" (UID: \"ee11bff0-fc98-47a5-9f2b-8a1f2675c98c\") " pod="openshift-insights/insights-runtime-extractor-wtc5s"
Jul 29 17:52:53.988171 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:53.988153    2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 17:52:53.988229 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:53.988221    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee11bff0-fc98-47a5-9f2b-8a1f2675c98c-insights-runtime-extractor-tls podName:ee11bff0-fc98-47a5-9f2b-8a1f2675c98c nodeName:}" failed. No retries permitted until 2026-07-29 17:52:57.988204947 +0000 UTC m=+57.857427731 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ee11bff0-fc98-47a5-9f2b-8a1f2675c98c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wtc5s" (UID: "ee11bff0-fc98-47a5-9f2b-8a1f2675c98c") : secret "insights-runtime-extractor-tls" not found
Jul 29 17:52:57.216382 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:57.216350    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a1be716c-91ad-4ab5-a476-32486b3c0506-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-5v6zw\" (UID: \"a1be716c-91ad-4ab5-a476-32486b3c0506\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-5v6zw"
Jul 29 17:52:57.218906 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:57.218889    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a1be716c-91ad-4ab5-a476-32486b3c0506-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-5v6zw\" (UID: \"a1be716c-91ad-4ab5-a476-32486b3c0506\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-5v6zw"
Jul 29 17:52:57.317603 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:57.317558    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3aad4b60-ee91-42d3-bbf1-865219602575-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-grlrl\" (UID: \"3aad4b60-ee91-42d3-bbf1-865219602575\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-grlrl"
Jul 29 17:52:57.317782 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:57.317705    2557 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 29 17:52:57.317831 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:57.317785    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3aad4b60-ee91-42d3-bbf1-865219602575-networking-console-plugin-cert podName:3aad4b60-ee91-42d3-bbf1-865219602575 nodeName:}" failed. No retries permitted until 2026-07-29 17:53:13.317769549 +0000 UTC m=+73.186992336 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3aad4b60-ee91-42d3-bbf1-865219602575-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-grlrl" (UID: "3aad4b60-ee91-42d3-bbf1-865219602575") : secret "networking-console-plugin-cert" not found
Jul 29 17:52:57.418219 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:57.418186    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0da2e7ec-9cb4-42dc-a537-a5b46687ee32-service-ca-bundle\") pod \"router-default-676c59fd4f-tvcgb\" (UID: \"0da2e7ec-9cb4-42dc-a537-a5b46687ee32\") " pod="openshift-ingress/router-default-676c59fd4f-tvcgb"
Jul 29 17:52:57.418326 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:57.418265    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0da2e7ec-9cb4-42dc-a537-a5b46687ee32-metrics-certs\") pod \"router-default-676c59fd4f-tvcgb\" (UID: \"0da2e7ec-9cb4-42dc-a537-a5b46687ee32\") " pod="openshift-ingress/router-default-676c59fd4f-tvcgb"
Jul 29 17:52:57.418326 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:57.418313    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/427a5e33-ee40-4ef0-a223-32a6ba9e52cb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-rgj9m\" (UID: \"427a5e33-ee40-4ef0-a223-32a6ba9e52cb\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rgj9m"
Jul 29 17:52:57.418405 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:57.418354    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0da2e7ec-9cb4-42dc-a537-a5b46687ee32-service-ca-bundle podName:0da2e7ec-9cb4-42dc-a537-a5b46687ee32 nodeName:}" failed. No retries permitted until 2026-07-29 17:53:13.418334494 +0000 UTC m=+73.287557281 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0da2e7ec-9cb4-42dc-a537-a5b46687ee32-service-ca-bundle") pod "router-default-676c59fd4f-tvcgb" (UID: "0da2e7ec-9cb4-42dc-a537-a5b46687ee32") : configmap references non-existent config key: service-ca.crt
Jul 29 17:52:57.418405 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:57.418397    2557 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 29 17:52:57.418494 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:57.418401    2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 29 17:52:57.418494 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:57.418452    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/427a5e33-ee40-4ef0-a223-32a6ba9e52cb-cluster-monitoring-operator-tls podName:427a5e33-ee40-4ef0-a223-32a6ba9e52cb nodeName:}" failed. No retries permitted until 2026-07-29 17:53:13.418443355 +0000 UTC m=+73.287666139 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/427a5e33-ee40-4ef0-a223-32a6ba9e52cb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-rgj9m" (UID: "427a5e33-ee40-4ef0-a223-32a6ba9e52cb") : secret "cluster-monitoring-operator-tls" not found
Jul 29 17:52:57.418494 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:57.418464    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0da2e7ec-9cb4-42dc-a537-a5b46687ee32-metrics-certs podName:0da2e7ec-9cb4-42dc-a537-a5b46687ee32 nodeName:}" failed. No retries permitted until 2026-07-29 17:53:13.418457945 +0000 UTC m=+73.287680729 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0da2e7ec-9cb4-42dc-a537-a5b46687ee32-metrics-certs") pod "router-default-676c59fd4f-tvcgb" (UID: "0da2e7ec-9cb4-42dc-a537-a5b46687ee32") : secret "router-metrics-certs-default" not found
Jul 29 17:52:57.485853 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:57.485792    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-5v6zw"
Jul 29 17:52:57.642840 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:57.642812    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-5v6zw"]
Jul 29 17:52:57.953321 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:57.953288    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-5v6zw" event={"ID":"a1be716c-91ad-4ab5-a476-32486b3c0506","Type":"ContainerStarted","Data":"9803e92b54b84c05cf2fb08a6ea2d5e9ea750183d4e1fd9d64e79105c3d8c26c"}
Jul 29 17:52:58.023105 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:58.023069    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ee11bff0-fc98-47a5-9f2b-8a1f2675c98c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wtc5s\" (UID: \"ee11bff0-fc98-47a5-9f2b-8a1f2675c98c\") " pod="openshift-insights/insights-runtime-extractor-wtc5s"
Jul 29 17:52:58.023268 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:58.023231    2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 17:52:58.023314 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:52:58.023301    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee11bff0-fc98-47a5-9f2b-8a1f2675c98c-insights-runtime-extractor-tls podName:ee11bff0-fc98-47a5-9f2b-8a1f2675c98c nodeName:}" failed. No retries permitted until 2026-07-29 17:53:06.023284504 +0000 UTC m=+65.892507292 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ee11bff0-fc98-47a5-9f2b-8a1f2675c98c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wtc5s" (UID: "ee11bff0-fc98-47a5-9f2b-8a1f2675c98c") : secret "insights-runtime-extractor-tls" not found
Jul 29 17:52:59.961107 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:59.961071    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-5v6zw" event={"ID":"a1be716c-91ad-4ab5-a476-32486b3c0506","Type":"ContainerStarted","Data":"f6727d927c232da4009cb58d74e9a37484da7196c41131049bfad41fedb96f3f"}
Jul 29 17:52:59.961466 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:59.961165    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-5v6zw" event={"ID":"a1be716c-91ad-4ab5-a476-32486b3c0506","Type":"ContainerStarted","Data":"5ee3ca2200e795e95d9d963ed4eeef8975ca129919acb8d2964b7f426e06a5c2"}
Jul 29 17:52:59.981843 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:52:59.981799    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-5v6zw" podStartSLOduration=17.391577221 podStartE2EDuration="18.981781023s" totalImagesPullingTime="1.590203802s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:52:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:52:59.981170329 +0000 UTC m=+59.850393135" watchObservedRunningTime="2026-07-29 17:52:59.981781023 +0000 UTC m=+59.851003829"
Jul 29 17:53:00.841625 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:00.841600    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-t9s7z"
Jul 29 17:53:03.646345 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:03.646313    2557 scope.go:117] "RemoveContainer" containerID="f53188d3eed2ed73dca8f879cd8ca3b0f1b702659324c4ea0728d494e81b6212"
Jul 29 17:53:03.975563 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:03.975482    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gg2wj_2bf0bb71-4047-45a7-bd8e-5d6efabd26bd/console-operator/2.log"
Jul 29 17:53:03.975905 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:03.975889    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gg2wj_2bf0bb71-4047-45a7-bd8e-5d6efabd26bd/console-operator/1.log"
Jul 29 17:53:03.975967 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:03.975925    2557 generic.go:356] "Generic (PLEG): container finished" podID="2bf0bb71-4047-45a7-bd8e-5d6efabd26bd" containerID="91b67afd574b1e0bfef430e95ae409e929649019adafa57585033864a831e850" exitCode=255
Jul 29 17:53:03.975967 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:03.975962    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-gg2wj" event={"ID":"2bf0bb71-4047-45a7-bd8e-5d6efabd26bd","Type":"ContainerDied","Data":"91b67afd574b1e0bfef430e95ae409e929649019adafa57585033864a831e850"}
Jul 29 17:53:03.976078 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:03.976010    2557 scope.go:117] "RemoveContainer" containerID="f53188d3eed2ed73dca8f879cd8ca3b0f1b702659324c4ea0728d494e81b6212"
Jul 29 17:53:03.976316 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:03.976288    2557 scope.go:117] "RemoveContainer" containerID="91b67afd574b1e0bfef430e95ae409e929649019adafa57585033864a831e850"
Jul 29 17:53:03.976483 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:53:03.976465    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-gg2wj_openshift-console-operator(2bf0bb71-4047-45a7-bd8e-5d6efabd26bd)\"" pod="openshift-console-operator/console-operator-584dcb6b54-gg2wj" podUID="2bf0bb71-4047-45a7-bd8e-5d6efabd26bd"
Jul 29 17:53:04.980081 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:04.980054    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gg2wj_2bf0bb71-4047-45a7-bd8e-5d6efabd26bd/console-operator/2.log"
Jul 29 17:53:06.095586 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:06.095556    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ee11bff0-fc98-47a5-9f2b-8a1f2675c98c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wtc5s\" (UID: \"ee11bff0-fc98-47a5-9f2b-8a1f2675c98c\") " pod="openshift-insights/insights-runtime-extractor-wtc5s"
Jul 29 17:53:06.097967 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:06.097942    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ee11bff0-fc98-47a5-9f2b-8a1f2675c98c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wtc5s\" (UID: \"ee11bff0-fc98-47a5-9f2b-8a1f2675c98c\") " pod="openshift-insights/insights-runtime-extractor-wtc5s"
Jul 29 17:53:06.159109 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:06.159086    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-fvz84\""
Jul 29 17:53:06.167186 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:06.167171    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-wtc5s"
Jul 29 17:53:06.292062 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:06.292027    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-wtc5s"]
Jul 29 17:53:06.295215 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:53:06.295183    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee11bff0_fc98_47a5_9f2b_8a1f2675c98c.slice/crio-83d8ea7a2b45e5bb069c2d686337ba438e1698d32dc143123a0eb52caeb45d17 WatchSource:0}: Error finding container 83d8ea7a2b45e5bb069c2d686337ba438e1698d32dc143123a0eb52caeb45d17: Status 404 returned error can't find the container with id 83d8ea7a2b45e5bb069c2d686337ba438e1698d32dc143123a0eb52caeb45d17
Jul 29 17:53:06.398243 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:06.398181    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znrl2\" (UniqueName: \"kubernetes.io/projected/5540549f-0d35-480f-9135-2e51348f86ea-kube-api-access-znrl2\") pod \"network-check-target-svfrm\" (UID: \"5540549f-0d35-480f-9135-2e51348f86ea\") " pod="openshift-network-diagnostics/network-check-target-svfrm"
Jul 29 17:53:06.398369 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:06.398257    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/905f9420-cd73-4e49-a85f-2163a04b8fcd-original-pull-secret\") pod \"global-pull-secret-syncer-tdktb\" (UID: \"905f9420-cd73-4e49-a85f-2163a04b8fcd\") " pod="kube-system/global-pull-secret-syncer-tdktb"
Jul 29 17:53:06.400860 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:06.400839    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-znrl2\" (UniqueName: \"kubernetes.io/projected/5540549f-0d35-480f-9135-2e51348f86ea-kube-api-access-znrl2\") pod \"network-check-target-svfrm\" (UID: \"5540549f-0d35-480f-9135-2e51348f86ea\") " pod="openshift-network-diagnostics/network-check-target-svfrm"
Jul 29 17:53:06.400962 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:06.400938    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 29 17:53:06.411314 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:06.411291    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/905f9420-cd73-4e49-a85f-2163a04b8fcd-original-pull-secret\") pod \"global-pull-secret-syncer-tdktb\" (UID: \"905f9420-cd73-4e49-a85f-2163a04b8fcd\") " pod="kube-system/global-pull-secret-syncer-tdktb"
Jul 29 17:53:06.499152 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:06.499127    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57afeaa9-b72d-4389-940b-f56a30b67995-metrics-certs\") pod \"network-metrics-daemon-kbx7n\" (UID: \"57afeaa9-b72d-4389-940b-f56a30b67995\") " pod="openshift-multus/network-metrics-daemon-kbx7n"
Jul 29 17:53:06.501899 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:06.501885    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 29 17:53:06.512292 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:06.512266    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57afeaa9-b72d-4389-940b-f56a30b67995-metrics-certs\") pod \"network-metrics-daemon-kbx7n\" (UID: \"57afeaa9-b72d-4389-940b-f56a30b67995\") " pod="openshift-multus/network-metrics-daemon-kbx7n"
Jul 29 17:53:06.560498 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:06.560478    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-w7s6p\""
Jul 29 17:53:06.563945 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:06.563927    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tdktb"
Jul 29 17:53:06.567607 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:06.567591    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kbx7n"
Jul 29 17:53:06.572300 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:06.572286    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-llfsb\""
Jul 29 17:53:06.581055 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:06.581035    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-svfrm"
Jul 29 17:53:06.714540 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:06.714048    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tdktb"]
Jul 29 17:53:06.716560 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:53:06.716533    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod905f9420_cd73_4e49_a85f_2163a04b8fcd.slice/crio-e64e0eb08d5fa2c98524364aaa0b473ce00b249aeb6137b8580250675d84439d WatchSource:0}: Error finding container e64e0eb08d5fa2c98524364aaa0b473ce00b249aeb6137b8580250675d84439d: Status 404 returned error can't find the container with id e64e0eb08d5fa2c98524364aaa0b473ce00b249aeb6137b8580250675d84439d
Jul 29 17:53:06.942967 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:06.942936    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-svfrm"]
Jul 29 17:53:06.944586 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:06.944527    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kbx7n"]
Jul 29 17:53:06.947583 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:53:06.947556    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57afeaa9_b72d_4389_940b_f56a30b67995.slice/crio-855695b091afbba721c6fce15d27dbdad8081ea19523370d82d0a3be63f4431a WatchSource:0}: Error finding container 855695b091afbba721c6fce15d27dbdad8081ea19523370d82d0a3be63f4431a: Status 404 returned error can't find the container with id 855695b091afbba721c6fce15d27dbdad8081ea19523370d82d0a3be63f4431a
Jul 29 17:53:06.951672 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:53:06.951646    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5540549f_0d35_480f_9135_2e51348f86ea.slice/crio-03457fc5f98905e87134f168ac2a459153736f00d7e2decf44defa4b72009b59 WatchSource:0}: Error finding container 03457fc5f98905e87134f168ac2a459153736f00d7e2decf44defa4b72009b59: Status 404 returned error can't find the container with id 03457fc5f98905e87134f168ac2a459153736f00d7e2decf44defa4b72009b59
Jul 29 17:53:06.989183 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:06.989120    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-svfrm" event={"ID":"5540549f-0d35-480f-9135-2e51348f86ea","Type":"ContainerStarted","Data":"03457fc5f98905e87134f168ac2a459153736f00d7e2decf44defa4b72009b59"}
Jul 29 17:53:06.990705 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:06.990673    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tdktb" event={"ID":"905f9420-cd73-4e49-a85f-2163a04b8fcd","Type":"ContainerStarted","Data":"e64e0eb08d5fa2c98524364aaa0b473ce00b249aeb6137b8580250675d84439d"}
Jul 29 17:53:06.992318 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:06.992292    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wtc5s" event={"ID":"ee11bff0-fc98-47a5-9f2b-8a1f2675c98c","Type":"ContainerStarted","Data":"83d8ea7a2b45e5bb069c2d686337ba438e1698d32dc143123a0eb52caeb45d17"}
Jul 29 17:53:06.993679 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:06.993624    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kbx7n" event={"ID":"57afeaa9-b72d-4389-940b-f56a30b67995","Type":"ContainerStarted","Data":"855695b091afbba721c6fce15d27dbdad8081ea19523370d82d0a3be63f4431a"}
Jul 29 17:53:07.410029 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:07.409992    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/09c3a81a-22ca-4700-af82-9c7e86f77f07-registry-tls\") pod \"image-registry-6774b48fd9-hrjxr\" (UID: \"09c3a81a-22ca-4700-af82-9c7e86f77f07\") " pod="openshift-image-registry/image-registry-6774b48fd9-hrjxr"
Jul 29 17:53:07.413273 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:07.413245    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/09c3a81a-22ca-4700-af82-9c7e86f77f07-registry-tls\") pod \"image-registry-6774b48fd9-hrjxr\" (UID: \"09c3a81a-22ca-4700-af82-9c7e86f77f07\") " pod="openshift-image-registry/image-registry-6774b48fd9-hrjxr"
Jul 29 17:53:07.510746 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:07.510697    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dedf5ed1-ad33-4658-b79c-ae80134b18c7-metrics-tls\") pod \"dns-default-bwjgj\" (UID: \"dedf5ed1-ad33-4658-b79c-ae80134b18c7\") " pod="openshift-dns/dns-default-bwjgj"
Jul 29 17:53:07.513322 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:07.513292    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dedf5ed1-ad33-4658-b79c-ae80134b18c7-metrics-tls\") pod \"dns-default-bwjgj\" (UID: \"dedf5ed1-ad33-4658-b79c-ae80134b18c7\") " pod="openshift-dns/dns-default-bwjgj"
Jul 29 17:53:07.635338 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:07.635158    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-w8nx9\""
Jul 29 17:53:07.642705 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:07.642452    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6774b48fd9-hrjxr"
Jul 29 17:53:07.671471 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:07.671295    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-tr8hk\""
Jul 29 17:53:07.678787 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:07.678764    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bwjgj"
Jul 29 17:53:07.808003 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:07.807967    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6774b48fd9-hrjxr"]
Jul 29 17:53:07.822564 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:53:07.822535    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09c3a81a_22ca_4700_af82_9c7e86f77f07.slice/crio-0ea0b496fd413a14aedb9fb62de58bd94a7b8d333cf15f3f22051ffbecbcb0a5 WatchSource:0}: Error finding container 0ea0b496fd413a14aedb9fb62de58bd94a7b8d333cf15f3f22051ffbecbcb0a5: Status 404 returned error can't find the container with id 0ea0b496fd413a14aedb9fb62de58bd94a7b8d333cf15f3f22051ffbecbcb0a5
Jul 29 17:53:07.870964 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:07.870634    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bwjgj"]
Jul 29 17:53:07.873612 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:53:07.873574    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddedf5ed1_ad33_4658_b79c_ae80134b18c7.slice/crio-3ba4b7b0b21fbfaac83b97ec76b10184ae8370a60041739a02e1b819d4e6c3d4 WatchSource:0}: Error finding container 3ba4b7b0b21fbfaac83b97ec76b10184ae8370a60041739a02e1b819d4e6c3d4: Status 404 returned error can't find the container with id 3ba4b7b0b21fbfaac83b97ec76b10184ae8370a60041739a02e1b819d4e6c3d4
Jul 29 17:53:08.005671 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:08.005634    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6774b48fd9-hrjxr" event={"ID":"09c3a81a-22ca-4700-af82-9c7e86f77f07","Type":"ContainerStarted","Data":"22713f1a107ff9e16a02849f8edcc98ba754888942e394a60bdca253251996a1"}
Jul 29 17:53:08.005858 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:08.005698    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6774b48fd9-hrjxr" event={"ID":"09c3a81a-22ca-4700-af82-9c7e86f77f07","Type":"ContainerStarted","Data":"0ea0b496fd413a14aedb9fb62de58bd94a7b8d333cf15f3f22051ffbecbcb0a5"}
Jul 29 17:53:08.006485 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:08.006461    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6774b48fd9-hrjxr"
Jul 29 17:53:08.008911 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:08.008884    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-svfrm" event={"ID":"5540549f-0d35-480f-9135-2e51348f86ea","Type":"ContainerStarted","Data":"8d3104b6ce513b6b83fd9e0938e7c3bae3318b7fd6c6506bdd45579f426e69b7"}
Jul 29 17:53:08.009353 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:08.009337    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-svfrm"
Jul 29 17:53:08.011644 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:08.011591    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bwjgj" event={"ID":"dedf5ed1-ad33-4658-b79c-ae80134b18c7","Type":"ContainerStarted","Data":"3ba4b7b0b21fbfaac83b97ec76b10184ae8370a60041739a02e1b819d4e6c3d4"}
Jul 29 17:53:08.028758 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:08.028693    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6774b48fd9-hrjxr" podStartSLOduration=67.028677825 podStartE2EDuration="1m7.028677825s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:52:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:53:08.027294685 +0000 UTC m=+67.896517493" watchObservedRunningTime="2026-07-29 17:53:08.028677825 +0000 UTC m=+67.897900654"
Jul 29 17:53:09.019226 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:09.019181    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kbx7n" event={"ID":"57afeaa9-b72d-4389-940b-f56a30b67995","Type":"ContainerStarted","Data":"311d484243c99ae9eea6a1dea7a07cdaabb823ebe51ebdc577d789c9657f530d"}
Jul 29 17:53:11.750741 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:11.750690    2557 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-gg2wj"
Jul 29 17:53:11.751179 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:11.750776    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-gg2wj"
Jul 29 17:53:11.751179 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:11.751128    2557 scope.go:117] "RemoveContainer" containerID="91b67afd574b1e0bfef430e95ae409e929649019adafa57585033864a831e850"
Jul 29 17:53:11.751355 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:53:11.751328    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-gg2wj_openshift-console-operator(2bf0bb71-4047-45a7-bd8e-5d6efabd26bd)\"" pod="openshift-console-operator/console-operator-584dcb6b54-gg2wj" podUID="2bf0bb71-4047-45a7-bd8e-5d6efabd26bd"
Jul 29 17:53:11.767716 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:11.767666    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-svfrm" podStartSLOduration=71.767651866 podStartE2EDuration="1m11.767651866s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:52:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:53:08.044380939 +0000 UTC m=+67.913603748" watchObservedRunningTime="2026-07-29 17:53:11.767651866 +0000 UTC m=+71.636874651"
Jul 29 17:53:13.374303 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:13.374266    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3aad4b60-ee91-42d3-bbf1-865219602575-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-grlrl\" (UID: \"3aad4b60-ee91-42d3-bbf1-865219602575\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-grlrl"
Jul 29 17:53:13.376717 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:13.376694    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3aad4b60-ee91-42d3-bbf1-865219602575-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-grlrl\" (UID: \"3aad4b60-ee91-42d3-bbf1-865219602575\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-grlrl"
Jul 29 17:53:13.475474 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:13.475439    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0da2e7ec-9cb4-42dc-a537-a5b46687ee32-service-ca-bundle\") pod \"router-default-676c59fd4f-tvcgb\" (UID: \"0da2e7ec-9cb4-42dc-a537-a5b46687ee32\") " pod="openshift-ingress/router-default-676c59fd4f-tvcgb"
Jul 29 17:53:13.475637 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:13.475510    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0da2e7ec-9cb4-42dc-a537-a5b46687ee32-metrics-certs\") pod \"router-default-676c59fd4f-tvcgb\" (UID: \"0da2e7ec-9cb4-42dc-a537-a5b46687ee32\") " pod="openshift-ingress/router-default-676c59fd4f-tvcgb"
Jul 29 17:53:13.475637 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:13.475545    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/427a5e33-ee40-4ef0-a223-32a6ba9e52cb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-rgj9m\" (UID: \"427a5e33-ee40-4ef0-a223-32a6ba9e52cb\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rgj9m"
Jul 29 17:53:13.476267 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:13.476188    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0da2e7ec-9cb4-42dc-a537-a5b46687ee32-service-ca-bundle\") pod \"router-default-676c59fd4f-tvcgb\" (UID: \"0da2e7ec-9cb4-42dc-a537-a5b46687ee32\") " pod="openshift-ingress/router-default-676c59fd4f-tvcgb"
Jul 29 17:53:13.478217 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:13.478188    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/427a5e33-ee40-4ef0-a223-32a6ba9e52cb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-rgj9m\" (UID: \"427a5e33-ee40-4ef0-a223-32a6ba9e52cb\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rgj9m"
Jul 29 17:53:13.478546 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:13.478525    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0da2e7ec-9cb4-42dc-a537-a5b46687ee32-metrics-certs\") pod \"router-default-676c59fd4f-tvcgb\" (UID: \"0da2e7ec-9cb4-42dc-a537-a5b46687ee32\") " pod="openshift-ingress/router-default-676c59fd4f-tvcgb"
Jul 29 17:53:13.573598 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:13.573565    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-flh7w\""
Jul 29 17:53:13.581859 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:13.581835    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-grlrl"
Jul 29 17:53:13.598524 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:13.598501    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-jd426\""
Jul 29 17:53:13.605977 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:13.605954    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rgj9m"
Jul 29 17:53:13.622906 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:13.622884    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-thk5g\""
Jul 29 17:53:13.631382 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:13.631331    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-676c59fd4f-tvcgb"
Jul 29 17:53:13.925981 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:13.925907    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-rgj9m"]
Jul 29 17:53:13.928830 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:53:13.928791    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod427a5e33_ee40_4ef0_a223_32a6ba9e52cb.slice/crio-7e4f876b4a0077d35f1aba97985ed5705661679c322c572fdd3c9e6024be6c8c WatchSource:0}: Error finding container 7e4f876b4a0077d35f1aba97985ed5705661679c322c572fdd3c9e6024be6c8c: Status 404 returned error can't find the container with id 7e4f876b4a0077d35f1aba97985ed5705661679c322c572fdd3c9e6024be6c8c
Jul 29 17:53:13.960784 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:13.960458    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-grlrl"]
Jul 29 17:53:13.963644 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:13.963586    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-676c59fd4f-tvcgb"]
Jul 29 17:53:14.045486 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:14.045446    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wtc5s" event={"ID":"ee11bff0-fc98-47a5-9f2b-8a1f2675c98c","Type":"ContainerStarted","Data":"202372a5351d875362df1783ad6f1d1fe8065bee36d891b13e6200d94c03c78b"}
Jul 29 17:53:14.047344 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:14.047294    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bwjgj" event={"ID":"dedf5ed1-ad33-4658-b79c-ae80134b18c7","Type":"ContainerStarted","Data":"a47447dd01c7c53d0c5930e073a3da86821bf3f978f108058287af7457508d61"}
Jul 29 17:53:14.050007 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:14.049962    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-676c59fd4f-tvcgb" event={"ID":"0da2e7ec-9cb4-42dc-a537-a5b46687ee32","Type":"ContainerStarted","Data":"45a93a8ac4754a16db2fe0514bfceb5840d9a3e7ae75e99fc8b34b7d2d2f3246"}
Jul 29 17:53:14.052632 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:14.052472    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kbx7n" event={"ID":"57afeaa9-b72d-4389-940b-f56a30b67995","Type":"ContainerStarted","Data":"8d24eb688a17410c1a7522184d6599bb1876435c5d0b237a3230a393fda4d9b7"}
Jul 29 17:53:14.053622 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:14.053603    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-grlrl" event={"ID":"3aad4b60-ee91-42d3-bbf1-865219602575","Type":"ContainerStarted","Data":"25d299d89ec52892db9089b4e320aaade04e97c97bc59cd9820323d21e4995b5"}
Jul 29 17:53:14.054786 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:14.054718    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rgj9m" event={"ID":"427a5e33-ee40-4ef0-a223-32a6ba9e52cb","Type":"ContainerStarted","Data":"7e4f876b4a0077d35f1aba97985ed5705661679c322c572fdd3c9e6024be6c8c"}
Jul 29 17:53:14.056076 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:14.056050    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tdktb" event={"ID":"905f9420-cd73-4e49-a85f-2163a04b8fcd","Type":"ContainerStarted","Data":"fd4c2bfa3584ef55c918e0f03d580980db55fc04872b6b45f00674b29c11b087"}
Jul 29 17:53:14.069273 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:14.069221    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-kbx7n" podStartSLOduration=72.195826864 podStartE2EDuration="1m14.069203482s" totalImagesPullingTime="1.873376618s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:52:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:53:14.067570821 +0000 UTC m=+73.936793629" watchObservedRunningTime="2026-07-29 17:53:14.069203482 +0000 UTC m=+73.938426289"
Jul 29 17:53:14.085641 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:14.085588    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-tdktb" podStartSLOduration=67.065777783 podStartE2EDuration="1m14.085571437s" totalImagesPullingTime="7.019793654s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 17:52:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:53:14.085202453 +0000 UTC m=+73.954425260" watchObservedRunningTime="2026-07-29 17:53:14.085571437 +0000 UTC m=+73.954794245"
Jul 29 17:53:15.066878 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:15.066309    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bwjgj" event={"ID":"dedf5ed1-ad33-4658-b79c-ae80134b18c7","Type":"ContainerStarted","Data":"70ea8b933c4fc299f09151fb72250bac2f4f42b18710329397355f0015f0d575"}
Jul 29 17:53:15.066878 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:15.066441    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-bwjgj"
Jul 29 17:53:15.069806 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:15.069636    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-676c59fd4f-tvcgb" event={"ID":"0da2e7ec-9cb4-42dc-a537-a5b46687ee32","Type":"ContainerStarted","Data":"a57488a1d62d2aea75ce3e79f041e1d7cdcc0ed1af8f776da110ee3d830093f4"}
Jul 29 17:53:15.087235 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:15.086041    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-bwjgj" podStartSLOduration=34.180482001 podStartE2EDuration="40.086020164s" totalImagesPullingTime="5.905538163s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:52:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:53:15.085401644 +0000 UTC m=+74.954624453" watchObservedRunningTime="2026-07-29 17:53:15.086020164 +0000 UTC m=+74.955242972"
Jul 29 17:53:15.103407 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:15.103346    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-676c59fd4f-tvcgb" podStartSLOduration=34.103328589 podStartE2EDuration="34.103328589s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:52:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:53:15.102925837 +0000 UTC m=+74.972148644" watchObservedRunningTime="2026-07-29 17:53:15.103328589 +0000 UTC m=+74.972551397"
Jul 29 17:53:15.632001 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:15.631934    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-676c59fd4f-tvcgb"
Jul 29 17:53:15.635012 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:15.634989    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-676c59fd4f-tvcgb"
Jul 29 17:53:16.074923 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:16.074882    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-grlrl" event={"ID":"3aad4b60-ee91-42d3-bbf1-865219602575","Type":"ContainerStarted","Data":"5e5c4b4a4b83aa5d7f0bda5fd6a15f808d6ff3b948acf7f367f416e096bcdf44"}
Jul 29 17:53:16.075438 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:16.075419    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-676c59fd4f-tvcgb"
Jul 29 17:53:16.076629 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:16.076602    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-676c59fd4f-tvcgb"
Jul 29 17:53:16.091911 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:16.091829    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-74c9f884b7-grlrl" podStartSLOduration=33.582435146 podStartE2EDuration="35.091810919s" totalImagesPullingTime="1.509375773s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:52:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:53:16.090559898 +0000 UTC m=+75.959782706" watchObservedRunningTime="2026-07-29 17:53:16.091810919 +0000 UTC m=+75.961033726"
Jul 29 17:53:16.777266 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:16.777237    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8898cd8f4-sbht6"]
Jul 29 17:53:16.979279 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:16.979253    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8898cd8f4-sbht6"]
Jul 29 17:53:16.979463 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:16.979368    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8898cd8f4-sbht6"
Jul 29 17:53:16.982077 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:16.982056    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 29 17:53:16.983492 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:16.983471    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 29 17:53:16.983586 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:16.983488    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 29 17:53:16.983586 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:16.983538    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-sxmdt\""
Jul 29 17:53:16.983586 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:16.983475    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 29 17:53:17.109786 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:17.109744    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gksqr\" (UniqueName: \"kubernetes.io/projected/bdb04cc5-a0e8-4b8c-a4d5-0d91a814a2fe-kube-api-access-gksqr\") pod \"managed-serviceaccount-addon-agent-8898cd8f4-sbht6\" (UID: \"bdb04cc5-a0e8-4b8c-a4d5-0d91a814a2fe\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8898cd8f4-sbht6"
Jul 29 17:53:17.110131 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:17.109817    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bdb04cc5-a0e8-4b8c-a4d5-0d91a814a2fe-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-8898cd8f4-sbht6\" (UID: \"bdb04cc5-a0e8-4b8c-a4d5-0d91a814a2fe\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8898cd8f4-sbht6"
Jul 29 17:53:17.211172 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:17.211140    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gksqr\" (UniqueName: \"kubernetes.io/projected/bdb04cc5-a0e8-4b8c-a4d5-0d91a814a2fe-kube-api-access-gksqr\") pod \"managed-serviceaccount-addon-agent-8898cd8f4-sbht6\" (UID: \"bdb04cc5-a0e8-4b8c-a4d5-0d91a814a2fe\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8898cd8f4-sbht6"
Jul 29 17:53:17.211508 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:17.211480    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bdb04cc5-a0e8-4b8c-a4d5-0d91a814a2fe-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-8898cd8f4-sbht6\" (UID: \"bdb04cc5-a0e8-4b8c-a4d5-0d91a814a2fe\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8898cd8f4-sbht6"
Jul 29 17:53:17.214271 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:17.214247    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bdb04cc5-a0e8-4b8c-a4d5-0d91a814a2fe-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-8898cd8f4-sbht6\" (UID: \"bdb04cc5-a0e8-4b8c-a4d5-0d91a814a2fe\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8898cd8f4-sbht6"
Jul 29 17:53:17.219490 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:17.219469    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gksqr\" (UniqueName: \"kubernetes.io/projected/bdb04cc5-a0e8-4b8c-a4d5-0d91a814a2fe-kube-api-access-gksqr\") pod \"managed-serviceaccount-addon-agent-8898cd8f4-sbht6\" (UID: \"bdb04cc5-a0e8-4b8c-a4d5-0d91a814a2fe\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8898cd8f4-sbht6"
Jul 29 17:53:17.297360 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:17.297319    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8898cd8f4-sbht6"
Jul 29 17:53:18.854916 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:18.854885    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8898cd8f4-sbht6"]
Jul 29 17:53:18.858114 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:53:18.858085    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbdb04cc5_a0e8_4b8c_a4d5_0d91a814a2fe.slice/crio-8f150588fea797ba2ec0159b0dee61e881ab578a66d53409e4d8f88a560aad40 WatchSource:0}: Error finding container 8f150588fea797ba2ec0159b0dee61e881ab578a66d53409e4d8f88a560aad40: Status 404 returned error can't find the container with id 8f150588fea797ba2ec0159b0dee61e881ab578a66d53409e4d8f88a560aad40
Jul 29 17:53:19.091343 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:19.091310    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rgj9m" event={"ID":"427a5e33-ee40-4ef0-a223-32a6ba9e52cb","Type":"ContainerStarted","Data":"e38da36d273cd3d67616cb9e10d183ec4210a64c3db157a4d0c395517fc045eb"}
Jul 29 17:53:19.092544 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:19.092523    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8898cd8f4-sbht6" event={"ID":"bdb04cc5-a0e8-4b8c-a4d5-0d91a814a2fe","Type":"ContainerStarted","Data":"8f150588fea797ba2ec0159b0dee61e881ab578a66d53409e4d8f88a560aad40"}
Jul 29 17:53:19.123022 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:19.122978    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rgj9m" podStartSLOduration=33.320445437 podStartE2EDuration="38.122962006s" totalImagesPullingTime="4.802516569s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:52:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:53:19.121297171 +0000 UTC m=+78.990519978" watchObservedRunningTime="2026-07-29 17:53:19.122962006 +0000 UTC m=+78.992184812"
Jul 29 17:53:19.354514 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:19.354481    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-cshwb"]
Jul 29 17:53:19.377758 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:19.377642    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-cshwb"]
Jul 29 17:53:19.377935 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:19.377771    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-cshwb"
Jul 29 17:53:19.383319 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:19.383294    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 29 17:53:19.383434 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:19.383303    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-hwlg8\""
Jul 29 17:53:19.432055 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:19.432029    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/527c7cfd-bdea-432e-886a-ccdefd744e0a-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-cshwb\" (UID: \"527c7cfd-bdea-432e-886a-ccdefd744e0a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-cshwb"
Jul 29 17:53:19.532414 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:19.532388    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/527c7cfd-bdea-432e-886a-ccdefd744e0a-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-cshwb\" (UID: \"527c7cfd-bdea-432e-886a-ccdefd744e0a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-cshwb"
Jul 29 17:53:19.532534 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:53:19.532517    2557 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Jul 29 17:53:19.532601 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:53:19.532572    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/527c7cfd-bdea-432e-886a-ccdefd744e0a-tls-certificates podName:527c7cfd-bdea-432e-886a-ccdefd744e0a nodeName:}" failed. No retries permitted until 2026-07-29 17:53:20.032553706 +0000 UTC m=+79.901776498 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/527c7cfd-bdea-432e-886a-ccdefd744e0a-tls-certificates") pod "prometheus-operator-admission-webhook-7f8d78f54b-cshwb" (UID: "527c7cfd-bdea-432e-886a-ccdefd744e0a") : secret "prometheus-operator-admission-webhook-tls" not found
Jul 29 17:53:20.036794 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:20.036710    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/527c7cfd-bdea-432e-886a-ccdefd744e0a-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-cshwb\" (UID: \"527c7cfd-bdea-432e-886a-ccdefd744e0a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-cshwb"
Jul 29 17:53:20.040460 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:20.040430    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/527c7cfd-bdea-432e-886a-ccdefd744e0a-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-cshwb\" (UID: \"527c7cfd-bdea-432e-886a-ccdefd744e0a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-cshwb"
Jul 29 17:53:20.101196 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:20.101157    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wtc5s" event={"ID":"ee11bff0-fc98-47a5-9f2b-8a1f2675c98c","Type":"ContainerStarted","Data":"83754828a79423f417209a90b478c77dc5451f6f7e8c2e936e62a6d87d925f92"}
Jul 29 17:53:20.287717 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:20.287622    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-cshwb"
Jul 29 17:53:20.460745 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:20.458906    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-cshwb"]
Jul 29 17:53:20.465821 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:53:20.465794    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod527c7cfd_bdea_432e_886a_ccdefd744e0a.slice/crio-cdcea395677c597da2b96f2578f3b069294b30326ec4342093a69ac266a95696 WatchSource:0}: Error finding container cdcea395677c597da2b96f2578f3b069294b30326ec4342093a69ac266a95696: Status 404 returned error can't find the container with id cdcea395677c597da2b96f2578f3b069294b30326ec4342093a69ac266a95696
Jul 29 17:53:21.106486 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:21.106449    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-cshwb" event={"ID":"527c7cfd-bdea-432e-886a-ccdefd744e0a","Type":"ContainerStarted","Data":"cdcea395677c597da2b96f2578f3b069294b30326ec4342093a69ac266a95696"}
Jul 29 17:53:23.647110 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:23.646976    2557 scope.go:117] "RemoveContainer" containerID="91b67afd574b1e0bfef430e95ae409e929649019adafa57585033864a831e850"
Jul 29 17:53:23.647593 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:53:23.647197    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-gg2wj_openshift-console-operator(2bf0bb71-4047-45a7-bd8e-5d6efabd26bd)\"" pod="openshift-console-operator/console-operator-584dcb6b54-gg2wj" podUID="2bf0bb71-4047-45a7-bd8e-5d6efabd26bd"
Jul 29 17:53:24.121405 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:24.121370    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wtc5s" event={"ID":"ee11bff0-fc98-47a5-9f2b-8a1f2675c98c","Type":"ContainerStarted","Data":"54e36cda00ffd9dfe97b159cc921eae40a6828abd692dda0c3feca57ad93b7cd"}
Jul 29 17:53:24.122990 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:24.122956    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8898cd8f4-sbht6" event={"ID":"bdb04cc5-a0e8-4b8c-a4d5-0d91a814a2fe","Type":"ContainerStarted","Data":"7f39801dd52f41816da55f5365d36c39ca96c6780d7f7fdeafd20dd1a063f20c"}
Jul 29 17:53:24.124402 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:24.124377    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-cshwb" event={"ID":"527c7cfd-bdea-432e-886a-ccdefd744e0a","Type":"ContainerStarted","Data":"60ee20247b15fa3aa2a96ef18095a487a33c5def0875230b24ce358c2b88fc5e"}
Jul 29 17:53:24.124589 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:24.124571    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-cshwb"
Jul 29 17:53:24.129546 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:24.129524    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-cshwb"
Jul 29 17:53:24.141984 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:24.141946    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-wtc5s" podStartSLOduration=16.858744104 podStartE2EDuration="34.141933541s" totalImagesPullingTime="17.283189437s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 17:52:50 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:53:24.139968211 +0000 UTC m=+84.009191017" watchObservedRunningTime="2026-07-29 17:53:24.141933541 +0000 UTC m=+84.011156387"
Jul 29 17:53:24.158025 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:24.157985    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8898cd8f4-sbht6" podStartSLOduration=3.1803727 podStartE2EDuration="8.157974648s" totalImagesPullingTime="4.977601948s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:53:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:53:24.156894509 +0000 UTC m=+84.026117317" watchObservedRunningTime="2026-07-29 17:53:24.157974648 +0000 UTC m=+84.027197454"
Jul 29 17:53:24.175043 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:24.174968    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-cshwb" podStartSLOduration=1.7636992660000002 podStartE2EDuration="5.174957008s" totalImagesPullingTime="3.411257742s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:53:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:53:24.174118203 +0000 UTC m=+84.043341009" watchObservedRunningTime="2026-07-29 17:53:24.174957008 +0000 UTC m=+84.044179815"
Jul 29 17:53:25.077829 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:25.077800    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-bwjgj"
Jul 29 17:53:30.028256 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:30.028224    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6774b48fd9-hrjxr"
Jul 29 17:53:35.829073 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:35.829042    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-8x4dk"]
Jul 29 17:53:35.835144 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:35.835122    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8x4dk"
Jul 29 17:53:35.837567 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:35.837543    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 29 17:53:35.837716 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:35.837608    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-lwjtf\""
Jul 29 17:53:35.837956 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:35.837940    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 29 17:53:35.838884 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:35.838868    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 29 17:53:35.838983 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:35.838893    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 29 17:53:35.967166 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:35.967124    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5e92be4a-f9b0-4f36-8a4d-09ae165c21e4-node-exporter-accelerators-collector-config\") pod \"node-exporter-8x4dk\" (UID: \"5e92be4a-f9b0-4f36-8a4d-09ae165c21e4\") " pod="openshift-monitoring/node-exporter-8x4dk"
Jul 29 17:53:35.967166 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:35.967166    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5e92be4a-f9b0-4f36-8a4d-09ae165c21e4-node-exporter-wtmp\") pod \"node-exporter-8x4dk\" (UID: \"5e92be4a-f9b0-4f36-8a4d-09ae165c21e4\") " pod="openshift-monitoring/node-exporter-8x4dk"
Jul 29 17:53:35.967441 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:35.967190    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5e92be4a-f9b0-4f36-8a4d-09ae165c21e4-node-exporter-tls\") pod \"node-exporter-8x4dk\" (UID: \"5e92be4a-f9b0-4f36-8a4d-09ae165c21e4\") " pod="openshift-monitoring/node-exporter-8x4dk"
Jul 29 17:53:35.967441 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:35.967288    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5e92be4a-f9b0-4f36-8a4d-09ae165c21e4-root\") pod \"node-exporter-8x4dk\" (UID: \"5e92be4a-f9b0-4f36-8a4d-09ae165c21e4\") " pod="openshift-monitoring/node-exporter-8x4dk"
Jul 29 17:53:35.967441 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:35.967330    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5e92be4a-f9b0-4f36-8a4d-09ae165c21e4-node-exporter-textfile\") pod \"node-exporter-8x4dk\" (UID: \"5e92be4a-f9b0-4f36-8a4d-09ae165c21e4\") " pod="openshift-monitoring/node-exporter-8x4dk"
Jul 29 17:53:35.967441 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:35.967376    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5e92be4a-f9b0-4f36-8a4d-09ae165c21e4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8x4dk\" (UID: \"5e92be4a-f9b0-4f36-8a4d-09ae165c21e4\") " pod="openshift-monitoring/node-exporter-8x4dk"
Jul 29 17:53:35.967441 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:35.967426    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5e92be4a-f9b0-4f36-8a4d-09ae165c21e4-sys\") pod \"node-exporter-8x4dk\" (UID: \"5e92be4a-f9b0-4f36-8a4d-09ae165c21e4\") " pod="openshift-monitoring/node-exporter-8x4dk"
Jul 29 17:53:35.967651 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:35.967469    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5e92be4a-f9b0-4f36-8a4d-09ae165c21e4-metrics-client-ca\") pod \"node-exporter-8x4dk\" (UID: \"5e92be4a-f9b0-4f36-8a4d-09ae165c21e4\") " pod="openshift-monitoring/node-exporter-8x4dk"
Jul 29 17:53:35.967651 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:35.967493    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgxgd\" (UniqueName: \"kubernetes.io/projected/5e92be4a-f9b0-4f36-8a4d-09ae165c21e4-kube-api-access-wgxgd\") pod \"node-exporter-8x4dk\" (UID: \"5e92be4a-f9b0-4f36-8a4d-09ae165c21e4\") " pod="openshift-monitoring/node-exporter-8x4dk"
Jul 29 17:53:36.068747 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:36.068681    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5e92be4a-f9b0-4f36-8a4d-09ae165c21e4-node-exporter-tls\") pod \"node-exporter-8x4dk\" (UID: \"5e92be4a-f9b0-4f36-8a4d-09ae165c21e4\") " pod="openshift-monitoring/node-exporter-8x4dk"
Jul 29 17:53:36.068946 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:36.068762    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5e92be4a-f9b0-4f36-8a4d-09ae165c21e4-root\") pod \"node-exporter-8x4dk\" (UID: \"5e92be4a-f9b0-4f36-8a4d-09ae165c21e4\") " pod="openshift-monitoring/node-exporter-8x4dk"
Jul 29 17:53:36.068946 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:36.068783    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5e92be4a-f9b0-4f36-8a4d-09ae165c21e4-node-exporter-textfile\") pod \"node-exporter-8x4dk\" (UID: \"5e92be4a-f9b0-4f36-8a4d-09ae165c21e4\") " pod="openshift-monitoring/node-exporter-8x4dk"
Jul 29 17:53:36.068946 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:36.068805    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5e92be4a-f9b0-4f36-8a4d-09ae165c21e4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8x4dk\" (UID: \"5e92be4a-f9b0-4f36-8a4d-09ae165c21e4\") " pod="openshift-monitoring/node-exporter-8x4dk"
Jul 29 17:53:36.068946 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:36.068829    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5e92be4a-f9b0-4f36-8a4d-09ae165c21e4-sys\") pod \"node-exporter-8x4dk\" (UID: \"5e92be4a-f9b0-4f36-8a4d-09ae165c21e4\") " pod="openshift-monitoring/node-exporter-8x4dk"
Jul 29 17:53:36.068946 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:36.068862    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5e92be4a-f9b0-4f36-8a4d-09ae165c21e4-metrics-client-ca\") pod \"node-exporter-8x4dk\" (UID: \"5e92be4a-f9b0-4f36-8a4d-09ae165c21e4\") " pod="openshift-monitoring/node-exporter-8x4dk"
Jul 29 17:53:36.068946 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:36.068886    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgxgd\" (UniqueName: \"kubernetes.io/projected/5e92be4a-f9b0-4f36-8a4d-09ae165c21e4-kube-api-access-wgxgd\") pod \"node-exporter-8x4dk\" (UID: \"5e92be4a-f9b0-4f36-8a4d-09ae165c21e4\") " pod="openshift-monitoring/node-exporter-8x4dk"
Jul 29 17:53:36.068946 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:36.068883    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5e92be4a-f9b0-4f36-8a4d-09ae165c21e4-root\") pod \"node-exporter-8x4dk\" (UID: \"5e92be4a-f9b0-4f36-8a4d-09ae165c21e4\") " pod="openshift-monitoring/node-exporter-8x4dk"
Jul 29 17:53:36.068946 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:36.068921    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5e92be4a-f9b0-4f36-8a4d-09ae165c21e4-node-exporter-accelerators-collector-config\") pod \"node-exporter-8x4dk\" (UID: \"5e92be4a-f9b0-4f36-8a4d-09ae165c21e4\") " pod="openshift-monitoring/node-exporter-8x4dk"
Jul 29 17:53:36.068946 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:36.068936    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5e92be4a-f9b0-4f36-8a4d-09ae165c21e4-sys\") pod \"node-exporter-8x4dk\" (UID: \"5e92be4a-f9b0-4f36-8a4d-09ae165c21e4\") " pod="openshift-monitoring/node-exporter-8x4dk"
Jul 29 17:53:36.069393 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:36.068952    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5e92be4a-f9b0-4f36-8a4d-09ae165c21e4-node-exporter-wtmp\") pod \"node-exporter-8x4dk\" (UID: \"5e92be4a-f9b0-4f36-8a4d-09ae165c21e4\") " pod="openshift-monitoring/node-exporter-8x4dk"
Jul 29 17:53:36.069393 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:36.069072    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5e92be4a-f9b0-4f36-8a4d-09ae165c21e4-node-exporter-wtmp\") pod \"node-exporter-8x4dk\" (UID: \"5e92be4a-f9b0-4f36-8a4d-09ae165c21e4\") " pod="openshift-monitoring/node-exporter-8x4dk"
Jul 29 17:53:36.069393 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:36.069220    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5e92be4a-f9b0-4f36-8a4d-09ae165c21e4-node-exporter-textfile\") pod \"node-exporter-8x4dk\" (UID: \"5e92be4a-f9b0-4f36-8a4d-09ae165c21e4\") " pod="openshift-monitoring/node-exporter-8x4dk"
Jul 29 17:53:36.069552 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:36.069452    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5e92be4a-f9b0-4f36-8a4d-09ae165c21e4-metrics-client-ca\") pod \"node-exporter-8x4dk\" (UID: \"5e92be4a-f9b0-4f36-8a4d-09ae165c21e4\") " pod="openshift-monitoring/node-exporter-8x4dk"
Jul 29 17:53:36.069552 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:36.069529    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5e92be4a-f9b0-4f36-8a4d-09ae165c21e4-node-exporter-accelerators-collector-config\") pod \"node-exporter-8x4dk\" (UID: \"5e92be4a-f9b0-4f36-8a4d-09ae165c21e4\") " pod="openshift-monitoring/node-exporter-8x4dk"
Jul 29 17:53:36.071676 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:36.071651    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5e92be4a-f9b0-4f36-8a4d-09ae165c21e4-node-exporter-tls\") pod \"node-exporter-8x4dk\" (UID: \"5e92be4a-f9b0-4f36-8a4d-09ae165c21e4\") " pod="openshift-monitoring/node-exporter-8x4dk"
Jul 29 17:53:36.071802 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:36.071688    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5e92be4a-f9b0-4f36-8a4d-09ae165c21e4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8x4dk\" (UID: \"5e92be4a-f9b0-4f36-8a4d-09ae165c21e4\") " pod="openshift-monitoring/node-exporter-8x4dk"
Jul 29 17:53:36.085007 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:36.084944    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgxgd\" (UniqueName: \"kubernetes.io/projected/5e92be4a-f9b0-4f36-8a4d-09ae165c21e4-kube-api-access-wgxgd\") pod \"node-exporter-8x4dk\" (UID: \"5e92be4a-f9b0-4f36-8a4d-09ae165c21e4\") " pod="openshift-monitoring/node-exporter-8x4dk"
Jul 29 17:53:36.144049 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:36.144013    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8x4dk"
Jul 29 17:53:36.157777 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:53:36.157747    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e92be4a_f9b0_4f36_8a4d_09ae165c21e4.slice/crio-95388d86307b3d147473495469efaabf4ef1880cc066a8dbe38067118a7f8c29 WatchSource:0}: Error finding container 95388d86307b3d147473495469efaabf4ef1880cc066a8dbe38067118a7f8c29: Status 404 returned error can't find the container with id 95388d86307b3d147473495469efaabf4ef1880cc066a8dbe38067118a7f8c29
Jul 29 17:53:36.165783 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:36.165755    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8x4dk" event={"ID":"5e92be4a-f9b0-4f36-8a4d-09ae165c21e4","Type":"ContainerStarted","Data":"95388d86307b3d147473495469efaabf4ef1880cc066a8dbe38067118a7f8c29"}
Jul 29 17:53:37.171999 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:37.171960    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8x4dk" event={"ID":"5e92be4a-f9b0-4f36-8a4d-09ae165c21e4","Type":"ContainerStarted","Data":"100b80d9468f4bfbf4b271438b7f726bf0aec08e099cc88888467279d4d2749a"}
Jul 29 17:53:38.178241 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:38.178208    2557 generic.go:356] "Generic (PLEG): container finished" podID="5e92be4a-f9b0-4f36-8a4d-09ae165c21e4" containerID="100b80d9468f4bfbf4b271438b7f726bf0aec08e099cc88888467279d4d2749a" exitCode=0
Jul 29 17:53:38.178623 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:38.178282    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8x4dk" event={"ID":"5e92be4a-f9b0-4f36-8a4d-09ae165c21e4","Type":"ContainerDied","Data":"100b80d9468f4bfbf4b271438b7f726bf0aec08e099cc88888467279d4d2749a"}
Jul 29 17:53:38.178623 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:38.178333    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8x4dk" event={"ID":"5e92be4a-f9b0-4f36-8a4d-09ae165c21e4","Type":"ContainerStarted","Data":"37033ffe54d0ccb22ee2c1535afd08046e36e41af8520426ae2b453e4bcf7868"}
Jul 29 17:53:38.178623 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:38.178350    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8x4dk" event={"ID":"5e92be4a-f9b0-4f36-8a4d-09ae165c21e4","Type":"ContainerStarted","Data":"cd1d8a9129552d0e24a1f01034c56822f180a0106505bda2f6fb7946b48acecf"}
Jul 29 17:53:38.199033 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:38.198990    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-8x4dk" podStartSLOduration=2.261342793 podStartE2EDuration="3.1989757s" totalImagesPullingTime="866.945935ms" totalInitContainerRuntime="70.686972ms" isStatefulPod=true podCreationTimestamp="2026-07-29 17:53:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:53:38.196809932 +0000 UTC m=+98.066032739" watchObservedRunningTime="2026-07-29 17:53:38.1989757 +0000 UTC m=+98.068198507"
Jul 29 17:53:38.655044 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:38.655015    2557 scope.go:117] "RemoveContainer" containerID="91b67afd574b1e0bfef430e95ae409e929649019adafa57585033864a831e850"
Jul 29 17:53:39.184239 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:39.184206    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gg2wj_2bf0bb71-4047-45a7-bd8e-5d6efabd26bd/console-operator/2.log"
Jul 29 17:53:39.184618 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:39.184334    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-gg2wj" event={"ID":"2bf0bb71-4047-45a7-bd8e-5d6efabd26bd","Type":"ContainerStarted","Data":"0a9f40734ae19a74f4c07722c07b957f22358ad0f35e7661534151a95ba5295d"}
Jul 29 17:53:39.184852 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:39.184824    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-gg2wj"
Jul 29 17:53:39.193649 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:39.193580    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-gg2wj"
Jul 29 17:53:39.207113 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:39.207047    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-gg2wj" podStartSLOduration=51.641736134 podStartE2EDuration="58.207023733s" totalImagesPullingTime="6.565287599s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:52:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:53:39.204692095 +0000 UTC m=+99.073914928" watchObservedRunningTime="2026-07-29 17:53:39.207023733 +0000 UTC m=+99.076246542"
Jul 29 17:53:40.025865 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:40.025835    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-svfrm"
Jul 29 17:53:44.677790 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:44.677759    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6774b48fd9-hrjxr"]
Jul 29 17:53:50.079416 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:50.079386    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-699fdb8b9b-2sf78"]
Jul 29 17:53:50.099930 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:50.099895    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-699fdb8b9b-2sf78"]
Jul 29 17:53:50.100081 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:50.099992    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-699fdb8b9b-2sf78"
Jul 29 17:53:50.102821 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:50.102793    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 29 17:53:50.102947 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:50.102796    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-8lgzp\""
Jul 29 17:53:50.103999 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:50.103979    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 29 17:53:50.104107 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:50.103980    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 29 17:53:50.104107 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:50.104003    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 29 17:53:50.104220 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:50.104164    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 29 17:53:50.104220 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:50.104039    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 29 17:53:50.104405 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:50.104388    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 29 17:53:50.182604 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:50.182569    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e122fa40-8c49-4b18-ac40-6d69288e2dc9-console-serving-cert\") pod \"console-699fdb8b9b-2sf78\" (UID: \"e122fa40-8c49-4b18-ac40-6d69288e2dc9\") " pod="openshift-console/console-699fdb8b9b-2sf78"
Jul 29 17:53:50.182604 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:50.182604    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e122fa40-8c49-4b18-ac40-6d69288e2dc9-oauth-serving-cert\") pod \"console-699fdb8b9b-2sf78\" (UID: \"e122fa40-8c49-4b18-ac40-6d69288e2dc9\") " pod="openshift-console/console-699fdb8b9b-2sf78"
Jul 29 17:53:50.182896 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:50.182623    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e122fa40-8c49-4b18-ac40-6d69288e2dc9-console-oauth-config\") pod \"console-699fdb8b9b-2sf78\" (UID: \"e122fa40-8c49-4b18-ac40-6d69288e2dc9\") " pod="openshift-console/console-699fdb8b9b-2sf78"
Jul 29 17:53:50.182896 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:50.182643    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e122fa40-8c49-4b18-ac40-6d69288e2dc9-console-config\") pod \"console-699fdb8b9b-2sf78\" (UID: \"e122fa40-8c49-4b18-ac40-6d69288e2dc9\") " pod="openshift-console/console-699fdb8b9b-2sf78"
Jul 29 17:53:50.182896 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:50.182816    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e122fa40-8c49-4b18-ac40-6d69288e2dc9-service-ca\") pod \"console-699fdb8b9b-2sf78\" (UID: \"e122fa40-8c49-4b18-ac40-6d69288e2dc9\") " pod="openshift-console/console-699fdb8b9b-2sf78"
Jul 29 17:53:50.182896 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:50.182872    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whcv8\" (UniqueName: \"kubernetes.io/projected/e122fa40-8c49-4b18-ac40-6d69288e2dc9-kube-api-access-whcv8\") pod \"console-699fdb8b9b-2sf78\" (UID: \"e122fa40-8c49-4b18-ac40-6d69288e2dc9\") " pod="openshift-console/console-699fdb8b9b-2sf78"
Jul 29 17:53:50.284130 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:50.284081    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e122fa40-8c49-4b18-ac40-6d69288e2dc9-console-serving-cert\") pod \"console-699fdb8b9b-2sf78\" (UID: \"e122fa40-8c49-4b18-ac40-6d69288e2dc9\") " pod="openshift-console/console-699fdb8b9b-2sf78"
Jul 29 17:53:50.284318 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:50.284144    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e122fa40-8c49-4b18-ac40-6d69288e2dc9-oauth-serving-cert\") pod \"console-699fdb8b9b-2sf78\" (UID: \"e122fa40-8c49-4b18-ac40-6d69288e2dc9\") " pod="openshift-console/console-699fdb8b9b-2sf78"
Jul 29 17:53:50.284318 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:50.284171    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e122fa40-8c49-4b18-ac40-6d69288e2dc9-console-oauth-config\") pod \"console-699fdb8b9b-2sf78\" (UID: \"e122fa40-8c49-4b18-ac40-6d69288e2dc9\") " pod="openshift-console/console-699fdb8b9b-2sf78"
Jul 29 17:53:50.284318 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:50.284194    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e122fa40-8c49-4b18-ac40-6d69288e2dc9-console-config\") pod \"console-699fdb8b9b-2sf78\" (UID: \"e122fa40-8c49-4b18-ac40-6d69288e2dc9\") " pod="openshift-console/console-699fdb8b9b-2sf78"
Jul 29 17:53:50.284318 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:50.284260    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e122fa40-8c49-4b18-ac40-6d69288e2dc9-service-ca\") pod \"console-699fdb8b9b-2sf78\" (UID: \"e122fa40-8c49-4b18-ac40-6d69288e2dc9\") " pod="openshift-console/console-699fdb8b9b-2sf78"
Jul 29 17:53:50.284318 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:50.284315    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whcv8\" (UniqueName: \"kubernetes.io/projected/e122fa40-8c49-4b18-ac40-6d69288e2dc9-kube-api-access-whcv8\") pod \"console-699fdb8b9b-2sf78\" (UID: \"e122fa40-8c49-4b18-ac40-6d69288e2dc9\") " pod="openshift-console/console-699fdb8b9b-2sf78"
Jul 29 17:53:50.285053 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:50.285026    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e122fa40-8c49-4b18-ac40-6d69288e2dc9-oauth-serving-cert\") pod \"console-699fdb8b9b-2sf78\" (UID: \"e122fa40-8c49-4b18-ac40-6d69288e2dc9\") " pod="openshift-console/console-699fdb8b9b-2sf78"
Jul 29 17:53:50.285168 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:50.285051    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e122fa40-8c49-4b18-ac40-6d69288e2dc9-console-config\") pod \"console-699fdb8b9b-2sf78\" (UID: \"e122fa40-8c49-4b18-ac40-6d69288e2dc9\") " pod="openshift-console/console-699fdb8b9b-2sf78"
Jul 29 17:53:50.285287 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:50.285265    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e122fa40-8c49-4b18-ac40-6d69288e2dc9-service-ca\") pod \"console-699fdb8b9b-2sf78\" (UID: \"e122fa40-8c49-4b18-ac40-6d69288e2dc9\") " pod="openshift-console/console-699fdb8b9b-2sf78"
Jul 29 17:53:50.286787 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:50.286761    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e122fa40-8c49-4b18-ac40-6d69288e2dc9-console-oauth-config\") pod \"console-699fdb8b9b-2sf78\" (UID: \"e122fa40-8c49-4b18-ac40-6d69288e2dc9\") " pod="openshift-console/console-699fdb8b9b-2sf78"
Jul 29 17:53:50.287047 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:50.287021    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e122fa40-8c49-4b18-ac40-6d69288e2dc9-console-serving-cert\") pod \"console-699fdb8b9b-2sf78\" (UID: \"e122fa40-8c49-4b18-ac40-6d69288e2dc9\") " pod="openshift-console/console-699fdb8b9b-2sf78"
Jul 29 17:53:50.292687 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:50.292666    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whcv8\" (UniqueName: \"kubernetes.io/projected/e122fa40-8c49-4b18-ac40-6d69288e2dc9-kube-api-access-whcv8\") pod \"console-699fdb8b9b-2sf78\" (UID: \"e122fa40-8c49-4b18-ac40-6d69288e2dc9\") " pod="openshift-console/console-699fdb8b9b-2sf78"
Jul 29 17:53:50.409865 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:50.409794    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-699fdb8b9b-2sf78"
Jul 29 17:53:50.579882 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:50.579859    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-699fdb8b9b-2sf78"]
Jul 29 17:53:50.581762 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:53:50.581719    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode122fa40_8c49_4b18_ac40_6d69288e2dc9.slice/crio-d8f26548aca9d9fa579880683feaa844748ee5a298917998b050476d28a4d75a WatchSource:0}: Error finding container d8f26548aca9d9fa579880683feaa844748ee5a298917998b050476d28a4d75a: Status 404 returned error can't find the container with id d8f26548aca9d9fa579880683feaa844748ee5a298917998b050476d28a4d75a
Jul 29 17:53:51.228523 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:51.228477    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-699fdb8b9b-2sf78" event={"ID":"e122fa40-8c49-4b18-ac40-6d69288e2dc9","Type":"ContainerStarted","Data":"d8f26548aca9d9fa579880683feaa844748ee5a298917998b050476d28a4d75a"}
Jul 29 17:53:55.245687 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:55.245651    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-699fdb8b9b-2sf78" event={"ID":"e122fa40-8c49-4b18-ac40-6d69288e2dc9","Type":"ContainerStarted","Data":"739be0a489e1d4e5743770b55a7d145d147b5468bf3e70b18b5adaede8c13940"}
Jul 29 17:53:55.264226 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:53:55.264176    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-699fdb8b9b-2sf78" podStartSLOduration=1.663311512 podStartE2EDuration="5.264161112s" totalImagesPullingTime="3.6008496s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:53:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:53:55.263976773 +0000 UTC m=+115.133199581" watchObservedRunningTime="2026-07-29 17:53:55.264161112 +0000 UTC m=+115.133383924"
Jul 29 17:54:00.300014 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:00.299981    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-69cd5dffd8-v7mh4"]
Jul 29 17:54:00.313991 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:00.313951    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-69cd5dffd8-v7mh4"]
Jul 29 17:54:00.313991 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:00.313972    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69cd5dffd8-v7mh4"
Jul 29 17:54:00.323190 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:00.323172    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 29 17:54:00.410610 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:00.410580    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-699fdb8b9b-2sf78"
Jul 29 17:54:00.410795 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:00.410632    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-699fdb8b9b-2sf78"
Jul 29 17:54:00.415390 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:00.415368    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-699fdb8b9b-2sf78"
Jul 29 17:54:00.462191 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:00.462163    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/709d21d9-aa24-484b-9456-84ef989351ee-service-ca\") pod \"console-69cd5dffd8-v7mh4\" (UID: \"709d21d9-aa24-484b-9456-84ef989351ee\") " pod="openshift-console/console-69cd5dffd8-v7mh4"
Jul 29 17:54:00.462315 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:00.462195    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/709d21d9-aa24-484b-9456-84ef989351ee-console-config\") pod \"console-69cd5dffd8-v7mh4\" (UID: \"709d21d9-aa24-484b-9456-84ef989351ee\") " pod="openshift-console/console-69cd5dffd8-v7mh4"
Jul 29 17:54:00.462315 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:00.462272    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/709d21d9-aa24-484b-9456-84ef989351ee-console-serving-cert\") pod \"console-69cd5dffd8-v7mh4\" (UID: \"709d21d9-aa24-484b-9456-84ef989351ee\") " pod="openshift-console/console-69cd5dffd8-v7mh4"
Jul 29 17:54:00.462315 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:00.462298    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwv67\" (UniqueName: \"kubernetes.io/projected/709d21d9-aa24-484b-9456-84ef989351ee-kube-api-access-wwv67\") pod \"console-69cd5dffd8-v7mh4\" (UID: \"709d21d9-aa24-484b-9456-84ef989351ee\") " pod="openshift-console/console-69cd5dffd8-v7mh4"
Jul 29 17:54:00.462315 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:00.462315    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/709d21d9-aa24-484b-9456-84ef989351ee-console-oauth-config\") pod \"console-69cd5dffd8-v7mh4\" (UID: \"709d21d9-aa24-484b-9456-84ef989351ee\") " pod="openshift-console/console-69cd5dffd8-v7mh4"
Jul 29 17:54:00.462467 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:00.462356    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/709d21d9-aa24-484b-9456-84ef989351ee-oauth-serving-cert\") pod \"console-69cd5dffd8-v7mh4\" (UID: \"709d21d9-aa24-484b-9456-84ef989351ee\") " pod="openshift-console/console-69cd5dffd8-v7mh4"
Jul 29 17:54:00.462467 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:00.462398    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/709d21d9-aa24-484b-9456-84ef989351ee-trusted-ca-bundle\") pod \"console-69cd5dffd8-v7mh4\" (UID: \"709d21d9-aa24-484b-9456-84ef989351ee\") " pod="openshift-console/console-69cd5dffd8-v7mh4"
Jul 29 17:54:00.562818 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:00.562750    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/709d21d9-aa24-484b-9456-84ef989351ee-service-ca\") pod \"console-69cd5dffd8-v7mh4\" (UID: \"709d21d9-aa24-484b-9456-84ef989351ee\") " pod="openshift-console/console-69cd5dffd8-v7mh4"
Jul 29 17:54:00.562818 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:00.562799    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/709d21d9-aa24-484b-9456-84ef989351ee-console-config\") pod \"console-69cd5dffd8-v7mh4\" (UID: \"709d21d9-aa24-484b-9456-84ef989351ee\") " pod="openshift-console/console-69cd5dffd8-v7mh4"
Jul 29 17:54:00.563005 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:00.562929    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/709d21d9-aa24-484b-9456-84ef989351ee-console-serving-cert\") pod \"console-69cd5dffd8-v7mh4\" (UID: \"709d21d9-aa24-484b-9456-84ef989351ee\") " pod="openshift-console/console-69cd5dffd8-v7mh4"
Jul 29 17:54:00.563436 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:00.563407    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/709d21d9-aa24-484b-9456-84ef989351ee-service-ca\") pod \"console-69cd5dffd8-v7mh4\" (UID: \"709d21d9-aa24-484b-9456-84ef989351ee\") " pod="openshift-console/console-69cd5dffd8-v7mh4"
Jul 29 17:54:00.563436 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:00.563422    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwv67\" (UniqueName: \"kubernetes.io/projected/709d21d9-aa24-484b-9456-84ef989351ee-kube-api-access-wwv67\") pod \"console-69cd5dffd8-v7mh4\" (UID: \"709d21d9-aa24-484b-9456-84ef989351ee\") " pod="openshift-console/console-69cd5dffd8-v7mh4"
Jul 29 17:54:00.563617 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:00.563474    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/709d21d9-aa24-484b-9456-84ef989351ee-console-oauth-config\") pod \"console-69cd5dffd8-v7mh4\" (UID: \"709d21d9-aa24-484b-9456-84ef989351ee\") " pod="openshift-console/console-69cd5dffd8-v7mh4"
Jul 29 17:54:00.563617 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:00.563501    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/709d21d9-aa24-484b-9456-84ef989351ee-oauth-serving-cert\") pod \"console-69cd5dffd8-v7mh4\" (UID: \"709d21d9-aa24-484b-9456-84ef989351ee\") " pod="openshift-console/console-69cd5dffd8-v7mh4"
Jul 29 17:54:00.563617 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:00.563527    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/709d21d9-aa24-484b-9456-84ef989351ee-console-config\") pod \"console-69cd5dffd8-v7mh4\" (UID: \"709d21d9-aa24-484b-9456-84ef989351ee\") " pod="openshift-console/console-69cd5dffd8-v7mh4"
Jul 29 17:54:00.563617 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:00.563540    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/709d21d9-aa24-484b-9456-84ef989351ee-trusted-ca-bundle\") pod \"console-69cd5dffd8-v7mh4\" (UID: \"709d21d9-aa24-484b-9456-84ef989351ee\") " pod="openshift-console/console-69cd5dffd8-v7mh4"
Jul 29 17:54:00.564335 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:00.564311    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/709d21d9-aa24-484b-9456-84ef989351ee-oauth-serving-cert\") pod \"console-69cd5dffd8-v7mh4\" (UID: \"709d21d9-aa24-484b-9456-84ef989351ee\") " pod="openshift-console/console-69cd5dffd8-v7mh4"
Jul 29 17:54:00.566182 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:00.566149    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/709d21d9-aa24-484b-9456-84ef989351ee-console-serving-cert\") pod \"console-69cd5dffd8-v7mh4\" (UID: \"709d21d9-aa24-484b-9456-84ef989351ee\") " pod="openshift-console/console-69cd5dffd8-v7mh4"
Jul 29 17:54:00.566303 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:00.566286    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/709d21d9-aa24-484b-9456-84ef989351ee-console-oauth-config\") pod \"console-69cd5dffd8-v7mh4\" (UID: \"709d21d9-aa24-484b-9456-84ef989351ee\") " pod="openshift-console/console-69cd5dffd8-v7mh4"
Jul 29 17:54:00.571662 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:00.571641    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 29 17:54:00.572672 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:00.572651    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwv67\" (UniqueName: \"kubernetes.io/projected/709d21d9-aa24-484b-9456-84ef989351ee-kube-api-access-wwv67\") pod \"console-69cd5dffd8-v7mh4\" (UID: \"709d21d9-aa24-484b-9456-84ef989351ee\") " pod="openshift-console/console-69cd5dffd8-v7mh4"
Jul 29 17:54:00.574500 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:00.574480    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/709d21d9-aa24-484b-9456-84ef989351ee-trusted-ca-bundle\") pod \"console-69cd5dffd8-v7mh4\" (UID: \"709d21d9-aa24-484b-9456-84ef989351ee\") " pod="openshift-console/console-69cd5dffd8-v7mh4"
Jul 29 17:54:00.623459 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:00.623429    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69cd5dffd8-v7mh4"
Jul 29 17:54:00.756592 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:00.756568    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-69cd5dffd8-v7mh4"]
Jul 29 17:54:00.759222 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:54:00.759191    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod709d21d9_aa24_484b_9456_84ef989351ee.slice/crio-7f4d5237cfac220d8f1e06209a24e7f602ddda0430a7dd27e485c24f03ba9dad WatchSource:0}: Error finding container 7f4d5237cfac220d8f1e06209a24e7f602ddda0430a7dd27e485c24f03ba9dad: Status 404 returned error can't find the container with id 7f4d5237cfac220d8f1e06209a24e7f602ddda0430a7dd27e485c24f03ba9dad
Jul 29 17:54:01.269351 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:01.269248    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69cd5dffd8-v7mh4" event={"ID":"709d21d9-aa24-484b-9456-84ef989351ee","Type":"ContainerStarted","Data":"398b39222c9e3896471b387ff62739205d37e6fd2ec7be80dfa43a9e6c91d1b4"}
Jul 29 17:54:01.269351 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:01.269331    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69cd5dffd8-v7mh4" event={"ID":"709d21d9-aa24-484b-9456-84ef989351ee","Type":"ContainerStarted","Data":"7f4d5237cfac220d8f1e06209a24e7f602ddda0430a7dd27e485c24f03ba9dad"}
Jul 29 17:54:01.273004 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:01.272980    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-699fdb8b9b-2sf78"
Jul 29 17:54:01.290953 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:01.290909    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-69cd5dffd8-v7mh4" podStartSLOduration=1.290896137 podStartE2EDuration="1.290896137s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:54:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:54:01.290062021 +0000 UTC m=+121.159284829" watchObservedRunningTime="2026-07-29 17:54:01.290896137 +0000 UTC m=+121.160119008"
Jul 29 17:54:05.286093 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:05.286056    2557 generic.go:356] "Generic (PLEG): container finished" podID="a9042be4-9aa4-4bc5-b9b8-364a7f782353" containerID="49e400a66020ca8c5f2c93ab06e319b3b36fd28b35ee109017ddc41c13c0c480" exitCode=0
Jul 29 17:54:05.286479 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:05.286133    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-4j8zv" event={"ID":"a9042be4-9aa4-4bc5-b9b8-364a7f782353","Type":"ContainerDied","Data":"49e400a66020ca8c5f2c93ab06e319b3b36fd28b35ee109017ddc41c13c0c480"}
Jul 29 17:54:05.286527 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:05.286499    2557 scope.go:117] "RemoveContainer" containerID="49e400a66020ca8c5f2c93ab06e319b3b36fd28b35ee109017ddc41c13c0c480"
Jul 29 17:54:06.291269 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:06.291236    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-4j8zv" event={"ID":"a9042be4-9aa4-4bc5-b9b8-364a7f782353","Type":"ContainerStarted","Data":"a73e2179eb55430fef9585ba7d4f27f1ca58617cf190dd9500b609e8b444d3dc"}
Jul 29 17:54:09.696813 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:09.696747    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6774b48fd9-hrjxr" podUID="09c3a81a-22ca-4700-af82-9c7e86f77f07" containerName="registry" containerID="cri-o://22713f1a107ff9e16a02849f8edcc98ba754888942e394a60bdca253251996a1" gracePeriod=30
Jul 29 17:54:09.939008 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:09.938984    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6774b48fd9-hrjxr"
Jul 29 17:54:10.041318 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:10.041292    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/09c3a81a-22ca-4700-af82-9c7e86f77f07-bound-sa-token\") pod \"09c3a81a-22ca-4700-af82-9c7e86f77f07\" (UID: \"09c3a81a-22ca-4700-af82-9c7e86f77f07\") "
Jul 29 17:54:10.041455 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:10.041323    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/09c3a81a-22ca-4700-af82-9c7e86f77f07-registry-certificates\") pod \"09c3a81a-22ca-4700-af82-9c7e86f77f07\" (UID: \"09c3a81a-22ca-4700-af82-9c7e86f77f07\") "
Jul 29 17:54:10.041455 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:10.041359    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/09c3a81a-22ca-4700-af82-9c7e86f77f07-trusted-ca\") pod \"09c3a81a-22ca-4700-af82-9c7e86f77f07\" (UID: \"09c3a81a-22ca-4700-af82-9c7e86f77f07\") "
Jul 29 17:54:10.041455 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:10.041410    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7cldh\" (UniqueName: \"kubernetes.io/projected/09c3a81a-22ca-4700-af82-9c7e86f77f07-kube-api-access-7cldh\") pod \"09c3a81a-22ca-4700-af82-9c7e86f77f07\" (UID: \"09c3a81a-22ca-4700-af82-9c7e86f77f07\") "
Jul 29 17:54:10.041455 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:10.041430    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/09c3a81a-22ca-4700-af82-9c7e86f77f07-installation-pull-secrets\") pod \"09c3a81a-22ca-4700-af82-9c7e86f77f07\" (UID: \"09c3a81a-22ca-4700-af82-9c7e86f77f07\") "
Jul 29 17:54:10.041455 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:10.041449    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/09c3a81a-22ca-4700-af82-9c7e86f77f07-ca-trust-extracted\") pod \"09c3a81a-22ca-4700-af82-9c7e86f77f07\" (UID: \"09c3a81a-22ca-4700-af82-9c7e86f77f07\") "
Jul 29 17:54:10.041761 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:10.041558    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/09c3a81a-22ca-4700-af82-9c7e86f77f07-image-registry-private-configuration\") pod \"09c3a81a-22ca-4700-af82-9c7e86f77f07\" (UID: \"09c3a81a-22ca-4700-af82-9c7e86f77f07\") "
Jul 29 17:54:10.041761 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:10.041609    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/09c3a81a-22ca-4700-af82-9c7e86f77f07-registry-tls\") pod \"09c3a81a-22ca-4700-af82-9c7e86f77f07\" (UID: \"09c3a81a-22ca-4700-af82-9c7e86f77f07\") "
Jul 29 17:54:10.041875 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:10.041776    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09c3a81a-22ca-4700-af82-9c7e86f77f07-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "09c3a81a-22ca-4700-af82-9c7e86f77f07" (UID: "09c3a81a-22ca-4700-af82-9c7e86f77f07"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 17:54:10.041933 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:10.041922    2557 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/09c3a81a-22ca-4700-af82-9c7e86f77f07-registry-certificates\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 17:54:10.042228 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:10.042203    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09c3a81a-22ca-4700-af82-9c7e86f77f07-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "09c3a81a-22ca-4700-af82-9c7e86f77f07" (UID: "09c3a81a-22ca-4700-af82-9c7e86f77f07"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 17:54:10.044113 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:10.044086    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09c3a81a-22ca-4700-af82-9c7e86f77f07-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "09c3a81a-22ca-4700-af82-9c7e86f77f07" (UID: "09c3a81a-22ca-4700-af82-9c7e86f77f07"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 17:54:10.044335 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:10.044314    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09c3a81a-22ca-4700-af82-9c7e86f77f07-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "09c3a81a-22ca-4700-af82-9c7e86f77f07" (UID: "09c3a81a-22ca-4700-af82-9c7e86f77f07"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 17:54:10.044431 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:10.044407    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09c3a81a-22ca-4700-af82-9c7e86f77f07-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "09c3a81a-22ca-4700-af82-9c7e86f77f07" (UID: "09c3a81a-22ca-4700-af82-9c7e86f77f07"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 17:54:10.044485 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:10.044449    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09c3a81a-22ca-4700-af82-9c7e86f77f07-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "09c3a81a-22ca-4700-af82-9c7e86f77f07" (UID: "09c3a81a-22ca-4700-af82-9c7e86f77f07"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 17:54:10.044535 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:10.044492    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09c3a81a-22ca-4700-af82-9c7e86f77f07-kube-api-access-7cldh" (OuterVolumeSpecName: "kube-api-access-7cldh") pod "09c3a81a-22ca-4700-af82-9c7e86f77f07" (UID: "09c3a81a-22ca-4700-af82-9c7e86f77f07"). InnerVolumeSpecName "kube-api-access-7cldh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 17:54:10.050361 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:10.050340    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09c3a81a-22ca-4700-af82-9c7e86f77f07-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "09c3a81a-22ca-4700-af82-9c7e86f77f07" (UID: "09c3a81a-22ca-4700-af82-9c7e86f77f07"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 17:54:10.142767 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:10.142743    2557 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/09c3a81a-22ca-4700-af82-9c7e86f77f07-ca-trust-extracted\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 17:54:10.142767 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:10.142766    2557 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/09c3a81a-22ca-4700-af82-9c7e86f77f07-image-registry-private-configuration\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 17:54:10.142921 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:10.142777    2557 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/09c3a81a-22ca-4700-af82-9c7e86f77f07-registry-tls\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 17:54:10.142921 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:10.142787    2557 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/09c3a81a-22ca-4700-af82-9c7e86f77f07-bound-sa-token\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 17:54:10.142921 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:10.142795    2557 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/09c3a81a-22ca-4700-af82-9c7e86f77f07-trusted-ca\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 17:54:10.142921 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:10.142803    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7cldh\" (UniqueName: \"kubernetes.io/projected/09c3a81a-22ca-4700-af82-9c7e86f77f07-kube-api-access-7cldh\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 17:54:10.142921 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:10.142811    2557 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/09c3a81a-22ca-4700-af82-9c7e86f77f07-installation-pull-secrets\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 17:54:10.306532 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:10.306453    2557 generic.go:356] "Generic (PLEG): container finished" podID="09c3a81a-22ca-4700-af82-9c7e86f77f07" containerID="22713f1a107ff9e16a02849f8edcc98ba754888942e394a60bdca253251996a1" exitCode=0
Jul 29 17:54:10.306532 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:10.306502    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6774b48fd9-hrjxr"
Jul 29 17:54:10.306704 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:10.306535    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6774b48fd9-hrjxr" event={"ID":"09c3a81a-22ca-4700-af82-9c7e86f77f07","Type":"ContainerDied","Data":"22713f1a107ff9e16a02849f8edcc98ba754888942e394a60bdca253251996a1"}
Jul 29 17:54:10.306704 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:10.306586    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6774b48fd9-hrjxr" event={"ID":"09c3a81a-22ca-4700-af82-9c7e86f77f07","Type":"ContainerDied","Data":"0ea0b496fd413a14aedb9fb62de58bd94a7b8d333cf15f3f22051ffbecbcb0a5"}
Jul 29 17:54:10.306704 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:10.306607    2557 scope.go:117] "RemoveContainer" containerID="22713f1a107ff9e16a02849f8edcc98ba754888942e394a60bdca253251996a1"
Jul 29 17:54:10.315636 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:10.315621    2557 scope.go:117] "RemoveContainer" containerID="22713f1a107ff9e16a02849f8edcc98ba754888942e394a60bdca253251996a1"
Jul 29 17:54:10.315916 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:54:10.315898    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22713f1a107ff9e16a02849f8edcc98ba754888942e394a60bdca253251996a1\": container with ID starting with 22713f1a107ff9e16a02849f8edcc98ba754888942e394a60bdca253251996a1 not found: ID does not exist" containerID="22713f1a107ff9e16a02849f8edcc98ba754888942e394a60bdca253251996a1"
Jul 29 17:54:10.315969 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:10.315925    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22713f1a107ff9e16a02849f8edcc98ba754888942e394a60bdca253251996a1"} err="failed to get container status \"22713f1a107ff9e16a02849f8edcc98ba754888942e394a60bdca253251996a1\": rpc error: code = NotFound desc = could not find container \"22713f1a107ff9e16a02849f8edcc98ba754888942e394a60bdca253251996a1\": container with ID starting with 22713f1a107ff9e16a02849f8edcc98ba754888942e394a60bdca253251996a1 not found: ID does not exist"
Jul 29 17:54:10.328587 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:10.328569    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6774b48fd9-hrjxr"]
Jul 29 17:54:10.334138 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:10.334118    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6774b48fd9-hrjxr"]
Jul 29 17:54:10.623964 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:10.623874    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-69cd5dffd8-v7mh4"
Jul 29 17:54:10.623964 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:10.623934    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-69cd5dffd8-v7mh4"
Jul 29 17:54:10.630611 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:10.630584    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-69cd5dffd8-v7mh4"
Jul 29 17:54:10.652225 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:10.652192    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09c3a81a-22ca-4700-af82-9c7e86f77f07" path="/var/lib/kubelet/pods/09c3a81a-22ca-4700-af82-9c7e86f77f07/volumes"
Jul 29 17:54:11.314859 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:11.314828    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-69cd5dffd8-v7mh4"
Jul 29 17:54:11.364054 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:11.364018    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-699fdb8b9b-2sf78"]
Jul 29 17:54:19.341634 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:19.341560    2557 generic.go:356] "Generic (PLEG): container finished" podID="a91229a9-7326-45f0-9bed-57c0f1868780" containerID="e9526e567f8bd964658ac572513d265bd31493c64fc1d6cc6d9dc37847b647c7" exitCode=0
Jul 29 17:54:19.342027 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:19.341630    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-hhwbp" event={"ID":"a91229a9-7326-45f0-9bed-57c0f1868780","Type":"ContainerDied","Data":"e9526e567f8bd964658ac572513d265bd31493c64fc1d6cc6d9dc37847b647c7"}
Jul 29 17:54:19.342027 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:19.341963    2557 scope.go:117] "RemoveContainer" containerID="e9526e567f8bd964658ac572513d265bd31493c64fc1d6cc6d9dc37847b647c7"
Jul 29 17:54:19.343343 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:19.343313    2557 generic.go:356] "Generic (PLEG): container finished" podID="fcb6432a-cde0-4f40-9e00-2523c43ca2c8" containerID="544d4adf675f1719be6927feebcafa2607a757fb323f000d524a6550b41317c7" exitCode=0
Jul 29 17:54:19.343463 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:19.343385    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-2fks5" event={"ID":"fcb6432a-cde0-4f40-9e00-2523c43ca2c8","Type":"ContainerDied","Data":"544d4adf675f1719be6927feebcafa2607a757fb323f000d524a6550b41317c7"}
Jul 29 17:54:19.343695 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:19.343677    2557 scope.go:117] "RemoveContainer" containerID="544d4adf675f1719be6927feebcafa2607a757fb323f000d524a6550b41317c7"
Jul 29 17:54:20.349853 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:20.349818    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-2fks5" event={"ID":"fcb6432a-cde0-4f40-9e00-2523c43ca2c8","Type":"ContainerStarted","Data":"e3a480d6b4f7c31d0cc1ccc0af1f6b2509cf17834ea477251ffe0a2bd4055847"}
Jul 29 17:54:20.351387 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:20.351369    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-hhwbp" event={"ID":"a91229a9-7326-45f0-9bed-57c0f1868780","Type":"ContainerStarted","Data":"03cfba536e5a61c9b9d93f14a1e440b67948f83a68fb0343e05c664a1755eb53"}
Jul 29 17:54:36.386910 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:36.386859    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-699fdb8b9b-2sf78" podUID="e122fa40-8c49-4b18-ac40-6d69288e2dc9" containerName="console" containerID="cri-o://739be0a489e1d4e5743770b55a7d145d147b5468bf3e70b18b5adaede8c13940" gracePeriod=15
Jul 29 17:54:36.631578 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:36.631554    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-699fdb8b9b-2sf78"
Jul 29 17:54:36.736951 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:36.736888    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e122fa40-8c49-4b18-ac40-6d69288e2dc9-console-oauth-config\") pod \"e122fa40-8c49-4b18-ac40-6d69288e2dc9\" (UID: \"e122fa40-8c49-4b18-ac40-6d69288e2dc9\") "
Jul 29 17:54:36.736951 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:36.736919    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e122fa40-8c49-4b18-ac40-6d69288e2dc9-console-serving-cert\") pod \"e122fa40-8c49-4b18-ac40-6d69288e2dc9\" (UID: \"e122fa40-8c49-4b18-ac40-6d69288e2dc9\") "
Jul 29 17:54:36.736951 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:36.736946    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e122fa40-8c49-4b18-ac40-6d69288e2dc9-service-ca\") pod \"e122fa40-8c49-4b18-ac40-6d69288e2dc9\" (UID: \"e122fa40-8c49-4b18-ac40-6d69288e2dc9\") "
Jul 29 17:54:36.737218 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:36.736975    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e122fa40-8c49-4b18-ac40-6d69288e2dc9-console-config\") pod \"e122fa40-8c49-4b18-ac40-6d69288e2dc9\" (UID: \"e122fa40-8c49-4b18-ac40-6d69288e2dc9\") "
Jul 29 17:54:36.737218 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:36.737008    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e122fa40-8c49-4b18-ac40-6d69288e2dc9-oauth-serving-cert\") pod \"e122fa40-8c49-4b18-ac40-6d69288e2dc9\" (UID: \"e122fa40-8c49-4b18-ac40-6d69288e2dc9\") "
Jul 29 17:54:36.737218 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:36.737042    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-whcv8\" (UniqueName: \"kubernetes.io/projected/e122fa40-8c49-4b18-ac40-6d69288e2dc9-kube-api-access-whcv8\") pod \"e122fa40-8c49-4b18-ac40-6d69288e2dc9\" (UID: \"e122fa40-8c49-4b18-ac40-6d69288e2dc9\") "
Jul 29 17:54:36.737417 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:36.737386    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e122fa40-8c49-4b18-ac40-6d69288e2dc9-console-config" (OuterVolumeSpecName: "console-config") pod "e122fa40-8c49-4b18-ac40-6d69288e2dc9" (UID: "e122fa40-8c49-4b18-ac40-6d69288e2dc9"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 17:54:36.737547 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:36.737392    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e122fa40-8c49-4b18-ac40-6d69288e2dc9-service-ca" (OuterVolumeSpecName: "service-ca") pod "e122fa40-8c49-4b18-ac40-6d69288e2dc9" (UID: "e122fa40-8c49-4b18-ac40-6d69288e2dc9"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 17:54:36.737698 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:36.737587    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e122fa40-8c49-4b18-ac40-6d69288e2dc9-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e122fa40-8c49-4b18-ac40-6d69288e2dc9" (UID: "e122fa40-8c49-4b18-ac40-6d69288e2dc9"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 17:54:36.739346 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:36.739324    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e122fa40-8c49-4b18-ac40-6d69288e2dc9-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e122fa40-8c49-4b18-ac40-6d69288e2dc9" (UID: "e122fa40-8c49-4b18-ac40-6d69288e2dc9"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 17:54:36.739831 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:36.739808    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e122fa40-8c49-4b18-ac40-6d69288e2dc9-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e122fa40-8c49-4b18-ac40-6d69288e2dc9" (UID: "e122fa40-8c49-4b18-ac40-6d69288e2dc9"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 17:54:36.739923 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:36.739846    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e122fa40-8c49-4b18-ac40-6d69288e2dc9-kube-api-access-whcv8" (OuterVolumeSpecName: "kube-api-access-whcv8") pod "e122fa40-8c49-4b18-ac40-6d69288e2dc9" (UID: "e122fa40-8c49-4b18-ac40-6d69288e2dc9"). InnerVolumeSpecName "kube-api-access-whcv8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 17:54:36.838291 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:36.838262    2557 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e122fa40-8c49-4b18-ac40-6d69288e2dc9-console-oauth-config\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 17:54:36.838291 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:36.838285    2557 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e122fa40-8c49-4b18-ac40-6d69288e2dc9-console-serving-cert\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 17:54:36.838434 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:36.838297    2557 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e122fa40-8c49-4b18-ac40-6d69288e2dc9-service-ca\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 17:54:36.838434 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:36.838309    2557 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e122fa40-8c49-4b18-ac40-6d69288e2dc9-console-config\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 17:54:36.838434 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:36.838321    2557 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e122fa40-8c49-4b18-ac40-6d69288e2dc9-oauth-serving-cert\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 17:54:36.838434 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:36.838332    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-whcv8\" (UniqueName: \"kubernetes.io/projected/e122fa40-8c49-4b18-ac40-6d69288e2dc9-kube-api-access-whcv8\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 17:54:37.413241 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:37.413203    2557 generic.go:356] "Generic (PLEG): container finished" podID="e122fa40-8c49-4b18-ac40-6d69288e2dc9" containerID="739be0a489e1d4e5743770b55a7d145d147b5468bf3e70b18b5adaede8c13940" exitCode=0
Jul 29 17:54:37.413668 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:37.413255    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-699fdb8b9b-2sf78"
Jul 29 17:54:37.413668 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:37.413260    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-699fdb8b9b-2sf78" event={"ID":"e122fa40-8c49-4b18-ac40-6d69288e2dc9","Type":"ContainerDied","Data":"739be0a489e1d4e5743770b55a7d145d147b5468bf3e70b18b5adaede8c13940"}
Jul 29 17:54:37.413668 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:37.413301    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-699fdb8b9b-2sf78" event={"ID":"e122fa40-8c49-4b18-ac40-6d69288e2dc9","Type":"ContainerDied","Data":"d8f26548aca9d9fa579880683feaa844748ee5a298917998b050476d28a4d75a"}
Jul 29 17:54:37.413668 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:37.413319    2557 scope.go:117] "RemoveContainer" containerID="739be0a489e1d4e5743770b55a7d145d147b5468bf3e70b18b5adaede8c13940"
Jul 29 17:54:37.423948 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:37.423931    2557 scope.go:117] "RemoveContainer" containerID="739be0a489e1d4e5743770b55a7d145d147b5468bf3e70b18b5adaede8c13940"
Jul 29 17:54:37.424199 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:54:37.424180    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"739be0a489e1d4e5743770b55a7d145d147b5468bf3e70b18b5adaede8c13940\": container with ID starting with 739be0a489e1d4e5743770b55a7d145d147b5468bf3e70b18b5adaede8c13940 not found: ID does not exist" containerID="739be0a489e1d4e5743770b55a7d145d147b5468bf3e70b18b5adaede8c13940"
Jul 29 17:54:37.424250 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:37.424206    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"739be0a489e1d4e5743770b55a7d145d147b5468bf3e70b18b5adaede8c13940"} err="failed to get container status \"739be0a489e1d4e5743770b55a7d145d147b5468bf3e70b18b5adaede8c13940\": rpc error: code = NotFound desc = could not find container \"739be0a489e1d4e5743770b55a7d145d147b5468bf3e70b18b5adaede8c13940\": container with ID starting with 739be0a489e1d4e5743770b55a7d145d147b5468bf3e70b18b5adaede8c13940 not found: ID does not exist"
Jul 29 17:54:37.436974 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:37.436941    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-699fdb8b9b-2sf78"]
Jul 29 17:54:37.443927 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:37.443901    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-699fdb8b9b-2sf78"]
Jul 29 17:54:38.651291 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:54:38.651253    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e122fa40-8c49-4b18-ac40-6d69288e2dc9" path="/var/lib/kubelet/pods/e122fa40-8c49-4b18-ac40-6d69288e2dc9/volumes"
Jul 29 17:55:03.955587 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:03.955551    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-59cf9545bd-dc6b8"]
Jul 29 17:55:03.956480 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:03.956455    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e122fa40-8c49-4b18-ac40-6d69288e2dc9" containerName="console"
Jul 29 17:55:03.956626 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:03.956612    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="e122fa40-8c49-4b18-ac40-6d69288e2dc9" containerName="console"
Jul 29 17:55:03.956787 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:03.956772    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09c3a81a-22ca-4700-af82-9c7e86f77f07" containerName="registry"
Jul 29 17:55:03.956893 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:03.956880    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="09c3a81a-22ca-4700-af82-9c7e86f77f07" containerName="registry"
Jul 29 17:55:03.957025 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:03.957008    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="e122fa40-8c49-4b18-ac40-6d69288e2dc9" containerName="console"
Jul 29 17:55:03.957093 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:03.957029    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="09c3a81a-22ca-4700-af82-9c7e86f77f07" containerName="registry"
Jul 29 17:55:03.962521 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:03.962487    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59cf9545bd-dc6b8"
Jul 29 17:55:03.971633 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:03.971606    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59cf9545bd-dc6b8"]
Jul 29 17:55:04.046090 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:04.046058    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4a24bae0-a6aa-4966-8bdb-0be98412fa2d-console-serving-cert\") pod \"console-59cf9545bd-dc6b8\" (UID: \"4a24bae0-a6aa-4966-8bdb-0be98412fa2d\") " pod="openshift-console/console-59cf9545bd-dc6b8"
Jul 29 17:55:04.046090 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:04.046088    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4a24bae0-a6aa-4966-8bdb-0be98412fa2d-service-ca\") pod \"console-59cf9545bd-dc6b8\" (UID: \"4a24bae0-a6aa-4966-8bdb-0be98412fa2d\") " pod="openshift-console/console-59cf9545bd-dc6b8"
Jul 29 17:55:04.046323 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:04.046107    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4a24bae0-a6aa-4966-8bdb-0be98412fa2d-oauth-serving-cert\") pod \"console-59cf9545bd-dc6b8\" (UID: \"4a24bae0-a6aa-4966-8bdb-0be98412fa2d\") " pod="openshift-console/console-59cf9545bd-dc6b8"
Jul 29 17:55:04.046323 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:04.046142    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4a24bae0-a6aa-4966-8bdb-0be98412fa2d-console-oauth-config\") pod \"console-59cf9545bd-dc6b8\" (UID: \"4a24bae0-a6aa-4966-8bdb-0be98412fa2d\") " pod="openshift-console/console-59cf9545bd-dc6b8"
Jul 29 17:55:04.046323 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:04.046227    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a24bae0-a6aa-4966-8bdb-0be98412fa2d-trusted-ca-bundle\") pod \"console-59cf9545bd-dc6b8\" (UID: \"4a24bae0-a6aa-4966-8bdb-0be98412fa2d\") " pod="openshift-console/console-59cf9545bd-dc6b8"
Jul 29 17:55:04.046323 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:04.046287    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7nmn\" (UniqueName: \"kubernetes.io/projected/4a24bae0-a6aa-4966-8bdb-0be98412fa2d-kube-api-access-k7nmn\") pod \"console-59cf9545bd-dc6b8\" (UID: \"4a24bae0-a6aa-4966-8bdb-0be98412fa2d\") " pod="openshift-console/console-59cf9545bd-dc6b8"
Jul 29 17:55:04.046323 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:04.046320    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4a24bae0-a6aa-4966-8bdb-0be98412fa2d-console-config\") pod \"console-59cf9545bd-dc6b8\" (UID: \"4a24bae0-a6aa-4966-8bdb-0be98412fa2d\") " pod="openshift-console/console-59cf9545bd-dc6b8"
Jul 29 17:55:04.147097 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:04.147067    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4a24bae0-a6aa-4966-8bdb-0be98412fa2d-console-oauth-config\") pod \"console-59cf9545bd-dc6b8\" (UID: \"4a24bae0-a6aa-4966-8bdb-0be98412fa2d\") " pod="openshift-console/console-59cf9545bd-dc6b8"
Jul 29 17:55:04.147097 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:04.147110    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a24bae0-a6aa-4966-8bdb-0be98412fa2d-trusted-ca-bundle\") pod \"console-59cf9545bd-dc6b8\" (UID: \"4a24bae0-a6aa-4966-8bdb-0be98412fa2d\") " pod="openshift-console/console-59cf9545bd-dc6b8"
Jul 29 17:55:04.147322 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:04.147154    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7nmn\" (UniqueName: \"kubernetes.io/projected/4a24bae0-a6aa-4966-8bdb-0be98412fa2d-kube-api-access-k7nmn\") pod \"console-59cf9545bd-dc6b8\" (UID: \"4a24bae0-a6aa-4966-8bdb-0be98412fa2d\") " pod="openshift-console/console-59cf9545bd-dc6b8"
Jul 29 17:55:04.147322 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:04.147175    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4a24bae0-a6aa-4966-8bdb-0be98412fa2d-console-config\") pod \"console-59cf9545bd-dc6b8\" (UID: \"4a24bae0-a6aa-4966-8bdb-0be98412fa2d\") " pod="openshift-console/console-59cf9545bd-dc6b8"
Jul 29 17:55:04.147322 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:04.147203    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4a24bae0-a6aa-4966-8bdb-0be98412fa2d-console-serving-cert\") pod \"console-59cf9545bd-dc6b8\" (UID: \"4a24bae0-a6aa-4966-8bdb-0be98412fa2d\") " pod="openshift-console/console-59cf9545bd-dc6b8"
Jul 29 17:55:04.147322 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:04.147226    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4a24bae0-a6aa-4966-8bdb-0be98412fa2d-service-ca\") pod \"console-59cf9545bd-dc6b8\" (UID: \"4a24bae0-a6aa-4966-8bdb-0be98412fa2d\") " pod="openshift-console/console-59cf9545bd-dc6b8"
Jul 29 17:55:04.147322 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:04.147243    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4a24bae0-a6aa-4966-8bdb-0be98412fa2d-oauth-serving-cert\") pod \"console-59cf9545bd-dc6b8\" (UID: \"4a24bae0-a6aa-4966-8bdb-0be98412fa2d\") " pod="openshift-console/console-59cf9545bd-dc6b8"
Jul 29 17:55:04.148083 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:04.148055    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4a24bae0-a6aa-4966-8bdb-0be98412fa2d-console-config\") pod \"console-59cf9545bd-dc6b8\" (UID: \"4a24bae0-a6aa-4966-8bdb-0be98412fa2d\") " pod="openshift-console/console-59cf9545bd-dc6b8"
Jul 29 17:55:04.148266 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:04.148131    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4a24bae0-a6aa-4966-8bdb-0be98412fa2d-oauth-serving-cert\") pod \"console-59cf9545bd-dc6b8\" (UID: \"4a24bae0-a6aa-4966-8bdb-0be98412fa2d\") " pod="openshift-console/console-59cf9545bd-dc6b8"
Jul 29 17:55:04.148367 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:04.148131    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4a24bae0-a6aa-4966-8bdb-0be98412fa2d-service-ca\") pod \"console-59cf9545bd-dc6b8\" (UID: \"4a24bae0-a6aa-4966-8bdb-0be98412fa2d\") " pod="openshift-console/console-59cf9545bd-dc6b8"
Jul 29 17:55:04.148406 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:04.148373    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a24bae0-a6aa-4966-8bdb-0be98412fa2d-trusted-ca-bundle\") pod \"console-59cf9545bd-dc6b8\" (UID: \"4a24bae0-a6aa-4966-8bdb-0be98412fa2d\") " pod="openshift-console/console-59cf9545bd-dc6b8"
Jul 29 17:55:04.149660 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:04.149643    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4a24bae0-a6aa-4966-8bdb-0be98412fa2d-console-oauth-config\") pod \"console-59cf9545bd-dc6b8\" (UID: \"4a24bae0-a6aa-4966-8bdb-0be98412fa2d\") " pod="openshift-console/console-59cf9545bd-dc6b8"
Jul 29 17:55:04.149811 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:04.149791    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4a24bae0-a6aa-4966-8bdb-0be98412fa2d-console-serving-cert\") pod \"console-59cf9545bd-dc6b8\" (UID: \"4a24bae0-a6aa-4966-8bdb-0be98412fa2d\") " pod="openshift-console/console-59cf9545bd-dc6b8"
Jul 29 17:55:04.156001 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:04.155977    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7nmn\" (UniqueName: \"kubernetes.io/projected/4a24bae0-a6aa-4966-8bdb-0be98412fa2d-kube-api-access-k7nmn\") pod \"console-59cf9545bd-dc6b8\" (UID: \"4a24bae0-a6aa-4966-8bdb-0be98412fa2d\") " pod="openshift-console/console-59cf9545bd-dc6b8"
Jul 29 17:55:04.274924 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:04.274850    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59cf9545bd-dc6b8"
Jul 29 17:55:04.410821 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:04.410776    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59cf9545bd-dc6b8"]
Jul 29 17:55:04.412826 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:55:04.412791    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a24bae0_a6aa_4966_8bdb_0be98412fa2d.slice/crio-8663ae28b4c10f229f39c7fbd8a95703ae4c1dd38a03e283da034b64a8422d85 WatchSource:0}: Error finding container 8663ae28b4c10f229f39c7fbd8a95703ae4c1dd38a03e283da034b64a8422d85: Status 404 returned error can't find the container with id 8663ae28b4c10f229f39c7fbd8a95703ae4c1dd38a03e283da034b64a8422d85
Jul 29 17:55:04.516199 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:04.516165    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59cf9545bd-dc6b8" event={"ID":"4a24bae0-a6aa-4966-8bdb-0be98412fa2d","Type":"ContainerStarted","Data":"27465c75143be889d18912ea56c71b6eeac3a3b04e4a36c973a0d8a83e7c1359"}
Jul 29 17:55:04.516383 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:04.516220    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59cf9545bd-dc6b8" event={"ID":"4a24bae0-a6aa-4966-8bdb-0be98412fa2d","Type":"ContainerStarted","Data":"8663ae28b4c10f229f39c7fbd8a95703ae4c1dd38a03e283da034b64a8422d85"}
Jul 29 17:55:04.533366 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:04.533257    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-59cf9545bd-dc6b8" podStartSLOduration=1.533235684 podStartE2EDuration="1.533235684s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:55:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:55:04.533103218 +0000 UTC m=+184.402326036" watchObservedRunningTime="2026-07-29 17:55:04.533235684 +0000 UTC m=+184.402458491"
Jul 29 17:55:14.275143 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:14.275108    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-59cf9545bd-dc6b8"
Jul 29 17:55:14.275660 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:14.275371    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-59cf9545bd-dc6b8"
Jul 29 17:55:14.280014 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:14.279995    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-59cf9545bd-dc6b8"
Jul 29 17:55:14.555695 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:14.555662    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-59cf9545bd-dc6b8"
Jul 29 17:55:14.593390 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:14.593353    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-69cd5dffd8-v7mh4"]
Jul 29 17:55:39.616783 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:39.616745    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-69cd5dffd8-v7mh4" podUID="709d21d9-aa24-484b-9456-84ef989351ee" containerName="console" containerID="cri-o://398b39222c9e3896471b387ff62739205d37e6fd2ec7be80dfa43a9e6c91d1b4" gracePeriod=15
Jul 29 17:55:39.863122 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:39.863094    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69cd5dffd8-v7mh4"
Jul 29 17:55:39.914660 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:39.914601    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wwv67\" (UniqueName: \"kubernetes.io/projected/709d21d9-aa24-484b-9456-84ef989351ee-kube-api-access-wwv67\") pod \"709d21d9-aa24-484b-9456-84ef989351ee\" (UID: \"709d21d9-aa24-484b-9456-84ef989351ee\") "
Jul 29 17:55:39.914660 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:39.914630    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/709d21d9-aa24-484b-9456-84ef989351ee-oauth-serving-cert\") pod \"709d21d9-aa24-484b-9456-84ef989351ee\" (UID: \"709d21d9-aa24-484b-9456-84ef989351ee\") "
Jul 29 17:55:39.914660 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:39.914654    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/709d21d9-aa24-484b-9456-84ef989351ee-console-config\") pod \"709d21d9-aa24-484b-9456-84ef989351ee\" (UID: \"709d21d9-aa24-484b-9456-84ef989351ee\") "
Jul 29 17:55:39.914882 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:39.914691    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/709d21d9-aa24-484b-9456-84ef989351ee-console-oauth-config\") pod \"709d21d9-aa24-484b-9456-84ef989351ee\" (UID: \"709d21d9-aa24-484b-9456-84ef989351ee\") "
Jul 29 17:55:39.914882 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:39.914750    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/709d21d9-aa24-484b-9456-84ef989351ee-console-serving-cert\") pod \"709d21d9-aa24-484b-9456-84ef989351ee\" (UID: \"709d21d9-aa24-484b-9456-84ef989351ee\") "
Jul 29 17:55:39.914882 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:39.914794    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/709d21d9-aa24-484b-9456-84ef989351ee-trusted-ca-bundle\") pod \"709d21d9-aa24-484b-9456-84ef989351ee\" (UID: \"709d21d9-aa24-484b-9456-84ef989351ee\") "
Jul 29 17:55:39.914882 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:39.914839    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/709d21d9-aa24-484b-9456-84ef989351ee-service-ca\") pod \"709d21d9-aa24-484b-9456-84ef989351ee\" (UID: \"709d21d9-aa24-484b-9456-84ef989351ee\") "
Jul 29 17:55:39.915178 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:39.915066    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/709d21d9-aa24-484b-9456-84ef989351ee-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "709d21d9-aa24-484b-9456-84ef989351ee" (UID: "709d21d9-aa24-484b-9456-84ef989351ee"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 17:55:39.915178 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:39.915158    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/709d21d9-aa24-484b-9456-84ef989351ee-console-config" (OuterVolumeSpecName: "console-config") pod "709d21d9-aa24-484b-9456-84ef989351ee" (UID: "709d21d9-aa24-484b-9456-84ef989351ee"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 17:55:39.915317 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:39.915278    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/709d21d9-aa24-484b-9456-84ef989351ee-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "709d21d9-aa24-484b-9456-84ef989351ee" (UID: "709d21d9-aa24-484b-9456-84ef989351ee"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 17:55:39.915317 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:39.915290    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/709d21d9-aa24-484b-9456-84ef989351ee-service-ca" (OuterVolumeSpecName: "service-ca") pod "709d21d9-aa24-484b-9456-84ef989351ee" (UID: "709d21d9-aa24-484b-9456-84ef989351ee"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 17:55:39.915671 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:39.915653    2557 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/709d21d9-aa24-484b-9456-84ef989351ee-oauth-serving-cert\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 17:55:39.915740 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:39.915677    2557 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/709d21d9-aa24-484b-9456-84ef989351ee-console-config\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 17:55:39.915740 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:39.915693    2557 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/709d21d9-aa24-484b-9456-84ef989351ee-trusted-ca-bundle\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 17:55:39.915740 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:39.915707    2557 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/709d21d9-aa24-484b-9456-84ef989351ee-service-ca\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 17:55:39.917024 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:39.917001    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/709d21d9-aa24-484b-9456-84ef989351ee-kube-api-access-wwv67" (OuterVolumeSpecName: "kube-api-access-wwv67") pod "709d21d9-aa24-484b-9456-84ef989351ee" (UID: "709d21d9-aa24-484b-9456-84ef989351ee"). InnerVolumeSpecName "kube-api-access-wwv67". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 17:55:39.917407 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:39.917382    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/709d21d9-aa24-484b-9456-84ef989351ee-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "709d21d9-aa24-484b-9456-84ef989351ee" (UID: "709d21d9-aa24-484b-9456-84ef989351ee"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 17:55:39.917407 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:39.917397    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/709d21d9-aa24-484b-9456-84ef989351ee-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "709d21d9-aa24-484b-9456-84ef989351ee" (UID: "709d21d9-aa24-484b-9456-84ef989351ee"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 17:55:40.016189 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:40.016150    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wwv67\" (UniqueName: \"kubernetes.io/projected/709d21d9-aa24-484b-9456-84ef989351ee-kube-api-access-wwv67\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 17:55:40.016189 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:40.016184    2557 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/709d21d9-aa24-484b-9456-84ef989351ee-console-oauth-config\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 17:55:40.016189 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:40.016194    2557 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/709d21d9-aa24-484b-9456-84ef989351ee-console-serving-cert\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 17:55:40.647979 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:40.647947    2557 generic.go:356] "Generic (PLEG): container finished" podID="709d21d9-aa24-484b-9456-84ef989351ee" containerID="398b39222c9e3896471b387ff62739205d37e6fd2ec7be80dfa43a9e6c91d1b4" exitCode=0
Jul 29 17:55:40.648369 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:40.648352    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69cd5dffd8-v7mh4"
Jul 29 17:55:40.656641 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:40.656607    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69cd5dffd8-v7mh4" event={"ID":"709d21d9-aa24-484b-9456-84ef989351ee","Type":"ContainerDied","Data":"398b39222c9e3896471b387ff62739205d37e6fd2ec7be80dfa43a9e6c91d1b4"}
Jul 29 17:55:40.656808 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:40.656658    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69cd5dffd8-v7mh4" event={"ID":"709d21d9-aa24-484b-9456-84ef989351ee","Type":"ContainerDied","Data":"7f4d5237cfac220d8f1e06209a24e7f602ddda0430a7dd27e485c24f03ba9dad"}
Jul 29 17:55:40.656808 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:40.656684    2557 scope.go:117] "RemoveContainer" containerID="398b39222c9e3896471b387ff62739205d37e6fd2ec7be80dfa43a9e6c91d1b4"
Jul 29 17:55:40.666462 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:40.666441    2557 scope.go:117] "RemoveContainer" containerID="398b39222c9e3896471b387ff62739205d37e6fd2ec7be80dfa43a9e6c91d1b4"
Jul 29 17:55:40.666710 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:55:40.666684    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"398b39222c9e3896471b387ff62739205d37e6fd2ec7be80dfa43a9e6c91d1b4\": container with ID starting with 398b39222c9e3896471b387ff62739205d37e6fd2ec7be80dfa43a9e6c91d1b4 not found: ID does not exist" containerID="398b39222c9e3896471b387ff62739205d37e6fd2ec7be80dfa43a9e6c91d1b4"
Jul 29 17:55:40.666806 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:40.666736    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"398b39222c9e3896471b387ff62739205d37e6fd2ec7be80dfa43a9e6c91d1b4"} err="failed to get container status \"398b39222c9e3896471b387ff62739205d37e6fd2ec7be80dfa43a9e6c91d1b4\": rpc error: code = NotFound desc = could not find container \"398b39222c9e3896471b387ff62739205d37e6fd2ec7be80dfa43a9e6c91d1b4\": container with ID starting with 398b39222c9e3896471b387ff62739205d37e6fd2ec7be80dfa43a9e6c91d1b4 not found: ID does not exist"
Jul 29 17:55:40.675696 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:40.675667    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-69cd5dffd8-v7mh4"]
Jul 29 17:55:40.677296 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:40.677278    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-69cd5dffd8-v7mh4"]
Jul 29 17:55:42.651417 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:55:42.651387    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="709d21d9-aa24-484b-9456-84ef989351ee" path="/var/lib/kubelet/pods/709d21d9-aa24-484b-9456-84ef989351ee/volumes"
Jul 29 17:56:13.693568 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:13.693536    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5f5c4f8dfb-8nlj8"]
Jul 29 17:56:13.693997 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:13.693864    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="709d21d9-aa24-484b-9456-84ef989351ee" containerName="console"
Jul 29 17:56:13.693997 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:13.693878    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="709d21d9-aa24-484b-9456-84ef989351ee" containerName="console"
Jul 29 17:56:13.693997 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:13.693931    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="709d21d9-aa24-484b-9456-84ef989351ee" containerName="console"
Jul 29 17:56:13.698430 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:13.698401    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f5c4f8dfb-8nlj8"
Jul 29 17:56:13.710796 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:13.710769    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f5c4f8dfb-8nlj8"]
Jul 29 17:56:13.758983 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:13.758948    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fc27aa4a-2b7e-4182-ad51-1eee720619dd-console-serving-cert\") pod \"console-5f5c4f8dfb-8nlj8\" (UID: \"fc27aa4a-2b7e-4182-ad51-1eee720619dd\") " pod="openshift-console/console-5f5c4f8dfb-8nlj8"
Jul 29 17:56:13.759175 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:13.758999    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fc27aa4a-2b7e-4182-ad51-1eee720619dd-service-ca\") pod \"console-5f5c4f8dfb-8nlj8\" (UID: \"fc27aa4a-2b7e-4182-ad51-1eee720619dd\") " pod="openshift-console/console-5f5c4f8dfb-8nlj8"
Jul 29 17:56:13.759175 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:13.759053    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvp2k\" (UniqueName: \"kubernetes.io/projected/fc27aa4a-2b7e-4182-ad51-1eee720619dd-kube-api-access-xvp2k\") pod \"console-5f5c4f8dfb-8nlj8\" (UID: \"fc27aa4a-2b7e-4182-ad51-1eee720619dd\") " pod="openshift-console/console-5f5c4f8dfb-8nlj8"
Jul 29 17:56:13.759175 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:13.759163    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc27aa4a-2b7e-4182-ad51-1eee720619dd-trusted-ca-bundle\") pod \"console-5f5c4f8dfb-8nlj8\" (UID: \"fc27aa4a-2b7e-4182-ad51-1eee720619dd\") " pod="openshift-console/console-5f5c4f8dfb-8nlj8"
Jul 29 17:56:13.759354 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:13.759187    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fc27aa4a-2b7e-4182-ad51-1eee720619dd-oauth-serving-cert\") pod \"console-5f5c4f8dfb-8nlj8\" (UID: \"fc27aa4a-2b7e-4182-ad51-1eee720619dd\") " pod="openshift-console/console-5f5c4f8dfb-8nlj8"
Jul 29 17:56:13.759354 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:13.759264    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fc27aa4a-2b7e-4182-ad51-1eee720619dd-console-oauth-config\") pod \"console-5f5c4f8dfb-8nlj8\" (UID: \"fc27aa4a-2b7e-4182-ad51-1eee720619dd\") " pod="openshift-console/console-5f5c4f8dfb-8nlj8"
Jul 29 17:56:13.759354 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:13.759324    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fc27aa4a-2b7e-4182-ad51-1eee720619dd-console-config\") pod \"console-5f5c4f8dfb-8nlj8\" (UID: \"fc27aa4a-2b7e-4182-ad51-1eee720619dd\") " pod="openshift-console/console-5f5c4f8dfb-8nlj8"
Jul 29 17:56:13.860056 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:13.860025    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fc27aa4a-2b7e-4182-ad51-1eee720619dd-console-oauth-config\") pod \"console-5f5c4f8dfb-8nlj8\" (UID: \"fc27aa4a-2b7e-4182-ad51-1eee720619dd\") " pod="openshift-console/console-5f5c4f8dfb-8nlj8"
Jul 29 17:56:13.860056 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:13.860059    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fc27aa4a-2b7e-4182-ad51-1eee720619dd-console-config\") pod \"console-5f5c4f8dfb-8nlj8\" (UID: \"fc27aa4a-2b7e-4182-ad51-1eee720619dd\") " pod="openshift-console/console-5f5c4f8dfb-8nlj8"
Jul 29 17:56:13.860245 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:13.860086    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fc27aa4a-2b7e-4182-ad51-1eee720619dd-console-serving-cert\") pod \"console-5f5c4f8dfb-8nlj8\" (UID: \"fc27aa4a-2b7e-4182-ad51-1eee720619dd\") " pod="openshift-console/console-5f5c4f8dfb-8nlj8"
Jul 29 17:56:13.860245 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:13.860121    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fc27aa4a-2b7e-4182-ad51-1eee720619dd-service-ca\") pod \"console-5f5c4f8dfb-8nlj8\" (UID: \"fc27aa4a-2b7e-4182-ad51-1eee720619dd\") " pod="openshift-console/console-5f5c4f8dfb-8nlj8"
Jul 29 17:56:13.860245 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:13.860152    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvp2k\" (UniqueName: \"kubernetes.io/projected/fc27aa4a-2b7e-4182-ad51-1eee720619dd-kube-api-access-xvp2k\") pod \"console-5f5c4f8dfb-8nlj8\" (UID: \"fc27aa4a-2b7e-4182-ad51-1eee720619dd\") " pod="openshift-console/console-5f5c4f8dfb-8nlj8"
Jul 29 17:56:13.860245 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:13.860217    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc27aa4a-2b7e-4182-ad51-1eee720619dd-trusted-ca-bundle\") pod \"console-5f5c4f8dfb-8nlj8\" (UID: \"fc27aa4a-2b7e-4182-ad51-1eee720619dd\") " pod="openshift-console/console-5f5c4f8dfb-8nlj8"
Jul 29 17:56:13.860245 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:13.860234    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fc27aa4a-2b7e-4182-ad51-1eee720619dd-oauth-serving-cert\") pod \"console-5f5c4f8dfb-8nlj8\" (UID: \"fc27aa4a-2b7e-4182-ad51-1eee720619dd\") " pod="openshift-console/console-5f5c4f8dfb-8nlj8"
Jul 29 17:56:13.860958 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:13.860923    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fc27aa4a-2b7e-4182-ad51-1eee720619dd-console-config\") pod \"console-5f5c4f8dfb-8nlj8\" (UID: \"fc27aa4a-2b7e-4182-ad51-1eee720619dd\") " pod="openshift-console/console-5f5c4f8dfb-8nlj8"
Jul 29 17:56:13.861119 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:13.861097    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fc27aa4a-2b7e-4182-ad51-1eee720619dd-oauth-serving-cert\") pod \"console-5f5c4f8dfb-8nlj8\" (UID: \"fc27aa4a-2b7e-4182-ad51-1eee720619dd\") " pod="openshift-console/console-5f5c4f8dfb-8nlj8"
Jul 29 17:56:13.861181 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:13.861117    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fc27aa4a-2b7e-4182-ad51-1eee720619dd-service-ca\") pod \"console-5f5c4f8dfb-8nlj8\" (UID: \"fc27aa4a-2b7e-4182-ad51-1eee720619dd\") " pod="openshift-console/console-5f5c4f8dfb-8nlj8"
Jul 29 17:56:13.861181 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:13.861134    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc27aa4a-2b7e-4182-ad51-1eee720619dd-trusted-ca-bundle\") pod \"console-5f5c4f8dfb-8nlj8\" (UID: \"fc27aa4a-2b7e-4182-ad51-1eee720619dd\") " pod="openshift-console/console-5f5c4f8dfb-8nlj8"
Jul 29 17:56:13.862751 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:13.862712    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fc27aa4a-2b7e-4182-ad51-1eee720619dd-console-oauth-config\") pod \"console-5f5c4f8dfb-8nlj8\" (UID: \"fc27aa4a-2b7e-4182-ad51-1eee720619dd\") " pod="openshift-console/console-5f5c4f8dfb-8nlj8"
Jul 29 17:56:13.862751 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:13.862715    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fc27aa4a-2b7e-4182-ad51-1eee720619dd-console-serving-cert\") pod \"console-5f5c4f8dfb-8nlj8\" (UID: \"fc27aa4a-2b7e-4182-ad51-1eee720619dd\") " pod="openshift-console/console-5f5c4f8dfb-8nlj8"
Jul 29 17:56:13.870431 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:13.870411    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvp2k\" (UniqueName: \"kubernetes.io/projected/fc27aa4a-2b7e-4182-ad51-1eee720619dd-kube-api-access-xvp2k\") pod \"console-5f5c4f8dfb-8nlj8\" (UID: \"fc27aa4a-2b7e-4182-ad51-1eee720619dd\") " pod="openshift-console/console-5f5c4f8dfb-8nlj8"
Jul 29 17:56:14.009248 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:14.009161    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f5c4f8dfb-8nlj8"
Jul 29 17:56:14.139686 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:14.139661    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f5c4f8dfb-8nlj8"]
Jul 29 17:56:14.141639 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:56:14.141607    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc27aa4a_2b7e_4182_ad51_1eee720619dd.slice/crio-dd057d7f4d3cb27967113465fd4e3f13c50964b9a561aff08a8b000377619344 WatchSource:0}: Error finding container dd057d7f4d3cb27967113465fd4e3f13c50964b9a561aff08a8b000377619344: Status 404 returned error can't find the container with id dd057d7f4d3cb27967113465fd4e3f13c50964b9a561aff08a8b000377619344
Jul 29 17:56:14.778117 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:14.778083    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f5c4f8dfb-8nlj8" event={"ID":"fc27aa4a-2b7e-4182-ad51-1eee720619dd","Type":"ContainerStarted","Data":"c214d08b6ef77665b019af6bf644fb4c8432103c2e8dd9393d44b3f578537d48"}
Jul 29 17:56:14.778512 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:14.778133    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f5c4f8dfb-8nlj8" event={"ID":"fc27aa4a-2b7e-4182-ad51-1eee720619dd","Type":"ContainerStarted","Data":"dd057d7f4d3cb27967113465fd4e3f13c50964b9a561aff08a8b000377619344"}
Jul 29 17:56:14.795683 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:14.795631    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5f5c4f8dfb-8nlj8" podStartSLOduration=1.795615776 podStartE2EDuration="1.795615776s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:56:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:56:14.794320191 +0000 UTC m=+254.663542999" watchObservedRunningTime="2026-07-29 17:56:14.795615776 +0000 UTC m=+254.664838583"
Jul 29 17:56:24.009638 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:24.009563    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5f5c4f8dfb-8nlj8"
Jul 29 17:56:24.009638 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:24.009618    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5f5c4f8dfb-8nlj8"
Jul 29 17:56:24.014369 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:24.014346    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5f5c4f8dfb-8nlj8"
Jul 29 17:56:24.818847 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:24.818819    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5f5c4f8dfb-8nlj8"
Jul 29 17:56:24.862056 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:24.862025    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59cf9545bd-dc6b8"]
Jul 29 17:56:49.886927 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:49.886866    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-59cf9545bd-dc6b8" podUID="4a24bae0-a6aa-4966-8bdb-0be98412fa2d" containerName="console" containerID="cri-o://27465c75143be889d18912ea56c71b6eeac3a3b04e4a36c973a0d8a83e7c1359" gracePeriod=15
Jul 29 17:56:50.127150 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:50.127126    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59cf9545bd-dc6b8"
Jul 29 17:56:50.162519 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:50.162443    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k7nmn\" (UniqueName: \"kubernetes.io/projected/4a24bae0-a6aa-4966-8bdb-0be98412fa2d-kube-api-access-k7nmn\") pod \"4a24bae0-a6aa-4966-8bdb-0be98412fa2d\" (UID: \"4a24bae0-a6aa-4966-8bdb-0be98412fa2d\") "
Jul 29 17:56:50.162681 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:50.162522    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a24bae0-a6aa-4966-8bdb-0be98412fa2d-trusted-ca-bundle\") pod \"4a24bae0-a6aa-4966-8bdb-0be98412fa2d\" (UID: \"4a24bae0-a6aa-4966-8bdb-0be98412fa2d\") "
Jul 29 17:56:50.162681 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:50.162545    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4a24bae0-a6aa-4966-8bdb-0be98412fa2d-oauth-serving-cert\") pod \"4a24bae0-a6aa-4966-8bdb-0be98412fa2d\" (UID: \"4a24bae0-a6aa-4966-8bdb-0be98412fa2d\") "
Jul 29 17:56:50.162681 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:50.162631    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4a24bae0-a6aa-4966-8bdb-0be98412fa2d-console-serving-cert\") pod \"4a24bae0-a6aa-4966-8bdb-0be98412fa2d\" (UID: \"4a24bae0-a6aa-4966-8bdb-0be98412fa2d\") "
Jul 29 17:56:50.162681 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:50.162677    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4a24bae0-a6aa-4966-8bdb-0be98412fa2d-service-ca\") pod \"4a24bae0-a6aa-4966-8bdb-0be98412fa2d\" (UID: \"4a24bae0-a6aa-4966-8bdb-0be98412fa2d\") "
Jul 29 17:56:50.162918 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:50.162701    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4a24bae0-a6aa-4966-8bdb-0be98412fa2d-console-oauth-config\") pod \"4a24bae0-a6aa-4966-8bdb-0be98412fa2d\" (UID: \"4a24bae0-a6aa-4966-8bdb-0be98412fa2d\") "
Jul 29 17:56:50.162918 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:50.162745    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4a24bae0-a6aa-4966-8bdb-0be98412fa2d-console-config\") pod \"4a24bae0-a6aa-4966-8bdb-0be98412fa2d\" (UID: \"4a24bae0-a6aa-4966-8bdb-0be98412fa2d\") "
Jul 29 17:56:50.163012 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:50.162941    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a24bae0-a6aa-4966-8bdb-0be98412fa2d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4a24bae0-a6aa-4966-8bdb-0be98412fa2d" (UID: "4a24bae0-a6aa-4966-8bdb-0be98412fa2d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 17:56:50.163012 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:50.162982    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a24bae0-a6aa-4966-8bdb-0be98412fa2d-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "4a24bae0-a6aa-4966-8bdb-0be98412fa2d" (UID: "4a24bae0-a6aa-4966-8bdb-0be98412fa2d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 17:56:50.163356 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:50.163323    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a24bae0-a6aa-4966-8bdb-0be98412fa2d-service-ca" (OuterVolumeSpecName: "service-ca") pod "4a24bae0-a6aa-4966-8bdb-0be98412fa2d" (UID: "4a24bae0-a6aa-4966-8bdb-0be98412fa2d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 17:56:50.163580 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:50.163493    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a24bae0-a6aa-4966-8bdb-0be98412fa2d-console-config" (OuterVolumeSpecName: "console-config") pod "4a24bae0-a6aa-4966-8bdb-0be98412fa2d" (UID: "4a24bae0-a6aa-4966-8bdb-0be98412fa2d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 17:56:50.164992 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:50.164970    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a24bae0-a6aa-4966-8bdb-0be98412fa2d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4a24bae0-a6aa-4966-8bdb-0be98412fa2d" (UID: "4a24bae0-a6aa-4966-8bdb-0be98412fa2d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 17:56:50.165085 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:50.164988    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a24bae0-a6aa-4966-8bdb-0be98412fa2d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4a24bae0-a6aa-4966-8bdb-0be98412fa2d" (UID: "4a24bae0-a6aa-4966-8bdb-0be98412fa2d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 17:56:50.165085 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:50.164992    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a24bae0-a6aa-4966-8bdb-0be98412fa2d-kube-api-access-k7nmn" (OuterVolumeSpecName: "kube-api-access-k7nmn") pod "4a24bae0-a6aa-4966-8bdb-0be98412fa2d" (UID: "4a24bae0-a6aa-4966-8bdb-0be98412fa2d"). InnerVolumeSpecName "kube-api-access-k7nmn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 17:56:50.263550 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:50.263511    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k7nmn\" (UniqueName: \"kubernetes.io/projected/4a24bae0-a6aa-4966-8bdb-0be98412fa2d-kube-api-access-k7nmn\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 17:56:50.263550 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:50.263545    2557 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a24bae0-a6aa-4966-8bdb-0be98412fa2d-trusted-ca-bundle\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 17:56:50.263550 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:50.263557    2557 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4a24bae0-a6aa-4966-8bdb-0be98412fa2d-oauth-serving-cert\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 17:56:50.263828 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:50.263569    2557 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4a24bae0-a6aa-4966-8bdb-0be98412fa2d-console-serving-cert\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 17:56:50.263828 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:50.263581    2557 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4a24bae0-a6aa-4966-8bdb-0be98412fa2d-service-ca\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 17:56:50.263828 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:50.263592    2557 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4a24bae0-a6aa-4966-8bdb-0be98412fa2d-console-oauth-config\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 17:56:50.263828 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:50.263603    2557 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4a24bae0-a6aa-4966-8bdb-0be98412fa2d-console-config\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 17:56:50.918895 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:50.918809    2557 generic.go:356] "Generic (PLEG): container finished" podID="4a24bae0-a6aa-4966-8bdb-0be98412fa2d" containerID="27465c75143be889d18912ea56c71b6eeac3a3b04e4a36c973a0d8a83e7c1359" exitCode=0
Jul 29 17:56:50.918895 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:50.918862    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59cf9545bd-dc6b8"
Jul 29 17:56:50.919416 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:50.918898    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59cf9545bd-dc6b8" event={"ID":"4a24bae0-a6aa-4966-8bdb-0be98412fa2d","Type":"ContainerDied","Data":"27465c75143be889d18912ea56c71b6eeac3a3b04e4a36c973a0d8a83e7c1359"}
Jul 29 17:56:50.919416 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:50.918951    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59cf9545bd-dc6b8" event={"ID":"4a24bae0-a6aa-4966-8bdb-0be98412fa2d","Type":"ContainerDied","Data":"8663ae28b4c10f229f39c7fbd8a95703ae4c1dd38a03e283da034b64a8422d85"}
Jul 29 17:56:50.919416 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:50.918973    2557 scope.go:117] "RemoveContainer" containerID="27465c75143be889d18912ea56c71b6eeac3a3b04e4a36c973a0d8a83e7c1359"
Jul 29 17:56:50.927131 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:50.927115    2557 scope.go:117] "RemoveContainer" containerID="27465c75143be889d18912ea56c71b6eeac3a3b04e4a36c973a0d8a83e7c1359"
Jul 29 17:56:50.927339 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:56:50.927322    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27465c75143be889d18912ea56c71b6eeac3a3b04e4a36c973a0d8a83e7c1359\": container with ID starting with 27465c75143be889d18912ea56c71b6eeac3a3b04e4a36c973a0d8a83e7c1359 not found: ID does not exist" containerID="27465c75143be889d18912ea56c71b6eeac3a3b04e4a36c973a0d8a83e7c1359"
Jul 29 17:56:50.927392 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:50.927346    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27465c75143be889d18912ea56c71b6eeac3a3b04e4a36c973a0d8a83e7c1359"} err="failed to get container status \"27465c75143be889d18912ea56c71b6eeac3a3b04e4a36c973a0d8a83e7c1359\": rpc error: code = NotFound desc = could not find container \"27465c75143be889d18912ea56c71b6eeac3a3b04e4a36c973a0d8a83e7c1359\": container with ID starting with 27465c75143be889d18912ea56c71b6eeac3a3b04e4a36c973a0d8a83e7c1359 not found: ID does not exist"
Jul 29 17:56:50.934494 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:50.934473    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59cf9545bd-dc6b8"]
Jul 29 17:56:50.938575 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:50.938554    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-59cf9545bd-dc6b8"]
Jul 29 17:56:52.652116 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:56:52.652089    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a24bae0-a6aa-4966-8bdb-0be98412fa2d" path="/var/lib/kubelet/pods/4a24bae0-a6aa-4966-8bdb-0be98412fa2d/volumes"
Jul 29 17:57:00.541392 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:00.541363    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gg2wj_2bf0bb71-4047-45a7-bd8e-5d6efabd26bd/console-operator/2.log"
Jul 29 17:57:00.552931 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:00.552909    2557 kubelet.go:1648] "Image garbage collection succeeded"
Jul 29 17:57:00.553512 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:00.553495    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gg2wj_2bf0bb71-4047-45a7-bd8e-5d6efabd26bd/console-operator/2.log"
Jul 29 17:57:15.119787 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:15.119750    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qsk9b"]
Jul 29 17:57:15.122194 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:15.120018    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a24bae0-a6aa-4966-8bdb-0be98412fa2d" containerName="console"
Jul 29 17:57:15.122194 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:15.120029    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a24bae0-a6aa-4966-8bdb-0be98412fa2d" containerName="console"
Jul 29 17:57:15.122194 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:15.120082    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a24bae0-a6aa-4966-8bdb-0be98412fa2d" containerName="console"
Jul 29 17:57:15.123040 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:15.123025    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qsk9b"
Jul 29 17:57:15.125685 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:15.125661    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-rrbkn\""
Jul 29 17:57:15.125903 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:15.125700    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 29 17:57:15.126798 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:15.126779    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 29 17:57:15.131798 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:15.131765    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qsk9b"]
Jul 29 17:57:15.243056 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:15.243022    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jqqc\" (UniqueName: \"kubernetes.io/projected/7f919ac3-ac25-40be-8309-72d6abd6cd44-kube-api-access-5jqqc\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qsk9b\" (UID: \"7f919ac3-ac25-40be-8309-72d6abd6cd44\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qsk9b"
Jul 29 17:57:15.243261 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:15.243080    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7f919ac3-ac25-40be-8309-72d6abd6cd44-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qsk9b\" (UID: \"7f919ac3-ac25-40be-8309-72d6abd6cd44\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qsk9b"
Jul 29 17:57:15.243261 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:15.243171    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7f919ac3-ac25-40be-8309-72d6abd6cd44-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qsk9b\" (UID: \"7f919ac3-ac25-40be-8309-72d6abd6cd44\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qsk9b"
Jul 29 17:57:15.343984 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:15.343931    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jqqc\" (UniqueName: \"kubernetes.io/projected/7f919ac3-ac25-40be-8309-72d6abd6cd44-kube-api-access-5jqqc\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qsk9b\" (UID: \"7f919ac3-ac25-40be-8309-72d6abd6cd44\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qsk9b"
Jul 29 17:57:15.344147 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:15.344017    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7f919ac3-ac25-40be-8309-72d6abd6cd44-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qsk9b\" (UID: \"7f919ac3-ac25-40be-8309-72d6abd6cd44\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qsk9b"
Jul 29 17:57:15.344147 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:15.344063    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7f919ac3-ac25-40be-8309-72d6abd6cd44-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qsk9b\" (UID: \"7f919ac3-ac25-40be-8309-72d6abd6cd44\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qsk9b"
Jul 29 17:57:15.344415 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:15.344396    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7f919ac3-ac25-40be-8309-72d6abd6cd44-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qsk9b\" (UID: \"7f919ac3-ac25-40be-8309-72d6abd6cd44\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qsk9b"
Jul 29 17:57:15.344454 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:15.344426    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7f919ac3-ac25-40be-8309-72d6abd6cd44-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qsk9b\" (UID: \"7f919ac3-ac25-40be-8309-72d6abd6cd44\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qsk9b"
Jul 29 17:57:15.352271 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:15.352246    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jqqc\" (UniqueName: \"kubernetes.io/projected/7f919ac3-ac25-40be-8309-72d6abd6cd44-kube-api-access-5jqqc\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qsk9b\" (UID: \"7f919ac3-ac25-40be-8309-72d6abd6cd44\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qsk9b"
Jul 29 17:57:15.432832 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:15.432763    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qsk9b"
Jul 29 17:57:15.557966 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:15.557942    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qsk9b"]
Jul 29 17:57:15.559819 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:57:15.559782    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f919ac3_ac25_40be_8309_72d6abd6cd44.slice/crio-8b12728216cf88b438b9b818da6be34b2ac3d615f596961b9e11e382c843f1d7 WatchSource:0}: Error finding container 8b12728216cf88b438b9b818da6be34b2ac3d615f596961b9e11e382c843f1d7: Status 404 returned error can't find the container with id 8b12728216cf88b438b9b818da6be34b2ac3d615f596961b9e11e382c843f1d7
Jul 29 17:57:15.561651 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:15.561635    2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 17:57:16.018895 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:16.018864    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qsk9b" event={"ID":"7f919ac3-ac25-40be-8309-72d6abd6cd44","Type":"ContainerStarted","Data":"8b12728216cf88b438b9b818da6be34b2ac3d615f596961b9e11e382c843f1d7"}
Jul 29 17:57:23.052113 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:23.052079    2557 generic.go:356] "Generic (PLEG): container finished" podID="7f919ac3-ac25-40be-8309-72d6abd6cd44" containerID="5671a2eab0b8b4a8902577b36fa653f4f371f47ac1d88f80e696e59fa9301c26" exitCode=0
Jul 29 17:57:23.052434 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:23.052132    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qsk9b" event={"ID":"7f919ac3-ac25-40be-8309-72d6abd6cd44","Type":"ContainerDied","Data":"5671a2eab0b8b4a8902577b36fa653f4f371f47ac1d88f80e696e59fa9301c26"}
Jul 29 17:57:26.066926 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:26.066889    2557 generic.go:356] "Generic (PLEG): container finished" podID="7f919ac3-ac25-40be-8309-72d6abd6cd44" containerID="7742af86c6aff2137f213d5d92f13aef3d442975af21a4f2a939e90932b74de1" exitCode=0
Jul 29 17:57:26.067354 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:26.066969    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qsk9b" event={"ID":"7f919ac3-ac25-40be-8309-72d6abd6cd44","Type":"ContainerDied","Data":"7742af86c6aff2137f213d5d92f13aef3d442975af21a4f2a939e90932b74de1"}
Jul 29 17:57:34.104079 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:34.104038    2557 generic.go:356] "Generic (PLEG): container finished" podID="7f919ac3-ac25-40be-8309-72d6abd6cd44" containerID="2bdab9ed9d1390c8c022bc95042cad3f917bd54cbf4d94ea7c6e7436c5226a0f" exitCode=0
Jul 29 17:57:34.104526 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:34.104124    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qsk9b" event={"ID":"7f919ac3-ac25-40be-8309-72d6abd6cd44","Type":"ContainerDied","Data":"2bdab9ed9d1390c8c022bc95042cad3f917bd54cbf4d94ea7c6e7436c5226a0f"}
Jul 29 17:57:35.230331 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:35.230308    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qsk9b"
Jul 29 17:57:35.414429 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:35.414327    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7f919ac3-ac25-40be-8309-72d6abd6cd44-util\") pod \"7f919ac3-ac25-40be-8309-72d6abd6cd44\" (UID: \"7f919ac3-ac25-40be-8309-72d6abd6cd44\") "
Jul 29 17:57:35.414429 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:35.414389    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7f919ac3-ac25-40be-8309-72d6abd6cd44-bundle\") pod \"7f919ac3-ac25-40be-8309-72d6abd6cd44\" (UID: \"7f919ac3-ac25-40be-8309-72d6abd6cd44\") "
Jul 29 17:57:35.414429 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:35.414415    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5jqqc\" (UniqueName: \"kubernetes.io/projected/7f919ac3-ac25-40be-8309-72d6abd6cd44-kube-api-access-5jqqc\") pod \"7f919ac3-ac25-40be-8309-72d6abd6cd44\" (UID: \"7f919ac3-ac25-40be-8309-72d6abd6cd44\") "
Jul 29 17:57:35.415202 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:35.415168    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f919ac3-ac25-40be-8309-72d6abd6cd44-bundle" (OuterVolumeSpecName: "bundle") pod "7f919ac3-ac25-40be-8309-72d6abd6cd44" (UID: "7f919ac3-ac25-40be-8309-72d6abd6cd44"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 17:57:35.416702 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:35.416678    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f919ac3-ac25-40be-8309-72d6abd6cd44-kube-api-access-5jqqc" (OuterVolumeSpecName: "kube-api-access-5jqqc") pod "7f919ac3-ac25-40be-8309-72d6abd6cd44" (UID: "7f919ac3-ac25-40be-8309-72d6abd6cd44"). InnerVolumeSpecName "kube-api-access-5jqqc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 17:57:35.419158 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:35.419139    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f919ac3-ac25-40be-8309-72d6abd6cd44-util" (OuterVolumeSpecName: "util") pod "7f919ac3-ac25-40be-8309-72d6abd6cd44" (UID: "7f919ac3-ac25-40be-8309-72d6abd6cd44"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 17:57:35.515241 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:35.515201    2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7f919ac3-ac25-40be-8309-72d6abd6cd44-util\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 17:57:35.515241 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:35.515233    2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7f919ac3-ac25-40be-8309-72d6abd6cd44-bundle\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 17:57:35.515241 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:35.515243    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5jqqc\" (UniqueName: \"kubernetes.io/projected/7f919ac3-ac25-40be-8309-72d6abd6cd44-kube-api-access-5jqqc\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 17:57:36.113576 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:36.113539    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qsk9b" event={"ID":"7f919ac3-ac25-40be-8309-72d6abd6cd44","Type":"ContainerDied","Data":"8b12728216cf88b438b9b818da6be34b2ac3d615f596961b9e11e382c843f1d7"}
Jul 29 17:57:36.113788 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:36.113608    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8b12728216cf88b438b9b818da6be34b2ac3d615f596961b9e11e382c843f1d7"
Jul 29 17:57:36.113788 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:36.113666    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qsk9b"
Jul 29 17:57:43.096664 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:43.096631    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-m2rm9"]
Jul 29 17:57:43.097133 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:43.096938    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f919ac3-ac25-40be-8309-72d6abd6cd44" containerName="extract"
Jul 29 17:57:43.097133 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:43.096950    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f919ac3-ac25-40be-8309-72d6abd6cd44" containerName="extract"
Jul 29 17:57:43.097133 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:43.096969    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f919ac3-ac25-40be-8309-72d6abd6cd44" containerName="util"
Jul 29 17:57:43.097133 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:43.096975    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f919ac3-ac25-40be-8309-72d6abd6cd44" containerName="util"
Jul 29 17:57:43.097133 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:43.096987    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f919ac3-ac25-40be-8309-72d6abd6cd44" containerName="pull"
Jul 29 17:57:43.097133 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:43.096992    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f919ac3-ac25-40be-8309-72d6abd6cd44" containerName="pull"
Jul 29 17:57:43.097133 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:43.097035    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f919ac3-ac25-40be-8309-72d6abd6cd44" containerName="extract"
Jul 29 17:57:43.102356 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:43.102337    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-m2rm9"
Jul 29 17:57:43.104961 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:43.104943    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 29 17:57:43.105213 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:43.105194    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-2f2gh\""
Jul 29 17:57:43.105416 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:43.105397    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 29 17:57:43.115876 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:43.115855    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-m2rm9"]
Jul 29 17:57:43.172742 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:43.172688    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d9fab3dd-1a95-44c9-aef0-2dcb6fa72512-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-m2rm9\" (UID: \"d9fab3dd-1a95-44c9-aef0-2dcb6fa72512\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-m2rm9"
Jul 29 17:57:43.172893 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:43.172773    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kch5\" (UniqueName: \"kubernetes.io/projected/d9fab3dd-1a95-44c9-aef0-2dcb6fa72512-kube-api-access-9kch5\") pod \"cert-manager-operator-controller-manager-55c444bf86-m2rm9\" (UID: \"d9fab3dd-1a95-44c9-aef0-2dcb6fa72512\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-m2rm9"
Jul 29 17:57:43.273615 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:43.273584    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d9fab3dd-1a95-44c9-aef0-2dcb6fa72512-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-m2rm9\" (UID: \"d9fab3dd-1a95-44c9-aef0-2dcb6fa72512\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-m2rm9"
Jul 29 17:57:43.273822 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:43.273650    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kch5\" (UniqueName: \"kubernetes.io/projected/d9fab3dd-1a95-44c9-aef0-2dcb6fa72512-kube-api-access-9kch5\") pod \"cert-manager-operator-controller-manager-55c444bf86-m2rm9\" (UID: \"d9fab3dd-1a95-44c9-aef0-2dcb6fa72512\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-m2rm9"
Jul 29 17:57:43.274004 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:43.273985    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d9fab3dd-1a95-44c9-aef0-2dcb6fa72512-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-m2rm9\" (UID: \"d9fab3dd-1a95-44c9-aef0-2dcb6fa72512\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-m2rm9"
Jul 29 17:57:43.283085 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:43.283062    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kch5\" (UniqueName: \"kubernetes.io/projected/d9fab3dd-1a95-44c9-aef0-2dcb6fa72512-kube-api-access-9kch5\") pod \"cert-manager-operator-controller-manager-55c444bf86-m2rm9\" (UID: \"d9fab3dd-1a95-44c9-aef0-2dcb6fa72512\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-m2rm9"
Jul 29 17:57:43.410761 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:43.410650    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-m2rm9"
Jul 29 17:57:43.574788 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:43.574755    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-m2rm9"]
Jul 29 17:57:43.578231 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:57:43.578206    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9fab3dd_1a95_44c9_aef0_2dcb6fa72512.slice/crio-ef85e3e92175c1d211b4364b17db99c4e2e21c4ecb415bb8c5a48058e6941b62 WatchSource:0}: Error finding container ef85e3e92175c1d211b4364b17db99c4e2e21c4ecb415bb8c5a48058e6941b62: Status 404 returned error can't find the container with id ef85e3e92175c1d211b4364b17db99c4e2e21c4ecb415bb8c5a48058e6941b62
Jul 29 17:57:44.144830 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:44.144798    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-m2rm9" event={"ID":"d9fab3dd-1a95-44c9-aef0-2dcb6fa72512","Type":"ContainerStarted","Data":"ef85e3e92175c1d211b4364b17db99c4e2e21c4ecb415bb8c5a48058e6941b62"}
Jul 29 17:57:46.155537 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:46.155499    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-m2rm9" event={"ID":"d9fab3dd-1a95-44c9-aef0-2dcb6fa72512","Type":"ContainerStarted","Data":"f096167637405f546c0fb8dc57a3c9004f1c79126363bd0bda5a4997e83597cb"}
Jul 29 17:57:46.178455 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:46.178410    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-m2rm9" podStartSLOduration=1.179272005 podStartE2EDuration="3.178396457s" totalImagesPullingTime="1.999124452s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:57:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:57:46.176924042 +0000 UTC m=+346.046146849" watchObservedRunningTime="2026-07-29 17:57:46.178396457 +0000 UTC m=+346.047619263"
Jul 29 17:57:47.004253 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:47.004217    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj96l"]
Jul 29 17:57:47.007513 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:47.007494    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj96l"
Jul 29 17:57:47.038132 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:47.038109    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 29 17:57:47.040631 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:47.040615    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 29 17:57:47.041504 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:47.041491    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-rrbkn\""
Jul 29 17:57:47.087544 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:47.087521    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj96l"]
Jul 29 17:57:47.103177 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:47.103154    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e6b4fb22-40a8-4073-928d-f2ab37495ca9-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj96l\" (UID: \"e6b4fb22-40a8-4073-928d-f2ab37495ca9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj96l"
Jul 29 17:57:47.103289 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:47.103189    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9sxw\" (UniqueName: \"kubernetes.io/projected/e6b4fb22-40a8-4073-928d-f2ab37495ca9-kube-api-access-v9sxw\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj96l\" (UID: \"e6b4fb22-40a8-4073-928d-f2ab37495ca9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj96l"
Jul 29 17:57:47.103289 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:47.103228    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e6b4fb22-40a8-4073-928d-f2ab37495ca9-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj96l\" (UID: \"e6b4fb22-40a8-4073-928d-f2ab37495ca9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj96l"
Jul 29 17:57:47.204182 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:47.204146    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e6b4fb22-40a8-4073-928d-f2ab37495ca9-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj96l\" (UID: \"e6b4fb22-40a8-4073-928d-f2ab37495ca9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj96l"
Jul 29 17:57:47.204650 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:47.204190    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9sxw\" (UniqueName: \"kubernetes.io/projected/e6b4fb22-40a8-4073-928d-f2ab37495ca9-kube-api-access-v9sxw\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj96l\" (UID: \"e6b4fb22-40a8-4073-928d-f2ab37495ca9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj96l"
Jul 29 17:57:47.204650 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:47.204246    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e6b4fb22-40a8-4073-928d-f2ab37495ca9-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj96l\" (UID: \"e6b4fb22-40a8-4073-928d-f2ab37495ca9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj96l"
Jul 29 17:57:47.204650 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:47.204580    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e6b4fb22-40a8-4073-928d-f2ab37495ca9-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj96l\" (UID: \"e6b4fb22-40a8-4073-928d-f2ab37495ca9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj96l"
Jul 29 17:57:47.204650 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:47.204607    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e6b4fb22-40a8-4073-928d-f2ab37495ca9-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj96l\" (UID: \"e6b4fb22-40a8-4073-928d-f2ab37495ca9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj96l"
Jul 29 17:57:47.213227 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:47.213206    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9sxw\" (UniqueName: \"kubernetes.io/projected/e6b4fb22-40a8-4073-928d-f2ab37495ca9-kube-api-access-v9sxw\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj96l\" (UID: \"e6b4fb22-40a8-4073-928d-f2ab37495ca9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj96l"
Jul 29 17:57:47.316522 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:47.316496    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj96l"
Jul 29 17:57:47.442747 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:47.442701    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj96l"]
Jul 29 17:57:47.444961 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:57:47.444931    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6b4fb22_40a8_4073_928d_f2ab37495ca9.slice/crio-5048304c049d59c9caaafba9bddbeeb05d1aca26daa41517b5c249a38f42188a WatchSource:0}: Error finding container 5048304c049d59c9caaafba9bddbeeb05d1aca26daa41517b5c249a38f42188a: Status 404 returned error can't find the container with id 5048304c049d59c9caaafba9bddbeeb05d1aca26daa41517b5c249a38f42188a
Jul 29 17:57:48.165278 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:48.165238    2557 generic.go:356] "Generic (PLEG): container finished" podID="e6b4fb22-40a8-4073-928d-f2ab37495ca9" containerID="00c612b4c414cec20b178433e687dca780dd1af95665846bb04761e58ba3fc98" exitCode=0
Jul 29 17:57:48.165474 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:48.165332    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj96l" event={"ID":"e6b4fb22-40a8-4073-928d-f2ab37495ca9","Type":"ContainerDied","Data":"00c612b4c414cec20b178433e687dca780dd1af95665846bb04761e58ba3fc98"}
Jul 29 17:57:48.165474 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:48.165374    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj96l" event={"ID":"e6b4fb22-40a8-4073-928d-f2ab37495ca9","Type":"ContainerStarted","Data":"5048304c049d59c9caaafba9bddbeeb05d1aca26daa41517b5c249a38f42188a"}
Jul 29 17:57:50.233633 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:50.233591    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-sxb4k"]
Jul 29 17:57:50.236865 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:50.236849    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-sxb4k"
Jul 29 17:57:50.239431 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:50.239392    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-vk42q\""
Jul 29 17:57:50.239431 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:50.239424    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 29 17:57:50.239612 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:50.239417    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 29 17:57:50.248665 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:50.248644    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-sxb4k"]
Jul 29 17:57:50.331840 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:50.331804    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pzcw\" (UniqueName: \"kubernetes.io/projected/c1b52722-835a-449a-ae3a-69705d060899-kube-api-access-5pzcw\") pod \"cert-manager-cainjector-559cdd8c87-sxb4k\" (UID: \"c1b52722-835a-449a-ae3a-69705d060899\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-sxb4k"
Jul 29 17:57:50.331840 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:50.331845    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c1b52722-835a-449a-ae3a-69705d060899-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-sxb4k\" (UID: \"c1b52722-835a-449a-ae3a-69705d060899\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-sxb4k"
Jul 29 17:57:50.432981 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:50.432953    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pzcw\" (UniqueName: \"kubernetes.io/projected/c1b52722-835a-449a-ae3a-69705d060899-kube-api-access-5pzcw\") pod \"cert-manager-cainjector-559cdd8c87-sxb4k\" (UID: \"c1b52722-835a-449a-ae3a-69705d060899\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-sxb4k"
Jul 29 17:57:50.433159 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:50.432999    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c1b52722-835a-449a-ae3a-69705d060899-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-sxb4k\" (UID: \"c1b52722-835a-449a-ae3a-69705d060899\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-sxb4k"
Jul 29 17:57:50.441070 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:50.441036    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c1b52722-835a-449a-ae3a-69705d060899-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-sxb4k\" (UID: \"c1b52722-835a-449a-ae3a-69705d060899\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-sxb4k"
Jul 29 17:57:50.441182 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:50.441117    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pzcw\" (UniqueName: \"kubernetes.io/projected/c1b52722-835a-449a-ae3a-69705d060899-kube-api-access-5pzcw\") pod \"cert-manager-cainjector-559cdd8c87-sxb4k\" (UID: \"c1b52722-835a-449a-ae3a-69705d060899\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-sxb4k"
Jul 29 17:57:50.545959 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:50.545929    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-sxb4k"
Jul 29 17:57:50.710961 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:50.710934    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-sxb4k"]
Jul 29 17:57:50.713206 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:57:50.713173    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc1b52722_835a_449a_ae3a_69705d060899.slice/crio-f5dd46d18463016de7d0b414deb08b5f40f0a2729d28a5eaf537f2bff9738ee9 WatchSource:0}: Error finding container f5dd46d18463016de7d0b414deb08b5f40f0a2729d28a5eaf537f2bff9738ee9: Status 404 returned error can't find the container with id f5dd46d18463016de7d0b414deb08b5f40f0a2729d28a5eaf537f2bff9738ee9
Jul 29 17:57:51.180348 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:51.180314    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-sxb4k" event={"ID":"c1b52722-835a-449a-ae3a-69705d060899","Type":"ContainerStarted","Data":"f5dd46d18463016de7d0b414deb08b5f40f0a2729d28a5eaf537f2bff9738ee9"}
Jul 29 17:57:52.187178 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:52.187108    2557 generic.go:356] "Generic (PLEG): container finished" podID="e6b4fb22-40a8-4073-928d-f2ab37495ca9" containerID="90b1d6fd48f32c6e83aea3c7facf66e12fc011e4d05886d2dd113269b4e7a78b" exitCode=0
Jul 29 17:57:52.187619 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:52.187208    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj96l" event={"ID":"e6b4fb22-40a8-4073-928d-f2ab37495ca9","Type":"ContainerDied","Data":"90b1d6fd48f32c6e83aea3c7facf66e12fc011e4d05886d2dd113269b4e7a78b"}
Jul 29 17:57:53.194691 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:53.194654    2557 generic.go:356] "Generic (PLEG): container finished" podID="e6b4fb22-40a8-4073-928d-f2ab37495ca9" containerID="a5e23e6b9acb0cae54446a81706fafb599cc235aa0670b8c4f9c8acd08e9e6b3" exitCode=0
Jul 29 17:57:53.195130 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:53.194755    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj96l" event={"ID":"e6b4fb22-40a8-4073-928d-f2ab37495ca9","Type":"ContainerDied","Data":"a5e23e6b9acb0cae54446a81706fafb599cc235aa0670b8c4f9c8acd08e9e6b3"}
Jul 29 17:57:54.201010 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:54.200966    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-sxb4k" event={"ID":"c1b52722-835a-449a-ae3a-69705d060899","Type":"ContainerStarted","Data":"9e40118e44a8e24a7e17f7db1b055b7204769eef2d8c3c1f21c74b4f99a06aa1"}
Jul 29 17:57:54.220010 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:54.219950    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-sxb4k" podStartSLOduration=0.874717371 podStartE2EDuration="4.219928842s" totalImagesPullingTime="3.345211471s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:57:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:57:54.218818171 +0000 UTC m=+354.088040981" watchObservedRunningTime="2026-07-29 17:57:54.219928842 +0000 UTC m=+354.089151650"
Jul 29 17:57:54.344978 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:54.344954    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj96l"
Jul 29 17:57:54.370958 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:54.370933    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v9sxw\" (UniqueName: \"kubernetes.io/projected/e6b4fb22-40a8-4073-928d-f2ab37495ca9-kube-api-access-v9sxw\") pod \"e6b4fb22-40a8-4073-928d-f2ab37495ca9\" (UID: \"e6b4fb22-40a8-4073-928d-f2ab37495ca9\") "
Jul 29 17:57:54.371094 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:54.370989    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e6b4fb22-40a8-4073-928d-f2ab37495ca9-bundle\") pod \"e6b4fb22-40a8-4073-928d-f2ab37495ca9\" (UID: \"e6b4fb22-40a8-4073-928d-f2ab37495ca9\") "
Jul 29 17:57:54.371094 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:54.371012    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e6b4fb22-40a8-4073-928d-f2ab37495ca9-util\") pod \"e6b4fb22-40a8-4073-928d-f2ab37495ca9\" (UID: \"e6b4fb22-40a8-4073-928d-f2ab37495ca9\") "
Jul 29 17:57:54.371411 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:54.371376    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6b4fb22-40a8-4073-928d-f2ab37495ca9-bundle" (OuterVolumeSpecName: "bundle") pod "e6b4fb22-40a8-4073-928d-f2ab37495ca9" (UID: "e6b4fb22-40a8-4073-928d-f2ab37495ca9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 17:57:54.373093 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:54.373063    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6b4fb22-40a8-4073-928d-f2ab37495ca9-kube-api-access-v9sxw" (OuterVolumeSpecName: "kube-api-access-v9sxw") pod "e6b4fb22-40a8-4073-928d-f2ab37495ca9" (UID: "e6b4fb22-40a8-4073-928d-f2ab37495ca9"). InnerVolumeSpecName "kube-api-access-v9sxw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 17:57:54.376673 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:54.376645    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6b4fb22-40a8-4073-928d-f2ab37495ca9-util" (OuterVolumeSpecName: "util") pod "e6b4fb22-40a8-4073-928d-f2ab37495ca9" (UID: "e6b4fb22-40a8-4073-928d-f2ab37495ca9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 17:57:54.472216 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:54.472176    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v9sxw\" (UniqueName: \"kubernetes.io/projected/e6b4fb22-40a8-4073-928d-f2ab37495ca9-kube-api-access-v9sxw\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 17:57:54.472216 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:54.472203    2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e6b4fb22-40a8-4073-928d-f2ab37495ca9-bundle\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 17:57:54.472216 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:54.472212    2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e6b4fb22-40a8-4073-928d-f2ab37495ca9-util\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 17:57:55.207097 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:55.207065    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj96l"
Jul 29 17:57:55.207097 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:55.207092    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj96l" event={"ID":"e6b4fb22-40a8-4073-928d-f2ab37495ca9","Type":"ContainerDied","Data":"5048304c049d59c9caaafba9bddbeeb05d1aca26daa41517b5c249a38f42188a"}
Jul 29 17:57:55.207636 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:57:55.207135    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5048304c049d59c9caaafba9bddbeeb05d1aca26daa41517b5c249a38f42188a"
Jul 29 17:58:10.061069 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:10.061033    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9fwf6"]
Jul 29 17:58:10.061444 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:10.061309    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6b4fb22-40a8-4073-928d-f2ab37495ca9" containerName="pull"
Jul 29 17:58:10.061444 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:10.061321    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6b4fb22-40a8-4073-928d-f2ab37495ca9" containerName="pull"
Jul 29 17:58:10.061444 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:10.061331    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6b4fb22-40a8-4073-928d-f2ab37495ca9" containerName="extract"
Jul 29 17:58:10.061444 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:10.061336    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6b4fb22-40a8-4073-928d-f2ab37495ca9" containerName="extract"
Jul 29 17:58:10.061444 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:10.061350    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6b4fb22-40a8-4073-928d-f2ab37495ca9" containerName="util"
Jul 29 17:58:10.061444 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:10.061356    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6b4fb22-40a8-4073-928d-f2ab37495ca9" containerName="util"
Jul 29 17:58:10.061444 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:10.061405    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6b4fb22-40a8-4073-928d-f2ab37495ca9" containerName="extract"
Jul 29 17:58:10.064374 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:10.064357    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9fwf6"
Jul 29 17:58:10.067023 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:10.066994    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 29 17:58:10.067143 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:10.067080    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 29 17:58:10.068209 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:10.068193    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-rrbkn\""
Jul 29 17:58:10.072253 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:10.072233    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9fwf6"]
Jul 29 17:58:10.196342 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:10.196304    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f71534bc-5c58-480a-8726-43af79e9f7e5-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9fwf6\" (UID: \"f71534bc-5c58-480a-8726-43af79e9f7e5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9fwf6"
Jul 29 17:58:10.196342 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:10.196349    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nb6b\" (UniqueName: \"kubernetes.io/projected/f71534bc-5c58-480a-8726-43af79e9f7e5-kube-api-access-8nb6b\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9fwf6\" (UID: \"f71534bc-5c58-480a-8726-43af79e9f7e5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9fwf6"
Jul 29 17:58:10.196584 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:10.196368    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f71534bc-5c58-480a-8726-43af79e9f7e5-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9fwf6\" (UID: \"f71534bc-5c58-480a-8726-43af79e9f7e5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9fwf6"
Jul 29 17:58:10.297669 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:10.297635    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f71534bc-5c58-480a-8726-43af79e9f7e5-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9fwf6\" (UID: \"f71534bc-5c58-480a-8726-43af79e9f7e5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9fwf6"
Jul 29 17:58:10.297871 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:10.297679    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nb6b\" (UniqueName: \"kubernetes.io/projected/f71534bc-5c58-480a-8726-43af79e9f7e5-kube-api-access-8nb6b\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9fwf6\" (UID: \"f71534bc-5c58-480a-8726-43af79e9f7e5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9fwf6"
Jul 29 17:58:10.297871 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:10.297701    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f71534bc-5c58-480a-8726-43af79e9f7e5-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9fwf6\" (UID: \"f71534bc-5c58-480a-8726-43af79e9f7e5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9fwf6"
Jul 29 17:58:10.298102 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:10.298079    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f71534bc-5c58-480a-8726-43af79e9f7e5-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9fwf6\" (UID: \"f71534bc-5c58-480a-8726-43af79e9f7e5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9fwf6"
Jul 29 17:58:10.298144 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:10.298089    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f71534bc-5c58-480a-8726-43af79e9f7e5-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9fwf6\" (UID: \"f71534bc-5c58-480a-8726-43af79e9f7e5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9fwf6"
Jul 29 17:58:10.306076 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:10.306051    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nb6b\" (UniqueName: \"kubernetes.io/projected/f71534bc-5c58-480a-8726-43af79e9f7e5-kube-api-access-8nb6b\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9fwf6\" (UID: \"f71534bc-5c58-480a-8726-43af79e9f7e5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9fwf6"
Jul 29 17:58:10.374133 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:10.374058    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9fwf6"
Jul 29 17:58:10.504075 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:10.504046    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9fwf6"]
Jul 29 17:58:10.506962 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:58:10.506928    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf71534bc_5c58_480a_8726_43af79e9f7e5.slice/crio-d26d90a3b565040606308f3f614a90d2fe058a29515d575f59355112f349bda0 WatchSource:0}: Error finding container d26d90a3b565040606308f3f614a90d2fe058a29515d575f59355112f349bda0: Status 404 returned error can't find the container with id d26d90a3b565040606308f3f614a90d2fe058a29515d575f59355112f349bda0
Jul 29 17:58:11.277127 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:11.277090    2557 generic.go:356] "Generic (PLEG): container finished" podID="f71534bc-5c58-480a-8726-43af79e9f7e5" containerID="6cec933e802993da440f7b63dfeb1f2c204644744b9b4b9448cf0bca55959130" exitCode=0
Jul 29 17:58:11.277525 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:11.277166    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9fwf6" event={"ID":"f71534bc-5c58-480a-8726-43af79e9f7e5","Type":"ContainerDied","Data":"6cec933e802993da440f7b63dfeb1f2c204644744b9b4b9448cf0bca55959130"}
Jul 29 17:58:11.277525 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:11.277210    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9fwf6" event={"ID":"f71534bc-5c58-480a-8726-43af79e9f7e5","Type":"ContainerStarted","Data":"d26d90a3b565040606308f3f614a90d2fe058a29515d575f59355112f349bda0"}
Jul 29 17:58:12.283389 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:12.283305    2557 generic.go:356] "Generic (PLEG): container finished" podID="f71534bc-5c58-480a-8726-43af79e9f7e5" containerID="21f9b8a21b6144d83ef7f523d01ca6f5b617e7a49072b7023a4dbc100746d17a" exitCode=0
Jul 29 17:58:12.283746 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:12.283386    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9fwf6" event={"ID":"f71534bc-5c58-480a-8726-43af79e9f7e5","Type":"ContainerDied","Data":"21f9b8a21b6144d83ef7f523d01ca6f5b617e7a49072b7023a4dbc100746d17a"}
Jul 29 17:58:13.289557 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:13.289522    2557 generic.go:356] "Generic (PLEG): container finished" podID="f71534bc-5c58-480a-8726-43af79e9f7e5" containerID="9d11bce8af636de45e31db83bf76baabeef633e1c330b4f40023331d26b88ae9" exitCode=0
Jul 29 17:58:13.289964 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:13.289599    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9fwf6" event={"ID":"f71534bc-5c58-480a-8726-43af79e9f7e5","Type":"ContainerDied","Data":"9d11bce8af636de45e31db83bf76baabeef633e1c330b4f40023331d26b88ae9"}
Jul 29 17:58:14.413707 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:14.413682    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9fwf6"
Jul 29 17:58:14.533057 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:14.533025    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f71534bc-5c58-480a-8726-43af79e9f7e5-util\") pod \"f71534bc-5c58-480a-8726-43af79e9f7e5\" (UID: \"f71534bc-5c58-480a-8726-43af79e9f7e5\") "
Jul 29 17:58:14.533213 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:14.533086    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8nb6b\" (UniqueName: \"kubernetes.io/projected/f71534bc-5c58-480a-8726-43af79e9f7e5-kube-api-access-8nb6b\") pod \"f71534bc-5c58-480a-8726-43af79e9f7e5\" (UID: \"f71534bc-5c58-480a-8726-43af79e9f7e5\") "
Jul 29 17:58:14.533213 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:14.533124    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f71534bc-5c58-480a-8726-43af79e9f7e5-bundle\") pod \"f71534bc-5c58-480a-8726-43af79e9f7e5\" (UID: \"f71534bc-5c58-480a-8726-43af79e9f7e5\") "
Jul 29 17:58:14.533834 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:14.533805    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f71534bc-5c58-480a-8726-43af79e9f7e5-bundle" (OuterVolumeSpecName: "bundle") pod "f71534bc-5c58-480a-8726-43af79e9f7e5" (UID: "f71534bc-5c58-480a-8726-43af79e9f7e5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 17:58:14.535266 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:14.535242    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f71534bc-5c58-480a-8726-43af79e9f7e5-kube-api-access-8nb6b" (OuterVolumeSpecName: "kube-api-access-8nb6b") pod "f71534bc-5c58-480a-8726-43af79e9f7e5" (UID: "f71534bc-5c58-480a-8726-43af79e9f7e5"). InnerVolumeSpecName "kube-api-access-8nb6b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 17:58:14.538318 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:14.538285    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f71534bc-5c58-480a-8726-43af79e9f7e5-util" (OuterVolumeSpecName: "util") pod "f71534bc-5c58-480a-8726-43af79e9f7e5" (UID: "f71534bc-5c58-480a-8726-43af79e9f7e5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 17:58:14.633941 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:14.633861    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8nb6b\" (UniqueName: \"kubernetes.io/projected/f71534bc-5c58-480a-8726-43af79e9f7e5-kube-api-access-8nb6b\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 17:58:14.633941 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:14.633889    2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f71534bc-5c58-480a-8726-43af79e9f7e5-bundle\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 17:58:14.633941 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:14.633899    2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f71534bc-5c58-480a-8726-43af79e9f7e5-util\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 17:58:15.299995 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:15.299967    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9fwf6"
Jul 29 17:58:15.299995 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:15.299973    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9fwf6" event={"ID":"f71534bc-5c58-480a-8726-43af79e9f7e5","Type":"ContainerDied","Data":"d26d90a3b565040606308f3f614a90d2fe058a29515d575f59355112f349bda0"}
Jul 29 17:58:15.300210 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:15.300013    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d26d90a3b565040606308f3f614a90d2fe058a29515d575f59355112f349bda0"
Jul 29 17:58:27.527158 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:27.527124    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d84c6c79c-qq57z"]
Jul 29 17:58:27.527661 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:27.527410    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f71534bc-5c58-480a-8726-43af79e9f7e5" containerName="pull"
Jul 29 17:58:27.527661 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:27.527420    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="f71534bc-5c58-480a-8726-43af79e9f7e5" containerName="pull"
Jul 29 17:58:27.527661 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:27.527431    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f71534bc-5c58-480a-8726-43af79e9f7e5" containerName="extract"
Jul 29 17:58:27.527661 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:27.527451    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="f71534bc-5c58-480a-8726-43af79e9f7e5" containerName="extract"
Jul 29 17:58:27.527661 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:27.527463    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f71534bc-5c58-480a-8726-43af79e9f7e5" containerName="util"
Jul 29 17:58:27.527661 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:27.527469    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="f71534bc-5c58-480a-8726-43af79e9f7e5" containerName="util"
Jul 29 17:58:27.527661 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:27.527518    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="f71534bc-5c58-480a-8726-43af79e9f7e5" containerName="extract"
Jul 29 17:58:27.532447 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:27.532428    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-qq57z"
Jul 29 17:58:27.536700 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:27.536679    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 29 17:58:27.536840 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:27.536713    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-9q8v4\""
Jul 29 17:58:27.537013 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:27.536995    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 29 17:58:27.537262 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:27.537249    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 29 17:58:27.538097 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:27.538081    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 29 17:58:27.548229 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:27.548205    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d84c6c79c-qq57z"]
Jul 29 17:58:27.641902 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:27.641869    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/49eb35f6-74a3-42ba-b8de-d69123427363-manifests\") pod \"opendatahub-operator-controller-manager-d84c6c79c-qq57z\" (UID: \"49eb35f6-74a3-42ba-b8de-d69123427363\") " pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-qq57z"
Jul 29 17:58:27.642104 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:27.641916    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58525\" (UniqueName: \"kubernetes.io/projected/49eb35f6-74a3-42ba-b8de-d69123427363-kube-api-access-58525\") pod \"opendatahub-operator-controller-manager-d84c6c79c-qq57z\" (UID: \"49eb35f6-74a3-42ba-b8de-d69123427363\") " pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-qq57z"
Jul 29 17:58:27.642104 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:27.641938    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/49eb35f6-74a3-42ba-b8de-d69123427363-tmp\") pod \"opendatahub-operator-controller-manager-d84c6c79c-qq57z\" (UID: \"49eb35f6-74a3-42ba-b8de-d69123427363\") " pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-qq57z"
Jul 29 17:58:27.642104 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:27.642055    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/49eb35f6-74a3-42ba-b8de-d69123427363-webhook-cert\") pod \"opendatahub-operator-controller-manager-d84c6c79c-qq57z\" (UID: \"49eb35f6-74a3-42ba-b8de-d69123427363\") " pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-qq57z"
Jul 29 17:58:27.642104 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:27.642089    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/49eb35f6-74a3-42ba-b8de-d69123427363-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d84c6c79c-qq57z\" (UID: \"49eb35f6-74a3-42ba-b8de-d69123427363\") " pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-qq57z"
Jul 29 17:58:27.742874 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:27.742841    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/49eb35f6-74a3-42ba-b8de-d69123427363-manifests\") pod \"opendatahub-operator-controller-manager-d84c6c79c-qq57z\" (UID: \"49eb35f6-74a3-42ba-b8de-d69123427363\") " pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-qq57z"
Jul 29 17:58:27.742874 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:27.742879    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58525\" (UniqueName: \"kubernetes.io/projected/49eb35f6-74a3-42ba-b8de-d69123427363-kube-api-access-58525\") pod \"opendatahub-operator-controller-manager-d84c6c79c-qq57z\" (UID: \"49eb35f6-74a3-42ba-b8de-d69123427363\") " pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-qq57z"
Jul 29 17:58:27.743130 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:27.742897    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/49eb35f6-74a3-42ba-b8de-d69123427363-tmp\") pod \"opendatahub-operator-controller-manager-d84c6c79c-qq57z\" (UID: \"49eb35f6-74a3-42ba-b8de-d69123427363\") " pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-qq57z"
Jul 29 17:58:27.743130 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:27.742981    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/49eb35f6-74a3-42ba-b8de-d69123427363-webhook-cert\") pod \"opendatahub-operator-controller-manager-d84c6c79c-qq57z\" (UID: \"49eb35f6-74a3-42ba-b8de-d69123427363\") " pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-qq57z"
Jul 29 17:58:27.743130 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:27.743010    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/49eb35f6-74a3-42ba-b8de-d69123427363-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d84c6c79c-qq57z\" (UID: \"49eb35f6-74a3-42ba-b8de-d69123427363\") " pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-qq57z"
Jul 29 17:58:27.743269 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:27.743247    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/49eb35f6-74a3-42ba-b8de-d69123427363-manifests\") pod \"opendatahub-operator-controller-manager-d84c6c79c-qq57z\" (UID: \"49eb35f6-74a3-42ba-b8de-d69123427363\") " pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-qq57z"
Jul 29 17:58:27.745353 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:27.745327    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/49eb35f6-74a3-42ba-b8de-d69123427363-tmp\") pod \"opendatahub-operator-controller-manager-d84c6c79c-qq57z\" (UID: \"49eb35f6-74a3-42ba-b8de-d69123427363\") " pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-qq57z"
Jul 29 17:58:27.745505 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:27.745488    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/49eb35f6-74a3-42ba-b8de-d69123427363-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d84c6c79c-qq57z\" (UID: \"49eb35f6-74a3-42ba-b8de-d69123427363\") " pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-qq57z"
Jul 29 17:58:27.745612 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:27.745592    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/49eb35f6-74a3-42ba-b8de-d69123427363-webhook-cert\") pod \"opendatahub-operator-controller-manager-d84c6c79c-qq57z\" (UID: \"49eb35f6-74a3-42ba-b8de-d69123427363\") " pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-qq57z"
Jul 29 17:58:27.750395 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:27.750377    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-58525\" (UniqueName: \"kubernetes.io/projected/49eb35f6-74a3-42ba-b8de-d69123427363-kube-api-access-58525\") pod \"opendatahub-operator-controller-manager-d84c6c79c-qq57z\" (UID: \"49eb35f6-74a3-42ba-b8de-d69123427363\") " pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-qq57z"
Jul 29 17:58:27.842883 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:27.842848    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-qq57z"
Jul 29 17:58:28.011074 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:28.011050    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d84c6c79c-qq57z"]
Jul 29 17:58:28.013073 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:58:28.013048    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49eb35f6_74a3_42ba_b8de_d69123427363.slice/crio-1d1922bb4d880ea1a9e38cee32ee5f6c1b065005a8bc7b1dbb53f4fd47867087 WatchSource:0}: Error finding container 1d1922bb4d880ea1a9e38cee32ee5f6c1b065005a8bc7b1dbb53f4fd47867087: Status 404 returned error can't find the container with id 1d1922bb4d880ea1a9e38cee32ee5f6c1b065005a8bc7b1dbb53f4fd47867087
Jul 29 17:58:28.356484 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:28.356447    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-qq57z" event={"ID":"49eb35f6-74a3-42ba-b8de-d69123427363","Type":"ContainerStarted","Data":"1d1922bb4d880ea1a9e38cee32ee5f6c1b065005a8bc7b1dbb53f4fd47867087"}
Jul 29 17:58:31.373521 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:31.373491    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-qq57z" event={"ID":"49eb35f6-74a3-42ba-b8de-d69123427363","Type":"ContainerStarted","Data":"fe9a8627a698ec05aacc67846ad3c146faa2ee4a8a0e0611b2f56905f0aee4eb"}
Jul 29 17:58:33.384532 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:33.384495    2557 generic.go:356] "Generic (PLEG): container finished" podID="49eb35f6-74a3-42ba-b8de-d69123427363" containerID="fe9a8627a698ec05aacc67846ad3c146faa2ee4a8a0e0611b2f56905f0aee4eb" exitCode=0
Jul 29 17:58:33.384532 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:33.384522    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-qq57z" event={"ID":"49eb35f6-74a3-42ba-b8de-d69123427363","Type":"ContainerDied","Data":"fe9a8627a698ec05aacc67846ad3c146faa2ee4a8a0e0611b2f56905f0aee4eb"}
Jul 29 17:58:34.391818 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:34.391785    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-qq57z" event={"ID":"49eb35f6-74a3-42ba-b8de-d69123427363","Type":"ContainerStarted","Data":"2cb056e5540825cb67bbc55cacc6ba038a7579c9eb8cf510bc2acde1aba7e672"}
Jul 29 17:58:34.392287 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:34.392009    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-qq57z"
Jul 29 17:58:34.419601 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:34.419547    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-qq57z" podStartSLOduration=2.767580545 podStartE2EDuration="7.419527136s" totalImagesPullingTime="3.564276366s" totalInitContainerRuntime="1.087670225s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:58:27 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:58:34.417020922 +0000 UTC m=+394.286243742" watchObservedRunningTime="2026-07-29 17:58:34.419527136 +0000 UTC m=+394.288749948"
Jul 29 17:58:45.399029 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:45.399001    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-qq57z"
Jul 29 17:58:47.646837 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:47.646795    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-869f956dcc-fp29f"]
Jul 29 17:58:47.654765 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:47.654745    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-869f956dcc-fp29f"
Jul 29 17:58:47.659008 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:47.658988    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 29 17:58:47.659123 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:47.658993    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 29 17:58:47.675099 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:47.675078    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-869f956dcc-fp29f"]
Jul 29 17:58:47.698507 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:47.698486    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkwbq\" (UniqueName: \"kubernetes.io/projected/3877a269-9027-424f-8903-b06036bbbcb2-kube-api-access-vkwbq\") pod \"kube-auth-proxy-869f956dcc-fp29f\" (UID: \"3877a269-9027-424f-8903-b06036bbbcb2\") " pod="openshift-ingress/kube-auth-proxy-869f956dcc-fp29f"
Jul 29 17:58:47.698606 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:47.698513    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3877a269-9027-424f-8903-b06036bbbcb2-tmp\") pod \"kube-auth-proxy-869f956dcc-fp29f\" (UID: \"3877a269-9027-424f-8903-b06036bbbcb2\") " pod="openshift-ingress/kube-auth-proxy-869f956dcc-fp29f"
Jul 29 17:58:47.698606 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:47.698594    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3877a269-9027-424f-8903-b06036bbbcb2-tls-certs\") pod \"kube-auth-proxy-869f956dcc-fp29f\" (UID: \"3877a269-9027-424f-8903-b06036bbbcb2\") " pod="openshift-ingress/kube-auth-proxy-869f956dcc-fp29f"
Jul 29 17:58:47.799901 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:47.799856    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3877a269-9027-424f-8903-b06036bbbcb2-tls-certs\") pod \"kube-auth-proxy-869f956dcc-fp29f\" (UID: \"3877a269-9027-424f-8903-b06036bbbcb2\") " pod="openshift-ingress/kube-auth-proxy-869f956dcc-fp29f"
Jul 29 17:58:47.800058 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:47.799944    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkwbq\" (UniqueName: \"kubernetes.io/projected/3877a269-9027-424f-8903-b06036bbbcb2-kube-api-access-vkwbq\") pod \"kube-auth-proxy-869f956dcc-fp29f\" (UID: \"3877a269-9027-424f-8903-b06036bbbcb2\") " pod="openshift-ingress/kube-auth-proxy-869f956dcc-fp29f"
Jul 29 17:58:47.800058 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:47.799969    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3877a269-9027-424f-8903-b06036bbbcb2-tmp\") pod \"kube-auth-proxy-869f956dcc-fp29f\" (UID: \"3877a269-9027-424f-8903-b06036bbbcb2\") " pod="openshift-ingress/kube-auth-proxy-869f956dcc-fp29f"
Jul 29 17:58:47.802262 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:47.802242    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3877a269-9027-424f-8903-b06036bbbcb2-tmp\") pod \"kube-auth-proxy-869f956dcc-fp29f\" (UID: \"3877a269-9027-424f-8903-b06036bbbcb2\") " pod="openshift-ingress/kube-auth-proxy-869f956dcc-fp29f"
Jul 29 17:58:47.802365 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:47.802348    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3877a269-9027-424f-8903-b06036bbbcb2-tls-certs\") pod \"kube-auth-proxy-869f956dcc-fp29f\" (UID: \"3877a269-9027-424f-8903-b06036bbbcb2\") " pod="openshift-ingress/kube-auth-proxy-869f956dcc-fp29f"
Jul 29 17:58:47.811372 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:47.811338    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkwbq\" (UniqueName: \"kubernetes.io/projected/3877a269-9027-424f-8903-b06036bbbcb2-kube-api-access-vkwbq\") pod \"kube-auth-proxy-869f956dcc-fp29f\" (UID: \"3877a269-9027-424f-8903-b06036bbbcb2\") " pod="openshift-ingress/kube-auth-proxy-869f956dcc-fp29f"
Jul 29 17:58:47.963681 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:47.963600    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-869f956dcc-fp29f"
Jul 29 17:58:47.965451 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:47.965425    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-672fv"]
Jul 29 17:58:47.968719 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:47.968701    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-672fv"
Jul 29 17:58:47.972963 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:47.972898    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 29 17:58:47.972963 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:47.972945    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-b7hnf\""
Jul 29 17:58:48.001441 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:48.001407    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5645b830-d2f9-4164-8e78-a7be8c9af2b5-cert\") pod \"odh-model-controller-7998c67dc4-672fv\" (UID: \"5645b830-d2f9-4164-8e78-a7be8c9af2b5\") " pod="opendatahub/odh-model-controller-7998c67dc4-672fv"
Jul 29 17:58:48.001576 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:48.001465    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzhw9\" (UniqueName: \"kubernetes.io/projected/5645b830-d2f9-4164-8e78-a7be8c9af2b5-kube-api-access-pzhw9\") pod \"odh-model-controller-7998c67dc4-672fv\" (UID: \"5645b830-d2f9-4164-8e78-a7be8c9af2b5\") " pod="opendatahub/odh-model-controller-7998c67dc4-672fv"
Jul 29 17:58:48.057301 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:48.057273    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-672fv"]
Jul 29 17:58:48.101858 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:48.101821    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzhw9\" (UniqueName: \"kubernetes.io/projected/5645b830-d2f9-4164-8e78-a7be8c9af2b5-kube-api-access-pzhw9\") pod \"odh-model-controller-7998c67dc4-672fv\" (UID: \"5645b830-d2f9-4164-8e78-a7be8c9af2b5\") " pod="opendatahub/odh-model-controller-7998c67dc4-672fv"
Jul 29 17:58:48.102021 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:48.101895    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5645b830-d2f9-4164-8e78-a7be8c9af2b5-cert\") pod \"odh-model-controller-7998c67dc4-672fv\" (UID: \"5645b830-d2f9-4164-8e78-a7be8c9af2b5\") " pod="opendatahub/odh-model-controller-7998c67dc4-672fv"
Jul 29 17:58:48.102082 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:58:48.102033    2557 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 29 17:58:48.102122 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:58:48.102095    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5645b830-d2f9-4164-8e78-a7be8c9af2b5-cert podName:5645b830-d2f9-4164-8e78-a7be8c9af2b5 nodeName:}" failed. No retries permitted until 2026-07-29 17:58:48.602079271 +0000 UTC m=+408.471302056 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/5645b830-d2f9-4164-8e78-a7be8c9af2b5-cert") pod "odh-model-controller-7998c67dc4-672fv" (UID: "5645b830-d2f9-4164-8e78-a7be8c9af2b5") : secret "odh-model-controller-webhook-cert" not found
Jul 29 17:58:48.115649 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:48.115627    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzhw9\" (UniqueName: \"kubernetes.io/projected/5645b830-d2f9-4164-8e78-a7be8c9af2b5-kube-api-access-pzhw9\") pod \"odh-model-controller-7998c67dc4-672fv\" (UID: \"5645b830-d2f9-4164-8e78-a7be8c9af2b5\") " pod="opendatahub/odh-model-controller-7998c67dc4-672fv"
Jul 29 17:58:48.122872 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:48.122853    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-869f956dcc-fp29f"]
Jul 29 17:58:48.124774 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:58:48.124754    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3877a269_9027_424f_8903_b06036bbbcb2.slice/crio-65ae49d8064964fff050cf987a3eae9a12efe061c79dfbda8d7070ad42b248a9 WatchSource:0}: Error finding container 65ae49d8064964fff050cf987a3eae9a12efe061c79dfbda8d7070ad42b248a9: Status 404 returned error can't find the container with id 65ae49d8064964fff050cf987a3eae9a12efe061c79dfbda8d7070ad42b248a9
Jul 29 17:58:48.454990 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:48.454954    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-869f956dcc-fp29f" event={"ID":"3877a269-9027-424f-8903-b06036bbbcb2","Type":"ContainerStarted","Data":"65ae49d8064964fff050cf987a3eae9a12efe061c79dfbda8d7070ad42b248a9"}
Jul 29 17:58:48.604835 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:48.604793    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5645b830-d2f9-4164-8e78-a7be8c9af2b5-cert\") pod \"odh-model-controller-7998c67dc4-672fv\" (UID: \"5645b830-d2f9-4164-8e78-a7be8c9af2b5\") " pod="opendatahub/odh-model-controller-7998c67dc4-672fv"
Jul 29 17:58:48.607281 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:48.607264    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5645b830-d2f9-4164-8e78-a7be8c9af2b5-cert\") pod \"odh-model-controller-7998c67dc4-672fv\" (UID: \"5645b830-d2f9-4164-8e78-a7be8c9af2b5\") " pod="opendatahub/odh-model-controller-7998c67dc4-672fv"
Jul 29 17:58:48.892693 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:48.892659    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-672fv"
Jul 29 17:58:48.986586 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:48.986547    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-mdsvb"]
Jul 29 17:58:48.991678 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:48.991645    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-mdsvb"
Jul 29 17:58:48.995805 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:48.995597    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 29 17:58:48.995805 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:48.995689    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-6thdc\""
Jul 29 17:58:49.009361 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:49.009293    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-mdsvb"]
Jul 29 17:58:49.009916 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:49.009893    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9x8j\" (UniqueName: \"kubernetes.io/projected/dad1b1dc-9f1a-4fbd-bbbc-860e74a95cd2-kube-api-access-p9x8j\") pod \"model-serving-api-54d89bd5fc-mdsvb\" (UID: \"dad1b1dc-9f1a-4fbd-bbbc-860e74a95cd2\") " pod="opendatahub/model-serving-api-54d89bd5fc-mdsvb"
Jul 29 17:58:49.010082 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:49.010059    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dad1b1dc-9f1a-4fbd-bbbc-860e74a95cd2-tls-certs\") pod \"model-serving-api-54d89bd5fc-mdsvb\" (UID: \"dad1b1dc-9f1a-4fbd-bbbc-860e74a95cd2\") " pod="opendatahub/model-serving-api-54d89bd5fc-mdsvb"
Jul 29 17:58:49.049636 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:49.049593    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-672fv"]
Jul 29 17:58:49.052014 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:58:49.051983    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5645b830_d2f9_4164_8e78_a7be8c9af2b5.slice/crio-e166181df8a2d457f32a5446048834a8cf90060acfd71e93cf29bc1cd138a053 WatchSource:0}: Error finding container e166181df8a2d457f32a5446048834a8cf90060acfd71e93cf29bc1cd138a053: Status 404 returned error can't find the container with id e166181df8a2d457f32a5446048834a8cf90060acfd71e93cf29bc1cd138a053
Jul 29 17:58:49.110979 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:49.110944    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9x8j\" (UniqueName: \"kubernetes.io/projected/dad1b1dc-9f1a-4fbd-bbbc-860e74a95cd2-kube-api-access-p9x8j\") pod \"model-serving-api-54d89bd5fc-mdsvb\" (UID: \"dad1b1dc-9f1a-4fbd-bbbc-860e74a95cd2\") " pod="opendatahub/model-serving-api-54d89bd5fc-mdsvb"
Jul 29 17:58:49.110979 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:49.110995    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dad1b1dc-9f1a-4fbd-bbbc-860e74a95cd2-tls-certs\") pod \"model-serving-api-54d89bd5fc-mdsvb\" (UID: \"dad1b1dc-9f1a-4fbd-bbbc-860e74a95cd2\") " pod="opendatahub/model-serving-api-54d89bd5fc-mdsvb"
Jul 29 17:58:49.111222 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:58:49.111151    2557 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 29 17:58:49.111222 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:58:49.111219    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dad1b1dc-9f1a-4fbd-bbbc-860e74a95cd2-tls-certs podName:dad1b1dc-9f1a-4fbd-bbbc-860e74a95cd2 nodeName:}" failed. No retries permitted until 2026-07-29 17:58:49.611200486 +0000 UTC m=+409.480423288 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/dad1b1dc-9f1a-4fbd-bbbc-860e74a95cd2-tls-certs") pod "model-serving-api-54d89bd5fc-mdsvb" (UID: "dad1b1dc-9f1a-4fbd-bbbc-860e74a95cd2") : secret "model-serving-api-tls" not found
Jul 29 17:58:49.133364 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:49.133337    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9x8j\" (UniqueName: \"kubernetes.io/projected/dad1b1dc-9f1a-4fbd-bbbc-860e74a95cd2-kube-api-access-p9x8j\") pod \"model-serving-api-54d89bd5fc-mdsvb\" (UID: \"dad1b1dc-9f1a-4fbd-bbbc-860e74a95cd2\") " pod="opendatahub/model-serving-api-54d89bd5fc-mdsvb"
Jul 29 17:58:49.461399 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:49.461350    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-672fv" event={"ID":"5645b830-d2f9-4164-8e78-a7be8c9af2b5","Type":"ContainerStarted","Data":"e166181df8a2d457f32a5446048834a8cf90060acfd71e93cf29bc1cd138a053"}
Jul 29 17:58:49.616199 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:49.616163    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dad1b1dc-9f1a-4fbd-bbbc-860e74a95cd2-tls-certs\") pod \"model-serving-api-54d89bd5fc-mdsvb\" (UID: \"dad1b1dc-9f1a-4fbd-bbbc-860e74a95cd2\") " pod="opendatahub/model-serving-api-54d89bd5fc-mdsvb"
Jul 29 17:58:49.619164 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:49.619135    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dad1b1dc-9f1a-4fbd-bbbc-860e74a95cd2-tls-certs\") pod \"model-serving-api-54d89bd5fc-mdsvb\" (UID: \"dad1b1dc-9f1a-4fbd-bbbc-860e74a95cd2\") " pod="opendatahub/model-serving-api-54d89bd5fc-mdsvb"
Jul 29 17:58:49.907603 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:49.907556    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-mdsvb"
Jul 29 17:58:50.078073 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:50.077608    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-mdsvb"]
Jul 29 17:58:50.809694 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:58:50.809655    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddad1b1dc_9f1a_4fbd_bbbc_860e74a95cd2.slice/crio-5c83968a7bee2b6b3029d0dbd4e19980ac2fd2203f660aa1436fcad3b6fe57b6 WatchSource:0}: Error finding container 5c83968a7bee2b6b3029d0dbd4e19980ac2fd2203f660aa1436fcad3b6fe57b6: Status 404 returned error can't find the container with id 5c83968a7bee2b6b3029d0dbd4e19980ac2fd2203f660aa1436fcad3b6fe57b6
Jul 29 17:58:51.474185 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:51.474152    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-mdsvb" event={"ID":"dad1b1dc-9f1a-4fbd-bbbc-860e74a95cd2","Type":"ContainerStarted","Data":"5c83968a7bee2b6b3029d0dbd4e19980ac2fd2203f660aa1436fcad3b6fe57b6"}
Jul 29 17:58:52.481306 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:52.481219    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-869f956dcc-fp29f" event={"ID":"3877a269-9027-424f-8903-b06036bbbcb2","Type":"ContainerStarted","Data":"627e07ef0c592d002413ed37da0c90db8e6eb30ec90e87b3a8cb6eaf4cb328ad"}
Jul 29 17:58:52.482804 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:52.482780    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-672fv" event={"ID":"5645b830-d2f9-4164-8e78-a7be8c9af2b5","Type":"ContainerStarted","Data":"ecc9bbc5ae1529e8c37b3cb7b1a04625940a810e15a5c8b5a193b5f2a116258a"}
Jul 29 17:58:52.482934 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:52.482831    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-672fv"
Jul 29 17:58:52.499661 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:52.499600    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-869f956dcc-fp29f" podStartSLOduration=1.565496761 podStartE2EDuration="5.499584848s" totalImagesPullingTime="3.934088087s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:58:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:58:52.499001843 +0000 UTC m=+412.368224642" watchObservedRunningTime="2026-07-29 17:58:52.499584848 +0000 UTC m=+412.368807656"
Jul 29 17:58:52.517014 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:52.516969    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-672fv" podStartSLOduration=2.455258067 podStartE2EDuration="5.516957699s" totalImagesPullingTime="3.061699632s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:58:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:58:52.515411739 +0000 UTC m=+412.384634546" watchObservedRunningTime="2026-07-29 17:58:52.516957699 +0000 UTC m=+412.386180530"
Jul 29 17:58:53.488373 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:53.488339    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-mdsvb" event={"ID":"dad1b1dc-9f1a-4fbd-bbbc-860e74a95cd2","Type":"ContainerStarted","Data":"86c36277ecd90436dfd09a5d05ab92f26eefed2dab634c34df7c1a4cf2ffb5e5"}
Jul 29 17:58:53.488747 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:53.488654    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-mdsvb"
Jul 29 17:58:53.528065 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:53.528022    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-mdsvb" podStartSLOduration=2.940640719 podStartE2EDuration="5.528008199s" totalImagesPullingTime="2.58736748s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:58:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:58:53.527216175 +0000 UTC m=+413.396438983" watchObservedRunningTime="2026-07-29 17:58:53.528008199 +0000 UTC m=+413.397231033"
Jul 29 17:58:54.036812 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:54.036781    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5"]
Jul 29 17:58:54.040045 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:54.040029    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5"
Jul 29 17:58:54.042715 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:54.042692    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 29 17:58:54.042832 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:54.042815    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-k4g2r\""
Jul 29 17:58:54.053233 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:54.053209    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5"]
Jul 29 17:58:54.155734 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:54.155700    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/920ab95a-edbf-425b-8f80-45937925ae16-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5\" (UID: \"920ab95a-edbf-425b-8f80-45937925ae16\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5"
Jul 29 17:58:54.155952 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:54.155803    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/920ab95a-edbf-425b-8f80-45937925ae16-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5\" (UID: \"920ab95a-edbf-425b-8f80-45937925ae16\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5"
Jul 29 17:58:54.155952 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:54.155827    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/920ab95a-edbf-425b-8f80-45937925ae16-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5\" (UID: \"920ab95a-edbf-425b-8f80-45937925ae16\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5"
Jul 29 17:58:54.156061 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:54.155953    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/920ab95a-edbf-425b-8f80-45937925ae16-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5\" (UID: \"920ab95a-edbf-425b-8f80-45937925ae16\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5"
Jul 29 17:58:54.156061 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:54.155979    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/920ab95a-edbf-425b-8f80-45937925ae16-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5\" (UID: \"920ab95a-edbf-425b-8f80-45937925ae16\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5"
Jul 29 17:58:54.156061 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:54.156017    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/920ab95a-edbf-425b-8f80-45937925ae16-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5\" (UID: \"920ab95a-edbf-425b-8f80-45937925ae16\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5"
Jul 29 17:58:54.156061 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:54.156037    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/920ab95a-edbf-425b-8f80-45937925ae16-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5\" (UID: \"920ab95a-edbf-425b-8f80-45937925ae16\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5"
Jul 29 17:58:54.156224 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:54.156061    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mv2d8\" (UniqueName: \"kubernetes.io/projected/920ab95a-edbf-425b-8f80-45937925ae16-kube-api-access-mv2d8\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5\" (UID: \"920ab95a-edbf-425b-8f80-45937925ae16\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5"
Jul 29 17:58:54.156224 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:54.156080    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/920ab95a-edbf-425b-8f80-45937925ae16-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5\" (UID: \"920ab95a-edbf-425b-8f80-45937925ae16\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5"
Jul 29 17:58:54.256968 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:54.256932    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/920ab95a-edbf-425b-8f80-45937925ae16-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5\" (UID: \"920ab95a-edbf-425b-8f80-45937925ae16\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5"
Jul 29 17:58:54.257126 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:54.256971    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/920ab95a-edbf-425b-8f80-45937925ae16-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5\" (UID: \"920ab95a-edbf-425b-8f80-45937925ae16\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5"
Jul 29 17:58:54.257126 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:54.257012    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/920ab95a-edbf-425b-8f80-45937925ae16-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5\" (UID: \"920ab95a-edbf-425b-8f80-45937925ae16\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5"
Jul 29 17:58:54.257126 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:54.257036    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/920ab95a-edbf-425b-8f80-45937925ae16-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5\" (UID: \"920ab95a-edbf-425b-8f80-45937925ae16\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5"
Jul 29 17:58:54.257126 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:54.257069    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/920ab95a-edbf-425b-8f80-45937925ae16-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5\" (UID: \"920ab95a-edbf-425b-8f80-45937925ae16\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5"
Jul 29 17:58:54.257126 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:54.257091    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/920ab95a-edbf-425b-8f80-45937925ae16-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5\" (UID: \"920ab95a-edbf-425b-8f80-45937925ae16\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5"
Jul 29 17:58:54.257126 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:54.257114    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mv2d8\" (UniqueName: \"kubernetes.io/projected/920ab95a-edbf-425b-8f80-45937925ae16-kube-api-access-mv2d8\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5\" (UID: \"920ab95a-edbf-425b-8f80-45937925ae16\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5"
Jul 29 17:58:54.257429 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:54.257140    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/920ab95a-edbf-425b-8f80-45937925ae16-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5\" (UID: \"920ab95a-edbf-425b-8f80-45937925ae16\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5"
Jul 29 17:58:54.257429 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:54.257214    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/920ab95a-edbf-425b-8f80-45937925ae16-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5\" (UID: \"920ab95a-edbf-425b-8f80-45937925ae16\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5"
Jul 29 17:58:54.257525 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:54.257448    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/920ab95a-edbf-425b-8f80-45937925ae16-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5\" (UID: \"920ab95a-edbf-425b-8f80-45937925ae16\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5"
Jul 29 17:58:54.257601 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:54.257576    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/920ab95a-edbf-425b-8f80-45937925ae16-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5\" (UID: \"920ab95a-edbf-425b-8f80-45937925ae16\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5"
Jul 29 17:58:54.257657 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:54.257577    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/920ab95a-edbf-425b-8f80-45937925ae16-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5\" (UID: \"920ab95a-edbf-425b-8f80-45937925ae16\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5"
Jul 29 17:58:54.257706 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:54.257683    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/920ab95a-edbf-425b-8f80-45937925ae16-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5\" (UID: \"920ab95a-edbf-425b-8f80-45937925ae16\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5"
Jul 29 17:58:54.257858 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:54.257835    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/920ab95a-edbf-425b-8f80-45937925ae16-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5\" (UID: \"920ab95a-edbf-425b-8f80-45937925ae16\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5"
Jul 29 17:58:54.259392 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:54.259374    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/920ab95a-edbf-425b-8f80-45937925ae16-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5\" (UID: \"920ab95a-edbf-425b-8f80-45937925ae16\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5"
Jul 29 17:58:54.259603 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:54.259588    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/920ab95a-edbf-425b-8f80-45937925ae16-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5\" (UID: \"920ab95a-edbf-425b-8f80-45937925ae16\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5"
Jul 29 17:58:54.266897 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:54.266876    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/920ab95a-edbf-425b-8f80-45937925ae16-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5\" (UID: \"920ab95a-edbf-425b-8f80-45937925ae16\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5"
Jul 29 17:58:54.267508 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:54.267478    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mv2d8\" (UniqueName: \"kubernetes.io/projected/920ab95a-edbf-425b-8f80-45937925ae16-kube-api-access-mv2d8\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5\" (UID: \"920ab95a-edbf-425b-8f80-45937925ae16\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5"
Jul 29 17:58:54.355873 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:54.355804    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5"
Jul 29 17:58:54.492252 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:54.492225    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5"]
Jul 29 17:58:54.494224 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:58:54.494193    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod920ab95a_edbf_425b_8f80_45937925ae16.slice/crio-d8338fcde494249ef83e22c7261316686efef5f96817bf54a2af7d58de7756e5 WatchSource:0}: Error finding container d8338fcde494249ef83e22c7261316686efef5f96817bf54a2af7d58de7756e5: Status 404 returned error can't find the container with id d8338fcde494249ef83e22c7261316686efef5f96817bf54a2af7d58de7756e5
Jul 29 17:58:55.501754 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:55.501703    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5" event={"ID":"920ab95a-edbf-425b-8f80-45937925ae16","Type":"ContainerStarted","Data":"d8338fcde494249ef83e22c7261316686efef5f96817bf54a2af7d58de7756e5"}
Jul 29 17:58:56.197408 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:56.197379    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-j5k56"]
Jul 29 17:58:56.200552 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:56.200537    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-j5k56"
Jul 29 17:58:56.205682 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:56.205657    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-lsscf\""
Jul 29 17:58:56.205833 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:56.205758    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 29 17:58:56.205833 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:56.205773    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 29 17:58:56.206556 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:56.206533    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 29 17:58:56.232533 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:56.232509    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-j5k56"]
Jul 29 17:58:56.273210 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:56.273185    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b9ee721-eb64-4ab6-a806-c6562ce45797-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-j5k56\" (UID: \"8b9ee721-eb64-4ab6-a806-c6562ce45797\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-j5k56"
Jul 29 17:58:56.273331 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:56.273213    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8b9ee721-eb64-4ab6-a806-c6562ce45797-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-j5k56\" (UID: \"8b9ee721-eb64-4ab6-a806-c6562ce45797\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-j5k56"
Jul 29 17:58:56.273331 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:56.273253    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b9ee721-eb64-4ab6-a806-c6562ce45797-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-j5k56\" (UID: \"8b9ee721-eb64-4ab6-a806-c6562ce45797\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-j5k56"
Jul 29 17:58:56.273331 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:56.273276    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8b9ee721-eb64-4ab6-a806-c6562ce45797-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-j5k56\" (UID: \"8b9ee721-eb64-4ab6-a806-c6562ce45797\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-j5k56"
Jul 29 17:58:56.273331 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:56.273300    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lv6wv\" (UniqueName: \"kubernetes.io/projected/8b9ee721-eb64-4ab6-a806-c6562ce45797-kube-api-access-lv6wv\") pod \"istiod-openshift-gateway-5677ccb49b-j5k56\" (UID: \"8b9ee721-eb64-4ab6-a806-c6562ce45797\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-j5k56"
Jul 29 17:58:56.273331 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:56.273327    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8b9ee721-eb64-4ab6-a806-c6562ce45797-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-j5k56\" (UID: \"8b9ee721-eb64-4ab6-a806-c6562ce45797\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-j5k56"
Jul 29 17:58:56.273494 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:56.273408    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8b9ee721-eb64-4ab6-a806-c6562ce45797-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-j5k56\" (UID: \"8b9ee721-eb64-4ab6-a806-c6562ce45797\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-j5k56"
Jul 29 17:58:56.374799 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:56.374770    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8b9ee721-eb64-4ab6-a806-c6562ce45797-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-j5k56\" (UID: \"8b9ee721-eb64-4ab6-a806-c6562ce45797\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-j5k56"
Jul 29 17:58:56.374958 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:56.374831    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8b9ee721-eb64-4ab6-a806-c6562ce45797-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-j5k56\" (UID: \"8b9ee721-eb64-4ab6-a806-c6562ce45797\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-j5k56"
Jul 29 17:58:56.374958 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:56.374944    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b9ee721-eb64-4ab6-a806-c6562ce45797-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-j5k56\" (UID: \"8b9ee721-eb64-4ab6-a806-c6562ce45797\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-j5k56"
Jul 29 17:58:56.375065 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:56.374972    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8b9ee721-eb64-4ab6-a806-c6562ce45797-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-j5k56\" (UID: \"8b9ee721-eb64-4ab6-a806-c6562ce45797\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-j5k56"
Jul 29 17:58:56.375065 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:56.375022    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b9ee721-eb64-4ab6-a806-c6562ce45797-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-j5k56\" (UID: \"8b9ee721-eb64-4ab6-a806-c6562ce45797\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-j5k56"
Jul 29 17:58:56.375175 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:56.375129    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8b9ee721-eb64-4ab6-a806-c6562ce45797-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-j5k56\" (UID: \"8b9ee721-eb64-4ab6-a806-c6562ce45797\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-j5k56"
Jul 29 17:58:56.375175 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:56.375165    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lv6wv\" (UniqueName: \"kubernetes.io/projected/8b9ee721-eb64-4ab6-a806-c6562ce45797-kube-api-access-lv6wv\") pod \"istiod-openshift-gateway-5677ccb49b-j5k56\" (UID: \"8b9ee721-eb64-4ab6-a806-c6562ce45797\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-j5k56"
Jul 29 17:58:56.375979 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:56.375948    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8b9ee721-eb64-4ab6-a806-c6562ce45797-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-j5k56\" (UID: \"8b9ee721-eb64-4ab6-a806-c6562ce45797\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-j5k56"
Jul 29 17:58:56.377258 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:56.377237    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b9ee721-eb64-4ab6-a806-c6562ce45797-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-j5k56\" (UID: \"8b9ee721-eb64-4ab6-a806-c6562ce45797\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-j5k56"
Jul 29 17:58:56.377346 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:56.377303    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8b9ee721-eb64-4ab6-a806-c6562ce45797-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-j5k56\" (UID: \"8b9ee721-eb64-4ab6-a806-c6562ce45797\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-j5k56"
Jul 29 17:58:56.377346 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:56.377314    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8b9ee721-eb64-4ab6-a806-c6562ce45797-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-j5k56\" (UID: \"8b9ee721-eb64-4ab6-a806-c6562ce45797\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-j5k56"
Jul 29 17:58:56.377431 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:56.377414    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8b9ee721-eb64-4ab6-a806-c6562ce45797-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-j5k56\" (UID: \"8b9ee721-eb64-4ab6-a806-c6562ce45797\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-j5k56"
Jul 29 17:58:56.383500 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:56.383480    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lv6wv\" (UniqueName: \"kubernetes.io/projected/8b9ee721-eb64-4ab6-a806-c6562ce45797-kube-api-access-lv6wv\") pod \"istiod-openshift-gateway-5677ccb49b-j5k56\" (UID: \"8b9ee721-eb64-4ab6-a806-c6562ce45797\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-j5k56"
Jul 29 17:58:56.383782 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:56.383760    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b9ee721-eb64-4ab6-a806-c6562ce45797-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-j5k56\" (UID: \"8b9ee721-eb64-4ab6-a806-c6562ce45797\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-j5k56"
Jul 29 17:58:56.511511 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:56.511438    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-j5k56"
Jul 29 17:58:56.663669 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:56.663638    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-j5k56"]
Jul 29 17:58:56.665955 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:58:56.665926    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b9ee721_eb64_4ab6_a806_c6562ce45797.slice/crio-a672b845a8c29a58048327dd0cdfc58b924a03636fd834e75b7b10ea966e1a2f WatchSource:0}: Error finding container a672b845a8c29a58048327dd0cdfc58b924a03636fd834e75b7b10ea966e1a2f: Status 404 returned error can't find the container with id a672b845a8c29a58048327dd0cdfc58b924a03636fd834e75b7b10ea966e1a2f
Jul 29 17:58:57.512284 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:57.512252    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-j5k56" event={"ID":"8b9ee721-eb64-4ab6-a806-c6562ce45797","Type":"ContainerStarted","Data":"a672b845a8c29a58048327dd0cdfc58b924a03636fd834e75b7b10ea966e1a2f"}
Jul 29 17:58:59.954617 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:59.954578    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 17:58:59.954950 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:59.954696    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 17:58:59.954950 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:59.954756    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 17:58:59.956145 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:59.956117    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 17:58:59.956262 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:58:59.956176    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 17:59:00.530962 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:59:00.530924    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-j5k56" event={"ID":"8b9ee721-eb64-4ab6-a806-c6562ce45797","Type":"ContainerStarted","Data":"7c2be2fbf03157f77206ee685897e17120e5b07a732ec273fe3e2013c6e4d4d1"}
Jul 29 17:59:00.531158 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:59:00.531018    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-j5k56"
Jul 29 17:59:00.532525 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:59:00.532495    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5" event={"ID":"920ab95a-edbf-425b-8f80-45937925ae16","Type":"ContainerStarted","Data":"1db5538632eaa70bcd3e1bb312a629e9dfd58a1e30a26e4178e659831a8b371a"}
Jul 29 17:59:00.559788 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:59:00.559706    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-j5k56" podStartSLOduration=1.271996996 podStartE2EDuration="4.559687726s" totalImagesPullingTime="3.28769073s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:58:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:59:00.558769966 +0000 UTC m=+420.427992773" watchObservedRunningTime="2026-07-29 17:59:00.559687726 +0000 UTC m=+420.428910539"
Jul 29 17:59:00.582432 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:59:00.582386    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5" podStartSLOduration=1.124706241 podStartE2EDuration="6.58236585s" totalImagesPullingTime="5.457659609s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:58:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:59:00.580893024 +0000 UTC m=+420.450115844" watchObservedRunningTime="2026-07-29 17:59:00.58236585 +0000 UTC m=+420.451588661"
Jul 29 17:59:01.356605 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:59:01.356562    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5"
Jul 29 17:59:01.361329 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:59:01.361304    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5"
Jul 29 17:59:01.537331 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:59:01.537297    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5"
Jul 29 17:59:01.538356 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:59:01.538333    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5"
Jul 29 17:59:01.539153 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:59:01.539134    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-j5k56"
Jul 29 17:59:03.491281 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:59:03.491247    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-672fv"
Jul 29 17:59:04.497947 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:59:04.497917    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-mdsvb"
Jul 29 17:59:49.022434 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:59:49.022396    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6p84z"]
Jul 29 17:59:49.036110 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:59:49.036082    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-6p84z"
Jul 29 17:59:49.036376 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:59:49.036350    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6p84z"]
Jul 29 17:59:49.038919 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:59:49.038893    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 29 17:59:49.040044 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:59:49.040023    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 29 17:59:49.040403 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:59:49.040057    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-4xt2r\""
Jul 29 17:59:49.116933 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:59:49.116903    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqdwp\" (UniqueName: \"kubernetes.io/projected/2f1f45bd-018d-4ac4-9331-1d6234d5a5eb-kube-api-access-qqdwp\") pod \"kuadrant-operator-catalog-6p84z\" (UID: \"2f1f45bd-018d-4ac4-9331-1d6234d5a5eb\") " pod="kuadrant-system/kuadrant-operator-catalog-6p84z"
Jul 29 17:59:49.218212 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:59:49.218185    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqdwp\" (UniqueName: \"kubernetes.io/projected/2f1f45bd-018d-4ac4-9331-1d6234d5a5eb-kube-api-access-qqdwp\") pod \"kuadrant-operator-catalog-6p84z\" (UID: \"2f1f45bd-018d-4ac4-9331-1d6234d5a5eb\") " pod="kuadrant-system/kuadrant-operator-catalog-6p84z"
Jul 29 17:59:49.226165 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:59:49.226137    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqdwp\" (UniqueName: \"kubernetes.io/projected/2f1f45bd-018d-4ac4-9331-1d6234d5a5eb-kube-api-access-qqdwp\") pod \"kuadrant-operator-catalog-6p84z\" (UID: \"2f1f45bd-018d-4ac4-9331-1d6234d5a5eb\") " pod="kuadrant-system/kuadrant-operator-catalog-6p84z"
Jul 29 17:59:49.348329 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:59:49.348303    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-6p84z"
Jul 29 17:59:49.373090 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:59:49.373062    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6p84z"]
Jul 29 17:59:49.478016 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:59:49.477994    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6p84z"]
Jul 29 17:59:49.479405 ip-10-0-142-156 kubenswrapper[2557]: W0729 17:59:49.479369    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f1f45bd_018d_4ac4_9331_1d6234d5a5eb.slice/crio-f9504afda6dc279d33933a0dacaf92736051960d6c2ffdbcbe0deb2a1cc2d110 WatchSource:0}: Error finding container f9504afda6dc279d33933a0dacaf92736051960d6c2ffdbcbe0deb2a1cc2d110: Status 404 returned error can't find the container with id f9504afda6dc279d33933a0dacaf92736051960d6c2ffdbcbe0deb2a1cc2d110
Jul 29 17:59:49.754319 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:59:49.754237    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-6p84z" event={"ID":"2f1f45bd-018d-4ac4-9331-1d6234d5a5eb","Type":"ContainerStarted","Data":"f9504afda6dc279d33933a0dacaf92736051960d6c2ffdbcbe0deb2a1cc2d110"}
Jul 29 17:59:52.771637 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:59:52.771604    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-6p84z" event={"ID":"2f1f45bd-018d-4ac4-9331-1d6234d5a5eb","Type":"ContainerStarted","Data":"2b8c599e189af6195ce5096da94a159ba6507e63241afd37f58d262b711c7a61"}
Jul 29 17:59:52.772065 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:59:52.771655    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-6p84z" podUID="2f1f45bd-018d-4ac4-9331-1d6234d5a5eb" containerName="registry-server" containerID="cri-o://2b8c599e189af6195ce5096da94a159ba6507e63241afd37f58d262b711c7a61" gracePeriod=2
Jul 29 17:59:52.787544 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:59:52.787485    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-6p84z" podStartSLOduration=1.55854282 podStartE2EDuration="3.787469411s" totalImagesPullingTime="2.228926591s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:59:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:59:52.785523831 +0000 UTC m=+472.654746642" watchObservedRunningTime="2026-07-29 17:59:52.787469411 +0000 UTC m=+472.656692195"
Jul 29 17:59:53.022099 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:59:53.022047    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-6p84z"
Jul 29 17:59:53.048669 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:59:53.048646    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qqdwp\" (UniqueName: \"kubernetes.io/projected/2f1f45bd-018d-4ac4-9331-1d6234d5a5eb-kube-api-access-qqdwp\") pod \"2f1f45bd-018d-4ac4-9331-1d6234d5a5eb\" (UID: \"2f1f45bd-018d-4ac4-9331-1d6234d5a5eb\") "
Jul 29 17:59:53.051040 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:59:53.051015    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f1f45bd-018d-4ac4-9331-1d6234d5a5eb-kube-api-access-qqdwp" (OuterVolumeSpecName: "kube-api-access-qqdwp") pod "2f1f45bd-018d-4ac4-9331-1d6234d5a5eb" (UID: "2f1f45bd-018d-4ac4-9331-1d6234d5a5eb"). InnerVolumeSpecName "kube-api-access-qqdwp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 17:59:53.149608 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:59:53.149580    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qqdwp\" (UniqueName: \"kubernetes.io/projected/2f1f45bd-018d-4ac4-9331-1d6234d5a5eb-kube-api-access-qqdwp\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 17:59:53.777239 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:59:53.777205    2557 generic.go:356] "Generic (PLEG): container finished" podID="2f1f45bd-018d-4ac4-9331-1d6234d5a5eb" containerID="2b8c599e189af6195ce5096da94a159ba6507e63241afd37f58d262b711c7a61" exitCode=0
Jul 29 17:59:53.777658 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:59:53.777246    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-6p84z"
Jul 29 17:59:53.777658 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:59:53.777291    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-6p84z" event={"ID":"2f1f45bd-018d-4ac4-9331-1d6234d5a5eb","Type":"ContainerDied","Data":"2b8c599e189af6195ce5096da94a159ba6507e63241afd37f58d262b711c7a61"}
Jul 29 17:59:53.777658 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:59:53.777341    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-6p84z" event={"ID":"2f1f45bd-018d-4ac4-9331-1d6234d5a5eb","Type":"ContainerDied","Data":"f9504afda6dc279d33933a0dacaf92736051960d6c2ffdbcbe0deb2a1cc2d110"}
Jul 29 17:59:53.777658 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:59:53.777362    2557 scope.go:117] "RemoveContainer" containerID="2b8c599e189af6195ce5096da94a159ba6507e63241afd37f58d262b711c7a61"
Jul 29 17:59:53.787011 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:59:53.786991    2557 scope.go:117] "RemoveContainer" containerID="2b8c599e189af6195ce5096da94a159ba6507e63241afd37f58d262b711c7a61"
Jul 29 17:59:53.787232 ip-10-0-142-156 kubenswrapper[2557]: E0729 17:59:53.787214    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b8c599e189af6195ce5096da94a159ba6507e63241afd37f58d262b711c7a61\": container with ID starting with 2b8c599e189af6195ce5096da94a159ba6507e63241afd37f58d262b711c7a61 not found: ID does not exist" containerID="2b8c599e189af6195ce5096da94a159ba6507e63241afd37f58d262b711c7a61"
Jul 29 17:59:53.787280 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:59:53.787241    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b8c599e189af6195ce5096da94a159ba6507e63241afd37f58d262b711c7a61"} err="failed to get container status \"2b8c599e189af6195ce5096da94a159ba6507e63241afd37f58d262b711c7a61\": rpc error: code = NotFound desc = could not find container \"2b8c599e189af6195ce5096da94a159ba6507e63241afd37f58d262b711c7a61\": container with ID starting with 2b8c599e189af6195ce5096da94a159ba6507e63241afd37f58d262b711c7a61 not found: ID does not exist"
Jul 29 17:59:53.800767 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:59:53.800746    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6p84z"]
Jul 29 17:59:53.804823 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:59:53.804804    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6p84z"]
Jul 29 17:59:54.652259 ip-10-0-142-156 kubenswrapper[2557]: I0729 17:59:54.652227    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f1f45bd-018d-4ac4-9331-1d6234d5a5eb" path="/var/lib/kubelet/pods/2f1f45bd-018d-4ac4-9331-1d6234d5a5eb/volumes"
Jul 29 18:00:06.262821 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:06.262787    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8brs"]
Jul 29 18:00:06.263179 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:06.263106    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f1f45bd-018d-4ac4-9331-1d6234d5a5eb" containerName="registry-server"
Jul 29 18:00:06.263179 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:06.263118    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f1f45bd-018d-4ac4-9331-1d6234d5a5eb" containerName="registry-server"
Jul 29 18:00:06.263291 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:06.263184    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f1f45bd-018d-4ac4-9331-1d6234d5a5eb" containerName="registry-server"
Jul 29 18:00:06.267690 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:06.267670    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8brs"
Jul 29 18:00:06.270188 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:06.270166    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 29 18:00:06.271375 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:06.271351    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 29 18:00:06.271510 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:06.271465    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-dz6bj\""
Jul 29 18:00:06.273851 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:06.273831    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8brs"]
Jul 29 18:00:06.359358 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:06.359328    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/26ba8c29-8db7-4faf-bbd9-187f92b54bab-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8brs\" (UID: \"26ba8c29-8db7-4faf-bbd9-187f92b54bab\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8brs"
Jul 29 18:00:06.359358 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:06.359361    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/26ba8c29-8db7-4faf-bbd9-187f92b54bab-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8brs\" (UID: \"26ba8c29-8db7-4faf-bbd9-187f92b54bab\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8brs"
Jul 29 18:00:06.359610 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:06.359396    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5g78\" (UniqueName: \"kubernetes.io/projected/26ba8c29-8db7-4faf-bbd9-187f92b54bab-kube-api-access-h5g78\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8brs\" (UID: \"26ba8c29-8db7-4faf-bbd9-187f92b54bab\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8brs"
Jul 29 18:00:06.460671 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:06.460629    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5g78\" (UniqueName: \"kubernetes.io/projected/26ba8c29-8db7-4faf-bbd9-187f92b54bab-kube-api-access-h5g78\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8brs\" (UID: \"26ba8c29-8db7-4faf-bbd9-187f92b54bab\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8brs"
Jul 29 18:00:06.460870 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:06.460766    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/26ba8c29-8db7-4faf-bbd9-187f92b54bab-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8brs\" (UID: \"26ba8c29-8db7-4faf-bbd9-187f92b54bab\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8brs"
Jul 29 18:00:06.460870 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:06.460796    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/26ba8c29-8db7-4faf-bbd9-187f92b54bab-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8brs\" (UID: \"26ba8c29-8db7-4faf-bbd9-187f92b54bab\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8brs"
Jul 29 18:00:06.461135 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:06.461116    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/26ba8c29-8db7-4faf-bbd9-187f92b54bab-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8brs\" (UID: \"26ba8c29-8db7-4faf-bbd9-187f92b54bab\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8brs"
Jul 29 18:00:06.461204 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:06.461145    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/26ba8c29-8db7-4faf-bbd9-187f92b54bab-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8brs\" (UID: \"26ba8c29-8db7-4faf-bbd9-187f92b54bab\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8brs"
Jul 29 18:00:06.470085 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:06.470064    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5g78\" (UniqueName: \"kubernetes.io/projected/26ba8c29-8db7-4faf-bbd9-187f92b54bab-kube-api-access-h5g78\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8brs\" (UID: \"26ba8c29-8db7-4faf-bbd9-187f92b54bab\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8brs"
Jul 29 18:00:06.577048 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:06.577016    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8brs"
Jul 29 18:00:06.584089 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:06.584065    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rsngv"]
Jul 29 18:00:06.587470 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:06.587445    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rsngv"
Jul 29 18:00:06.595651 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:06.595623    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rsngv"]
Jul 29 18:00:06.663133 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:06.663098    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtrbn\" (UniqueName: \"kubernetes.io/projected/da62e93b-16c5-4747-b35e-628e7636fcc8-kube-api-access-gtrbn\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rsngv\" (UID: \"da62e93b-16c5-4747-b35e-628e7636fcc8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rsngv"
Jul 29 18:00:06.663303 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:06.663186    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/da62e93b-16c5-4747-b35e-628e7636fcc8-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rsngv\" (UID: \"da62e93b-16c5-4747-b35e-628e7636fcc8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rsngv"
Jul 29 18:00:06.663303 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:06.663225    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/da62e93b-16c5-4747-b35e-628e7636fcc8-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rsngv\" (UID: \"da62e93b-16c5-4747-b35e-628e7636fcc8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rsngv"
Jul 29 18:00:06.711654 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:06.711628    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8brs"]
Jul 29 18:00:06.713740 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:00:06.713687    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26ba8c29_8db7_4faf_bbd9_187f92b54bab.slice/crio-cbf84f7020522036f329e69f5f011ca76635ae7e02abfd755a2e39bf3d5d745a WatchSource:0}: Error finding container cbf84f7020522036f329e69f5f011ca76635ae7e02abfd755a2e39bf3d5d745a: Status 404 returned error can't find the container with id cbf84f7020522036f329e69f5f011ca76635ae7e02abfd755a2e39bf3d5d745a
Jul 29 18:00:06.764819 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:06.764789    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtrbn\" (UniqueName: \"kubernetes.io/projected/da62e93b-16c5-4747-b35e-628e7636fcc8-kube-api-access-gtrbn\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rsngv\" (UID: \"da62e93b-16c5-4747-b35e-628e7636fcc8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rsngv"
Jul 29 18:00:06.764920 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:06.764893    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/da62e93b-16c5-4747-b35e-628e7636fcc8-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rsngv\" (UID: \"da62e93b-16c5-4747-b35e-628e7636fcc8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rsngv"
Jul 29 18:00:06.764994 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:06.764934    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/da62e93b-16c5-4747-b35e-628e7636fcc8-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rsngv\" (UID: \"da62e93b-16c5-4747-b35e-628e7636fcc8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rsngv"
Jul 29 18:00:06.765341 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:06.765319    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/da62e93b-16c5-4747-b35e-628e7636fcc8-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rsngv\" (UID: \"da62e93b-16c5-4747-b35e-628e7636fcc8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rsngv"
Jul 29 18:00:06.765440 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:06.765366    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/da62e93b-16c5-4747-b35e-628e7636fcc8-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rsngv\" (UID: \"da62e93b-16c5-4747-b35e-628e7636fcc8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rsngv"
Jul 29 18:00:06.773101 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:06.773074    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtrbn\" (UniqueName: \"kubernetes.io/projected/da62e93b-16c5-4747-b35e-628e7636fcc8-kube-api-access-gtrbn\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rsngv\" (UID: \"da62e93b-16c5-4747-b35e-628e7636fcc8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rsngv"
Jul 29 18:00:06.837215 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:06.837184    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8brs" event={"ID":"26ba8c29-8db7-4faf-bbd9-187f92b54bab","Type":"ContainerStarted","Data":"b5db0a71077f343d6ad5d5e0e6b94929141783f6a5d4f611106c51f20f5dbe49"}
Jul 29 18:00:06.837433 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:06.837268    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8brs" event={"ID":"26ba8c29-8db7-4faf-bbd9-187f92b54bab","Type":"ContainerStarted","Data":"cbf84f7020522036f329e69f5f011ca76635ae7e02abfd755a2e39bf3d5d745a"}
Jul 29 18:00:06.911574 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:06.911538    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rsngv"
Jul 29 18:00:06.994382 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:06.994351    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r8j6"]
Jul 29 18:00:06.998333 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:06.998312    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r8j6"
Jul 29 18:00:07.006977 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:07.006945    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r8j6"]
Jul 29 18:00:07.044117 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:07.044091    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rsngv"]
Jul 29 18:00:07.045684 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:00:07.045654    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda62e93b_16c5_4747_b35e_628e7636fcc8.slice/crio-b55fc9712c37e68392cf3f9766d42b3b001ec404a43bef17bdfc3c994525cc7e WatchSource:0}: Error finding container b55fc9712c37e68392cf3f9766d42b3b001ec404a43bef17bdfc3c994525cc7e: Status 404 returned error can't find the container with id b55fc9712c37e68392cf3f9766d42b3b001ec404a43bef17bdfc3c994525cc7e
Jul 29 18:00:07.066767 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:07.066741    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frcld\" (UniqueName: \"kubernetes.io/projected/ece83ae2-c4e0-4df9-95fd-616c5ab8990e-kube-api-access-frcld\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r8j6\" (UID: \"ece83ae2-c4e0-4df9-95fd-616c5ab8990e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r8j6"
Jul 29 18:00:07.066857 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:07.066780    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ece83ae2-c4e0-4df9-95fd-616c5ab8990e-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r8j6\" (UID: \"ece83ae2-c4e0-4df9-95fd-616c5ab8990e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r8j6"
Jul 29 18:00:07.066857 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:07.066804    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ece83ae2-c4e0-4df9-95fd-616c5ab8990e-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r8j6\" (UID: \"ece83ae2-c4e0-4df9-95fd-616c5ab8990e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r8j6"
Jul 29 18:00:07.167990 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:07.167963    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frcld\" (UniqueName: \"kubernetes.io/projected/ece83ae2-c4e0-4df9-95fd-616c5ab8990e-kube-api-access-frcld\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r8j6\" (UID: \"ece83ae2-c4e0-4df9-95fd-616c5ab8990e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r8j6"
Jul 29 18:00:07.168138 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:07.168001    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ece83ae2-c4e0-4df9-95fd-616c5ab8990e-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r8j6\" (UID: \"ece83ae2-c4e0-4df9-95fd-616c5ab8990e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r8j6"
Jul 29 18:00:07.168138 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:07.168020    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ece83ae2-c4e0-4df9-95fd-616c5ab8990e-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r8j6\" (UID: \"ece83ae2-c4e0-4df9-95fd-616c5ab8990e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r8j6"
Jul 29 18:00:07.168347 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:07.168325    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ece83ae2-c4e0-4df9-95fd-616c5ab8990e-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r8j6\" (UID: \"ece83ae2-c4e0-4df9-95fd-616c5ab8990e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r8j6"
Jul 29 18:00:07.168347 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:07.168340    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ece83ae2-c4e0-4df9-95fd-616c5ab8990e-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r8j6\" (UID: \"ece83ae2-c4e0-4df9-95fd-616c5ab8990e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r8j6"
Jul 29 18:00:07.176428 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:07.176408    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-frcld\" (UniqueName: \"kubernetes.io/projected/ece83ae2-c4e0-4df9-95fd-616c5ab8990e-kube-api-access-frcld\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r8j6\" (UID: \"ece83ae2-c4e0-4df9-95fd-616c5ab8990e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r8j6"
Jul 29 18:00:07.308818 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:07.308789    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r8j6"
Jul 29 18:00:07.388083 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:07.388055    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gjbh"]
Jul 29 18:00:07.392402 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:07.392378    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gjbh"
Jul 29 18:00:07.401346 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:07.401154    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gjbh"]
Jul 29 18:00:07.442071 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:07.442047    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r8j6"]
Jul 29 18:00:07.443750 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:00:07.443707    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podece83ae2_c4e0_4df9_95fd_616c5ab8990e.slice/crio-e412e043ed83b167a021f724d60cc5ad344343eccb5d3f9d9a7639e3b371325b WatchSource:0}: Error finding container e412e043ed83b167a021f724d60cc5ad344343eccb5d3f9d9a7639e3b371325b: Status 404 returned error can't find the container with id e412e043ed83b167a021f724d60cc5ad344343eccb5d3f9d9a7639e3b371325b
Jul 29 18:00:07.471483 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:07.471461    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lddk\" (UniqueName: \"kubernetes.io/projected/5f32a898-ddb9-4a13-8e36-83c7533a9755-kube-api-access-2lddk\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gjbh\" (UID: \"5f32a898-ddb9-4a13-8e36-83c7533a9755\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gjbh"
Jul 29 18:00:07.471622 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:07.471525    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5f32a898-ddb9-4a13-8e36-83c7533a9755-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gjbh\" (UID: \"5f32a898-ddb9-4a13-8e36-83c7533a9755\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gjbh"
Jul 29 18:00:07.471692 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:07.471649    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5f32a898-ddb9-4a13-8e36-83c7533a9755-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gjbh\" (UID: \"5f32a898-ddb9-4a13-8e36-83c7533a9755\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gjbh"
Jul 29 18:00:07.572744 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:07.572704    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lddk\" (UniqueName: \"kubernetes.io/projected/5f32a898-ddb9-4a13-8e36-83c7533a9755-kube-api-access-2lddk\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gjbh\" (UID: \"5f32a898-ddb9-4a13-8e36-83c7533a9755\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gjbh"
Jul 29 18:00:07.572862 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:07.572778    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5f32a898-ddb9-4a13-8e36-83c7533a9755-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gjbh\" (UID: \"5f32a898-ddb9-4a13-8e36-83c7533a9755\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gjbh"
Jul 29 18:00:07.572862 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:07.572825    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5f32a898-ddb9-4a13-8e36-83c7533a9755-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gjbh\" (UID: \"5f32a898-ddb9-4a13-8e36-83c7533a9755\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gjbh"
Jul 29 18:00:07.573115 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:07.573099    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5f32a898-ddb9-4a13-8e36-83c7533a9755-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gjbh\" (UID: \"5f32a898-ddb9-4a13-8e36-83c7533a9755\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gjbh"
Jul 29 18:00:07.573185 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:07.573168    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5f32a898-ddb9-4a13-8e36-83c7533a9755-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gjbh\" (UID: \"5f32a898-ddb9-4a13-8e36-83c7533a9755\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gjbh"
Jul 29 18:00:07.581101 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:07.581075    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lddk\" (UniqueName: \"kubernetes.io/projected/5f32a898-ddb9-4a13-8e36-83c7533a9755-kube-api-access-2lddk\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gjbh\" (UID: \"5f32a898-ddb9-4a13-8e36-83c7533a9755\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gjbh"
Jul 29 18:00:07.704860 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:07.704791    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gjbh"
Jul 29 18:00:07.838119 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:07.838079    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gjbh"]
Jul 29 18:00:07.839684 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:00:07.839660    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f32a898_ddb9_4a13_8e36_83c7533a9755.slice/crio-ddf8ac947eecc47dbe35e2eb2b3482cb1765ae4aa4ba70afb7d1c7fc17c125a4 WatchSource:0}: Error finding container ddf8ac947eecc47dbe35e2eb2b3482cb1765ae4aa4ba70afb7d1c7fc17c125a4: Status 404 returned error can't find the container with id ddf8ac947eecc47dbe35e2eb2b3482cb1765ae4aa4ba70afb7d1c7fc17c125a4
Jul 29 18:00:07.843993 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:07.843969    2557 generic.go:356] "Generic (PLEG): container finished" podID="da62e93b-16c5-4747-b35e-628e7636fcc8" containerID="ca4bdcc240664417c020c78a13a2c347b61c59f28bfaed43bb66878a003b4a12" exitCode=0
Jul 29 18:00:07.844101 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:07.844059    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rsngv" event={"ID":"da62e93b-16c5-4747-b35e-628e7636fcc8","Type":"ContainerDied","Data":"ca4bdcc240664417c020c78a13a2c347b61c59f28bfaed43bb66878a003b4a12"}
Jul 29 18:00:07.844165 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:07.844126    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rsngv" event={"ID":"da62e93b-16c5-4747-b35e-628e7636fcc8","Type":"ContainerStarted","Data":"b55fc9712c37e68392cf3f9766d42b3b001ec404a43bef17bdfc3c994525cc7e"}
Jul 29 18:00:07.845523 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:07.845503    2557 generic.go:356] "Generic (PLEG): container finished" podID="26ba8c29-8db7-4faf-bbd9-187f92b54bab" containerID="b5db0a71077f343d6ad5d5e0e6b94929141783f6a5d4f611106c51f20f5dbe49" exitCode=0
Jul 29 18:00:07.845609 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:07.845564    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8brs" event={"ID":"26ba8c29-8db7-4faf-bbd9-187f92b54bab","Type":"ContainerDied","Data":"b5db0a71077f343d6ad5d5e0e6b94929141783f6a5d4f611106c51f20f5dbe49"}
Jul 29 18:00:07.847557 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:07.847446    2557 generic.go:356] "Generic (PLEG): container finished" podID="ece83ae2-c4e0-4df9-95fd-616c5ab8990e" containerID="e2a8e134608161d44b1f2a1c171a2edafccbcefce4bcb4c37fb0dea22c73c3a5" exitCode=0
Jul 29 18:00:07.847557 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:07.847534    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r8j6" event={"ID":"ece83ae2-c4e0-4df9-95fd-616c5ab8990e","Type":"ContainerDied","Data":"e2a8e134608161d44b1f2a1c171a2edafccbcefce4bcb4c37fb0dea22c73c3a5"}
Jul 29 18:00:07.847697 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:07.847575    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r8j6" event={"ID":"ece83ae2-c4e0-4df9-95fd-616c5ab8990e","Type":"ContainerStarted","Data":"e412e043ed83b167a021f724d60cc5ad344343eccb5d3f9d9a7639e3b371325b"}
Jul 29 18:00:08.854469 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:08.854435    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r8j6" event={"ID":"ece83ae2-c4e0-4df9-95fd-616c5ab8990e","Type":"ContainerStarted","Data":"45740928ebf34e4c8487c74783a8df21510e7826ad30085b84d8fda826c8abd2"}
Jul 29 18:00:08.856109 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:08.856080    2557 generic.go:356] "Generic (PLEG): container finished" podID="5f32a898-ddb9-4a13-8e36-83c7533a9755" containerID="5dd923c0b535381aaf5a9762a105adc8df5ded6d124171faa4f40b08a89a9dd5" exitCode=0
Jul 29 18:00:08.856228 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:08.856161    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gjbh" event={"ID":"5f32a898-ddb9-4a13-8e36-83c7533a9755","Type":"ContainerDied","Data":"5dd923c0b535381aaf5a9762a105adc8df5ded6d124171faa4f40b08a89a9dd5"}
Jul 29 18:00:08.856228 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:08.856209    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gjbh" event={"ID":"5f32a898-ddb9-4a13-8e36-83c7533a9755","Type":"ContainerStarted","Data":"ddf8ac947eecc47dbe35e2eb2b3482cb1765ae4aa4ba70afb7d1c7fc17c125a4"}
Jul 29 18:00:09.480142 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:09.480104    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-54dbc47d44-n8w42"]
Jul 29 18:00:09.483790 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:09.483763    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54dbc47d44-n8w42"
Jul 29 18:00:09.495356 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:09.495322    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-54dbc47d44-n8w42"]
Jul 29 18:00:09.590528 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:09.590506    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5d8bb6be-286e-4058-a065-e8c037015e0c-console-config\") pod \"console-54dbc47d44-n8w42\" (UID: \"5d8bb6be-286e-4058-a065-e8c037015e0c\") " pod="openshift-console/console-54dbc47d44-n8w42"
Jul 29 18:00:09.590621 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:09.590533    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d8bb6be-286e-4058-a065-e8c037015e0c-trusted-ca-bundle\") pod \"console-54dbc47d44-n8w42\" (UID: \"5d8bb6be-286e-4058-a065-e8c037015e0c\") " pod="openshift-console/console-54dbc47d44-n8w42"
Jul 29 18:00:09.590621 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:09.590552    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2xm9\" (UniqueName: \"kubernetes.io/projected/5d8bb6be-286e-4058-a065-e8c037015e0c-kube-api-access-t2xm9\") pod \"console-54dbc47d44-n8w42\" (UID: \"5d8bb6be-286e-4058-a065-e8c037015e0c\") " pod="openshift-console/console-54dbc47d44-n8w42"
Jul 29 18:00:09.590699 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:09.590631    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5d8bb6be-286e-4058-a065-e8c037015e0c-console-serving-cert\") pod \"console-54dbc47d44-n8w42\" (UID: \"5d8bb6be-286e-4058-a065-e8c037015e0c\") " pod="openshift-console/console-54dbc47d44-n8w42"
Jul 29 18:00:09.590699 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:09.590684    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5d8bb6be-286e-4058-a065-e8c037015e0c-service-ca\") pod \"console-54dbc47d44-n8w42\" (UID: \"5d8bb6be-286e-4058-a065-e8c037015e0c\") " pod="openshift-console/console-54dbc47d44-n8w42"
Jul 29 18:00:09.590790 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:09.590706    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5d8bb6be-286e-4058-a065-e8c037015e0c-console-oauth-config\") pod \"console-54dbc47d44-n8w42\" (UID: \"5d8bb6be-286e-4058-a065-e8c037015e0c\") " pod="openshift-console/console-54dbc47d44-n8w42"
Jul 29 18:00:09.590790 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:09.590750    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5d8bb6be-286e-4058-a065-e8c037015e0c-oauth-serving-cert\") pod \"console-54dbc47d44-n8w42\" (UID: \"5d8bb6be-286e-4058-a065-e8c037015e0c\") " pod="openshift-console/console-54dbc47d44-n8w42"
Jul 29 18:00:09.691425 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:09.691360    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5d8bb6be-286e-4058-a065-e8c037015e0c-console-config\") pod \"console-54dbc47d44-n8w42\" (UID: \"5d8bb6be-286e-4058-a065-e8c037015e0c\") " pod="openshift-console/console-54dbc47d44-n8w42"
Jul 29 18:00:09.691545 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:09.691443    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d8bb6be-286e-4058-a065-e8c037015e0c-trusted-ca-bundle\") pod \"console-54dbc47d44-n8w42\" (UID: \"5d8bb6be-286e-4058-a065-e8c037015e0c\") " pod="openshift-console/console-54dbc47d44-n8w42"
Jul 29 18:00:09.691545 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:09.691470    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2xm9\" (UniqueName: \"kubernetes.io/projected/5d8bb6be-286e-4058-a065-e8c037015e0c-kube-api-access-t2xm9\") pod \"console-54dbc47d44-n8w42\" (UID: \"5d8bb6be-286e-4058-a065-e8c037015e0c\") " pod="openshift-console/console-54dbc47d44-n8w42"
Jul 29 18:00:09.691545 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:09.691502    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5d8bb6be-286e-4058-a065-e8c037015e0c-console-serving-cert\") pod \"console-54dbc47d44-n8w42\" (UID: \"5d8bb6be-286e-4058-a065-e8c037015e0c\") " pod="openshift-console/console-54dbc47d44-n8w42"
Jul 29 18:00:09.691689 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:09.691551    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5d8bb6be-286e-4058-a065-e8c037015e0c-service-ca\") pod \"console-54dbc47d44-n8w42\" (UID: \"5d8bb6be-286e-4058-a065-e8c037015e0c\") " pod="openshift-console/console-54dbc47d44-n8w42"
Jul 29 18:00:09.691689 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:09.691579    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5d8bb6be-286e-4058-a065-e8c037015e0c-console-oauth-config\") pod \"console-54dbc47d44-n8w42\" (UID: \"5d8bb6be-286e-4058-a065-e8c037015e0c\") " pod="openshift-console/console-54dbc47d44-n8w42"
Jul 29 18:00:09.691809 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:09.691782    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5d8bb6be-286e-4058-a065-e8c037015e0c-oauth-serving-cert\") pod \"console-54dbc47d44-n8w42\" (UID: \"5d8bb6be-286e-4058-a065-e8c037015e0c\") " pod="openshift-console/console-54dbc47d44-n8w42"
Jul 29 18:00:09.692218 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:09.692167    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5d8bb6be-286e-4058-a065-e8c037015e0c-console-config\") pod \"console-54dbc47d44-n8w42\" (UID: \"5d8bb6be-286e-4058-a065-e8c037015e0c\") " pod="openshift-console/console-54dbc47d44-n8w42"
Jul 29 18:00:09.692370 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:09.692348    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5d8bb6be-286e-4058-a065-e8c037015e0c-service-ca\") pod \"console-54dbc47d44-n8w42\" (UID: \"5d8bb6be-286e-4058-a065-e8c037015e0c\") " pod="openshift-console/console-54dbc47d44-n8w42"
Jul 29 18:00:09.692438 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:09.692413    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5d8bb6be-286e-4058-a065-e8c037015e0c-oauth-serving-cert\") pod \"console-54dbc47d44-n8w42\" (UID: \"5d8bb6be-286e-4058-a065-e8c037015e0c\") " pod="openshift-console/console-54dbc47d44-n8w42"
Jul 29 18:00:09.692527 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:09.692438    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d8bb6be-286e-4058-a065-e8c037015e0c-trusted-ca-bundle\") pod \"console-54dbc47d44-n8w42\" (UID: \"5d8bb6be-286e-4058-a065-e8c037015e0c\") " pod="openshift-console/console-54dbc47d44-n8w42"
Jul 29 18:00:09.694508 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:09.694491    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5d8bb6be-286e-4058-a065-e8c037015e0c-console-oauth-config\") pod \"console-54dbc47d44-n8w42\" (UID: \"5d8bb6be-286e-4058-a065-e8c037015e0c\") " pod="openshift-console/console-54dbc47d44-n8w42"
Jul 29 18:00:09.694587 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:09.694548    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5d8bb6be-286e-4058-a065-e8c037015e0c-console-serving-cert\") pod \"console-54dbc47d44-n8w42\" (UID: \"5d8bb6be-286e-4058-a065-e8c037015e0c\") " pod="openshift-console/console-54dbc47d44-n8w42"
Jul 29 18:00:09.700428 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:09.700408    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2xm9\" (UniqueName: \"kubernetes.io/projected/5d8bb6be-286e-4058-a065-e8c037015e0c-kube-api-access-t2xm9\") pod \"console-54dbc47d44-n8w42\" (UID: \"5d8bb6be-286e-4058-a065-e8c037015e0c\") " pod="openshift-console/console-54dbc47d44-n8w42"
Jul 29 18:00:09.794576 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:09.794549    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54dbc47d44-n8w42"
Jul 29 18:00:09.864161 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:09.864128    2557 generic.go:356] "Generic (PLEG): container finished" podID="5f32a898-ddb9-4a13-8e36-83c7533a9755" containerID="edab98c1b5774208e03da35055e7f07c2768b146fafe49cbaf3f61ab332dc881" exitCode=0
Jul 29 18:00:09.864496 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:09.864277    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gjbh" event={"ID":"5f32a898-ddb9-4a13-8e36-83c7533a9755","Type":"ContainerDied","Data":"edab98c1b5774208e03da35055e7f07c2768b146fafe49cbaf3f61ab332dc881"}
Jul 29 18:00:09.866654 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:09.866631    2557 generic.go:356] "Generic (PLEG): container finished" podID="da62e93b-16c5-4747-b35e-628e7636fcc8" containerID="ece790f0ad7da9cf7bdcd9dafed02d1ac60bea4a794250dc08d52ac5a912bb3d" exitCode=0
Jul 29 18:00:09.866766 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:09.866708    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rsngv" event={"ID":"da62e93b-16c5-4747-b35e-628e7636fcc8","Type":"ContainerDied","Data":"ece790f0ad7da9cf7bdcd9dafed02d1ac60bea4a794250dc08d52ac5a912bb3d"}
Jul 29 18:00:09.869198 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:09.869171    2557 generic.go:356] "Generic (PLEG): container finished" podID="ece83ae2-c4e0-4df9-95fd-616c5ab8990e" containerID="45740928ebf34e4c8487c74783a8df21510e7826ad30085b84d8fda826c8abd2" exitCode=0
Jul 29 18:00:09.869303 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:09.869263    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r8j6" event={"ID":"ece83ae2-c4e0-4df9-95fd-616c5ab8990e","Type":"ContainerDied","Data":"45740928ebf34e4c8487c74783a8df21510e7826ad30085b84d8fda826c8abd2"}
Jul 29 18:00:10.005198 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:10.005123    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-54dbc47d44-n8w42"]
Jul 29 18:00:10.875793 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:10.875757    2557 generic.go:356] "Generic (PLEG): container finished" podID="da62e93b-16c5-4747-b35e-628e7636fcc8" containerID="bb9b45de9bc5ec39dbd2615aba1415860f230efb59c8038bb2ee79dc897da5ab" exitCode=0
Jul 29 18:00:10.876459 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:10.875839    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rsngv" event={"ID":"da62e93b-16c5-4747-b35e-628e7636fcc8","Type":"ContainerDied","Data":"bb9b45de9bc5ec39dbd2615aba1415860f230efb59c8038bb2ee79dc897da5ab"}
Jul 29 18:00:10.877654 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:10.877632    2557 generic.go:356] "Generic (PLEG): container finished" podID="26ba8c29-8db7-4faf-bbd9-187f92b54bab" containerID="e440c8daffc03a4b0220e594f8369dd5ecb5fe29e2751114280e3be7f2f727d8" exitCode=0
Jul 29 18:00:10.877804 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:10.877701    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8brs" event={"ID":"26ba8c29-8db7-4faf-bbd9-187f92b54bab","Type":"ContainerDied","Data":"e440c8daffc03a4b0220e594f8369dd5ecb5fe29e2751114280e3be7f2f727d8"}
Jul 29 18:00:10.879923 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:10.879901    2557 generic.go:356] "Generic (PLEG): container finished" podID="ece83ae2-c4e0-4df9-95fd-616c5ab8990e" containerID="a86778c89025192023694310b6a1345966d29ab23e59a95f9e927feb0a6b078e" exitCode=0
Jul 29 18:00:10.880017 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:10.879983    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r8j6" event={"ID":"ece83ae2-c4e0-4df9-95fd-616c5ab8990e","Type":"ContainerDied","Data":"a86778c89025192023694310b6a1345966d29ab23e59a95f9e927feb0a6b078e"}
Jul 29 18:00:10.881459 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:10.881441    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54dbc47d44-n8w42" event={"ID":"5d8bb6be-286e-4058-a065-e8c037015e0c","Type":"ContainerStarted","Data":"3b7afe2b2a990a8ca2dcf31649c7297d7bc6802a731e23064d9b2bddfbf9143f"}
Jul 29 18:00:10.881550 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:10.881472    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54dbc47d44-n8w42" event={"ID":"5d8bb6be-286e-4058-a065-e8c037015e0c","Type":"ContainerStarted","Data":"b89475454044fc2c46aee7c21bc5b4eee0c81366b33b6e02c7b1cf1b8bc391fa"}
Jul 29 18:00:10.883509 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:10.883480    2557 generic.go:356] "Generic (PLEG): container finished" podID="5f32a898-ddb9-4a13-8e36-83c7533a9755" containerID="af049fd5479afa6e98847113bb4d0c480b2696ad5c1ee926d8c97318af157f04" exitCode=0
Jul 29 18:00:10.883582 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:10.883546    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gjbh" event={"ID":"5f32a898-ddb9-4a13-8e36-83c7533a9755","Type":"ContainerDied","Data":"af049fd5479afa6e98847113bb4d0c480b2696ad5c1ee926d8c97318af157f04"}
Jul 29 18:00:10.968171 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:10.968122    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-54dbc47d44-n8w42" podStartSLOduration=1.968105205 podStartE2EDuration="1.968105205s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:00:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:00:10.967301776 +0000 UTC m=+490.836524585" watchObservedRunningTime="2026-07-29 18:00:10.968105205 +0000 UTC m=+490.837328012"
Jul 29 18:00:11.890683 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:11.890649    2557 generic.go:356] "Generic (PLEG): container finished" podID="26ba8c29-8db7-4faf-bbd9-187f92b54bab" containerID="1e9f877065c1aae78815d5cbacb89313ee3679aee7b00b6d7d5f77a4bc10d50b" exitCode=0
Jul 29 18:00:11.891205 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:11.890712    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8brs" event={"ID":"26ba8c29-8db7-4faf-bbd9-187f92b54bab","Type":"ContainerDied","Data":"1e9f877065c1aae78815d5cbacb89313ee3679aee7b00b6d7d5f77a4bc10d50b"}
Jul 29 18:00:12.069058 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:12.069033    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rsngv"
Jul 29 18:00:12.071967 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:12.071946    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r8j6"
Jul 29 18:00:12.075314 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:12.075296    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gjbh"
Jul 29 18:00:12.212654 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:12.212580    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ece83ae2-c4e0-4df9-95fd-616c5ab8990e-bundle\") pod \"ece83ae2-c4e0-4df9-95fd-616c5ab8990e\" (UID: \"ece83ae2-c4e0-4df9-95fd-616c5ab8990e\") "
Jul 29 18:00:12.212654 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:12.212619    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ece83ae2-c4e0-4df9-95fd-616c5ab8990e-util\") pod \"ece83ae2-c4e0-4df9-95fd-616c5ab8990e\" (UID: \"ece83ae2-c4e0-4df9-95fd-616c5ab8990e\") "
Jul 29 18:00:12.212654 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:12.212647    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/da62e93b-16c5-4747-b35e-628e7636fcc8-util\") pod \"da62e93b-16c5-4747-b35e-628e7636fcc8\" (UID: \"da62e93b-16c5-4747-b35e-628e7636fcc8\") "
Jul 29 18:00:12.212956 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:12.212666    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5f32a898-ddb9-4a13-8e36-83c7533a9755-bundle\") pod \"5f32a898-ddb9-4a13-8e36-83c7533a9755\" (UID: \"5f32a898-ddb9-4a13-8e36-83c7533a9755\") "
Jul 29 18:00:12.212956 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:12.212694    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/da62e93b-16c5-4747-b35e-628e7636fcc8-bundle\") pod \"da62e93b-16c5-4747-b35e-628e7636fcc8\" (UID: \"da62e93b-16c5-4747-b35e-628e7636fcc8\") "
Jul 29 18:00:12.212956 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:12.212755    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5f32a898-ddb9-4a13-8e36-83c7533a9755-util\") pod \"5f32a898-ddb9-4a13-8e36-83c7533a9755\" (UID: \"5f32a898-ddb9-4a13-8e36-83c7533a9755\") "
Jul 29 18:00:12.212956 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:12.212787    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2lddk\" (UniqueName: \"kubernetes.io/projected/5f32a898-ddb9-4a13-8e36-83c7533a9755-kube-api-access-2lddk\") pod \"5f32a898-ddb9-4a13-8e36-83c7533a9755\" (UID: \"5f32a898-ddb9-4a13-8e36-83c7533a9755\") "
Jul 29 18:00:12.212956 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:12.212828    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-frcld\" (UniqueName: \"kubernetes.io/projected/ece83ae2-c4e0-4df9-95fd-616c5ab8990e-kube-api-access-frcld\") pod \"ece83ae2-c4e0-4df9-95fd-616c5ab8990e\" (UID: \"ece83ae2-c4e0-4df9-95fd-616c5ab8990e\") "
Jul 29 18:00:12.212956 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:12.212887    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gtrbn\" (UniqueName: \"kubernetes.io/projected/da62e93b-16c5-4747-b35e-628e7636fcc8-kube-api-access-gtrbn\") pod \"da62e93b-16c5-4747-b35e-628e7636fcc8\" (UID: \"da62e93b-16c5-4747-b35e-628e7636fcc8\") "
Jul 29 18:00:12.213628 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:12.213360    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ece83ae2-c4e0-4df9-95fd-616c5ab8990e-bundle" (OuterVolumeSpecName: "bundle") pod "ece83ae2-c4e0-4df9-95fd-616c5ab8990e" (UID: "ece83ae2-c4e0-4df9-95fd-616c5ab8990e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:00:12.213628 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:12.213468    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f32a898-ddb9-4a13-8e36-83c7533a9755-bundle" (OuterVolumeSpecName: "bundle") pod "5f32a898-ddb9-4a13-8e36-83c7533a9755" (UID: "5f32a898-ddb9-4a13-8e36-83c7533a9755"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:00:12.213628 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:12.213604    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da62e93b-16c5-4747-b35e-628e7636fcc8-bundle" (OuterVolumeSpecName: "bundle") pod "da62e93b-16c5-4747-b35e-628e7636fcc8" (UID: "da62e93b-16c5-4747-b35e-628e7636fcc8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:00:12.215416 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:12.215388    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f32a898-ddb9-4a13-8e36-83c7533a9755-kube-api-access-2lddk" (OuterVolumeSpecName: "kube-api-access-2lddk") pod "5f32a898-ddb9-4a13-8e36-83c7533a9755" (UID: "5f32a898-ddb9-4a13-8e36-83c7533a9755"). InnerVolumeSpecName "kube-api-access-2lddk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:00:12.215750 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:12.215705    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da62e93b-16c5-4747-b35e-628e7636fcc8-kube-api-access-gtrbn" (OuterVolumeSpecName: "kube-api-access-gtrbn") pod "da62e93b-16c5-4747-b35e-628e7636fcc8" (UID: "da62e93b-16c5-4747-b35e-628e7636fcc8"). InnerVolumeSpecName "kube-api-access-gtrbn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:00:12.215912 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:12.215889    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ece83ae2-c4e0-4df9-95fd-616c5ab8990e-kube-api-access-frcld" (OuterVolumeSpecName: "kube-api-access-frcld") pod "ece83ae2-c4e0-4df9-95fd-616c5ab8990e" (UID: "ece83ae2-c4e0-4df9-95fd-616c5ab8990e"). InnerVolumeSpecName "kube-api-access-frcld". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:00:12.218701 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:12.218682    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ece83ae2-c4e0-4df9-95fd-616c5ab8990e-util" (OuterVolumeSpecName: "util") pod "ece83ae2-c4e0-4df9-95fd-616c5ab8990e" (UID: "ece83ae2-c4e0-4df9-95fd-616c5ab8990e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:00:12.219714 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:12.219696    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f32a898-ddb9-4a13-8e36-83c7533a9755-util" (OuterVolumeSpecName: "util") pod "5f32a898-ddb9-4a13-8e36-83c7533a9755" (UID: "5f32a898-ddb9-4a13-8e36-83c7533a9755"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:00:12.221248 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:12.221225    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da62e93b-16c5-4747-b35e-628e7636fcc8-util" (OuterVolumeSpecName: "util") pod "da62e93b-16c5-4747-b35e-628e7636fcc8" (UID: "da62e93b-16c5-4747-b35e-628e7636fcc8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:00:12.314251 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:12.314227    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gtrbn\" (UniqueName: \"kubernetes.io/projected/da62e93b-16c5-4747-b35e-628e7636fcc8-kube-api-access-gtrbn\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:00:12.314251 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:12.314250    2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ece83ae2-c4e0-4df9-95fd-616c5ab8990e-bundle\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:00:12.314394 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:12.314261    2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ece83ae2-c4e0-4df9-95fd-616c5ab8990e-util\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:00:12.314394 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:12.314269    2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/da62e93b-16c5-4747-b35e-628e7636fcc8-util\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:00:12.314394 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:12.314277    2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5f32a898-ddb9-4a13-8e36-83c7533a9755-bundle\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:00:12.314394 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:12.314284    2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/da62e93b-16c5-4747-b35e-628e7636fcc8-bundle\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:00:12.314394 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:12.314291    2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5f32a898-ddb9-4a13-8e36-83c7533a9755-util\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:00:12.314394 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:12.314309    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2lddk\" (UniqueName: \"kubernetes.io/projected/5f32a898-ddb9-4a13-8e36-83c7533a9755-kube-api-access-2lddk\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:00:12.314394 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:12.314319    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-frcld\" (UniqueName: \"kubernetes.io/projected/ece83ae2-c4e0-4df9-95fd-616c5ab8990e-kube-api-access-frcld\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:00:12.897112 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:12.897082    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rsngv" event={"ID":"da62e93b-16c5-4747-b35e-628e7636fcc8","Type":"ContainerDied","Data":"b55fc9712c37e68392cf3f9766d42b3b001ec404a43bef17bdfc3c994525cc7e"}
Jul 29 18:00:12.897112 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:12.897098    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rsngv"
Jul 29 18:00:12.897614 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:12.897131    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b55fc9712c37e68392cf3f9766d42b3b001ec404a43bef17bdfc3c994525cc7e"
Jul 29 18:00:12.899402 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:12.899380    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r8j6"
Jul 29 18:00:12.899520 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:12.899399    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r8j6" event={"ID":"ece83ae2-c4e0-4df9-95fd-616c5ab8990e","Type":"ContainerDied","Data":"e412e043ed83b167a021f724d60cc5ad344343eccb5d3f9d9a7639e3b371325b"}
Jul 29 18:00:12.899520 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:12.899440    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e412e043ed83b167a021f724d60cc5ad344343eccb5d3f9d9a7639e3b371325b"
Jul 29 18:00:12.901417 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:12.901364    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gjbh"
Jul 29 18:00:12.901417 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:12.901371    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gjbh" event={"ID":"5f32a898-ddb9-4a13-8e36-83c7533a9755","Type":"ContainerDied","Data":"ddf8ac947eecc47dbe35e2eb2b3482cb1765ae4aa4ba70afb7d1c7fc17c125a4"}
Jul 29 18:00:12.901417 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:12.901410    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ddf8ac947eecc47dbe35e2eb2b3482cb1765ae4aa4ba70afb7d1c7fc17c125a4"
Jul 29 18:00:13.030218 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:13.030193    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8brs"
Jul 29 18:00:13.121922 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:13.121896    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/26ba8c29-8db7-4faf-bbd9-187f92b54bab-util\") pod \"26ba8c29-8db7-4faf-bbd9-187f92b54bab\" (UID: \"26ba8c29-8db7-4faf-bbd9-187f92b54bab\") "
Jul 29 18:00:13.122062 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:13.121984    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h5g78\" (UniqueName: \"kubernetes.io/projected/26ba8c29-8db7-4faf-bbd9-187f92b54bab-kube-api-access-h5g78\") pod \"26ba8c29-8db7-4faf-bbd9-187f92b54bab\" (UID: \"26ba8c29-8db7-4faf-bbd9-187f92b54bab\") "
Jul 29 18:00:13.122062 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:13.122006    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/26ba8c29-8db7-4faf-bbd9-187f92b54bab-bundle\") pod \"26ba8c29-8db7-4faf-bbd9-187f92b54bab\" (UID: \"26ba8c29-8db7-4faf-bbd9-187f92b54bab\") "
Jul 29 18:00:13.122587 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:13.122566    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26ba8c29-8db7-4faf-bbd9-187f92b54bab-bundle" (OuterVolumeSpecName: "bundle") pod "26ba8c29-8db7-4faf-bbd9-187f92b54bab" (UID: "26ba8c29-8db7-4faf-bbd9-187f92b54bab"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:00:13.124430 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:13.124395    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26ba8c29-8db7-4faf-bbd9-187f92b54bab-kube-api-access-h5g78" (OuterVolumeSpecName: "kube-api-access-h5g78") pod "26ba8c29-8db7-4faf-bbd9-187f92b54bab" (UID: "26ba8c29-8db7-4faf-bbd9-187f92b54bab"). InnerVolumeSpecName "kube-api-access-h5g78". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:00:13.127185 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:13.127162    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26ba8c29-8db7-4faf-bbd9-187f92b54bab-util" (OuterVolumeSpecName: "util") pod "26ba8c29-8db7-4faf-bbd9-187f92b54bab" (UID: "26ba8c29-8db7-4faf-bbd9-187f92b54bab"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:00:13.223326 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:13.223269    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h5g78\" (UniqueName: \"kubernetes.io/projected/26ba8c29-8db7-4faf-bbd9-187f92b54bab-kube-api-access-h5g78\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:00:13.223326 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:13.223291    2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/26ba8c29-8db7-4faf-bbd9-187f92b54bab-bundle\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:00:13.223326 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:13.223302    2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/26ba8c29-8db7-4faf-bbd9-187f92b54bab-util\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:00:13.908515 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:13.908484    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8brs"
Jul 29 18:00:13.908515 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:13.908490    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8brs" event={"ID":"26ba8c29-8db7-4faf-bbd9-187f92b54bab","Type":"ContainerDied","Data":"cbf84f7020522036f329e69f5f011ca76635ae7e02abfd755a2e39bf3d5d745a"}
Jul 29 18:00:13.909001 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:13.908539    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cbf84f7020522036f329e69f5f011ca76635ae7e02abfd755a2e39bf3d5d745a"
Jul 29 18:00:19.795604 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:19.795571    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-54dbc47d44-n8w42"
Jul 29 18:00:19.796019 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:19.795632    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-54dbc47d44-n8w42"
Jul 29 18:00:19.800229 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:19.800207    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-54dbc47d44-n8w42"
Jul 29 18:00:19.945809 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:19.945786    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-54dbc47d44-n8w42"
Jul 29 18:00:19.994642 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:19.994607    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f5c4f8dfb-8nlj8"]
Jul 29 18:00:27.123216 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:27.123177    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bbnbh"]
Jul 29 18:00:27.123674 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:27.123546    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26ba8c29-8db7-4faf-bbd9-187f92b54bab" containerName="util"
Jul 29 18:00:27.123674 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:27.123559    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="26ba8c29-8db7-4faf-bbd9-187f92b54bab" containerName="util"
Jul 29 18:00:27.123674 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:27.123573    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da62e93b-16c5-4747-b35e-628e7636fcc8" containerName="extract"
Jul 29 18:00:27.123674 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:27.123582    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="da62e93b-16c5-4747-b35e-628e7636fcc8" containerName="extract"
Jul 29 18:00:27.123674 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:27.123592    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ece83ae2-c4e0-4df9-95fd-616c5ab8990e" containerName="extract"
Jul 29 18:00:27.123674 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:27.123597    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="ece83ae2-c4e0-4df9-95fd-616c5ab8990e" containerName="extract"
Jul 29 18:00:27.123674 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:27.123606    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26ba8c29-8db7-4faf-bbd9-187f92b54bab" containerName="pull"
Jul 29 18:00:27.123674 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:27.123611    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="26ba8c29-8db7-4faf-bbd9-187f92b54bab" containerName="pull"
Jul 29 18:00:27.123674 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:27.123618    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da62e93b-16c5-4747-b35e-628e7636fcc8" containerName="util"
Jul 29 18:00:27.123674 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:27.123623    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="da62e93b-16c5-4747-b35e-628e7636fcc8" containerName="util"
Jul 29 18:00:27.123674 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:27.123631    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26ba8c29-8db7-4faf-bbd9-187f92b54bab" containerName="extract"
Jul 29 18:00:27.123674 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:27.123636    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="26ba8c29-8db7-4faf-bbd9-187f92b54bab" containerName="extract"
Jul 29 18:00:27.123674 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:27.123643    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f32a898-ddb9-4a13-8e36-83c7533a9755" containerName="pull"
Jul 29 18:00:27.123674 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:27.123647    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f32a898-ddb9-4a13-8e36-83c7533a9755" containerName="pull"
Jul 29 18:00:27.123674 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:27.123658    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ece83ae2-c4e0-4df9-95fd-616c5ab8990e" containerName="util"
Jul 29 18:00:27.123674 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:27.123666    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="ece83ae2-c4e0-4df9-95fd-616c5ab8990e" containerName="util"
Jul 29 18:00:27.123674 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:27.123675    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f32a898-ddb9-4a13-8e36-83c7533a9755" containerName="extract"
Jul 29 18:00:27.123674 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:27.123680    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f32a898-ddb9-4a13-8e36-83c7533a9755" containerName="extract"
Jul 29 18:00:27.124225 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:27.123687    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f32a898-ddb9-4a13-8e36-83c7533a9755" containerName="util"
Jul 29 18:00:27.124225 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:27.123692    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f32a898-ddb9-4a13-8e36-83c7533a9755" containerName="util"
Jul 29 18:00:27.124225 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:27.123696    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ece83ae2-c4e0-4df9-95fd-616c5ab8990e" containerName="pull"
Jul 29 18:00:27.124225 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:27.123701    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="ece83ae2-c4e0-4df9-95fd-616c5ab8990e" containerName="pull"
Jul 29 18:00:27.124225 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:27.123706    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da62e93b-16c5-4747-b35e-628e7636fcc8" containerName="pull"
Jul 29 18:00:27.124225 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:27.123711    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="da62e93b-16c5-4747-b35e-628e7636fcc8" containerName="pull"
Jul 29 18:00:27.124225 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:27.123787    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="26ba8c29-8db7-4faf-bbd9-187f92b54bab" containerName="extract"
Jul 29 18:00:27.124225 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:27.123797    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="da62e93b-16c5-4747-b35e-628e7636fcc8" containerName="extract"
Jul 29 18:00:27.124225 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:27.123805    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="ece83ae2-c4e0-4df9-95fd-616c5ab8990e" containerName="extract"
Jul 29 18:00:27.124225 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:27.123812    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f32a898-ddb9-4a13-8e36-83c7533a9755" containerName="extract"
Jul 29 18:00:27.129376 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:27.129356    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bbnbh"
Jul 29 18:00:27.132407 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:27.132383    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 29 18:00:27.132534 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:27.132502    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 29 18:00:27.132601 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:27.132506    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-nnb89\""
Jul 29 18:00:27.143294 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:27.143270    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bbnbh"]
Jul 29 18:00:27.243811 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:27.243770    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bcc23a48-61e4-4357-af15-20462ae7af92-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-bbnbh\" (UID: \"bcc23a48-61e4-4357-af15-20462ae7af92\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bbnbh"
Jul 29 18:00:27.244000 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:27.243861    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqg75\" (UniqueName: \"kubernetes.io/projected/bcc23a48-61e4-4357-af15-20462ae7af92-kube-api-access-tqg75\") pod \"kuadrant-operator-controller-manager-8684c9ff49-bbnbh\" (UID: \"bcc23a48-61e4-4357-af15-20462ae7af92\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bbnbh"
Jul 29 18:00:27.344688 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:27.344650    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bcc23a48-61e4-4357-af15-20462ae7af92-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-bbnbh\" (UID: \"bcc23a48-61e4-4357-af15-20462ae7af92\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bbnbh"
Jul 29 18:00:27.344875 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:27.344712    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqg75\" (UniqueName: \"kubernetes.io/projected/bcc23a48-61e4-4357-af15-20462ae7af92-kube-api-access-tqg75\") pod \"kuadrant-operator-controller-manager-8684c9ff49-bbnbh\" (UID: \"bcc23a48-61e4-4357-af15-20462ae7af92\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bbnbh"
Jul 29 18:00:27.345040 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:27.345018    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bcc23a48-61e4-4357-af15-20462ae7af92-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-bbnbh\" (UID: \"bcc23a48-61e4-4357-af15-20462ae7af92\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bbnbh"
Jul 29 18:00:27.354495 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:27.354464    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqg75\" (UniqueName: \"kubernetes.io/projected/bcc23a48-61e4-4357-af15-20462ae7af92-kube-api-access-tqg75\") pod \"kuadrant-operator-controller-manager-8684c9ff49-bbnbh\" (UID: \"bcc23a48-61e4-4357-af15-20462ae7af92\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bbnbh"
Jul 29 18:00:27.439544 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:27.439445    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bbnbh"
Jul 29 18:00:27.588710 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:27.588683    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bbnbh"]
Jul 29 18:00:27.590118 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:00:27.590092    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcc23a48_61e4_4357_af15_20462ae7af92.slice/crio-faab115352ffa3f39c7561f9100f1e8bc28a8cbd101b99b51700c9a596dedf45 WatchSource:0}: Error finding container faab115352ffa3f39c7561f9100f1e8bc28a8cbd101b99b51700c9a596dedf45: Status 404 returned error can't find the container with id faab115352ffa3f39c7561f9100f1e8bc28a8cbd101b99b51700c9a596dedf45
Jul 29 18:00:27.981437 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:27.981399    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bbnbh" event={"ID":"bcc23a48-61e4-4357-af15-20462ae7af92","Type":"ContainerStarted","Data":"faab115352ffa3f39c7561f9100f1e8bc28a8cbd101b99b51700c9a596dedf45"}
Jul 29 18:00:33.135958 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:33.135340    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-l8cgv"]
Jul 29 18:00:33.140183 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:33.140161    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-l8cgv"
Jul 29 18:00:33.143058 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:33.143031    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-9cfhm\""
Jul 29 18:00:33.143266 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:33.143250    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 29 18:00:33.158610 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:33.158588    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-l8cgv"]
Jul 29 18:00:33.192879 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:33.192856    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8bnr\" (UniqueName: \"kubernetes.io/projected/d1921098-bba3-4024-9220-fdefd73975ea-kube-api-access-b8bnr\") pod \"dns-operator-controller-manager-864dd78c4c-l8cgv\" (UID: \"d1921098-bba3-4024-9220-fdefd73975ea\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-l8cgv"
Jul 29 18:00:33.294081 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:33.294048    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8bnr\" (UniqueName: \"kubernetes.io/projected/d1921098-bba3-4024-9220-fdefd73975ea-kube-api-access-b8bnr\") pod \"dns-operator-controller-manager-864dd78c4c-l8cgv\" (UID: \"d1921098-bba3-4024-9220-fdefd73975ea\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-l8cgv"
Jul 29 18:00:33.302172 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:33.302148    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8bnr\" (UniqueName: \"kubernetes.io/projected/d1921098-bba3-4024-9220-fdefd73975ea-kube-api-access-b8bnr\") pod \"dns-operator-controller-manager-864dd78c4c-l8cgv\" (UID: \"d1921098-bba3-4024-9220-fdefd73975ea\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-l8cgv"
Jul 29 18:00:33.451966 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:33.451894    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-l8cgv"
Jul 29 18:00:33.585669 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:33.585626    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-l8cgv"]
Jul 29 18:00:33.587572 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:00:33.587537    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1921098_bba3_4024_9220_fdefd73975ea.slice/crio-59c57600ad24a4df13331a343928dee4bff9a90465109262d8aab7ad40227b57 WatchSource:0}: Error finding container 59c57600ad24a4df13331a343928dee4bff9a90465109262d8aab7ad40227b57: Status 404 returned error can't find the container with id 59c57600ad24a4df13331a343928dee4bff9a90465109262d8aab7ad40227b57
Jul 29 18:00:34.016465 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:34.016430    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-l8cgv" event={"ID":"d1921098-bba3-4024-9220-fdefd73975ea","Type":"ContainerStarted","Data":"59c57600ad24a4df13331a343928dee4bff9a90465109262d8aab7ad40227b57"}
Jul 29 18:00:34.017973 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:34.017947    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bbnbh" event={"ID":"bcc23a48-61e4-4357-af15-20462ae7af92","Type":"ContainerStarted","Data":"bd3583d17ce33447caa109bce23cb510a13100bc9312f6a3e2c93f0a3b5e05c6"}
Jul 29 18:00:34.018120 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:34.018105    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bbnbh"
Jul 29 18:00:34.039448 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:34.039387    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bbnbh" podStartSLOduration=1.539340058 podStartE2EDuration="7.03936716s" totalImagesPullingTime="5.500027102s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:00:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:00:34.037977817 +0000 UTC m=+513.907200625" watchObservedRunningTime="2026-07-29 18:00:34.03936716 +0000 UTC m=+513.908589969"
Jul 29 18:00:36.031487 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:36.031433    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-l8cgv" event={"ID":"d1921098-bba3-4024-9220-fdefd73975ea","Type":"ContainerStarted","Data":"d1ad9eae6c8915c29dd7283f071a49cb46ceffafe6fd2d08329c7f765826d473"}
Jul 29 18:00:36.031876 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:36.031632    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-l8cgv"
Jul 29 18:00:36.067040 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:36.066992    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-l8cgv" podStartSLOduration=0.757811225 podStartE2EDuration="3.066976163s" totalImagesPullingTime="2.309164938s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:00:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:00:36.064747946 +0000 UTC m=+515.933970752" watchObservedRunningTime="2026-07-29 18:00:36.066976163 +0000 UTC m=+515.936198973"
Jul 29 18:00:45.016858 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:45.016819    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5f5c4f8dfb-8nlj8" podUID="fc27aa4a-2b7e-4182-ad51-1eee720619dd" containerName="console" containerID="cri-o://c214d08b6ef77665b019af6bf644fb4c8432103c2e8dd9393d44b3f578537d48" gracePeriod=15
Jul 29 18:00:45.026440 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:45.026417    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bbnbh"
Jul 29 18:00:45.268695 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:45.268635    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f5c4f8dfb-8nlj8"
Jul 29 18:00:45.395599 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:45.395565    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fc27aa4a-2b7e-4182-ad51-1eee720619dd-oauth-serving-cert\") pod \"fc27aa4a-2b7e-4182-ad51-1eee720619dd\" (UID: \"fc27aa4a-2b7e-4182-ad51-1eee720619dd\") "
Jul 29 18:00:45.395800 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:45.395641    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc27aa4a-2b7e-4182-ad51-1eee720619dd-trusted-ca-bundle\") pod \"fc27aa4a-2b7e-4182-ad51-1eee720619dd\" (UID: \"fc27aa4a-2b7e-4182-ad51-1eee720619dd\") "
Jul 29 18:00:45.395800 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:45.395671    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fc27aa4a-2b7e-4182-ad51-1eee720619dd-console-oauth-config\") pod \"fc27aa4a-2b7e-4182-ad51-1eee720619dd\" (UID: \"fc27aa4a-2b7e-4182-ad51-1eee720619dd\") "
Jul 29 18:00:45.395800 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:45.395690    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fc27aa4a-2b7e-4182-ad51-1eee720619dd-console-serving-cert\") pod \"fc27aa4a-2b7e-4182-ad51-1eee720619dd\" (UID: \"fc27aa4a-2b7e-4182-ad51-1eee720619dd\") "
Jul 29 18:00:45.395800 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:45.395712    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xvp2k\" (UniqueName: \"kubernetes.io/projected/fc27aa4a-2b7e-4182-ad51-1eee720619dd-kube-api-access-xvp2k\") pod \"fc27aa4a-2b7e-4182-ad51-1eee720619dd\" (UID: \"fc27aa4a-2b7e-4182-ad51-1eee720619dd\") "
Jul 29 18:00:45.395800 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:45.395759    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fc27aa4a-2b7e-4182-ad51-1eee720619dd-service-ca\") pod \"fc27aa4a-2b7e-4182-ad51-1eee720619dd\" (UID: \"fc27aa4a-2b7e-4182-ad51-1eee720619dd\") "
Jul 29 18:00:45.396059 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:45.395914    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fc27aa4a-2b7e-4182-ad51-1eee720619dd-console-config\") pod \"fc27aa4a-2b7e-4182-ad51-1eee720619dd\" (UID: \"fc27aa4a-2b7e-4182-ad51-1eee720619dd\") "
Jul 29 18:00:45.396059 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:45.395994    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc27aa4a-2b7e-4182-ad51-1eee720619dd-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "fc27aa4a-2b7e-4182-ad51-1eee720619dd" (UID: "fc27aa4a-2b7e-4182-ad51-1eee720619dd"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:00:45.396189 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:45.396153    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc27aa4a-2b7e-4182-ad51-1eee720619dd-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "fc27aa4a-2b7e-4182-ad51-1eee720619dd" (UID: "fc27aa4a-2b7e-4182-ad51-1eee720619dd"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:00:45.396244 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:45.396185    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc27aa4a-2b7e-4182-ad51-1eee720619dd-service-ca" (OuterVolumeSpecName: "service-ca") pod "fc27aa4a-2b7e-4182-ad51-1eee720619dd" (UID: "fc27aa4a-2b7e-4182-ad51-1eee720619dd"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:00:45.396244 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:45.396232    2557 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fc27aa4a-2b7e-4182-ad51-1eee720619dd-oauth-serving-cert\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:00:45.396334 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:45.396249    2557 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc27aa4a-2b7e-4182-ad51-1eee720619dd-trusted-ca-bundle\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:00:45.396334 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:45.396260    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc27aa4a-2b7e-4182-ad51-1eee720619dd-console-config" (OuterVolumeSpecName: "console-config") pod "fc27aa4a-2b7e-4182-ad51-1eee720619dd" (UID: "fc27aa4a-2b7e-4182-ad51-1eee720619dd"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:00:45.398148 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:45.398116    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc27aa4a-2b7e-4182-ad51-1eee720619dd-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "fc27aa4a-2b7e-4182-ad51-1eee720619dd" (UID: "fc27aa4a-2b7e-4182-ad51-1eee720619dd"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:00:45.398253 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:45.398167    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc27aa4a-2b7e-4182-ad51-1eee720619dd-kube-api-access-xvp2k" (OuterVolumeSpecName: "kube-api-access-xvp2k") pod "fc27aa4a-2b7e-4182-ad51-1eee720619dd" (UID: "fc27aa4a-2b7e-4182-ad51-1eee720619dd"). InnerVolumeSpecName "kube-api-access-xvp2k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:00:45.398253 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:45.398182    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc27aa4a-2b7e-4182-ad51-1eee720619dd-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "fc27aa4a-2b7e-4182-ad51-1eee720619dd" (UID: "fc27aa4a-2b7e-4182-ad51-1eee720619dd"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:00:45.496846 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:45.496814    2557 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fc27aa4a-2b7e-4182-ad51-1eee720619dd-console-oauth-config\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:00:45.496846 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:45.496840    2557 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fc27aa4a-2b7e-4182-ad51-1eee720619dd-console-serving-cert\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:00:45.497018 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:45.496855    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xvp2k\" (UniqueName: \"kubernetes.io/projected/fc27aa4a-2b7e-4182-ad51-1eee720619dd-kube-api-access-xvp2k\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:00:45.497018 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:45.496867    2557 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fc27aa4a-2b7e-4182-ad51-1eee720619dd-service-ca\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:00:45.497018 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:45.496879    2557 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fc27aa4a-2b7e-4182-ad51-1eee720619dd-console-config\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:00:46.087691 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:46.087658    2557 generic.go:356] "Generic (PLEG): container finished" podID="fc27aa4a-2b7e-4182-ad51-1eee720619dd" containerID="c214d08b6ef77665b019af6bf644fb4c8432103c2e8dd9393d44b3f578537d48" exitCode=0
Jul 29 18:00:46.088165 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:46.087699    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f5c4f8dfb-8nlj8"
Jul 29 18:00:46.088165 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:46.087742    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f5c4f8dfb-8nlj8" event={"ID":"fc27aa4a-2b7e-4182-ad51-1eee720619dd","Type":"ContainerDied","Data":"c214d08b6ef77665b019af6bf644fb4c8432103c2e8dd9393d44b3f578537d48"}
Jul 29 18:00:46.088165 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:46.087796    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f5c4f8dfb-8nlj8" event={"ID":"fc27aa4a-2b7e-4182-ad51-1eee720619dd","Type":"ContainerDied","Data":"dd057d7f4d3cb27967113465fd4e3f13c50964b9a561aff08a8b000377619344"}
Jul 29 18:00:46.088165 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:46.087817    2557 scope.go:117] "RemoveContainer" containerID="c214d08b6ef77665b019af6bf644fb4c8432103c2e8dd9393d44b3f578537d48"
Jul 29 18:00:46.097569 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:46.097552    2557 scope.go:117] "RemoveContainer" containerID="c214d08b6ef77665b019af6bf644fb4c8432103c2e8dd9393d44b3f578537d48"
Jul 29 18:00:46.097824 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:00:46.097806    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c214d08b6ef77665b019af6bf644fb4c8432103c2e8dd9393d44b3f578537d48\": container with ID starting with c214d08b6ef77665b019af6bf644fb4c8432103c2e8dd9393d44b3f578537d48 not found: ID does not exist" containerID="c214d08b6ef77665b019af6bf644fb4c8432103c2e8dd9393d44b3f578537d48"
Jul 29 18:00:46.097886 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:46.097832    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c214d08b6ef77665b019af6bf644fb4c8432103c2e8dd9393d44b3f578537d48"} err="failed to get container status \"c214d08b6ef77665b019af6bf644fb4c8432103c2e8dd9393d44b3f578537d48\": rpc error: code = NotFound desc = could not find container \"c214d08b6ef77665b019af6bf644fb4c8432103c2e8dd9393d44b3f578537d48\": container with ID starting with c214d08b6ef77665b019af6bf644fb4c8432103c2e8dd9393d44b3f578537d48 not found: ID does not exist"
Jul 29 18:00:46.111443 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:46.111417    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f5c4f8dfb-8nlj8"]
Jul 29 18:00:46.115303 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:46.115285    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5f5c4f8dfb-8nlj8"]
Jul 29 18:00:46.652325 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:46.652280    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc27aa4a-2b7e-4182-ad51-1eee720619dd" path="/var/lib/kubelet/pods/fc27aa4a-2b7e-4182-ad51-1eee720619dd/volumes"
Jul 29 18:00:46.652600 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:46.652582    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bbnbh"]
Jul 29 18:00:46.652836 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:46.652807    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bbnbh" podUID="bcc23a48-61e4-4357-af15-20462ae7af92" containerName="manager" containerID="cri-o://bd3583d17ce33447caa109bce23cb510a13100bc9312f6a3e2c93f0a3b5e05c6" gracePeriod=2
Jul 29 18:00:46.654550 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:46.654525    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bbnbh"]
Jul 29 18:00:46.655349 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:00:46.655319    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-bbnbh\" is forbidden: User \"system:node:ip-10-0-142-156.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-156.ec2.internal' and this object" podUID="bcc23a48-61e4-4357-af15-20462ae7af92" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bbnbh"
Jul 29 18:00:46.672780 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:46.672752    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v2hs5"]
Jul 29 18:00:46.673247 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:46.673229    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc27aa4a-2b7e-4182-ad51-1eee720619dd" containerName="console"
Jul 29 18:00:46.673247 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:46.673249    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc27aa4a-2b7e-4182-ad51-1eee720619dd" containerName="console"
Jul 29 18:00:46.673386 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:46.673258    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcc23a48-61e4-4357-af15-20462ae7af92" containerName="manager"
Jul 29 18:00:46.673386 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:46.673264    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcc23a48-61e4-4357-af15-20462ae7af92" containerName="manager"
Jul 29 18:00:46.673386 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:46.673336    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc27aa4a-2b7e-4182-ad51-1eee720619dd" containerName="console"
Jul 29 18:00:46.673386 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:46.673344    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="bcc23a48-61e4-4357-af15-20462ae7af92" containerName="manager"
Jul 29 18:00:46.676168 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:46.676152    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v2hs5"
Jul 29 18:00:46.708048 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:46.708021    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/53896010-7de3-43dd-b081-c821518fe7da-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-v2hs5\" (UID: \"53896010-7de3-43dd-b081-c821518fe7da\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v2hs5"
Jul 29 18:00:46.708162 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:46.708061    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f84wp\" (UniqueName: \"kubernetes.io/projected/53896010-7de3-43dd-b081-c821518fe7da-kube-api-access-f84wp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-v2hs5\" (UID: \"53896010-7de3-43dd-b081-c821518fe7da\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v2hs5"
Jul 29 18:00:46.709422 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:46.709400    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v2hs5"]
Jul 29 18:00:46.733477 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:00:46.733451    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-bbnbh\" is forbidden: User \"system:node:ip-10-0-142-156.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-156.ec2.internal' and this object" podUID="bcc23a48-61e4-4357-af15-20462ae7af92" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bbnbh"
Jul 29 18:00:46.809009 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:46.808969    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/53896010-7de3-43dd-b081-c821518fe7da-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-v2hs5\" (UID: \"53896010-7de3-43dd-b081-c821518fe7da\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v2hs5"
Jul 29 18:00:46.809164 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:46.809020    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f84wp\" (UniqueName: \"kubernetes.io/projected/53896010-7de3-43dd-b081-c821518fe7da-kube-api-access-f84wp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-v2hs5\" (UID: \"53896010-7de3-43dd-b081-c821518fe7da\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v2hs5"
Jul 29 18:00:46.809409 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:46.809390    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/53896010-7de3-43dd-b081-c821518fe7da-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-v2hs5\" (UID: \"53896010-7de3-43dd-b081-c821518fe7da\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v2hs5"
Jul 29 18:00:46.822432 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:46.822211    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f84wp\" (UniqueName: \"kubernetes.io/projected/53896010-7de3-43dd-b081-c821518fe7da-kube-api-access-f84wp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-v2hs5\" (UID: \"53896010-7de3-43dd-b081-c821518fe7da\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v2hs5"
Jul 29 18:00:46.888816 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:46.888793    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bbnbh"
Jul 29 18:00:46.891361 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:00:46.891338    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-bbnbh\" is forbidden: User \"system:node:ip-10-0-142-156.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-156.ec2.internal' and this object" podUID="bcc23a48-61e4-4357-af15-20462ae7af92" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bbnbh"
Jul 29 18:00:46.987887 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:46.987817    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-45kjh"]
Jul 29 18:00:46.990932 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:46.990916    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-45kjh"
Jul 29 18:00:47.009820 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:47.009795    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-45kjh"]
Jul 29 18:00:47.010412 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:47.010390    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bcc23a48-61e4-4357-af15-20462ae7af92-extensions-socket-volume\") pod \"bcc23a48-61e4-4357-af15-20462ae7af92\" (UID: \"bcc23a48-61e4-4357-af15-20462ae7af92\") "
Jul 29 18:00:47.010581 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:47.010567    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tqg75\" (UniqueName: \"kubernetes.io/projected/bcc23a48-61e4-4357-af15-20462ae7af92-kube-api-access-tqg75\") pod \"bcc23a48-61e4-4357-af15-20462ae7af92\" (UID: \"bcc23a48-61e4-4357-af15-20462ae7af92\") "
Jul 29 18:00:47.014990 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:47.014958    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcc23a48-61e4-4357-af15-20462ae7af92-kube-api-access-tqg75" (OuterVolumeSpecName: "kube-api-access-tqg75") pod "bcc23a48-61e4-4357-af15-20462ae7af92" (UID: "bcc23a48-61e4-4357-af15-20462ae7af92"). InnerVolumeSpecName "kube-api-access-tqg75". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:00:47.015086 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:47.015016    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcc23a48-61e4-4357-af15-20462ae7af92-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "bcc23a48-61e4-4357-af15-20462ae7af92" (UID: "bcc23a48-61e4-4357-af15-20462ae7af92"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:00:47.032786 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:47.032765    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v2hs5"
Jul 29 18:00:47.038361 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:47.038338    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-l8cgv"
Jul 29 18:00:47.044809 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:00:47.044781    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-bbnbh\" is forbidden: User \"system:node:ip-10-0-142-156.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-156.ec2.internal' and this object" podUID="bcc23a48-61e4-4357-af15-20462ae7af92" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bbnbh"
Jul 29 18:00:47.047953 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:00:47.047928    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-bbnbh\" is forbidden: User \"system:node:ip-10-0-142-156.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-156.ec2.internal' and this object" podUID="bcc23a48-61e4-4357-af15-20462ae7af92" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bbnbh"
Jul 29 18:00:47.100282 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:47.100239    2557 generic.go:356] "Generic (PLEG): container finished" podID="bcc23a48-61e4-4357-af15-20462ae7af92" containerID="bd3583d17ce33447caa109bce23cb510a13100bc9312f6a3e2c93f0a3b5e05c6" exitCode=0
Jul 29 18:00:47.100687 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:47.100328    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bbnbh"
Jul 29 18:00:47.100687 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:47.100400    2557 scope.go:117] "RemoveContainer" containerID="bd3583d17ce33447caa109bce23cb510a13100bc9312f6a3e2c93f0a3b5e05c6"
Jul 29 18:00:47.103033 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:00:47.102989    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-bbnbh\" is forbidden: User \"system:node:ip-10-0-142-156.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-156.ec2.internal' and this object" podUID="bcc23a48-61e4-4357-af15-20462ae7af92" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bbnbh"
Jul 29 18:00:47.111277 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:47.111253    2557 scope.go:117] "RemoveContainer" containerID="bd3583d17ce33447caa109bce23cb510a13100bc9312f6a3e2c93f0a3b5e05c6"
Jul 29 18:00:47.111455 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:47.111431    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g265l\" (UniqueName: \"kubernetes.io/projected/d0899957-7e30-4059-abf6-00245ce7f130-kube-api-access-g265l\") pod \"kuadrant-operator-controller-manager-86878c769f-45kjh\" (UID: \"d0899957-7e30-4059-abf6-00245ce7f130\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-45kjh"
Jul 29 18:00:47.111601 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:47.111583    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d0899957-7e30-4059-abf6-00245ce7f130-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-45kjh\" (UID: \"d0899957-7e30-4059-abf6-00245ce7f130\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-45kjh"
Jul 29 18:00:47.111673 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:00:47.111600    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd3583d17ce33447caa109bce23cb510a13100bc9312f6a3e2c93f0a3b5e05c6\": container with ID starting with bd3583d17ce33447caa109bce23cb510a13100bc9312f6a3e2c93f0a3b5e05c6 not found: ID does not exist" containerID="bd3583d17ce33447caa109bce23cb510a13100bc9312f6a3e2c93f0a3b5e05c6"
Jul 29 18:00:47.111673 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:47.111632    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd3583d17ce33447caa109bce23cb510a13100bc9312f6a3e2c93f0a3b5e05c6"} err="failed to get container status \"bd3583d17ce33447caa109bce23cb510a13100bc9312f6a3e2c93f0a3b5e05c6\": rpc error: code = NotFound desc = could not find container \"bd3583d17ce33447caa109bce23cb510a13100bc9312f6a3e2c93f0a3b5e05c6\": container with ID starting with bd3583d17ce33447caa109bce23cb510a13100bc9312f6a3e2c93f0a3b5e05c6 not found: ID does not exist"
Jul 29 18:00:47.111867 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:47.111686    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tqg75\" (UniqueName: \"kubernetes.io/projected/bcc23a48-61e4-4357-af15-20462ae7af92-kube-api-access-tqg75\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:00:47.111867 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:47.111707    2557 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bcc23a48-61e4-4357-af15-20462ae7af92-extensions-socket-volume\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:00:47.114316 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:00:47.114271    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-bbnbh\" is forbidden: User \"system:node:ip-10-0-142-156.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-156.ec2.internal' and this object" podUID="bcc23a48-61e4-4357-af15-20462ae7af92" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bbnbh"
Jul 29 18:00:47.185371 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:47.185342    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v2hs5"]
Jul 29 18:00:47.186286 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:00:47.186259    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53896010_7de3_43dd_b081_c821518fe7da.slice/crio-1cfff8b285f30244185fb480ff682030e89c2e3f081a143574cd6fc685a991b0 WatchSource:0}: Error finding container 1cfff8b285f30244185fb480ff682030e89c2e3f081a143574cd6fc685a991b0: Status 404 returned error can't find the container with id 1cfff8b285f30244185fb480ff682030e89c2e3f081a143574cd6fc685a991b0
Jul 29 18:00:47.212113 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:47.212087    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g265l\" (UniqueName: \"kubernetes.io/projected/d0899957-7e30-4059-abf6-00245ce7f130-kube-api-access-g265l\") pod \"kuadrant-operator-controller-manager-86878c769f-45kjh\" (UID: \"d0899957-7e30-4059-abf6-00245ce7f130\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-45kjh"
Jul 29 18:00:47.212213 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:47.212146    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d0899957-7e30-4059-abf6-00245ce7f130-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-45kjh\" (UID: \"d0899957-7e30-4059-abf6-00245ce7f130\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-45kjh"
Jul 29 18:00:47.212433 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:47.212417    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d0899957-7e30-4059-abf6-00245ce7f130-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-45kjh\" (UID: \"d0899957-7e30-4059-abf6-00245ce7f130\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-45kjh"
Jul 29 18:00:47.223225 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:47.223205    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g265l\" (UniqueName: \"kubernetes.io/projected/d0899957-7e30-4059-abf6-00245ce7f130-kube-api-access-g265l\") pod \"kuadrant-operator-controller-manager-86878c769f-45kjh\" (UID: \"d0899957-7e30-4059-abf6-00245ce7f130\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-45kjh"
Jul 29 18:00:47.300688 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:47.300657    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-45kjh"
Jul 29 18:00:47.431367 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:47.431341    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-45kjh"]
Jul 29 18:00:47.432816 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:00:47.432785    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0899957_7e30_4059_abf6_00245ce7f130.slice/crio-d2fd77496dbea9beb92c06c65f6ae064d9855d4ecd185630853bd3da6d22e9ac WatchSource:0}: Error finding container d2fd77496dbea9beb92c06c65f6ae064d9855d4ecd185630853bd3da6d22e9ac: Status 404 returned error can't find the container with id d2fd77496dbea9beb92c06c65f6ae064d9855d4ecd185630853bd3da6d22e9ac
Jul 29 18:00:48.107762 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:48.107703    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-45kjh" event={"ID":"d0899957-7e30-4059-abf6-00245ce7f130","Type":"ContainerStarted","Data":"11089e31f476c2cfe9094b383871c50de328128db8fda0af8950c165e6572aa5"}
Jul 29 18:00:48.108204 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:48.107782    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-45kjh" event={"ID":"d0899957-7e30-4059-abf6-00245ce7f130","Type":"ContainerStarted","Data":"d2fd77496dbea9beb92c06c65f6ae064d9855d4ecd185630853bd3da6d22e9ac"}
Jul 29 18:00:48.108204 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:48.107810    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-45kjh"
Jul 29 18:00:48.110126 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:48.110102    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v2hs5" event={"ID":"53896010-7de3-43dd-b081-c821518fe7da","Type":"ContainerStarted","Data":"4f6876a1580a9aa3b54dbe3081b83bf51110d0404d8fe5869194662f230e9063"}
Jul 29 18:00:48.110227 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:48.110150    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v2hs5" event={"ID":"53896010-7de3-43dd-b081-c821518fe7da","Type":"ContainerStarted","Data":"1cfff8b285f30244185fb480ff682030e89c2e3f081a143574cd6fc685a991b0"}
Jul 29 18:00:48.110227 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:48.110182    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v2hs5"
Jul 29 18:00:48.130169 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:48.130129    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-45kjh" podStartSLOduration=2.130112831 podStartE2EDuration="2.130112831s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:00:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:00:48.128680792 +0000 UTC m=+527.997903612" watchObservedRunningTime="2026-07-29 18:00:48.130112831 +0000 UTC m=+527.999335640"
Jul 29 18:00:48.131083 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:00:48.131059    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-bbnbh\" is forbidden: User \"system:node:ip-10-0-142-156.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-156.ec2.internal' and this object" podUID="bcc23a48-61e4-4357-af15-20462ae7af92" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bbnbh"
Jul 29 18:00:48.155393 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:48.155343    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v2hs5" podStartSLOduration=2.15533104 podStartE2EDuration="2.15533104s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:00:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:00:48.15392689 +0000 UTC m=+528.023149700" watchObservedRunningTime="2026-07-29 18:00:48.15533104 +0000 UTC m=+528.024553846"
Jul 29 18:00:48.156046 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:00:48.156019    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-bbnbh\" is forbidden: User \"system:node:ip-10-0-142-156.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-156.ec2.internal' and this object" podUID="bcc23a48-61e4-4357-af15-20462ae7af92" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bbnbh"
Jul 29 18:00:48.652677 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:48.652642    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bcc23a48-61e4-4357-af15-20462ae7af92" path="/var/lib/kubelet/pods/bcc23a48-61e4-4357-af15-20462ae7af92/volumes"
Jul 29 18:00:59.117694 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:59.117663    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-45kjh"
Jul 29 18:00:59.118168 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:59.117762    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v2hs5"
Jul 29 18:00:59.197937 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:59.197907    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v2hs5"]
Jul 29 18:00:59.198163 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:59.198122    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v2hs5" podUID="53896010-7de3-43dd-b081-c821518fe7da" containerName="manager" containerID="cri-o://4f6876a1580a9aa3b54dbe3081b83bf51110d0404d8fe5869194662f230e9063" gracePeriod=10
Jul 29 18:00:59.442177 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:59.442154    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v2hs5"
Jul 29 18:00:59.519547 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:59.519521    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/53896010-7de3-43dd-b081-c821518fe7da-extensions-socket-volume\") pod \"53896010-7de3-43dd-b081-c821518fe7da\" (UID: \"53896010-7de3-43dd-b081-c821518fe7da\") "
Jul 29 18:00:59.519688 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:59.519617    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f84wp\" (UniqueName: \"kubernetes.io/projected/53896010-7de3-43dd-b081-c821518fe7da-kube-api-access-f84wp\") pod \"53896010-7de3-43dd-b081-c821518fe7da\" (UID: \"53896010-7de3-43dd-b081-c821518fe7da\") "
Jul 29 18:00:59.519912 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:59.519888    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53896010-7de3-43dd-b081-c821518fe7da-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "53896010-7de3-43dd-b081-c821518fe7da" (UID: "53896010-7de3-43dd-b081-c821518fe7da"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:00:59.521668 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:59.521643    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53896010-7de3-43dd-b081-c821518fe7da-kube-api-access-f84wp" (OuterVolumeSpecName: "kube-api-access-f84wp") pod "53896010-7de3-43dd-b081-c821518fe7da" (UID: "53896010-7de3-43dd-b081-c821518fe7da"). InnerVolumeSpecName "kube-api-access-f84wp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:00:59.621207 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:59.621122    2557 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/53896010-7de3-43dd-b081-c821518fe7da-extensions-socket-volume\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:00:59.621207 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:00:59.621156    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f84wp\" (UniqueName: \"kubernetes.io/projected/53896010-7de3-43dd-b081-c821518fe7da-kube-api-access-f84wp\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:01:00.180047 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:00.180012    2557 generic.go:356] "Generic (PLEG): container finished" podID="53896010-7de3-43dd-b081-c821518fe7da" containerID="4f6876a1580a9aa3b54dbe3081b83bf51110d0404d8fe5869194662f230e9063" exitCode=0
Jul 29 18:01:00.180518 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:00.180064    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v2hs5"
Jul 29 18:01:00.180518 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:00.180098    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v2hs5" event={"ID":"53896010-7de3-43dd-b081-c821518fe7da","Type":"ContainerDied","Data":"4f6876a1580a9aa3b54dbe3081b83bf51110d0404d8fe5869194662f230e9063"}
Jul 29 18:01:00.180518 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:00.180147    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v2hs5" event={"ID":"53896010-7de3-43dd-b081-c821518fe7da","Type":"ContainerDied","Data":"1cfff8b285f30244185fb480ff682030e89c2e3f081a143574cd6fc685a991b0"}
Jul 29 18:01:00.180518 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:00.180169    2557 scope.go:117] "RemoveContainer" containerID="4f6876a1580a9aa3b54dbe3081b83bf51110d0404d8fe5869194662f230e9063"
Jul 29 18:01:00.189482 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:00.189463    2557 scope.go:117] "RemoveContainer" containerID="4f6876a1580a9aa3b54dbe3081b83bf51110d0404d8fe5869194662f230e9063"
Jul 29 18:01:00.189762 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:01:00.189712    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f6876a1580a9aa3b54dbe3081b83bf51110d0404d8fe5869194662f230e9063\": container with ID starting with 4f6876a1580a9aa3b54dbe3081b83bf51110d0404d8fe5869194662f230e9063 not found: ID does not exist" containerID="4f6876a1580a9aa3b54dbe3081b83bf51110d0404d8fe5869194662f230e9063"
Jul 29 18:01:00.189825 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:00.189766    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f6876a1580a9aa3b54dbe3081b83bf51110d0404d8fe5869194662f230e9063"} err="failed to get container status \"4f6876a1580a9aa3b54dbe3081b83bf51110d0404d8fe5869194662f230e9063\": rpc error: code = NotFound desc = could not find container \"4f6876a1580a9aa3b54dbe3081b83bf51110d0404d8fe5869194662f230e9063\": container with ID starting with 4f6876a1580a9aa3b54dbe3081b83bf51110d0404d8fe5869194662f230e9063 not found: ID does not exist"
Jul 29 18:01:00.224486 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:00.224454    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v2hs5"]
Jul 29 18:01:00.234216 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:00.234189    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v2hs5"]
Jul 29 18:01:00.652023 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:00.651993    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53896010-7de3-43dd-b081-c821518fe7da" path="/var/lib/kubelet/pods/53896010-7de3-43dd-b081-c821518fe7da/volumes"
Jul 29 18:01:15.552933 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:15.552902    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vzxzx"]
Jul 29 18:01:15.553337 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:15.553249    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53896010-7de3-43dd-b081-c821518fe7da" containerName="manager"
Jul 29 18:01:15.553337 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:15.553261    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="53896010-7de3-43dd-b081-c821518fe7da" containerName="manager"
Jul 29 18:01:15.553337 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:15.553337    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="53896010-7de3-43dd-b081-c821518fe7da" containerName="manager"
Jul 29 18:01:15.558279 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:15.558260    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vzxzx"
Jul 29 18:01:15.563612 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:15.563590    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-cqknd\""
Jul 29 18:01:15.591624 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:15.591598    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vzxzx"]
Jul 29 18:01:15.657368 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:15.657335    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5bb4f610-8260-474c-bae4-e5d9ed9a5280-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vzxzx\" (UID: \"5bb4f610-8260-474c-bae4-e5d9ed9a5280\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vzxzx"
Jul 29 18:01:15.657555 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:15.657378    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7d6kc\" (UniqueName: \"kubernetes.io/projected/5bb4f610-8260-474c-bae4-e5d9ed9a5280-kube-api-access-7d6kc\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vzxzx\" (UID: \"5bb4f610-8260-474c-bae4-e5d9ed9a5280\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vzxzx"
Jul 29 18:01:15.657555 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:15.657410    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bb4f610-8260-474c-bae4-e5d9ed9a5280-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vzxzx\" (UID: \"5bb4f610-8260-474c-bae4-e5d9ed9a5280\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vzxzx"
Jul 29 18:01:15.657555 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:15.657434    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bb4f610-8260-474c-bae4-e5d9ed9a5280-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vzxzx\" (UID: \"5bb4f610-8260-474c-bae4-e5d9ed9a5280\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vzxzx"
Jul 29 18:01:15.657555 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:15.657495    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5bb4f610-8260-474c-bae4-e5d9ed9a5280-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vzxzx\" (UID: \"5bb4f610-8260-474c-bae4-e5d9ed9a5280\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vzxzx"
Jul 29 18:01:15.657555 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:15.657521    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5bb4f610-8260-474c-bae4-e5d9ed9a5280-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vzxzx\" (UID: \"5bb4f610-8260-474c-bae4-e5d9ed9a5280\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vzxzx"
Jul 29 18:01:15.657800 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:15.657582    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5bb4f610-8260-474c-bae4-e5d9ed9a5280-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vzxzx\" (UID: \"5bb4f610-8260-474c-bae4-e5d9ed9a5280\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vzxzx"
Jul 29 18:01:15.657800 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:15.657609    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5bb4f610-8260-474c-bae4-e5d9ed9a5280-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vzxzx\" (UID: \"5bb4f610-8260-474c-bae4-e5d9ed9a5280\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vzxzx"
Jul 29 18:01:15.657800 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:15.657674    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5bb4f610-8260-474c-bae4-e5d9ed9a5280-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vzxzx\" (UID: \"5bb4f610-8260-474c-bae4-e5d9ed9a5280\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vzxzx"
Jul 29 18:01:15.758248 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:15.758214    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5bb4f610-8260-474c-bae4-e5d9ed9a5280-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vzxzx\" (UID: \"5bb4f610-8260-474c-bae4-e5d9ed9a5280\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vzxzx"
Jul 29 18:01:15.758442 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:15.758263    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5bb4f610-8260-474c-bae4-e5d9ed9a5280-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vzxzx\" (UID: \"5bb4f610-8260-474c-bae4-e5d9ed9a5280\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vzxzx"
Jul 29 18:01:15.758442 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:15.758320    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5bb4f610-8260-474c-bae4-e5d9ed9a5280-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vzxzx\" (UID: \"5bb4f610-8260-474c-bae4-e5d9ed9a5280\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vzxzx"
Jul 29 18:01:15.758442 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:15.758344    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5bb4f610-8260-474c-bae4-e5d9ed9a5280-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vzxzx\" (UID: \"5bb4f610-8260-474c-bae4-e5d9ed9a5280\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vzxzx"
Jul 29 18:01:15.758602 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:15.758532    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5bb4f610-8260-474c-bae4-e5d9ed9a5280-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vzxzx\" (UID: \"5bb4f610-8260-474c-bae4-e5d9ed9a5280\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vzxzx"
Jul 29 18:01:15.758661 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:15.758606    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5bb4f610-8260-474c-bae4-e5d9ed9a5280-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vzxzx\" (UID: \"5bb4f610-8260-474c-bae4-e5d9ed9a5280\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vzxzx"
Jul 29 18:01:15.758754 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:15.758663    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7d6kc\" (UniqueName: \"kubernetes.io/projected/5bb4f610-8260-474c-bae4-e5d9ed9a5280-kube-api-access-7d6kc\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vzxzx\" (UID: \"5bb4f610-8260-474c-bae4-e5d9ed9a5280\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vzxzx"
Jul 29 18:01:15.758827 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:15.758770    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bb4f610-8260-474c-bae4-e5d9ed9a5280-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vzxzx\" (UID: \"5bb4f610-8260-474c-bae4-e5d9ed9a5280\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vzxzx"
Jul 29 18:01:15.758827 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:15.758805    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bb4f610-8260-474c-bae4-e5d9ed9a5280-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vzxzx\" (UID: \"5bb4f610-8260-474c-bae4-e5d9ed9a5280\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vzxzx"
Jul 29 18:01:15.759147 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:15.759117    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5bb4f610-8260-474c-bae4-e5d9ed9a5280-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vzxzx\" (UID: \"5bb4f610-8260-474c-bae4-e5d9ed9a5280\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vzxzx"
Jul 29 18:01:15.759282 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:15.759124    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bb4f610-8260-474c-bae4-e5d9ed9a5280-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vzxzx\" (UID: \"5bb4f610-8260-474c-bae4-e5d9ed9a5280\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vzxzx"
Jul 29 18:01:15.759282 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:15.759162    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bb4f610-8260-474c-bae4-e5d9ed9a5280-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vzxzx\" (UID: \"5bb4f610-8260-474c-bae4-e5d9ed9a5280\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vzxzx"
Jul 29 18:01:15.759282 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:15.758772    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5bb4f610-8260-474c-bae4-e5d9ed9a5280-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vzxzx\" (UID: \"5bb4f610-8260-474c-bae4-e5d9ed9a5280\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vzxzx"
Jul 29 18:01:15.759282 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:15.759271    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5bb4f610-8260-474c-bae4-e5d9ed9a5280-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vzxzx\" (UID: \"5bb4f610-8260-474c-bae4-e5d9ed9a5280\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vzxzx"
Jul 29 18:01:15.761062 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:15.761041    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5bb4f610-8260-474c-bae4-e5d9ed9a5280-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vzxzx\" (UID: \"5bb4f610-8260-474c-bae4-e5d9ed9a5280\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vzxzx"
Jul 29 18:01:15.761171 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:15.761106    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5bb4f610-8260-474c-bae4-e5d9ed9a5280-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vzxzx\" (UID: \"5bb4f610-8260-474c-bae4-e5d9ed9a5280\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vzxzx"
Jul 29 18:01:15.775001 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:15.774978    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7d6kc\" (UniqueName: \"kubernetes.io/projected/5bb4f610-8260-474c-bae4-e5d9ed9a5280-kube-api-access-7d6kc\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vzxzx\" (UID: \"5bb4f610-8260-474c-bae4-e5d9ed9a5280\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vzxzx"
Jul 29 18:01:15.775138 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:15.775122    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5bb4f610-8260-474c-bae4-e5d9ed9a5280-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vzxzx\" (UID: \"5bb4f610-8260-474c-bae4-e5d9ed9a5280\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vzxzx"
Jul 29 18:01:15.868688 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:15.868604    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vzxzx"
Jul 29 18:01:16.013324 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:16.013300    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vzxzx"]
Jul 29 18:01:16.014977 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:01:16.014947    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bb4f610_8260_474c_bae4_e5d9ed9a5280.slice/crio-3f4037cfc743b39cfd06fd2c970b6d2ed76c0d5a1f238be71b3ef9b011d687fa WatchSource:0}: Error finding container 3f4037cfc743b39cfd06fd2c970b6d2ed76c0d5a1f238be71b3ef9b011d687fa: Status 404 returned error can't find the container with id 3f4037cfc743b39cfd06fd2c970b6d2ed76c0d5a1f238be71b3ef9b011d687fa
Jul 29 18:01:16.017018 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:16.016988    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:01:16.017078 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:16.017049    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:01:16.017127 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:16.017079    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:01:16.268222 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:16.268127    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vzxzx" event={"ID":"5bb4f610-8260-474c-bae4-e5d9ed9a5280","Type":"ContainerStarted","Data":"3c3e20439b758103ea68df71d31094300c410b460397aaff5e2500cd4d072baf"}
Jul 29 18:01:16.268222 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:16.268185    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vzxzx" event={"ID":"5bb4f610-8260-474c-bae4-e5d9ed9a5280","Type":"ContainerStarted","Data":"3f4037cfc743b39cfd06fd2c970b6d2ed76c0d5a1f238be71b3ef9b011d687fa"}
Jul 29 18:01:16.297965 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:16.297906    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vzxzx" podStartSLOduration=1.29788755 podStartE2EDuration="1.29788755s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:01:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:01:16.295622366 +0000 UTC m=+556.164845175" watchObservedRunningTime="2026-07-29 18:01:16.29788755 +0000 UTC m=+556.167110362"
Jul 29 18:01:16.868873 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:16.868843    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vzxzx"
Jul 29 18:01:17.335872 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:17.335842    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-lkbfh"]
Jul 29 18:01:17.339377 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:17.339360    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-lkbfh"
Jul 29 18:01:17.343395 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:17.343367    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-dz6bj\""
Jul 29 18:01:17.343515 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:17.343497    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 29 18:01:17.351697 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:17.351676    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-lkbfh"]
Jul 29 18:01:17.423579 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:17.423551    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-lkbfh"]
Jul 29 18:01:17.474198 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:17.474162    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ceb6a366-4224-4820-b3b6-6630e87a8332-config-file\") pod \"limitador-limitador-64c94fd64f-lkbfh\" (UID: \"ceb6a366-4224-4820-b3b6-6630e87a8332\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-lkbfh"
Jul 29 18:01:17.474341 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:17.474205    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xt978\" (UniqueName: \"kubernetes.io/projected/ceb6a366-4224-4820-b3b6-6630e87a8332-kube-api-access-xt978\") pod \"limitador-limitador-64c94fd64f-lkbfh\" (UID: \"ceb6a366-4224-4820-b3b6-6630e87a8332\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-lkbfh"
Jul 29 18:01:17.575717 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:17.575683    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ceb6a366-4224-4820-b3b6-6630e87a8332-config-file\") pod \"limitador-limitador-64c94fd64f-lkbfh\" (UID: \"ceb6a366-4224-4820-b3b6-6630e87a8332\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-lkbfh"
Jul 29 18:01:17.575717 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:17.575744    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xt978\" (UniqueName: \"kubernetes.io/projected/ceb6a366-4224-4820-b3b6-6630e87a8332-kube-api-access-xt978\") pod \"limitador-limitador-64c94fd64f-lkbfh\" (UID: \"ceb6a366-4224-4820-b3b6-6630e87a8332\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-lkbfh"
Jul 29 18:01:17.576359 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:17.576333    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ceb6a366-4224-4820-b3b6-6630e87a8332-config-file\") pod \"limitador-limitador-64c94fd64f-lkbfh\" (UID: \"ceb6a366-4224-4820-b3b6-6630e87a8332\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-lkbfh"
Jul 29 18:01:17.586119 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:17.586066    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xt978\" (UniqueName: \"kubernetes.io/projected/ceb6a366-4224-4820-b3b6-6630e87a8332-kube-api-access-xt978\") pod \"limitador-limitador-64c94fd64f-lkbfh\" (UID: \"ceb6a366-4224-4820-b3b6-6630e87a8332\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-lkbfh"
Jul 29 18:01:17.650012 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:17.649987    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-lkbfh"
Jul 29 18:01:17.783232 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:17.783201    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-lkbfh"]
Jul 29 18:01:17.784069 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:01:17.784043    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podceb6a366_4224_4820_b3b6_6630e87a8332.slice/crio-cbfe2224560c976097a37817bd52009a2339db42d2e28789d0b9c742e2759cf9 WatchSource:0}: Error finding container cbfe2224560c976097a37817bd52009a2339db42d2e28789d0b9c742e2759cf9: Status 404 returned error can't find the container with id cbfe2224560c976097a37817bd52009a2339db42d2e28789d0b9c742e2759cf9
Jul 29 18:01:17.873926 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:17.873855    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vzxzx"
Jul 29 18:01:18.279220 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:18.279139    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-lkbfh" event={"ID":"ceb6a366-4224-4820-b3b6-6630e87a8332","Type":"ContainerStarted","Data":"cbfe2224560c976097a37817bd52009a2339db42d2e28789d0b9c742e2759cf9"}
Jul 29 18:01:18.279384 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:18.279368    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vzxzx"
Jul 29 18:01:18.280465 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:18.280442    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vzxzx"
Jul 29 18:01:21.302344 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:21.302302    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-lkbfh" event={"ID":"ceb6a366-4224-4820-b3b6-6630e87a8332","Type":"ContainerStarted","Data":"37eb8108ddf92dbe5dd27b158edcd70261464b474e275a96676585099f220da1"}
Jul 29 18:01:21.302740 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:21.302369    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-lkbfh"
Jul 29 18:01:21.322570 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:21.322522    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-lkbfh" podStartSLOduration=1.401392936 podStartE2EDuration="4.322508081s" totalImagesPullingTime="2.921115145s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:01:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:01:21.321315383 +0000 UTC m=+561.190538191" watchObservedRunningTime="2026-07-29 18:01:21.322508081 +0000 UTC m=+561.191730888"
Jul 29 18:01:31.861767 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:31.861717    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-lkbfh"]
Jul 29 18:01:31.862258 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:31.861913    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-lkbfh" podUID="ceb6a366-4224-4820-b3b6-6630e87a8332" containerName="limitador" containerID="cri-o://37eb8108ddf92dbe5dd27b158edcd70261464b474e275a96676585099f220da1" gracePeriod=30
Jul 29 18:01:31.862660 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:31.862639    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-lkbfh"
Jul 29 18:01:32.360261 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:32.360227    2557 generic.go:356] "Generic (PLEG): container finished" podID="ceb6a366-4224-4820-b3b6-6630e87a8332" containerID="37eb8108ddf92dbe5dd27b158edcd70261464b474e275a96676585099f220da1" exitCode=0
Jul 29 18:01:32.360391 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:32.360288    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-lkbfh" event={"ID":"ceb6a366-4224-4820-b3b6-6630e87a8332","Type":"ContainerDied","Data":"37eb8108ddf92dbe5dd27b158edcd70261464b474e275a96676585099f220da1"}
Jul 29 18:01:32.415694 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:32.415673    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-lkbfh"
Jul 29 18:01:32.488852 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:32.488795    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ceb6a366-4224-4820-b3b6-6630e87a8332-config-file\") pod \"ceb6a366-4224-4820-b3b6-6630e87a8332\" (UID: \"ceb6a366-4224-4820-b3b6-6630e87a8332\") "
Jul 29 18:01:32.488982 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:32.488856    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xt978\" (UniqueName: \"kubernetes.io/projected/ceb6a366-4224-4820-b3b6-6630e87a8332-kube-api-access-xt978\") pod \"ceb6a366-4224-4820-b3b6-6630e87a8332\" (UID: \"ceb6a366-4224-4820-b3b6-6630e87a8332\") "
Jul 29 18:01:32.489194 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:32.489171    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ceb6a366-4224-4820-b3b6-6630e87a8332-config-file" (OuterVolumeSpecName: "config-file") pod "ceb6a366-4224-4820-b3b6-6630e87a8332" (UID: "ceb6a366-4224-4820-b3b6-6630e87a8332"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:01:32.490957 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:32.490936    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ceb6a366-4224-4820-b3b6-6630e87a8332-kube-api-access-xt978" (OuterVolumeSpecName: "kube-api-access-xt978") pod "ceb6a366-4224-4820-b3b6-6630e87a8332" (UID: "ceb6a366-4224-4820-b3b6-6630e87a8332"). InnerVolumeSpecName "kube-api-access-xt978". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:01:32.590218 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:32.590188    2557 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ceb6a366-4224-4820-b3b6-6630e87a8332-config-file\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:01:32.590218 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:32.590213    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xt978\" (UniqueName: \"kubernetes.io/projected/ceb6a366-4224-4820-b3b6-6630e87a8332-kube-api-access-xt978\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:01:33.367510 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:33.367485    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-lkbfh"
Jul 29 18:01:33.367510 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:33.367498    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-lkbfh" event={"ID":"ceb6a366-4224-4820-b3b6-6630e87a8332","Type":"ContainerDied","Data":"cbfe2224560c976097a37817bd52009a2339db42d2e28789d0b9c742e2759cf9"}
Jul 29 18:01:33.367992 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:33.367593    2557 scope.go:117] "RemoveContainer" containerID="37eb8108ddf92dbe5dd27b158edcd70261464b474e275a96676585099f220da1"
Jul 29 18:01:33.386984 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:33.386955    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-lkbfh"]
Jul 29 18:01:33.391985 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:33.391963    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-lkbfh"]
Jul 29 18:01:33.869108 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:33.869079    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-4cbzt"]
Jul 29 18:01:33.869467 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:33.869444    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ceb6a366-4224-4820-b3b6-6630e87a8332" containerName="limitador"
Jul 29 18:01:33.869467 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:33.869461    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="ceb6a366-4224-4820-b3b6-6630e87a8332" containerName="limitador"
Jul 29 18:01:33.869623 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:33.869525    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="ceb6a366-4224-4820-b3b6-6630e87a8332" containerName="limitador"
Jul 29 18:01:33.873784 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:33.873768    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-4cbzt"
Jul 29 18:01:33.876682 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:33.876662    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-mcn79\""
Jul 29 18:01:33.877992 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:33.877973    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 29 18:01:33.878112 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:33.878007    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 29 18:01:33.878112 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:33.877974    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 29 18:01:33.881853 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:33.881831    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-4cbzt"]
Jul 29 18:01:33.908619 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:33.908596    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/9353b89d-9f69-4bca-ba18-a0ee0bef1631-data\") pod \"postgres-69fb4594cf-4cbzt\" (UID: \"9353b89d-9f69-4bca-ba18-a0ee0bef1631\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-4cbzt"
Jul 29 18:01:33.908739 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:33.908641    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvsxp\" (UniqueName: \"kubernetes.io/projected/9353b89d-9f69-4bca-ba18-a0ee0bef1631-kube-api-access-kvsxp\") pod \"postgres-69fb4594cf-4cbzt\" (UID: \"9353b89d-9f69-4bca-ba18-a0ee0bef1631\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-4cbzt"
Jul 29 18:01:34.009755 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:34.009694    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvsxp\" (UniqueName: \"kubernetes.io/projected/9353b89d-9f69-4bca-ba18-a0ee0bef1631-kube-api-access-kvsxp\") pod \"postgres-69fb4594cf-4cbzt\" (UID: \"9353b89d-9f69-4bca-ba18-a0ee0bef1631\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-4cbzt"
Jul 29 18:01:34.009932 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:34.009836    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/9353b89d-9f69-4bca-ba18-a0ee0bef1631-data\") pod \"postgres-69fb4594cf-4cbzt\" (UID: \"9353b89d-9f69-4bca-ba18-a0ee0bef1631\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-4cbzt"
Jul 29 18:01:34.010178 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:34.010162    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/9353b89d-9f69-4bca-ba18-a0ee0bef1631-data\") pod \"postgres-69fb4594cf-4cbzt\" (UID: \"9353b89d-9f69-4bca-ba18-a0ee0bef1631\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-4cbzt"
Jul 29 18:01:34.018077 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:34.018048    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvsxp\" (UniqueName: \"kubernetes.io/projected/9353b89d-9f69-4bca-ba18-a0ee0bef1631-kube-api-access-kvsxp\") pod \"postgres-69fb4594cf-4cbzt\" (UID: \"9353b89d-9f69-4bca-ba18-a0ee0bef1631\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-4cbzt"
Jul 29 18:01:34.184115 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:34.184028    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-4cbzt"
Jul 29 18:01:34.314886 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:34.314859    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-4cbzt"]
Jul 29 18:01:34.315981 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:01:34.315954    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9353b89d_9f69_4bca_ba18_a0ee0bef1631.slice/crio-cf73384c7de8e8154fc44479c031021c6aefe6caa2d32f16543b4bb7260ab466 WatchSource:0}: Error finding container cf73384c7de8e8154fc44479c031021c6aefe6caa2d32f16543b4bb7260ab466: Status 404 returned error can't find the container with id cf73384c7de8e8154fc44479c031021c6aefe6caa2d32f16543b4bb7260ab466
Jul 29 18:01:34.374382 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:34.374357    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-4cbzt" event={"ID":"9353b89d-9f69-4bca-ba18-a0ee0bef1631","Type":"ContainerStarted","Data":"cf73384c7de8e8154fc44479c031021c6aefe6caa2d32f16543b4bb7260ab466"}
Jul 29 18:01:34.652379 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:34.652343    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ceb6a366-4224-4820-b3b6-6630e87a8332" path="/var/lib/kubelet/pods/ceb6a366-4224-4820-b3b6-6630e87a8332/volumes"
Jul 29 18:01:39.409341 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:39.409267    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-4cbzt" event={"ID":"9353b89d-9f69-4bca-ba18-a0ee0bef1631","Type":"ContainerStarted","Data":"d7f6f8661112bb0a6b6348f2de86690d82ad90af620761a53bae06a7c66dbc8d"}
Jul 29 18:01:39.409341 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:39.409329    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-4cbzt"
Jul 29 18:01:39.430226 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:39.430179    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-4cbzt" podStartSLOduration=1.567751216 podStartE2EDuration="6.430165176s" totalImagesPullingTime="4.86241396s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:01:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:01:39.427700786 +0000 UTC m=+579.296923592" watchObservedRunningTime="2026-07-29 18:01:39.430165176 +0000 UTC m=+579.299388027"
Jul 29 18:01:45.443059 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:45.443029    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-4cbzt"
Jul 29 18:01:53.156029 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:53.155997    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-86dc5fff6f-ww8cr"]
Jul 29 18:01:53.166933 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:53.166903    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-86dc5fff6f-ww8cr"
Jul 29 18:01:53.170199 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:53.170164    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 29 18:01:53.171343 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:53.171318    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-6j7nb\""
Jul 29 18:01:53.171504 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:53.171318    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 29 18:01:53.173984 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:53.173965    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-86dc5fff6f-ww8cr"]
Jul 29 18:01:53.278431 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:53.278397    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dn7wf\" (UniqueName: \"kubernetes.io/projected/d6d0c4fd-99a5-4cdb-9af2-2f82bbbc539d-kube-api-access-dn7wf\") pod \"maas-controller-86dc5fff6f-ww8cr\" (UID: \"d6d0c4fd-99a5-4cdb-9af2-2f82bbbc539d\") " pod="opendatahub/maas-controller-86dc5fff6f-ww8cr"
Jul 29 18:01:53.278588 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:53.278477    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d6d0c4fd-99a5-4cdb-9af2-2f82bbbc539d-webhook-cert\") pod \"maas-controller-86dc5fff6f-ww8cr\" (UID: \"d6d0c4fd-99a5-4cdb-9af2-2f82bbbc539d\") " pod="opendatahub/maas-controller-86dc5fff6f-ww8cr"
Jul 29 18:01:53.345837 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:53.345803    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5c67f77dd8-6bsc2"]
Jul 29 18:01:53.349220 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:53.349204    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c67f77dd8-6bsc2"
Jul 29 18:01:53.360966 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:53.360945    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5c67f77dd8-6bsc2"]
Jul 29 18:01:53.379934 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:53.379910    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d6d0c4fd-99a5-4cdb-9af2-2f82bbbc539d-webhook-cert\") pod \"maas-controller-86dc5fff6f-ww8cr\" (UID: \"d6d0c4fd-99a5-4cdb-9af2-2f82bbbc539d\") " pod="opendatahub/maas-controller-86dc5fff6f-ww8cr"
Jul 29 18:01:53.380067 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:53.379956    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dn7wf\" (UniqueName: \"kubernetes.io/projected/d6d0c4fd-99a5-4cdb-9af2-2f82bbbc539d-kube-api-access-dn7wf\") pod \"maas-controller-86dc5fff6f-ww8cr\" (UID: \"d6d0c4fd-99a5-4cdb-9af2-2f82bbbc539d\") " pod="opendatahub/maas-controller-86dc5fff6f-ww8cr"
Jul 29 18:01:53.382373 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:53.382350    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d6d0c4fd-99a5-4cdb-9af2-2f82bbbc539d-webhook-cert\") pod \"maas-controller-86dc5fff6f-ww8cr\" (UID: \"d6d0c4fd-99a5-4cdb-9af2-2f82bbbc539d\") " pod="opendatahub/maas-controller-86dc5fff6f-ww8cr"
Jul 29 18:01:53.388114 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:53.388091    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dn7wf\" (UniqueName: \"kubernetes.io/projected/d6d0c4fd-99a5-4cdb-9af2-2f82bbbc539d-kube-api-access-dn7wf\") pod \"maas-controller-86dc5fff6f-ww8cr\" (UID: \"d6d0c4fd-99a5-4cdb-9af2-2f82bbbc539d\") " pod="opendatahub/maas-controller-86dc5fff6f-ww8cr"
Jul 29 18:01:53.479310 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:53.479253    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-86dc5fff6f-ww8cr"
Jul 29 18:01:53.480683 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:53.480665    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bce97daa-2099-4347-8369-0e1035532aaa-webhook-cert\") pod \"maas-controller-5c67f77dd8-6bsc2\" (UID: \"bce97daa-2099-4347-8369-0e1035532aaa\") " pod="opendatahub/maas-controller-5c67f77dd8-6bsc2"
Jul 29 18:01:53.480775 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:53.480714    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbdlb\" (UniqueName: \"kubernetes.io/projected/bce97daa-2099-4347-8369-0e1035532aaa-kube-api-access-hbdlb\") pod \"maas-controller-5c67f77dd8-6bsc2\" (UID: \"bce97daa-2099-4347-8369-0e1035532aaa\") " pod="opendatahub/maas-controller-5c67f77dd8-6bsc2"
Jul 29 18:01:53.581835 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:53.581318    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bce97daa-2099-4347-8369-0e1035532aaa-webhook-cert\") pod \"maas-controller-5c67f77dd8-6bsc2\" (UID: \"bce97daa-2099-4347-8369-0e1035532aaa\") " pod="opendatahub/maas-controller-5c67f77dd8-6bsc2"
Jul 29 18:01:53.581835 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:53.581415    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbdlb\" (UniqueName: \"kubernetes.io/projected/bce97daa-2099-4347-8369-0e1035532aaa-kube-api-access-hbdlb\") pod \"maas-controller-5c67f77dd8-6bsc2\" (UID: \"bce97daa-2099-4347-8369-0e1035532aaa\") " pod="opendatahub/maas-controller-5c67f77dd8-6bsc2"
Jul 29 18:01:53.584375 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:53.584321    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bce97daa-2099-4347-8369-0e1035532aaa-webhook-cert\") pod \"maas-controller-5c67f77dd8-6bsc2\" (UID: \"bce97daa-2099-4347-8369-0e1035532aaa\") " pod="opendatahub/maas-controller-5c67f77dd8-6bsc2"
Jul 29 18:01:53.590717 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:53.590691    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbdlb\" (UniqueName: \"kubernetes.io/projected/bce97daa-2099-4347-8369-0e1035532aaa-kube-api-access-hbdlb\") pod \"maas-controller-5c67f77dd8-6bsc2\" (UID: \"bce97daa-2099-4347-8369-0e1035532aaa\") " pod="opendatahub/maas-controller-5c67f77dd8-6bsc2"
Jul 29 18:01:53.615100 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:53.615080    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-86dc5fff6f-ww8cr"]
Jul 29 18:01:53.615865 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:01:53.615842    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6d0c4fd_99a5_4cdb_9af2_2f82bbbc539d.slice/crio-f5a9907069c63f7bf9b997268ff5650511b1bcb19685f75bd259c6bf648f8065 WatchSource:0}: Error finding container f5a9907069c63f7bf9b997268ff5650511b1bcb19685f75bd259c6bf648f8065: Status 404 returned error can't find the container with id f5a9907069c63f7bf9b997268ff5650511b1bcb19685f75bd259c6bf648f8065
Jul 29 18:01:53.659525 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:53.659496    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c67f77dd8-6bsc2"
Jul 29 18:01:53.793047 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:53.792985    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5c67f77dd8-6bsc2"]
Jul 29 18:01:53.795290 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:01:53.795258    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbce97daa_2099_4347_8369_0e1035532aaa.slice/crio-12bf8276c04121280a7ed96251f0bf003542ab40c8bc3541d3ef27d772978720 WatchSource:0}: Error finding container 12bf8276c04121280a7ed96251f0bf003542ab40c8bc3541d3ef27d772978720: Status 404 returned error can't find the container with id 12bf8276c04121280a7ed96251f0bf003542ab40c8bc3541d3ef27d772978720
Jul 29 18:01:54.489406 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:54.489299    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-86dc5fff6f-ww8cr" event={"ID":"d6d0c4fd-99a5-4cdb-9af2-2f82bbbc539d","Type":"ContainerStarted","Data":"f5a9907069c63f7bf9b997268ff5650511b1bcb19685f75bd259c6bf648f8065"}
Jul 29 18:01:54.491179 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:54.491147    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c67f77dd8-6bsc2" event={"ID":"bce97daa-2099-4347-8369-0e1035532aaa","Type":"ContainerStarted","Data":"12bf8276c04121280a7ed96251f0bf003542ab40c8bc3541d3ef27d772978720"}
Jul 29 18:01:56.504410 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:56.504307    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-86dc5fff6f-ww8cr" event={"ID":"d6d0c4fd-99a5-4cdb-9af2-2f82bbbc539d","Type":"ContainerStarted","Data":"a5c8c863c173fb381c945d60d8027945a5f1b26f8547f6817f7434ec9dad7cce"}
Jul 29 18:01:56.504865 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:56.504523    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-86dc5fff6f-ww8cr"
Jul 29 18:01:56.505934 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:56.505904    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c67f77dd8-6bsc2" event={"ID":"bce97daa-2099-4347-8369-0e1035532aaa","Type":"ContainerStarted","Data":"b32f1637a09892be10ee663204fa8eae6289e73968d50a2483e1c673454f3656"}
Jul 29 18:01:56.506069 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:56.505949    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5c67f77dd8-6bsc2"
Jul 29 18:01:56.539714 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:56.539642    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-86dc5fff6f-ww8cr" podStartSLOduration=1.075896578 podStartE2EDuration="3.539620273s" totalImagesPullingTime="2.463723695s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:01:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:01:56.538300265 +0000 UTC m=+596.407523073" watchObservedRunningTime="2026-07-29 18:01:56.539620273 +0000 UTC m=+596.408843080"
Jul 29 18:01:56.594273 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:01:56.594217    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5c67f77dd8-6bsc2" podStartSLOduration=1.302787744 podStartE2EDuration="3.59420097s" totalImagesPullingTime="2.291413226s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:01:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:01:56.591202442 +0000 UTC m=+596.460425249" watchObservedRunningTime="2026-07-29 18:01:56.59420097 +0000 UTC m=+596.463423776"
Jul 29 18:02:00.587791 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:00.587765    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gg2wj_2bf0bb71-4047-45a7-bd8e-5d6efabd26bd/console-operator/2.log"
Jul 29 18:02:00.594243 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:00.594216    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gg2wj_2bf0bb71-4047-45a7-bd8e-5d6efabd26bd/console-operator/2.log"
Jul 29 18:02:07.513125 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:07.513089    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5c67f77dd8-6bsc2"
Jul 29 18:02:07.513588 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:07.513172    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-86dc5fff6f-ww8cr"
Jul 29 18:02:07.575557 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:07.575519    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-86dc5fff6f-ww8cr"]
Jul 29 18:02:07.575789 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:07.575742    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-86dc5fff6f-ww8cr" podUID="d6d0c4fd-99a5-4cdb-9af2-2f82bbbc539d" containerName="manager" containerID="cri-o://a5c8c863c173fb381c945d60d8027945a5f1b26f8547f6817f7434ec9dad7cce" gracePeriod=30
Jul 29 18:02:07.825472 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:07.825449    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-86dc5fff6f-ww8cr"
Jul 29 18:02:08.007704 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:08.007662    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d6d0c4fd-99a5-4cdb-9af2-2f82bbbc539d-webhook-cert\") pod \"d6d0c4fd-99a5-4cdb-9af2-2f82bbbc539d\" (UID: \"d6d0c4fd-99a5-4cdb-9af2-2f82bbbc539d\") "
Jul 29 18:02:08.007863 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:08.007745    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dn7wf\" (UniqueName: \"kubernetes.io/projected/d6d0c4fd-99a5-4cdb-9af2-2f82bbbc539d-kube-api-access-dn7wf\") pod \"d6d0c4fd-99a5-4cdb-9af2-2f82bbbc539d\" (UID: \"d6d0c4fd-99a5-4cdb-9af2-2f82bbbc539d\") "
Jul 29 18:02:08.010117 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:08.010084    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6d0c4fd-99a5-4cdb-9af2-2f82bbbc539d-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "d6d0c4fd-99a5-4cdb-9af2-2f82bbbc539d" (UID: "d6d0c4fd-99a5-4cdb-9af2-2f82bbbc539d"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:02:08.010117 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:08.010098    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6d0c4fd-99a5-4cdb-9af2-2f82bbbc539d-kube-api-access-dn7wf" (OuterVolumeSpecName: "kube-api-access-dn7wf") pod "d6d0c4fd-99a5-4cdb-9af2-2f82bbbc539d" (UID: "d6d0c4fd-99a5-4cdb-9af2-2f82bbbc539d"). InnerVolumeSpecName "kube-api-access-dn7wf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:02:08.108672 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:08.108574    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dn7wf\" (UniqueName: \"kubernetes.io/projected/d6d0c4fd-99a5-4cdb-9af2-2f82bbbc539d-kube-api-access-dn7wf\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:02:08.108672 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:08.108618    2557 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d6d0c4fd-99a5-4cdb-9af2-2f82bbbc539d-webhook-cert\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:02:08.572272 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:08.572240    2557 generic.go:356] "Generic (PLEG): container finished" podID="d6d0c4fd-99a5-4cdb-9af2-2f82bbbc539d" containerID="a5c8c863c173fb381c945d60d8027945a5f1b26f8547f6817f7434ec9dad7cce" exitCode=0
Jul 29 18:02:08.572655 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:08.572280    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-86dc5fff6f-ww8cr"
Jul 29 18:02:08.572655 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:08.572315    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-86dc5fff6f-ww8cr" event={"ID":"d6d0c4fd-99a5-4cdb-9af2-2f82bbbc539d","Type":"ContainerDied","Data":"a5c8c863c173fb381c945d60d8027945a5f1b26f8547f6817f7434ec9dad7cce"}
Jul 29 18:02:08.572655 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:08.572353    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-86dc5fff6f-ww8cr" event={"ID":"d6d0c4fd-99a5-4cdb-9af2-2f82bbbc539d","Type":"ContainerDied","Data":"f5a9907069c63f7bf9b997268ff5650511b1bcb19685f75bd259c6bf648f8065"}
Jul 29 18:02:08.572655 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:08.572374    2557 scope.go:117] "RemoveContainer" containerID="a5c8c863c173fb381c945d60d8027945a5f1b26f8547f6817f7434ec9dad7cce"
Jul 29 18:02:08.581883 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:08.581719    2557 scope.go:117] "RemoveContainer" containerID="a5c8c863c173fb381c945d60d8027945a5f1b26f8547f6817f7434ec9dad7cce"
Jul 29 18:02:08.581969 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:02:08.581952    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5c8c863c173fb381c945d60d8027945a5f1b26f8547f6817f7434ec9dad7cce\": container with ID starting with a5c8c863c173fb381c945d60d8027945a5f1b26f8547f6817f7434ec9dad7cce not found: ID does not exist" containerID="a5c8c863c173fb381c945d60d8027945a5f1b26f8547f6817f7434ec9dad7cce"
Jul 29 18:02:08.582016 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:08.581977    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5c8c863c173fb381c945d60d8027945a5f1b26f8547f6817f7434ec9dad7cce"} err="failed to get container status \"a5c8c863c173fb381c945d60d8027945a5f1b26f8547f6817f7434ec9dad7cce\": rpc error: code = NotFound desc = could not find container \"a5c8c863c173fb381c945d60d8027945a5f1b26f8547f6817f7434ec9dad7cce\": container with ID starting with a5c8c863c173fb381c945d60d8027945a5f1b26f8547f6817f7434ec9dad7cce not found: ID does not exist"
Jul 29 18:02:08.594555 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:08.594534    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-86dc5fff6f-ww8cr"]
Jul 29 18:02:08.598070 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:08.598036    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-86dc5fff6f-ww8cr"]
Jul 29 18:02:08.652424 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:08.652396    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6d0c4fd-99a5-4cdb-9af2-2f82bbbc539d" path="/var/lib/kubelet/pods/d6d0c4fd-99a5-4cdb-9af2-2f82bbbc539d/volumes"
Jul 29 18:02:33.000217 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:33.000140    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-85d97bb9f9-r6vwc"]
Jul 29 18:02:33.007231 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:33.000656    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6d0c4fd-99a5-4cdb-9af2-2f82bbbc539d" containerName="manager"
Jul 29 18:02:33.007231 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:33.000671    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6d0c4fd-99a5-4cdb-9af2-2f82bbbc539d" containerName="manager"
Jul 29 18:02:33.007231 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:33.000757    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6d0c4fd-99a5-4cdb-9af2-2f82bbbc539d" containerName="manager"
Jul 29 18:02:33.007740 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:33.007708    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-85d97bb9f9-r6vwc"
Jul 29 18:02:33.010514 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:33.010492    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 29 18:02:33.010610 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:33.010590    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-dc8mg\""
Jul 29 18:02:33.014587 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:33.014569    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-85d97bb9f9-r6vwc"]
Jul 29 18:02:33.089073 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:33.089033    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a852fdcf-7a6d-41ee-ab42-00e93f24e51f-maas-api-tls\") pod \"maas-api-85d97bb9f9-r6vwc\" (UID: \"a852fdcf-7a6d-41ee-ab42-00e93f24e51f\") " pod="odh-ai-gateway-infra/maas-api-85d97bb9f9-r6vwc"
Jul 29 18:02:33.089073 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:33.089075    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a852fdcf-7a6d-41ee-ab42-00e93f24e51f-openshift-service-ca\") pod \"maas-api-85d97bb9f9-r6vwc\" (UID: \"a852fdcf-7a6d-41ee-ab42-00e93f24e51f\") " pod="odh-ai-gateway-infra/maas-api-85d97bb9f9-r6vwc"
Jul 29 18:02:33.089275 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:33.089113    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqftm\" (UniqueName: \"kubernetes.io/projected/a852fdcf-7a6d-41ee-ab42-00e93f24e51f-kube-api-access-mqftm\") pod \"maas-api-85d97bb9f9-r6vwc\" (UID: \"a852fdcf-7a6d-41ee-ab42-00e93f24e51f\") " pod="odh-ai-gateway-infra/maas-api-85d97bb9f9-r6vwc"
Jul 29 18:02:33.190213 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:33.190179    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a852fdcf-7a6d-41ee-ab42-00e93f24e51f-maas-api-tls\") pod \"maas-api-85d97bb9f9-r6vwc\" (UID: \"a852fdcf-7a6d-41ee-ab42-00e93f24e51f\") " pod="odh-ai-gateway-infra/maas-api-85d97bb9f9-r6vwc"
Jul 29 18:02:33.190398 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:33.190222    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a852fdcf-7a6d-41ee-ab42-00e93f24e51f-openshift-service-ca\") pod \"maas-api-85d97bb9f9-r6vwc\" (UID: \"a852fdcf-7a6d-41ee-ab42-00e93f24e51f\") " pod="odh-ai-gateway-infra/maas-api-85d97bb9f9-r6vwc"
Jul 29 18:02:33.190398 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:33.190260    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqftm\" (UniqueName: \"kubernetes.io/projected/a852fdcf-7a6d-41ee-ab42-00e93f24e51f-kube-api-access-mqftm\") pod \"maas-api-85d97bb9f9-r6vwc\" (UID: \"a852fdcf-7a6d-41ee-ab42-00e93f24e51f\") " pod="odh-ai-gateway-infra/maas-api-85d97bb9f9-r6vwc"
Jul 29 18:02:33.190973 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:33.190949    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a852fdcf-7a6d-41ee-ab42-00e93f24e51f-openshift-service-ca\") pod \"maas-api-85d97bb9f9-r6vwc\" (UID: \"a852fdcf-7a6d-41ee-ab42-00e93f24e51f\") " pod="odh-ai-gateway-infra/maas-api-85d97bb9f9-r6vwc"
Jul 29 18:02:33.192661 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:33.192632    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a852fdcf-7a6d-41ee-ab42-00e93f24e51f-maas-api-tls\") pod \"maas-api-85d97bb9f9-r6vwc\" (UID: \"a852fdcf-7a6d-41ee-ab42-00e93f24e51f\") " pod="odh-ai-gateway-infra/maas-api-85d97bb9f9-r6vwc"
Jul 29 18:02:33.198751 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:33.198699    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqftm\" (UniqueName: \"kubernetes.io/projected/a852fdcf-7a6d-41ee-ab42-00e93f24e51f-kube-api-access-mqftm\") pod \"maas-api-85d97bb9f9-r6vwc\" (UID: \"a852fdcf-7a6d-41ee-ab42-00e93f24e51f\") " pod="odh-ai-gateway-infra/maas-api-85d97bb9f9-r6vwc"
Jul 29 18:02:33.319699 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:33.319670    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-85d97bb9f9-r6vwc"
Jul 29 18:02:33.494359 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:33.494326    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-85d97bb9f9-r6vwc"]
Jul 29 18:02:33.494858 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:02:33.494833    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda852fdcf_7a6d_41ee_ab42_00e93f24e51f.slice/crio-a4fa77bf422a514460448f12ff7e6036acf7889060daa345c0f2147aaf21da21 WatchSource:0}: Error finding container a4fa77bf422a514460448f12ff7e6036acf7889060daa345c0f2147aaf21da21: Status 404 returned error can't find the container with id a4fa77bf422a514460448f12ff7e6036acf7889060daa345c0f2147aaf21da21
Jul 29 18:02:33.496164 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:33.496143    2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 18:02:33.623004 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:33.622930    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-tkpf5"]
Jul 29 18:02:33.627985 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:33.627966    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-tkpf5"
Jul 29 18:02:33.631848 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:33.631829    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 29 18:02:33.637286 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:33.637265    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-tkpf5"]
Jul 29 18:02:33.694274 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:33.694238    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9b7h\" (UniqueName: \"kubernetes.io/projected/4be56e67-7714-47b7-b0c5-254141f6b6c5-kube-api-access-v9b7h\") pod \"payload-pre-processing-687dbb6cf6-tkpf5\" (UID: \"4be56e67-7714-47b7-b0c5-254141f6b6c5\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-tkpf5"
Jul 29 18:02:33.694450 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:33.694422    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4be56e67-7714-47b7-b0c5-254141f6b6c5-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-tkpf5\" (UID: \"4be56e67-7714-47b7-b0c5-254141f6b6c5\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-tkpf5"
Jul 29 18:02:33.707287 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:33.707263    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-85d97bb9f9-r6vwc" event={"ID":"a852fdcf-7a6d-41ee-ab42-00e93f24e51f","Type":"ContainerStarted","Data":"a4fa77bf422a514460448f12ff7e6036acf7889060daa345c0f2147aaf21da21"}
Jul 29 18:02:33.765394 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:33.765361    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-p6kkw"]
Jul 29 18:02:33.769063 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:33.769046    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-p6kkw"
Jul 29 18:02:33.772050 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:33.772026    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-4fw8z\""
Jul 29 18:02:33.780024 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:33.779995    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-p6kkw"]
Jul 29 18:02:33.795527 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:33.795496    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4be56e67-7714-47b7-b0c5-254141f6b6c5-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-tkpf5\" (UID: \"4be56e67-7714-47b7-b0c5-254141f6b6c5\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-tkpf5"
Jul 29 18:02:33.795701 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:33.795606    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0d65e55f-0460-4fd4-b46c-6d0bb6724295-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-p6kkw\" (UID: \"0d65e55f-0460-4fd4-b46c-6d0bb6724295\") " pod="openshift-ingress/payload-processing-5cc6cd444b-p6kkw"
Jul 29 18:02:33.795701 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:33.795648    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9b7h\" (UniqueName: \"kubernetes.io/projected/4be56e67-7714-47b7-b0c5-254141f6b6c5-kube-api-access-v9b7h\") pod \"payload-pre-processing-687dbb6cf6-tkpf5\" (UID: \"4be56e67-7714-47b7-b0c5-254141f6b6c5\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-tkpf5"
Jul 29 18:02:33.795701 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:33.795692    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pn64m\" (UniqueName: \"kubernetes.io/projected/0d65e55f-0460-4fd4-b46c-6d0bb6724295-kube-api-access-pn64m\") pod \"payload-processing-5cc6cd444b-p6kkw\" (UID: \"0d65e55f-0460-4fd4-b46c-6d0bb6724295\") " pod="openshift-ingress/payload-processing-5cc6cd444b-p6kkw"
Jul 29 18:02:33.796235 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:33.796216    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4be56e67-7714-47b7-b0c5-254141f6b6c5-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-tkpf5\" (UID: \"4be56e67-7714-47b7-b0c5-254141f6b6c5\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-tkpf5"
Jul 29 18:02:33.807649 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:33.807622    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9b7h\" (UniqueName: \"kubernetes.io/projected/4be56e67-7714-47b7-b0c5-254141f6b6c5-kube-api-access-v9b7h\") pod \"payload-pre-processing-687dbb6cf6-tkpf5\" (UID: \"4be56e67-7714-47b7-b0c5-254141f6b6c5\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-tkpf5"
Jul 29 18:02:33.897093 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:33.897003    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0d65e55f-0460-4fd4-b46c-6d0bb6724295-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-p6kkw\" (UID: \"0d65e55f-0460-4fd4-b46c-6d0bb6724295\") " pod="openshift-ingress/payload-processing-5cc6cd444b-p6kkw"
Jul 29 18:02:33.897093 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:33.897068    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pn64m\" (UniqueName: \"kubernetes.io/projected/0d65e55f-0460-4fd4-b46c-6d0bb6724295-kube-api-access-pn64m\") pod \"payload-processing-5cc6cd444b-p6kkw\" (UID: \"0d65e55f-0460-4fd4-b46c-6d0bb6724295\") " pod="openshift-ingress/payload-processing-5cc6cd444b-p6kkw"
Jul 29 18:02:33.897736 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:33.897698    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0d65e55f-0460-4fd4-b46c-6d0bb6724295-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-p6kkw\" (UID: \"0d65e55f-0460-4fd4-b46c-6d0bb6724295\") " pod="openshift-ingress/payload-processing-5cc6cd444b-p6kkw"
Jul 29 18:02:33.907538 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:33.907510    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pn64m\" (UniqueName: \"kubernetes.io/projected/0d65e55f-0460-4fd4-b46c-6d0bb6724295-kube-api-access-pn64m\") pod \"payload-processing-5cc6cd444b-p6kkw\" (UID: \"0d65e55f-0460-4fd4-b46c-6d0bb6724295\") " pod="openshift-ingress/payload-processing-5cc6cd444b-p6kkw"
Jul 29 18:02:33.941251 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:33.941217    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-tkpf5"
Jul 29 18:02:34.081566 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:34.081525    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-p6kkw"
Jul 29 18:02:34.084418 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:34.084388    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-tkpf5"]
Jul 29 18:02:34.085326 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:02:34.085303    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4be56e67_7714_47b7_b0c5_254141f6b6c5.slice/crio-01c1be33d8717d1617682ebfab35119d2252142a3bed08d9b89c65bf8fa794d5 WatchSource:0}: Error finding container 01c1be33d8717d1617682ebfab35119d2252142a3bed08d9b89c65bf8fa794d5: Status 404 returned error can't find the container with id 01c1be33d8717d1617682ebfab35119d2252142a3bed08d9b89c65bf8fa794d5
Jul 29 18:02:34.256837 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:34.256790    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-p6kkw"]
Jul 29 18:02:34.261066 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:02:34.261026    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d65e55f_0460_4fd4_b46c_6d0bb6724295.slice/crio-e5c4db36e0bf5b4f7dd27e5b1f7fe168434d9bd4522a56cd3cf8fa79c7a9d292 WatchSource:0}: Error finding container e5c4db36e0bf5b4f7dd27e5b1f7fe168434d9bd4522a56cd3cf8fa79c7a9d292: Status 404 returned error can't find the container with id e5c4db36e0bf5b4f7dd27e5b1f7fe168434d9bd4522a56cd3cf8fa79c7a9d292
Jul 29 18:02:34.714084 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:34.714051    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-tkpf5" event={"ID":"4be56e67-7714-47b7-b0c5-254141f6b6c5","Type":"ContainerStarted","Data":"01c1be33d8717d1617682ebfab35119d2252142a3bed08d9b89c65bf8fa794d5"}
Jul 29 18:02:34.715282 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:34.715258    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-p6kkw" event={"ID":"0d65e55f-0460-4fd4-b46c-6d0bb6724295","Type":"ContainerStarted","Data":"e5c4db36e0bf5b4f7dd27e5b1f7fe168434d9bd4522a56cd3cf8fa79c7a9d292"}
Jul 29 18:02:37.740063 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:37.740019    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-tkpf5" event={"ID":"4be56e67-7714-47b7-b0c5-254141f6b6c5","Type":"ContainerStarted","Data":"2c2b9ffdaa04996bfdc55caee084cdf0a082607bc2e6005e9d60567b2c796397"}
Jul 29 18:02:37.740814 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:37.740789    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-tkpf5"
Jul 29 18:02:37.742973 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:37.742939    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-85d97bb9f9-r6vwc" event={"ID":"a852fdcf-7a6d-41ee-ab42-00e93f24e51f","Type":"ContainerStarted","Data":"595047701a8fe13d049881ce1cf94b90ef5e75c6bd21d5c0dcbae7327064ecf5"}
Jul 29 18:02:37.743474 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:37.743440    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-85d97bb9f9-r6vwc"
Jul 29 18:02:37.750930 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:37.750905    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-p6kkw" event={"ID":"0d65e55f-0460-4fd4-b46c-6d0bb6724295","Type":"ContainerStarted","Data":"e2114eba0ba93bda85ec8fa1d92477934a68fc9ad0a7d8be80b8fa63e4419d06"}
Jul 29 18:02:37.751032 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:37.751003    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5cc6cd444b-p6kkw"
Jul 29 18:02:37.760918 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:37.760847    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-tkpf5" podStartSLOduration=1.6508598920000002 podStartE2EDuration="4.760828669s" totalImagesPullingTime="3.109968777s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:02:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:02:37.754951101 +0000 UTC m=+637.624173908" watchObservedRunningTime="2026-07-29 18:02:37.760828669 +0000 UTC m=+637.630051478"
Jul 29 18:02:37.772352 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:37.772307    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5cc6cd444b-p6kkw" podStartSLOduration=1.832770042 podStartE2EDuration="4.772295154s" totalImagesPullingTime="2.939525112s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:02:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:02:37.769549852 +0000 UTC m=+637.638772661" watchObservedRunningTime="2026-07-29 18:02:37.772295154 +0000 UTC m=+637.641517960"
Jul 29 18:02:37.788483 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:37.788438    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-85d97bb9f9-r6vwc" podStartSLOduration=2.087665336 podStartE2EDuration="5.78842568s" totalImagesPullingTime="3.700760344s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:02:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:02:37.785839679 +0000 UTC m=+637.655062490" watchObservedRunningTime="2026-07-29 18:02:37.78842568 +0000 UTC m=+637.657648487"
Jul 29 18:02:41.780710 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:41.780628    2557 generic.go:356] "Generic (PLEG): container finished" podID="5bb4f610-8260-474c-bae4-e5d9ed9a5280" containerID="3c3e20439b758103ea68df71d31094300c410b460397aaff5e2500cd4d072baf" exitCode=0
Jul 29 18:02:41.781148 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:41.780704    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vzxzx" event={"ID":"5bb4f610-8260-474c-bae4-e5d9ed9a5280","Type":"ContainerDied","Data":"3c3e20439b758103ea68df71d31094300c410b460397aaff5e2500cd4d072baf"}
Jul 29 18:02:41.781148 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:41.781068    2557 scope.go:117] "RemoveContainer" containerID="3c3e20439b758103ea68df71d31094300c410b460397aaff5e2500cd4d072baf"
Jul 29 18:02:41.782045 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:41.782015    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:02:41.782149 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:41.782095    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:02:41.782218 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:41.782145    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:02:42.788121 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:42.788088    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vzxzx" event={"ID":"5bb4f610-8260-474c-bae4-e5d9ed9a5280","Type":"ContainerStarted","Data":"43897883a9eb3c06d10bd317d445e4ec818e1f88fc101eab3dde8525da1ede8a"}
Jul 29 18:02:42.868960 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:42.868926    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vzxzx"
Jul 29 18:02:42.869411 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:42.869378    2557 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-vzxzx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Jul 29 18:02:42.869538 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:42.869430    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vzxzx" podUID="5bb4f610-8260-474c-bae4-e5d9ed9a5280" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Jul 29 18:02:43.869961 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:43.869923    2557 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-vzxzx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Jul 29 18:02:43.870351 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:43.869980    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vzxzx" podUID="5bb4f610-8260-474c-bae4-e5d9ed9a5280" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Jul 29 18:02:44.769783 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:44.769753    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-85d97bb9f9-r6vwc"
Jul 29 18:02:44.869239 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:44.869207    2557 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-vzxzx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Jul 29 18:02:44.869400 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:44.869276    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vzxzx" podUID="5bb4f610-8260-474c-bae4-e5d9ed9a5280" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Jul 29 18:02:45.869548 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:45.869508    2557 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-vzxzx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Jul 29 18:02:45.869968 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:45.869581    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vzxzx" podUID="5bb4f610-8260-474c-bae4-e5d9ed9a5280" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Jul 29 18:02:46.873832 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:46.873806    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vzxzx"
Jul 29 18:02:46.874224 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:46.874053    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vzxzx"
Jul 29 18:02:46.874944 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:46.874925    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vzxzx"
Jul 29 18:02:48.760262 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:48.760236    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5cc6cd444b-p6kkw"
Jul 29 18:02:49.766429 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:49.766390    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-tkpf5" podUID="4be56e67-7714-47b7-b0c5-254141f6b6c5" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.132.0.53:9004: connect: connection refused"
Jul 29 18:02:53.942636 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:53.942594    2557 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-tkpf5" podUID="4be56e67-7714-47b7-b0c5-254141f6b6c5" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.132.0.53:9004: connect: connection refused"
Jul 29 18:02:59.765922 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:02:59.765890    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-tkpf5"
Jul 29 18:03:06.615889 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:03:06.615854    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-2b6c6"]
Jul 29 18:03:06.621908 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:03:06.621883    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2b6c6"
Jul 29 18:03:06.624621 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:03:06.624600    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 29 18:03:06.626111 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:03:06.625774    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-w4f67\""
Jul 29 18:03:06.626111 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:03:06.625805    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 29 18:03:06.626111 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:03:06.626071    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 29 18:03:06.630209 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:03:06.630185    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-2b6c6"]
Jul 29 18:03:06.681574 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:03:06.681538    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/335de777-811d-421e-b05d-729871464949-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2b6c6\" (UID: \"335de777-811d-421e-b05d-729871464949\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2b6c6"
Jul 29 18:03:06.681773 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:03:06.681587    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/335de777-811d-421e-b05d-729871464949-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2b6c6\" (UID: \"335de777-811d-421e-b05d-729871464949\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2b6c6"
Jul 29 18:03:06.681773 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:03:06.681620    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4vlh\" (UniqueName: \"kubernetes.io/projected/335de777-811d-421e-b05d-729871464949-kube-api-access-l4vlh\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2b6c6\" (UID: \"335de777-811d-421e-b05d-729871464949\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2b6c6"
Jul 29 18:03:06.681773 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:03:06.681665    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/335de777-811d-421e-b05d-729871464949-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2b6c6\" (UID: \"335de777-811d-421e-b05d-729871464949\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2b6c6"
Jul 29 18:03:06.681773 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:03:06.681685    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/335de777-811d-421e-b05d-729871464949-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2b6c6\" (UID: \"335de777-811d-421e-b05d-729871464949\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2b6c6"
Jul 29 18:03:06.681773 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:03:06.681762    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/335de777-811d-421e-b05d-729871464949-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2b6c6\" (UID: \"335de777-811d-421e-b05d-729871464949\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2b6c6"
Jul 29 18:03:06.782495 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:03:06.782459    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/335de777-811d-421e-b05d-729871464949-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2b6c6\" (UID: \"335de777-811d-421e-b05d-729871464949\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2b6c6"
Jul 29 18:03:06.782676 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:03:06.782504    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4vlh\" (UniqueName: \"kubernetes.io/projected/335de777-811d-421e-b05d-729871464949-kube-api-access-l4vlh\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2b6c6\" (UID: \"335de777-811d-421e-b05d-729871464949\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2b6c6"
Jul 29 18:03:06.782676 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:03:06.782540    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/335de777-811d-421e-b05d-729871464949-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2b6c6\" (UID: \"335de777-811d-421e-b05d-729871464949\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2b6c6"
Jul 29 18:03:06.782676 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:03:06.782561    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/335de777-811d-421e-b05d-729871464949-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2b6c6\" (UID: \"335de777-811d-421e-b05d-729871464949\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2b6c6"
Jul 29 18:03:06.782676 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:03:06.782579    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/335de777-811d-421e-b05d-729871464949-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2b6c6\" (UID: \"335de777-811d-421e-b05d-729871464949\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2b6c6"
Jul 29 18:03:06.782676 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:03:06.782604    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/335de777-811d-421e-b05d-729871464949-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2b6c6\" (UID: \"335de777-811d-421e-b05d-729871464949\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2b6c6"
Jul 29 18:03:06.782973 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:03:06.782933    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/335de777-811d-421e-b05d-729871464949-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2b6c6\" (UID: \"335de777-811d-421e-b05d-729871464949\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2b6c6"
Jul 29 18:03:06.782973 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:03:06.782948    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/335de777-811d-421e-b05d-729871464949-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2b6c6\" (UID: \"335de777-811d-421e-b05d-729871464949\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2b6c6"
Jul 29 18:03:06.783076 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:03:06.783047    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/335de777-811d-421e-b05d-729871464949-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2b6c6\" (UID: \"335de777-811d-421e-b05d-729871464949\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2b6c6"
Jul 29 18:03:06.784958 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:03:06.784938    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/335de777-811d-421e-b05d-729871464949-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2b6c6\" (UID: \"335de777-811d-421e-b05d-729871464949\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2b6c6"
Jul 29 18:03:06.785346 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:03:06.785325    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/335de777-811d-421e-b05d-729871464949-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2b6c6\" (UID: \"335de777-811d-421e-b05d-729871464949\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2b6c6"
Jul 29 18:03:06.791295 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:03:06.791276    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4vlh\" (UniqueName: \"kubernetes.io/projected/335de777-811d-421e-b05d-729871464949-kube-api-access-l4vlh\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2b6c6\" (UID: \"335de777-811d-421e-b05d-729871464949\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2b6c6"
Jul 29 18:03:06.935226 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:03:06.935131    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2b6c6"
Jul 29 18:03:07.076590 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:03:07.076564    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-2b6c6"]
Jul 29 18:03:07.078233 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:03:07.078206    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod335de777_811d_421e_b05d_729871464949.slice/crio-e31460759c2995c71d302a5a35644a8f371d9836e8ca641f056156b402035931 WatchSource:0}: Error finding container e31460759c2995c71d302a5a35644a8f371d9836e8ca641f056156b402035931: Status 404 returned error can't find the container with id e31460759c2995c71d302a5a35644a8f371d9836e8ca641f056156b402035931
Jul 29 18:03:07.925686 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:03:07.925641    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2b6c6" event={"ID":"335de777-811d-421e-b05d-729871464949","Type":"ContainerStarted","Data":"e31460759c2995c71d302a5a35644a8f371d9836e8ca641f056156b402035931"}
Jul 29 18:03:08.932404 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:03:08.932367    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2b6c6" event={"ID":"335de777-811d-421e-b05d-729871464949","Type":"ContainerStarted","Data":"ac4cf7b1585afa9703c6064c2133c3671022b6a5ae3d0afb4a4de7745f8d18b6"}
Jul 29 18:03:08.951458 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:03:08.951408    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2b6c6" podStartSLOduration=1.8630176330000001 podStartE2EDuration="2.951391767s" totalImagesPullingTime="1.088374134s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:03:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:03:08.949045381 +0000 UTC m=+668.818268203" watchObservedRunningTime="2026-07-29 18:03:08.951391767 +0000 UTC m=+668.820614574"
Jul 29 18:03:16.936180 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:03:16.936141    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2b6c6"
Jul 29 18:03:16.936642 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:03:16.936202    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2b6c6"
Jul 29 18:03:16.948556 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:03:16.948529    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2b6c6"
Jul 29 18:03:16.987169 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:03:16.987141    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2b6c6"
Jul 29 18:05:58.200308 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:05:58.200273    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5c67f77dd8-6bsc2"]
Jul 29 18:05:58.200787 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:05:58.200458    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5c67f77dd8-6bsc2" podUID="bce97daa-2099-4347-8369-0e1035532aaa" containerName="manager" containerID="cri-o://b32f1637a09892be10ee663204fa8eae6289e73968d50a2483e1c673454f3656" gracePeriod=30
Jul 29 18:05:58.462437 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:05:58.462376    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c67f77dd8-6bsc2"
Jul 29 18:05:58.521385 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:05:58.521355    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bce97daa-2099-4347-8369-0e1035532aaa-webhook-cert\") pod \"bce97daa-2099-4347-8369-0e1035532aaa\" (UID: \"bce97daa-2099-4347-8369-0e1035532aaa\") "
Jul 29 18:05:58.521542 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:05:58.521394    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hbdlb\" (UniqueName: \"kubernetes.io/projected/bce97daa-2099-4347-8369-0e1035532aaa-kube-api-access-hbdlb\") pod \"bce97daa-2099-4347-8369-0e1035532aaa\" (UID: \"bce97daa-2099-4347-8369-0e1035532aaa\") "
Jul 29 18:05:58.523616 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:05:58.523585    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bce97daa-2099-4347-8369-0e1035532aaa-kube-api-access-hbdlb" (OuterVolumeSpecName: "kube-api-access-hbdlb") pod "bce97daa-2099-4347-8369-0e1035532aaa" (UID: "bce97daa-2099-4347-8369-0e1035532aaa"). InnerVolumeSpecName "kube-api-access-hbdlb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:05:58.523616 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:05:58.523606    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bce97daa-2099-4347-8369-0e1035532aaa-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "bce97daa-2099-4347-8369-0e1035532aaa" (UID: "bce97daa-2099-4347-8369-0e1035532aaa"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:05:58.622662 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:05:58.622629    2557 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bce97daa-2099-4347-8369-0e1035532aaa-webhook-cert\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:05:58.622662 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:05:58.622658    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hbdlb\" (UniqueName: \"kubernetes.io/projected/bce97daa-2099-4347-8369-0e1035532aaa-kube-api-access-hbdlb\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:05:58.854509 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:05:58.854474    2557 generic.go:356] "Generic (PLEG): container finished" podID="bce97daa-2099-4347-8369-0e1035532aaa" containerID="b32f1637a09892be10ee663204fa8eae6289e73968d50a2483e1c673454f3656" exitCode=0
Jul 29 18:05:58.854679 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:05:58.854519    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c67f77dd8-6bsc2"
Jul 29 18:05:58.854679 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:05:58.854557    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c67f77dd8-6bsc2" event={"ID":"bce97daa-2099-4347-8369-0e1035532aaa","Type":"ContainerDied","Data":"b32f1637a09892be10ee663204fa8eae6289e73968d50a2483e1c673454f3656"}
Jul 29 18:05:58.854679 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:05:58.854611    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c67f77dd8-6bsc2" event={"ID":"bce97daa-2099-4347-8369-0e1035532aaa","Type":"ContainerDied","Data":"12bf8276c04121280a7ed96251f0bf003542ab40c8bc3541d3ef27d772978720"}
Jul 29 18:05:58.854679 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:05:58.854639    2557 scope.go:117] "RemoveContainer" containerID="b32f1637a09892be10ee663204fa8eae6289e73968d50a2483e1c673454f3656"
Jul 29 18:05:58.863485 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:05:58.863469    2557 scope.go:117] "RemoveContainer" containerID="b32f1637a09892be10ee663204fa8eae6289e73968d50a2483e1c673454f3656"
Jul 29 18:05:58.863770 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:05:58.863748    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b32f1637a09892be10ee663204fa8eae6289e73968d50a2483e1c673454f3656\": container with ID starting with b32f1637a09892be10ee663204fa8eae6289e73968d50a2483e1c673454f3656 not found: ID does not exist" containerID="b32f1637a09892be10ee663204fa8eae6289e73968d50a2483e1c673454f3656"
Jul 29 18:05:58.863828 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:05:58.863778    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b32f1637a09892be10ee663204fa8eae6289e73968d50a2483e1c673454f3656"} err="failed to get container status \"b32f1637a09892be10ee663204fa8eae6289e73968d50a2483e1c673454f3656\": rpc error: code = NotFound desc = could not find container \"b32f1637a09892be10ee663204fa8eae6289e73968d50a2483e1c673454f3656\": container with ID starting with b32f1637a09892be10ee663204fa8eae6289e73968d50a2483e1c673454f3656 not found: ID does not exist"
Jul 29 18:05:58.871321 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:05:58.871302    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5c67f77dd8-6bsc2"]
Jul 29 18:05:58.875125 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:05:58.875106    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5c67f77dd8-6bsc2"]
Jul 29 18:06:00.653467 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:06:00.653433    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bce97daa-2099-4347-8369-0e1035532aaa" path="/var/lib/kubelet/pods/bce97daa-2099-4347-8369-0e1035532aaa/volumes"
Jul 29 18:06:59.654979 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:06:59.654946    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5c67f77dd8-chsdh"]
Jul 29 18:06:59.655382 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:06:59.655331    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bce97daa-2099-4347-8369-0e1035532aaa" containerName="manager"
Jul 29 18:06:59.655382 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:06:59.655342    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="bce97daa-2099-4347-8369-0e1035532aaa" containerName="manager"
Jul 29 18:06:59.655463 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:06:59.655420    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="bce97daa-2099-4347-8369-0e1035532aaa" containerName="manager"
Jul 29 18:06:59.658409 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:06:59.658392    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c67f77dd8-chsdh"
Jul 29 18:06:59.660931 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:06:59.660910    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 29 18:06:59.661924 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:06:59.661904    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-6j7nb\""
Jul 29 18:06:59.661924 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:06:59.661917    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 29 18:06:59.666869 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:06:59.666849    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5c67f77dd8-chsdh"]
Jul 29 18:06:59.669320 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:06:59.669044    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9a48fe9c-3581-4cd1-9350-acc1e699a0f4-webhook-cert\") pod \"maas-controller-5c67f77dd8-chsdh\" (UID: \"9a48fe9c-3581-4cd1-9350-acc1e699a0f4\") " pod="opendatahub/maas-controller-5c67f77dd8-chsdh"
Jul 29 18:06:59.669320 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:06:59.669087    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvkbj\" (UniqueName: \"kubernetes.io/projected/9a48fe9c-3581-4cd1-9350-acc1e699a0f4-kube-api-access-hvkbj\") pod \"maas-controller-5c67f77dd8-chsdh\" (UID: \"9a48fe9c-3581-4cd1-9350-acc1e699a0f4\") " pod="opendatahub/maas-controller-5c67f77dd8-chsdh"
Jul 29 18:06:59.769744 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:06:59.769705    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9a48fe9c-3581-4cd1-9350-acc1e699a0f4-webhook-cert\") pod \"maas-controller-5c67f77dd8-chsdh\" (UID: \"9a48fe9c-3581-4cd1-9350-acc1e699a0f4\") " pod="opendatahub/maas-controller-5c67f77dd8-chsdh"
Jul 29 18:06:59.769933 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:06:59.769760    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvkbj\" (UniqueName: \"kubernetes.io/projected/9a48fe9c-3581-4cd1-9350-acc1e699a0f4-kube-api-access-hvkbj\") pod \"maas-controller-5c67f77dd8-chsdh\" (UID: \"9a48fe9c-3581-4cd1-9350-acc1e699a0f4\") " pod="opendatahub/maas-controller-5c67f77dd8-chsdh"
Jul 29 18:06:59.772283 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:06:59.772260    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9a48fe9c-3581-4cd1-9350-acc1e699a0f4-webhook-cert\") pod \"maas-controller-5c67f77dd8-chsdh\" (UID: \"9a48fe9c-3581-4cd1-9350-acc1e699a0f4\") " pod="opendatahub/maas-controller-5c67f77dd8-chsdh"
Jul 29 18:06:59.777829 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:06:59.777803    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvkbj\" (UniqueName: \"kubernetes.io/projected/9a48fe9c-3581-4cd1-9350-acc1e699a0f4-kube-api-access-hvkbj\") pod \"maas-controller-5c67f77dd8-chsdh\" (UID: \"9a48fe9c-3581-4cd1-9350-acc1e699a0f4\") " pod="opendatahub/maas-controller-5c67f77dd8-chsdh"
Jul 29 18:06:59.970287 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:06:59.970196    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c67f77dd8-chsdh"
Jul 29 18:07:00.110496 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:07:00.110467    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5c67f77dd8-chsdh"]
Jul 29 18:07:00.111469 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:07:00.111425    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a48fe9c_3581_4cd1_9350_acc1e699a0f4.slice/crio-c357b70065e0b3c79b894ac3aa6da4eb8fabfbebfd49118b335887096cf27131 WatchSource:0}: Error finding container c357b70065e0b3c79b894ac3aa6da4eb8fabfbebfd49118b335887096cf27131: Status 404 returned error can't find the container with id c357b70065e0b3c79b894ac3aa6da4eb8fabfbebfd49118b335887096cf27131
Jul 29 18:07:00.191573 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:07:00.191535    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c67f77dd8-chsdh" event={"ID":"9a48fe9c-3581-4cd1-9350-acc1e699a0f4","Type":"ContainerStarted","Data":"c357b70065e0b3c79b894ac3aa6da4eb8fabfbebfd49118b335887096cf27131"}
Jul 29 18:07:00.577384 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:07:00.577361    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 29 18:07:00.629678 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:07:00.629654    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gg2wj_2bf0bb71-4047-45a7-bd8e-5d6efabd26bd/console-operator/2.log"
Jul 29 18:07:00.632399 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:07:00.632371    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gg2wj_2bf0bb71-4047-45a7-bd8e-5d6efabd26bd/console-operator/2.log"
Jul 29 18:07:01.198256 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:07:01.198218    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c67f77dd8-chsdh" event={"ID":"9a48fe9c-3581-4cd1-9350-acc1e699a0f4","Type":"ContainerStarted","Data":"6ce1347b65316b2c5ffe56a8a63756895f3b4320b4f2ea4dae5f7d6da18697e1"}
Jul 29 18:07:01.198675 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:07:01.198338    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5c67f77dd8-chsdh"
Jul 29 18:07:01.217011 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:07:01.216959    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5c67f77dd8-chsdh" podStartSLOduration=1.7550209049999999 podStartE2EDuration="2.2169457s" totalImagesPullingTime="461.924795ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:06:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:07:01.215696236 +0000 UTC m=+901.084919045" watchObservedRunningTime="2026-07-29 18:07:01.2169457 +0000 UTC m=+901.086168518"
Jul 29 18:07:12.205869 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:07:12.205835    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5c67f77dd8-chsdh"
Jul 29 18:07:16.825416 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:07:16.825378    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5c67f77dd8-chsdh"]
Jul 29 18:07:16.825964 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:07:16.825619    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5c67f77dd8-chsdh" podUID="9a48fe9c-3581-4cd1-9350-acc1e699a0f4" containerName="manager" containerID="cri-o://6ce1347b65316b2c5ffe56a8a63756895f3b4320b4f2ea4dae5f7d6da18697e1" gracePeriod=30
Jul 29 18:07:17.078088 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:07:17.078027    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c67f77dd8-chsdh"
Jul 29 18:07:17.128409 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:07:17.128382    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9a48fe9c-3581-4cd1-9350-acc1e699a0f4-webhook-cert\") pod \"9a48fe9c-3581-4cd1-9350-acc1e699a0f4\" (UID: \"9a48fe9c-3581-4cd1-9350-acc1e699a0f4\") "
Jul 29 18:07:17.128565 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:07:17.128509    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hvkbj\" (UniqueName: \"kubernetes.io/projected/9a48fe9c-3581-4cd1-9350-acc1e699a0f4-kube-api-access-hvkbj\") pod \"9a48fe9c-3581-4cd1-9350-acc1e699a0f4\" (UID: \"9a48fe9c-3581-4cd1-9350-acc1e699a0f4\") "
Jul 29 18:07:17.130613 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:07:17.130575    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a48fe9c-3581-4cd1-9350-acc1e699a0f4-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "9a48fe9c-3581-4cd1-9350-acc1e699a0f4" (UID: "9a48fe9c-3581-4cd1-9350-acc1e699a0f4"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:07:17.130769 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:07:17.130666    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a48fe9c-3581-4cd1-9350-acc1e699a0f4-kube-api-access-hvkbj" (OuterVolumeSpecName: "kube-api-access-hvkbj") pod "9a48fe9c-3581-4cd1-9350-acc1e699a0f4" (UID: "9a48fe9c-3581-4cd1-9350-acc1e699a0f4"). InnerVolumeSpecName "kube-api-access-hvkbj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:07:17.230031 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:07:17.229997    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hvkbj\" (UniqueName: \"kubernetes.io/projected/9a48fe9c-3581-4cd1-9350-acc1e699a0f4-kube-api-access-hvkbj\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:07:17.230031 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:07:17.230023    2557 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9a48fe9c-3581-4cd1-9350-acc1e699a0f4-webhook-cert\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:07:17.289508 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:07:17.289471    2557 generic.go:356] "Generic (PLEG): container finished" podID="9a48fe9c-3581-4cd1-9350-acc1e699a0f4" containerID="6ce1347b65316b2c5ffe56a8a63756895f3b4320b4f2ea4dae5f7d6da18697e1" exitCode=0
Jul 29 18:07:17.289643 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:07:17.289516    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c67f77dd8-chsdh"
Jul 29 18:07:17.289643 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:07:17.289561    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c67f77dd8-chsdh" event={"ID":"9a48fe9c-3581-4cd1-9350-acc1e699a0f4","Type":"ContainerDied","Data":"6ce1347b65316b2c5ffe56a8a63756895f3b4320b4f2ea4dae5f7d6da18697e1"}
Jul 29 18:07:17.289643 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:07:17.289612    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c67f77dd8-chsdh" event={"ID":"9a48fe9c-3581-4cd1-9350-acc1e699a0f4","Type":"ContainerDied","Data":"c357b70065e0b3c79b894ac3aa6da4eb8fabfbebfd49118b335887096cf27131"}
Jul 29 18:07:17.289643 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:07:17.289634    2557 scope.go:117] "RemoveContainer" containerID="6ce1347b65316b2c5ffe56a8a63756895f3b4320b4f2ea4dae5f7d6da18697e1"
Jul 29 18:07:17.301686 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:07:17.301669    2557 scope.go:117] "RemoveContainer" containerID="6ce1347b65316b2c5ffe56a8a63756895f3b4320b4f2ea4dae5f7d6da18697e1"
Jul 29 18:07:17.302403 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:07:17.302373    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ce1347b65316b2c5ffe56a8a63756895f3b4320b4f2ea4dae5f7d6da18697e1\": container with ID starting with 6ce1347b65316b2c5ffe56a8a63756895f3b4320b4f2ea4dae5f7d6da18697e1 not found: ID does not exist" containerID="6ce1347b65316b2c5ffe56a8a63756895f3b4320b4f2ea4dae5f7d6da18697e1"
Jul 29 18:07:17.302504 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:07:17.302413    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ce1347b65316b2c5ffe56a8a63756895f3b4320b4f2ea4dae5f7d6da18697e1"} err="failed to get container status \"6ce1347b65316b2c5ffe56a8a63756895f3b4320b4f2ea4dae5f7d6da18697e1\": rpc error: code = NotFound desc = could not find container \"6ce1347b65316b2c5ffe56a8a63756895f3b4320b4f2ea4dae5f7d6da18697e1\": container with ID starting with 6ce1347b65316b2c5ffe56a8a63756895f3b4320b4f2ea4dae5f7d6da18697e1 not found: ID does not exist"
Jul 29 18:07:17.310926 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:07:17.310907    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5c67f77dd8-chsdh"]
Jul 29 18:07:17.314062 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:07:17.314042    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5c67f77dd8-chsdh"]
Jul 29 18:07:18.076311 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:07:18.076279    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5c67f77dd8-9kj8h"]
Jul 29 18:07:18.076782 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:07:18.076713    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a48fe9c-3581-4cd1-9350-acc1e699a0f4" containerName="manager"
Jul 29 18:07:18.076782 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:07:18.076743    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a48fe9c-3581-4cd1-9350-acc1e699a0f4" containerName="manager"
Jul 29 18:07:18.076920 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:07:18.076843    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a48fe9c-3581-4cd1-9350-acc1e699a0f4" containerName="manager"
Jul 29 18:07:18.081102 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:07:18.081086    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c67f77dd8-9kj8h"
Jul 29 18:07:18.083814 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:07:18.083792    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 29 18:07:18.084934 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:07:18.084918    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 29 18:07:18.084986 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:07:18.084934    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-6j7nb\""
Jul 29 18:07:18.089780 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:07:18.089760    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5c67f77dd8-9kj8h"]
Jul 29 18:07:18.138991 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:07:18.138964    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xt5cx\" (UniqueName: \"kubernetes.io/projected/10654f46-4178-4840-81e6-52d53aacb1ec-kube-api-access-xt5cx\") pod \"maas-controller-5c67f77dd8-9kj8h\" (UID: \"10654f46-4178-4840-81e6-52d53aacb1ec\") " pod="opendatahub/maas-controller-5c67f77dd8-9kj8h"
Jul 29 18:07:18.139148 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:07:18.139015    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/10654f46-4178-4840-81e6-52d53aacb1ec-webhook-cert\") pod \"maas-controller-5c67f77dd8-9kj8h\" (UID: \"10654f46-4178-4840-81e6-52d53aacb1ec\") " pod="opendatahub/maas-controller-5c67f77dd8-9kj8h"
Jul 29 18:07:18.239863 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:07:18.239831    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xt5cx\" (UniqueName: \"kubernetes.io/projected/10654f46-4178-4840-81e6-52d53aacb1ec-kube-api-access-xt5cx\") pod \"maas-controller-5c67f77dd8-9kj8h\" (UID: \"10654f46-4178-4840-81e6-52d53aacb1ec\") " pod="opendatahub/maas-controller-5c67f77dd8-9kj8h"
Jul 29 18:07:18.240044 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:07:18.239884    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/10654f46-4178-4840-81e6-52d53aacb1ec-webhook-cert\") pod \"maas-controller-5c67f77dd8-9kj8h\" (UID: \"10654f46-4178-4840-81e6-52d53aacb1ec\") " pod="opendatahub/maas-controller-5c67f77dd8-9kj8h"
Jul 29 18:07:18.242514 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:07:18.242489    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/10654f46-4178-4840-81e6-52d53aacb1ec-webhook-cert\") pod \"maas-controller-5c67f77dd8-9kj8h\" (UID: \"10654f46-4178-4840-81e6-52d53aacb1ec\") " pod="opendatahub/maas-controller-5c67f77dd8-9kj8h"
Jul 29 18:07:18.247602 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:07:18.247580    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xt5cx\" (UniqueName: \"kubernetes.io/projected/10654f46-4178-4840-81e6-52d53aacb1ec-kube-api-access-xt5cx\") pod \"maas-controller-5c67f77dd8-9kj8h\" (UID: \"10654f46-4178-4840-81e6-52d53aacb1ec\") " pod="opendatahub/maas-controller-5c67f77dd8-9kj8h"
Jul 29 18:07:18.392917 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:07:18.392833    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c67f77dd8-9kj8h"
Jul 29 18:07:18.539972 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:07:18.539941    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5c67f77dd8-9kj8h"]
Jul 29 18:07:18.540489 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:07:18.540453    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10654f46_4178_4840_81e6_52d53aacb1ec.slice/crio-a264ca355f2a81a17c0d8391d64c1c1a55e9b5c142d3beeed1730c46728c6804 WatchSource:0}: Error finding container a264ca355f2a81a17c0d8391d64c1c1a55e9b5c142d3beeed1730c46728c6804: Status 404 returned error can't find the container with id a264ca355f2a81a17c0d8391d64c1c1a55e9b5c142d3beeed1730c46728c6804
Jul 29 18:07:18.652799 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:07:18.652701    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a48fe9c-3581-4cd1-9350-acc1e699a0f4" path="/var/lib/kubelet/pods/9a48fe9c-3581-4cd1-9350-acc1e699a0f4/volumes"
Jul 29 18:07:19.303461 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:07:19.303428    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c67f77dd8-9kj8h" event={"ID":"10654f46-4178-4840-81e6-52d53aacb1ec","Type":"ContainerStarted","Data":"5accc5850baf6fa153723842acbd38a3bb027162ffee2dda5642cd0b73f5f8fb"}
Jul 29 18:07:19.303915 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:07:19.303586    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5c67f77dd8-9kj8h"
Jul 29 18:07:19.303915 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:07:19.303630    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c67f77dd8-9kj8h" event={"ID":"10654f46-4178-4840-81e6-52d53aacb1ec","Type":"ContainerStarted","Data":"a264ca355f2a81a17c0d8391d64c1c1a55e9b5c142d3beeed1730c46728c6804"}
Jul 29 18:07:19.320559 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:07:19.320511    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5c67f77dd8-9kj8h" podStartSLOduration=0.839848981 podStartE2EDuration="1.320496344s" totalImagesPullingTime="480.647363ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:07:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:07:19.31940018 +0000 UTC m=+919.188623011" watchObservedRunningTime="2026-07-29 18:07:19.320496344 +0000 UTC m=+919.189719150"
Jul 29 18:07:30.311193 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:07:30.311159    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5c67f77dd8-9kj8h"
Jul 29 18:12:00.673322 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:12:00.673293    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gg2wj_2bf0bb71-4047-45a7-bd8e-5d6efabd26bd/console-operator/2.log"
Jul 29 18:12:00.674023 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:12:00.674005    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gg2wj_2bf0bb71-4047-45a7-bd8e-5d6efabd26bd/console-operator/2.log"
Jul 29 18:14:18.529927 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:14:18.529892    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-45kjh"]
Jul 29 18:14:18.532257 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:14:18.530097    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-45kjh" podUID="d0899957-7e30-4059-abf6-00245ce7f130" containerName="manager" containerID="cri-o://11089e31f476c2cfe9094b383871c50de328128db8fda0af8950c165e6572aa5" gracePeriod=10
Jul 29 18:14:18.788133 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:14:18.788110    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-45kjh"
Jul 29 18:14:18.803489 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:14:18.803464    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d0899957-7e30-4059-abf6-00245ce7f130-extensions-socket-volume\") pod \"d0899957-7e30-4059-abf6-00245ce7f130\" (UID: \"d0899957-7e30-4059-abf6-00245ce7f130\") "
Jul 29 18:14:18.803613 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:14:18.803523    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g265l\" (UniqueName: \"kubernetes.io/projected/d0899957-7e30-4059-abf6-00245ce7f130-kube-api-access-g265l\") pod \"d0899957-7e30-4059-abf6-00245ce7f130\" (UID: \"d0899957-7e30-4059-abf6-00245ce7f130\") "
Jul 29 18:14:18.803820 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:14:18.803800    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0899957-7e30-4059-abf6-00245ce7f130-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d0899957-7e30-4059-abf6-00245ce7f130" (UID: "d0899957-7e30-4059-abf6-00245ce7f130"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:14:18.805828 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:14:18.805784    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0899957-7e30-4059-abf6-00245ce7f130-kube-api-access-g265l" (OuterVolumeSpecName: "kube-api-access-g265l") pod "d0899957-7e30-4059-abf6-00245ce7f130" (UID: "d0899957-7e30-4059-abf6-00245ce7f130"). InnerVolumeSpecName "kube-api-access-g265l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:14:18.904895 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:14:18.904866    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g265l\" (UniqueName: \"kubernetes.io/projected/d0899957-7e30-4059-abf6-00245ce7f130-kube-api-access-g265l\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:14:18.904895 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:14:18.904895    2557 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d0899957-7e30-4059-abf6-00245ce7f130-extensions-socket-volume\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:14:19.636226 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:14:19.636194    2557 generic.go:356] "Generic (PLEG): container finished" podID="d0899957-7e30-4059-abf6-00245ce7f130" containerID="11089e31f476c2cfe9094b383871c50de328128db8fda0af8950c165e6572aa5" exitCode=0
Jul 29 18:14:19.636699 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:14:19.636237    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-45kjh" event={"ID":"d0899957-7e30-4059-abf6-00245ce7f130","Type":"ContainerDied","Data":"11089e31f476c2cfe9094b383871c50de328128db8fda0af8950c165e6572aa5"}
Jul 29 18:14:19.636699 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:14:19.636240    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-45kjh"
Jul 29 18:14:19.636699 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:14:19.636280    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-45kjh" event={"ID":"d0899957-7e30-4059-abf6-00245ce7f130","Type":"ContainerDied","Data":"d2fd77496dbea9beb92c06c65f6ae064d9855d4ecd185630853bd3da6d22e9ac"}
Jul 29 18:14:19.636699 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:14:19.636302    2557 scope.go:117] "RemoveContainer" containerID="11089e31f476c2cfe9094b383871c50de328128db8fda0af8950c165e6572aa5"
Jul 29 18:14:19.647212 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:14:19.647194    2557 scope.go:117] "RemoveContainer" containerID="11089e31f476c2cfe9094b383871c50de328128db8fda0af8950c165e6572aa5"
Jul 29 18:14:19.647466 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:14:19.647444    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11089e31f476c2cfe9094b383871c50de328128db8fda0af8950c165e6572aa5\": container with ID starting with 11089e31f476c2cfe9094b383871c50de328128db8fda0af8950c165e6572aa5 not found: ID does not exist" containerID="11089e31f476c2cfe9094b383871c50de328128db8fda0af8950c165e6572aa5"
Jul 29 18:14:19.647538 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:14:19.647475    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11089e31f476c2cfe9094b383871c50de328128db8fda0af8950c165e6572aa5"} err="failed to get container status \"11089e31f476c2cfe9094b383871c50de328128db8fda0af8950c165e6572aa5\": rpc error: code = NotFound desc = could not find container \"11089e31f476c2cfe9094b383871c50de328128db8fda0af8950c165e6572aa5\": container with ID starting with 11089e31f476c2cfe9094b383871c50de328128db8fda0af8950c165e6572aa5 not found: ID does not exist"
Jul 29 18:14:19.659207 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:14:19.659180    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-45kjh"]
Jul 29 18:14:19.664139 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:14:19.664116    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-45kjh"]
Jul 29 18:14:20.652376 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:14:20.652341    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0899957-7e30-4059-abf6-00245ce7f130" path="/var/lib/kubelet/pods/d0899957-7e30-4059-abf6-00245ce7f130/volumes"
Jul 29 18:15:00.145865 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:15:00.145824    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755815-584jp"]
Jul 29 18:15:00.146462 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:15:00.146397    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0899957-7e30-4059-abf6-00245ce7f130" containerName="manager"
Jul 29 18:15:00.146462 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:15:00.146418    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0899957-7e30-4059-abf6-00245ce7f130" containerName="manager"
Jul 29 18:15:00.146588 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:15:00.146529    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0899957-7e30-4059-abf6-00245ce7f130" containerName="manager"
Jul 29 18:15:00.149582 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:15:00.149561    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755815-584jp"
Jul 29 18:15:00.152199 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:15:00.152179    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qrbhl\""
Jul 29 18:15:00.157282 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:15:00.157256    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755815-584jp"]
Jul 29 18:15:00.161703 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:15:00.161681    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755815-584jp"
Jul 29 18:15:00.293125 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:15:00.293096    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755815-584jp"]
Jul 29 18:15:00.294768 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:15:00.294744    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89eea13b_8c96_44cb_8269_0d60628bd39c.slice/crio-49565f7588ebbac224fd3b2dc7301660adab7d7fd2af23b59c4edbcf9ef5738c WatchSource:0}: Error finding container 49565f7588ebbac224fd3b2dc7301660adab7d7fd2af23b59c4edbcf9ef5738c: Status 404 returned error can't find the container with id 49565f7588ebbac224fd3b2dc7301660adab7d7fd2af23b59c4edbcf9ef5738c
Jul 29 18:15:00.297201 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:15:00.297183    2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 18:15:00.861298 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:15:00.861263    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755815-584jp" event={"ID":"89eea13b-8c96-44cb-8269-0d60628bd39c","Type":"ContainerStarted","Data":"49565f7588ebbac224fd3b2dc7301660adab7d7fd2af23b59c4edbcf9ef5738c"}
Jul 29 18:15:10.919530 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:15:10.919482    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755815-584jp" event={"ID":"89eea13b-8c96-44cb-8269-0d60628bd39c","Type":"ContainerStarted","Data":"558c151349384852c9dd1cd3cf7a966d7d944e1a1472a8ba0ed745758578a1b1"}
Jul 29 18:15:10.937170 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:15:10.937111    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755815-584jp" podStartSLOduration=1.101367747 podStartE2EDuration="10.937092802s" totalImagesPullingTime="9.835725055s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:15:10.933530764 +0000 UTC m=+1390.802753570" watchObservedRunningTime="2026-07-29 18:15:10.937092802 +0000 UTC m=+1390.806315611"
Jul 29 18:15:11.926221 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:15:11.926183    2557 generic.go:356] "Generic (PLEG): container finished" podID="89eea13b-8c96-44cb-8269-0d60628bd39c" containerID="558c151349384852c9dd1cd3cf7a966d7d944e1a1472a8ba0ed745758578a1b1" exitCode=0
Jul 29 18:15:11.926609 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:15:11.926250    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755815-584jp" event={"ID":"89eea13b-8c96-44cb-8269-0d60628bd39c","Type":"ContainerDied","Data":"558c151349384852c9dd1cd3cf7a966d7d944e1a1472a8ba0ed745758578a1b1"}
Jul 29 18:15:13.071082 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:15:13.071060    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755815-584jp"
Jul 29 18:15:13.938692 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:15:13.938664    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755815-584jp"
Jul 29 18:15:13.938692 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:15:13.938672    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755815-584jp" event={"ID":"89eea13b-8c96-44cb-8269-0d60628bd39c","Type":"ContainerDied","Data":"49565f7588ebbac224fd3b2dc7301660adab7d7fd2af23b59c4edbcf9ef5738c"}
Jul 29 18:15:13.938899 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:15:13.938716    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="49565f7588ebbac224fd3b2dc7301660adab7d7fd2af23b59c4edbcf9ef5738c"
Jul 29 18:15:19.190863 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:15:19.190830    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4p58n"]
Jul 29 18:15:19.191239 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:15:19.191218    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89eea13b-8c96-44cb-8269-0d60628bd39c" containerName="cleanup"
Jul 29 18:15:19.191239 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:15:19.191228    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="89eea13b-8c96-44cb-8269-0d60628bd39c" containerName="cleanup"
Jul 29 18:15:19.191313 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:15:19.191290    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="89eea13b-8c96-44cb-8269-0d60628bd39c" containerName="cleanup"
Jul 29 18:15:19.194341 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:15:19.194324    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4p58n"
Jul 29 18:15:19.196965 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:15:19.196949    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-nnb89\""
Jul 29 18:15:19.206422 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:15:19.206401    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4p58n"]
Jul 29 18:15:19.225040 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:15:19.225011    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6c0ee61e-c00b-4e2c-964d-9460da93537a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-4p58n\" (UID: \"6c0ee61e-c00b-4e2c-964d-9460da93537a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4p58n"
Jul 29 18:15:19.225135 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:15:19.225063    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lv6w\" (UniqueName: \"kubernetes.io/projected/6c0ee61e-c00b-4e2c-964d-9460da93537a-kube-api-access-6lv6w\") pod \"kuadrant-operator-controller-manager-86878c769f-4p58n\" (UID: \"6c0ee61e-c00b-4e2c-964d-9460da93537a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4p58n"
Jul 29 18:15:19.325928 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:15:19.325904    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lv6w\" (UniqueName: \"kubernetes.io/projected/6c0ee61e-c00b-4e2c-964d-9460da93537a-kube-api-access-6lv6w\") pod \"kuadrant-operator-controller-manager-86878c769f-4p58n\" (UID: \"6c0ee61e-c00b-4e2c-964d-9460da93537a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4p58n"
Jul 29 18:15:19.326032 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:15:19.325988    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6c0ee61e-c00b-4e2c-964d-9460da93537a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-4p58n\" (UID: \"6c0ee61e-c00b-4e2c-964d-9460da93537a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4p58n"
Jul 29 18:15:19.326303 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:15:19.326287    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6c0ee61e-c00b-4e2c-964d-9460da93537a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-4p58n\" (UID: \"6c0ee61e-c00b-4e2c-964d-9460da93537a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4p58n"
Jul 29 18:15:19.334207 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:15:19.334181    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lv6w\" (UniqueName: \"kubernetes.io/projected/6c0ee61e-c00b-4e2c-964d-9460da93537a-kube-api-access-6lv6w\") pod \"kuadrant-operator-controller-manager-86878c769f-4p58n\" (UID: \"6c0ee61e-c00b-4e2c-964d-9460da93537a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4p58n"
Jul 29 18:15:19.505863 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:15:19.505798    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4p58n"
Jul 29 18:15:19.646415 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:15:19.646389    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4p58n"]
Jul 29 18:15:19.647690 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:15:19.647665    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c0ee61e_c00b_4e2c_964d_9460da93537a.slice/crio-95ebe873b913e7b93069a74d416d60417bf4a4a00e605cbbb7c8f3ec7bfd9f48 WatchSource:0}: Error finding container 95ebe873b913e7b93069a74d416d60417bf4a4a00e605cbbb7c8f3ec7bfd9f48: Status 404 returned error can't find the container with id 95ebe873b913e7b93069a74d416d60417bf4a4a00e605cbbb7c8f3ec7bfd9f48
Jul 29 18:15:19.973106 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:15:19.973066    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4p58n" event={"ID":"6c0ee61e-c00b-4e2c-964d-9460da93537a","Type":"ContainerStarted","Data":"b19711cd09b1f8bce7d37b84cf3d2926a1b4ec84d62327e2f14d38a68ca8d61f"}
Jul 29 18:15:19.973276 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:15:19.973126    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4p58n"
Jul 29 18:15:19.973276 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:15:19.973142    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4p58n" event={"ID":"6c0ee61e-c00b-4e2c-964d-9460da93537a","Type":"ContainerStarted","Data":"95ebe873b913e7b93069a74d416d60417bf4a4a00e605cbbb7c8f3ec7bfd9f48"}
Jul 29 18:15:19.996101 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:15:19.996054    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4p58n" podStartSLOduration=0.996038923 podStartE2EDuration="996.038923ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:15:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:15:19.993131956 +0000 UTC m=+1399.862354759" watchObservedRunningTime="2026-07-29 18:15:19.996038923 +0000 UTC m=+1399.865261730"
Jul 29 18:15:30.982058 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:15:30.982028    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4p58n"
Jul 29 18:17:00.716667 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:17:00.716643    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gg2wj_2bf0bb71-4047-45a7-bd8e-5d6efabd26bd/console-operator/2.log"
Jul 29 18:17:00.717839 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:17:00.717819    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gg2wj_2bf0bb71-4047-45a7-bd8e-5d6efabd26bd/console-operator/2.log"
Jul 29 18:22:00.755364 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:22:00.755332    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gg2wj_2bf0bb71-4047-45a7-bd8e-5d6efabd26bd/console-operator/2.log"
Jul 29 18:22:00.757826 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:22:00.757795    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gg2wj_2bf0bb71-4047-45a7-bd8e-5d6efabd26bd/console-operator/2.log"
Jul 29 18:24:35.227505 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:35.227475    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6"]
Jul 29 18:24:35.231036 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:35.231018    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6"
Jul 29 18:24:35.235281 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:35.235254    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-0174a53b-openshift-default-dockercfg-9jjkj\""
Jul 29 18:24:35.284214 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:35.284186    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6"]
Jul 29 18:24:35.363835 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:35.363803    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0407fda-71a1-435e-8904-b80c41e2a466-workload-certs\") pod \"e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6\" (UID: \"c0407fda-71a1-435e-8904-b80c41e2a466\") " pod="openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6"
Jul 29 18:24:35.363988 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:35.363842    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0407fda-71a1-435e-8904-b80c41e2a466-istio-data\") pod \"e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6\" (UID: \"c0407fda-71a1-435e-8904-b80c41e2a466\") " pod="openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6"
Jul 29 18:24:35.363988 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:35.363858    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0407fda-71a1-435e-8904-b80c41e2a466-istiod-ca-cert\") pod \"e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6\" (UID: \"c0407fda-71a1-435e-8904-b80c41e2a466\") " pod="openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6"
Jul 29 18:24:35.364079 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:35.363991    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0407fda-71a1-435e-8904-b80c41e2a466-istio-token\") pod \"e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6\" (UID: \"c0407fda-71a1-435e-8904-b80c41e2a466\") " pod="openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6"
Jul 29 18:24:35.364079 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:35.364021    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0407fda-71a1-435e-8904-b80c41e2a466-workload-socket\") pod \"e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6\" (UID: \"c0407fda-71a1-435e-8904-b80c41e2a466\") " pod="openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6"
Jul 29 18:24:35.364079 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:35.364061    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0407fda-71a1-435e-8904-b80c41e2a466-credential-socket\") pod \"e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6\" (UID: \"c0407fda-71a1-435e-8904-b80c41e2a466\") " pod="openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6"
Jul 29 18:24:35.364193 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:35.364081    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzvq7\" (UniqueName: \"kubernetes.io/projected/c0407fda-71a1-435e-8904-b80c41e2a466-kube-api-access-vzvq7\") pod \"e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6\" (UID: \"c0407fda-71a1-435e-8904-b80c41e2a466\") " pod="openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6"
Jul 29 18:24:35.364193 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:35.364107    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0407fda-71a1-435e-8904-b80c41e2a466-istio-envoy\") pod \"e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6\" (UID: \"c0407fda-71a1-435e-8904-b80c41e2a466\") " pod="openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6"
Jul 29 18:24:35.364193 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:35.364157    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0407fda-71a1-435e-8904-b80c41e2a466-istio-podinfo\") pod \"e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6\" (UID: \"c0407fda-71a1-435e-8904-b80c41e2a466\") " pod="openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6"
Jul 29 18:24:35.465195 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:35.465165    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0407fda-71a1-435e-8904-b80c41e2a466-workload-certs\") pod \"e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6\" (UID: \"c0407fda-71a1-435e-8904-b80c41e2a466\") " pod="openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6"
Jul 29 18:24:35.465195 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:35.465203    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0407fda-71a1-435e-8904-b80c41e2a466-istio-data\") pod \"e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6\" (UID: \"c0407fda-71a1-435e-8904-b80c41e2a466\") " pod="openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6"
Jul 29 18:24:35.465425 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:35.465222    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0407fda-71a1-435e-8904-b80c41e2a466-istiod-ca-cert\") pod \"e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6\" (UID: \"c0407fda-71a1-435e-8904-b80c41e2a466\") " pod="openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6"
Jul 29 18:24:35.465425 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:35.465280    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0407fda-71a1-435e-8904-b80c41e2a466-istio-token\") pod \"e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6\" (UID: \"c0407fda-71a1-435e-8904-b80c41e2a466\") " pod="openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6"
Jul 29 18:24:35.465425 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:35.465306    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0407fda-71a1-435e-8904-b80c41e2a466-workload-socket\") pod \"e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6\" (UID: \"c0407fda-71a1-435e-8904-b80c41e2a466\") " pod="openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6"
Jul 29 18:24:35.465425 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:35.465347    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0407fda-71a1-435e-8904-b80c41e2a466-credential-socket\") pod \"e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6\" (UID: \"c0407fda-71a1-435e-8904-b80c41e2a466\") " pod="openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6"
Jul 29 18:24:35.465425 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:35.465373    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzvq7\" (UniqueName: \"kubernetes.io/projected/c0407fda-71a1-435e-8904-b80c41e2a466-kube-api-access-vzvq7\") pod \"e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6\" (UID: \"c0407fda-71a1-435e-8904-b80c41e2a466\") " pod="openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6"
Jul 29 18:24:35.465425 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:35.465409    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0407fda-71a1-435e-8904-b80c41e2a466-istio-envoy\") pod \"e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6\" (UID: \"c0407fda-71a1-435e-8904-b80c41e2a466\") " pod="openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6"
Jul 29 18:24:35.465770 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:35.465442    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0407fda-71a1-435e-8904-b80c41e2a466-istio-podinfo\") pod \"e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6\" (UID: \"c0407fda-71a1-435e-8904-b80c41e2a466\") " pod="openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6"
Jul 29 18:24:35.465770 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:35.465601    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0407fda-71a1-435e-8904-b80c41e2a466-workload-certs\") pod \"e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6\" (UID: \"c0407fda-71a1-435e-8904-b80c41e2a466\") " pod="openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6"
Jul 29 18:24:35.465770 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:35.465717    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0407fda-71a1-435e-8904-b80c41e2a466-workload-socket\") pod \"e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6\" (UID: \"c0407fda-71a1-435e-8904-b80c41e2a466\") " pod="openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6"
Jul 29 18:24:35.465945 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:35.465871    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0407fda-71a1-435e-8904-b80c41e2a466-istio-data\") pod \"e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6\" (UID: \"c0407fda-71a1-435e-8904-b80c41e2a466\") " pod="openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6"
Jul 29 18:24:35.465945 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:35.465922    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0407fda-71a1-435e-8904-b80c41e2a466-credential-socket\") pod \"e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6\" (UID: \"c0407fda-71a1-435e-8904-b80c41e2a466\") " pod="openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6"
Jul 29 18:24:35.466209 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:35.466192    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0407fda-71a1-435e-8904-b80c41e2a466-istiod-ca-cert\") pod \"e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6\" (UID: \"c0407fda-71a1-435e-8904-b80c41e2a466\") " pod="openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6"
Jul 29 18:24:35.467618 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:35.467599    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0407fda-71a1-435e-8904-b80c41e2a466-istio-envoy\") pod \"e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6\" (UID: \"c0407fda-71a1-435e-8904-b80c41e2a466\") " pod="openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6"
Jul 29 18:24:35.467942 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:35.467925    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0407fda-71a1-435e-8904-b80c41e2a466-istio-podinfo\") pod \"e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6\" (UID: \"c0407fda-71a1-435e-8904-b80c41e2a466\") " pod="openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6"
Jul 29 18:24:35.474004 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:35.473972    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzvq7\" (UniqueName: \"kubernetes.io/projected/c0407fda-71a1-435e-8904-b80c41e2a466-kube-api-access-vzvq7\") pod \"e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6\" (UID: \"c0407fda-71a1-435e-8904-b80c41e2a466\") " pod="openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6"
Jul 29 18:24:35.474420 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:35.474376    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0407fda-71a1-435e-8904-b80c41e2a466-istio-token\") pod \"e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6\" (UID: \"c0407fda-71a1-435e-8904-b80c41e2a466\") " pod="openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6"
Jul 29 18:24:35.541623 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:35.541596    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6"
Jul 29 18:24:35.682597 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:35.682557    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6"]
Jul 29 18:24:35.682896 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:24:35.682874    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0407fda_71a1_435e_8904_b80c41e2a466.slice/crio-a93f5b9b2df4d8890d88f498de0694404a7f6cd230ba9e68745285e7e2bd1087 WatchSource:0}: Error finding container a93f5b9b2df4d8890d88f498de0694404a7f6cd230ba9e68745285e7e2bd1087: Status 404 returned error can't find the container with id a93f5b9b2df4d8890d88f498de0694404a7f6cd230ba9e68745285e7e2bd1087
Jul 29 18:24:35.685022 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:35.684995    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:24:35.685130 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:35.685055    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:24:35.685130 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:35.685090    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:24:36.039597 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:36.039556    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6" event={"ID":"c0407fda-71a1-435e-8904-b80c41e2a466","Type":"ContainerStarted","Data":"b8d6912aca6f0324802276ae30c3c9229f0537646b8e6de8441e239ad1c91abb"}
Jul 29 18:24:36.039815 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:36.039614    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6" event={"ID":"c0407fda-71a1-435e-8904-b80c41e2a466","Type":"ContainerStarted","Data":"a93f5b9b2df4d8890d88f498de0694404a7f6cd230ba9e68745285e7e2bd1087"}
Jul 29 18:24:36.063352 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:36.063287    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6" podStartSLOduration=1.063268088 podStartE2EDuration="1.063268088s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:24:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:24:36.062067696 +0000 UTC m=+1955.931290517" watchObservedRunningTime="2026-07-29 18:24:36.063268088 +0000 UTC m=+1955.932490895"
Jul 29 18:24:36.542369 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:36.542333    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6"
Jul 29 18:24:36.547470 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:36.547445    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6"
Jul 29 18:24:37.046424 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:37.046392    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6"
Jul 29 18:24:37.047311 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:37.047293    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6"
Jul 29 18:24:37.613056 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:37.613022    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0174a53b-646d79b5cf-n44gw"]
Jul 29 18:24:37.620788 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:37.620764    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0174a53b-646d79b5cf-n44gw"]
Jul 29 18:24:37.620911 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:37.620850    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0174a53b-646d79b5cf-n44gw"
Jul 29 18:24:37.623530 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:37.623508    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-0174a53b-serving-cert\""
Jul 29 18:24:37.793137 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:37.793100    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8c21693-4de2-43e5-a3e5-640d68a2d1f1-openshift-service-ca\") pod \"maas-api-e2e-ait-0174a53b-646d79b5cf-n44gw\" (UID: \"f8c21693-4de2-43e5-a3e5-640d68a2d1f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0174a53b-646d79b5cf-n44gw"
Jul 29 18:24:37.793338 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:37.793147    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8c21693-4de2-43e5-a3e5-640d68a2d1f1-maas-api-tls\") pod \"maas-api-e2e-ait-0174a53b-646d79b5cf-n44gw\" (UID: \"f8c21693-4de2-43e5-a3e5-640d68a2d1f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0174a53b-646d79b5cf-n44gw"
Jul 29 18:24:37.793338 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:37.793284    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsnps\" (UniqueName: \"kubernetes.io/projected/f8c21693-4de2-43e5-a3e5-640d68a2d1f1-kube-api-access-hsnps\") pod \"maas-api-e2e-ait-0174a53b-646d79b5cf-n44gw\" (UID: \"f8c21693-4de2-43e5-a3e5-640d68a2d1f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0174a53b-646d79b5cf-n44gw"
Jul 29 18:24:37.894190 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:37.894109    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsnps\" (UniqueName: \"kubernetes.io/projected/f8c21693-4de2-43e5-a3e5-640d68a2d1f1-kube-api-access-hsnps\") pod \"maas-api-e2e-ait-0174a53b-646d79b5cf-n44gw\" (UID: \"f8c21693-4de2-43e5-a3e5-640d68a2d1f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0174a53b-646d79b5cf-n44gw"
Jul 29 18:24:37.894338 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:37.894196    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8c21693-4de2-43e5-a3e5-640d68a2d1f1-openshift-service-ca\") pod \"maas-api-e2e-ait-0174a53b-646d79b5cf-n44gw\" (UID: \"f8c21693-4de2-43e5-a3e5-640d68a2d1f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0174a53b-646d79b5cf-n44gw"
Jul 29 18:24:37.894338 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:37.894220    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8c21693-4de2-43e5-a3e5-640d68a2d1f1-maas-api-tls\") pod \"maas-api-e2e-ait-0174a53b-646d79b5cf-n44gw\" (UID: \"f8c21693-4de2-43e5-a3e5-640d68a2d1f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0174a53b-646d79b5cf-n44gw"
Jul 29 18:24:37.894338 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:24:37.894318    2557 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-0174a53b-serving-cert: secret "maas-api-e2e-ait-0174a53b-serving-cert" not found
Jul 29 18:24:37.894469 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:24:37.894391    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8c21693-4de2-43e5-a3e5-640d68a2d1f1-maas-api-tls podName:f8c21693-4de2-43e5-a3e5-640d68a2d1f1 nodeName:}" failed. No retries permitted until 2026-07-29 18:24:38.394373746 +0000 UTC m=+1958.263596532 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f8c21693-4de2-43e5-a3e5-640d68a2d1f1-maas-api-tls") pod "maas-api-e2e-ait-0174a53b-646d79b5cf-n44gw" (UID: "f8c21693-4de2-43e5-a3e5-640d68a2d1f1") : secret "maas-api-e2e-ait-0174a53b-serving-cert" not found
Jul 29 18:24:37.894880 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:37.894862    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8c21693-4de2-43e5-a3e5-640d68a2d1f1-openshift-service-ca\") pod \"maas-api-e2e-ait-0174a53b-646d79b5cf-n44gw\" (UID: \"f8c21693-4de2-43e5-a3e5-640d68a2d1f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0174a53b-646d79b5cf-n44gw"
Jul 29 18:24:37.903694 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:37.903673    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsnps\" (UniqueName: \"kubernetes.io/projected/f8c21693-4de2-43e5-a3e5-640d68a2d1f1-kube-api-access-hsnps\") pod \"maas-api-e2e-ait-0174a53b-646d79b5cf-n44gw\" (UID: \"f8c21693-4de2-43e5-a3e5-640d68a2d1f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0174a53b-646d79b5cf-n44gw"
Jul 29 18:24:38.399238 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:38.399201    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8c21693-4de2-43e5-a3e5-640d68a2d1f1-maas-api-tls\") pod \"maas-api-e2e-ait-0174a53b-646d79b5cf-n44gw\" (UID: \"f8c21693-4de2-43e5-a3e5-640d68a2d1f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0174a53b-646d79b5cf-n44gw"
Jul 29 18:24:38.401955 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:38.401916    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8c21693-4de2-43e5-a3e5-640d68a2d1f1-maas-api-tls\") pod \"maas-api-e2e-ait-0174a53b-646d79b5cf-n44gw\" (UID: \"f8c21693-4de2-43e5-a3e5-640d68a2d1f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0174a53b-646d79b5cf-n44gw"
Jul 29 18:24:38.532134 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:38.532104    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0174a53b-646d79b5cf-n44gw"
Jul 29 18:24:38.670992 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:38.670962    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0174a53b-646d79b5cf-n44gw"]
Jul 29 18:24:38.672670 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:24:38.672642    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8c21693_4de2_43e5_a3e5_640d68a2d1f1.slice/crio-910032c16da66cf21c998a372219ceb5f1579e716582be4515de89399a7b3427 WatchSource:0}: Error finding container 910032c16da66cf21c998a372219ceb5f1579e716582be4515de89399a7b3427: Status 404 returned error can't find the container with id 910032c16da66cf21c998a372219ceb5f1579e716582be4515de89399a7b3427
Jul 29 18:24:38.674134 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:38.674114    2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 18:24:39.059632 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:39.059594    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0174a53b-646d79b5cf-n44gw" event={"ID":"f8c21693-4de2-43e5-a3e5-640d68a2d1f1","Type":"ContainerStarted","Data":"910032c16da66cf21c998a372219ceb5f1579e716582be4515de89399a7b3427"}
Jul 29 18:24:40.066949 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:40.066914    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0174a53b-646d79b5cf-n44gw" event={"ID":"f8c21693-4de2-43e5-a3e5-640d68a2d1f1","Type":"ContainerStarted","Data":"b62cb030b593fd25d65b61c87340dbe68e1a8b2d9694690d2fa56a4b12f6e1e5"}
Jul 29 18:24:40.067405 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:40.066982    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0174a53b-646d79b5cf-n44gw"
Jul 29 18:24:40.085178 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:40.085136    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0174a53b-646d79b5cf-n44gw" podStartSLOduration=2.58220013 podStartE2EDuration="3.085120386s" totalImagesPullingTime="502.920256ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:24:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:24:40.083200986 +0000 UTC m=+1959.952423820" watchObservedRunningTime="2026-07-29 18:24:40.085120386 +0000 UTC m=+1959.954343190"
Jul 29 18:24:46.078189 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:46.078158    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0174a53b-646d79b5cf-n44gw"
Jul 29 18:24:52.841423 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:52.841330    2557 patch_prober.go:28] interesting pod/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": read tcp 10.132.0.2:59414->10.132.0.60:15021: read: connection reset by peer" start-of-body=
Jul 29 18:24:52.842126 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:52.841420    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6" podUID="c0407fda-71a1-435e-8904-b80c41e2a466" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": read tcp 10.132.0.2:59414->10.132.0.60:15021: read: connection reset by peer"
Jul 29 18:24:53.144071 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:53.143984    2557 generic.go:356] "Generic (PLEG): container finished" podID="c0407fda-71a1-435e-8904-b80c41e2a466" containerID="b8d6912aca6f0324802276ae30c3c9229f0537646b8e6de8441e239ad1c91abb" exitCode=0
Jul 29 18:24:53.144218 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:53.144069    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6" event={"ID":"c0407fda-71a1-435e-8904-b80c41e2a466","Type":"ContainerDied","Data":"b8d6912aca6f0324802276ae30c3c9229f0537646b8e6de8441e239ad1c91abb"}
Jul 29 18:24:53.144460 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:53.144446    2557 scope.go:117] "RemoveContainer" containerID="b8d6912aca6f0324802276ae30c3c9229f0537646b8e6de8441e239ad1c91abb"
Jul 29 18:24:53.145513 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:53.145489    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:24:53.145574 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:53.145556    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:24:53.145610 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:53.145585    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:24:54.159088 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:54.159041    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6" event={"ID":"c0407fda-71a1-435e-8904-b80c41e2a466","Type":"ContainerStarted","Data":"ebe2acce81a7cca2b74b83d49a14962a312c9305e2c883f17b8b3f20141942ea"}
Jul 29 18:24:54.542805 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:54.542771    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6"
Jul 29 18:24:54.543206 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:54.543113    2557 patch_prober.go:28] interesting pod/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Jul 29 18:24:54.543391 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:54.543369    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6" podUID="c0407fda-71a1-435e-8904-b80c41e2a466" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Jul 29 18:24:55.542994 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:55.542549    2557 patch_prober.go:28] interesting pod/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Jul 29 18:24:55.542994 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:55.542609    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6" podUID="c0407fda-71a1-435e-8904-b80c41e2a466" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Jul 29 18:24:56.543162 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:56.543109    2557 patch_prober.go:28] interesting pod/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Jul 29 18:24:56.543679 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:56.543184    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6" podUID="c0407fda-71a1-435e-8904-b80c41e2a466" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Jul 29 18:24:57.543148 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:57.543105    2557 patch_prober.go:28] interesting pod/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Jul 29 18:24:57.543367 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:57.543186    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6" podUID="c0407fda-71a1-435e-8904-b80c41e2a466" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Jul 29 18:24:58.545541 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:58.545509    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6"
Jul 29 18:24:58.546045 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:58.545909    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6"
Jul 29 18:24:58.546498 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:24:58.546482    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6"
Jul 29 18:25:18.389403 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:18.389365    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0174a53b-646d79b5cf-n44gw"]
Jul 29 18:25:18.390022 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:18.389631    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0174a53b-646d79b5cf-n44gw" podUID="f8c21693-4de2-43e5-a3e5-640d68a2d1f1" containerName="maas-api" containerID="cri-o://b62cb030b593fd25d65b61c87340dbe68e1a8b2d9694690d2fa56a4b12f6e1e5" gracePeriod=30
Jul 29 18:25:18.652017 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:18.651948    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0174a53b-646d79b5cf-n44gw"
Jul 29 18:25:18.730769 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:18.730738    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6"]
Jul 29 18:25:18.730968 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:18.730945    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6" podUID="c0407fda-71a1-435e-8904-b80c41e2a466" containerName="istio-proxy" containerID="cri-o://ebe2acce81a7cca2b74b83d49a14962a312c9305e2c883f17b8b3f20141942ea" gracePeriod=30
Jul 29 18:25:18.775863 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:18.775831    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8c21693-4de2-43e5-a3e5-640d68a2d1f1-maas-api-tls\") pod \"f8c21693-4de2-43e5-a3e5-640d68a2d1f1\" (UID: \"f8c21693-4de2-43e5-a3e5-640d68a2d1f1\") "
Jul 29 18:25:18.776023 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:18.775984    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8c21693-4de2-43e5-a3e5-640d68a2d1f1-openshift-service-ca\") pod \"f8c21693-4de2-43e5-a3e5-640d68a2d1f1\" (UID: \"f8c21693-4de2-43e5-a3e5-640d68a2d1f1\") "
Jul 29 18:25:18.776072 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:18.776031    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hsnps\" (UniqueName: \"kubernetes.io/projected/f8c21693-4de2-43e5-a3e5-640d68a2d1f1-kube-api-access-hsnps\") pod \"f8c21693-4de2-43e5-a3e5-640d68a2d1f1\" (UID: \"f8c21693-4de2-43e5-a3e5-640d68a2d1f1\") "
Jul 29 18:25:18.776415 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:18.776385    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8c21693-4de2-43e5-a3e5-640d68a2d1f1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f8c21693-4de2-43e5-a3e5-640d68a2d1f1" (UID: "f8c21693-4de2-43e5-a3e5-640d68a2d1f1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:25:18.778937 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:18.778906    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8c21693-4de2-43e5-a3e5-640d68a2d1f1-kube-api-access-hsnps" (OuterVolumeSpecName: "kube-api-access-hsnps") pod "f8c21693-4de2-43e5-a3e5-640d68a2d1f1" (UID: "f8c21693-4de2-43e5-a3e5-640d68a2d1f1"). InnerVolumeSpecName "kube-api-access-hsnps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:25:18.779292 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:18.779003    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8c21693-4de2-43e5-a3e5-640d68a2d1f1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f8c21693-4de2-43e5-a3e5-640d68a2d1f1" (UID: "f8c21693-4de2-43e5-a3e5-640d68a2d1f1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:25:18.877596 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:18.877556    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8c21693-4de2-43e5-a3e5-640d68a2d1f1-openshift-service-ca\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:25:18.877596 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:18.877590    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hsnps\" (UniqueName: \"kubernetes.io/projected/f8c21693-4de2-43e5-a3e5-640d68a2d1f1-kube-api-access-hsnps\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:25:18.877596 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:18.877602    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8c21693-4de2-43e5-a3e5-640d68a2d1f1-maas-api-tls\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:25:19.298158 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:19.298119    2557 generic.go:356] "Generic (PLEG): container finished" podID="f8c21693-4de2-43e5-a3e5-640d68a2d1f1" containerID="b62cb030b593fd25d65b61c87340dbe68e1a8b2d9694690d2fa56a4b12f6e1e5" exitCode=0
Jul 29 18:25:19.298331 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:19.298165    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0174a53b-646d79b5cf-n44gw"
Jul 29 18:25:19.298331 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:19.298196    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0174a53b-646d79b5cf-n44gw" event={"ID":"f8c21693-4de2-43e5-a3e5-640d68a2d1f1","Type":"ContainerDied","Data":"b62cb030b593fd25d65b61c87340dbe68e1a8b2d9694690d2fa56a4b12f6e1e5"}
Jul 29 18:25:19.298331 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:19.298257    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0174a53b-646d79b5cf-n44gw" event={"ID":"f8c21693-4de2-43e5-a3e5-640d68a2d1f1","Type":"ContainerDied","Data":"910032c16da66cf21c998a372219ceb5f1579e716582be4515de89399a7b3427"}
Jul 29 18:25:19.298331 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:19.298281    2557 scope.go:117] "RemoveContainer" containerID="b62cb030b593fd25d65b61c87340dbe68e1a8b2d9694690d2fa56a4b12f6e1e5"
Jul 29 18:25:19.308021 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:19.308005    2557 scope.go:117] "RemoveContainer" containerID="b62cb030b593fd25d65b61c87340dbe68e1a8b2d9694690d2fa56a4b12f6e1e5"
Jul 29 18:25:19.308284 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:25:19.308264    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b62cb030b593fd25d65b61c87340dbe68e1a8b2d9694690d2fa56a4b12f6e1e5\": container with ID starting with b62cb030b593fd25d65b61c87340dbe68e1a8b2d9694690d2fa56a4b12f6e1e5 not found: ID does not exist" containerID="b62cb030b593fd25d65b61c87340dbe68e1a8b2d9694690d2fa56a4b12f6e1e5"
Jul 29 18:25:19.308352 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:19.308292    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b62cb030b593fd25d65b61c87340dbe68e1a8b2d9694690d2fa56a4b12f6e1e5"} err="failed to get container status \"b62cb030b593fd25d65b61c87340dbe68e1a8b2d9694690d2fa56a4b12f6e1e5\": rpc error: code = NotFound desc = could not find container \"b62cb030b593fd25d65b61c87340dbe68e1a8b2d9694690d2fa56a4b12f6e1e5\": container with ID starting with b62cb030b593fd25d65b61c87340dbe68e1a8b2d9694690d2fa56a4b12f6e1e5 not found: ID does not exist"
Jul 29 18:25:19.321712 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:19.321680    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0174a53b-646d79b5cf-n44gw"]
Jul 29 18:25:19.324081 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:19.324061    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0174a53b-646d79b5cf-n44gw"]
Jul 29 18:25:19.744154 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:19.744077    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm"]
Jul 29 18:25:19.744529 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:19.744515    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8c21693-4de2-43e5-a3e5-640d68a2d1f1" containerName="maas-api"
Jul 29 18:25:19.744579 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:19.744532    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8c21693-4de2-43e5-a3e5-640d68a2d1f1" containerName="maas-api"
Jul 29 18:25:19.744620 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:19.744612    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8c21693-4de2-43e5-a3e5-640d68a2d1f1" containerName="maas-api"
Jul 29 18:25:19.748946 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:19.748928    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm"
Jul 29 18:25:19.751534 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:19.751508    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-fa2af080-gw-openshift-default-dockercfg-tkvsc\""
Jul 29 18:25:19.759964 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:19.759941    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm"]
Jul 29 18:25:19.887228 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:19.887194    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/693f4250-a184-4e82-85fa-7463cf5b74ae-istio-podinfo\") pod \"e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm\" (UID: \"693f4250-a184-4e82-85fa-7463cf5b74ae\") " pod="openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm"
Jul 29 18:25:19.887228 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:19.887233    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/693f4250-a184-4e82-85fa-7463cf5b74ae-istiod-ca-cert\") pod \"e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm\" (UID: \"693f4250-a184-4e82-85fa-7463cf5b74ae\") " pod="openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm"
Jul 29 18:25:19.887446 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:19.887252    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6ftc\" (UniqueName: \"kubernetes.io/projected/693f4250-a184-4e82-85fa-7463cf5b74ae-kube-api-access-x6ftc\") pod \"e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm\" (UID: \"693f4250-a184-4e82-85fa-7463cf5b74ae\") " pod="openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm"
Jul 29 18:25:19.887446 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:19.887277    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/693f4250-a184-4e82-85fa-7463cf5b74ae-istio-data\") pod \"e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm\" (UID: \"693f4250-a184-4e82-85fa-7463cf5b74ae\") " pod="openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm"
Jul 29 18:25:19.887446 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:19.887330    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/693f4250-a184-4e82-85fa-7463cf5b74ae-credential-socket\") pod \"e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm\" (UID: \"693f4250-a184-4e82-85fa-7463cf5b74ae\") " pod="openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm"
Jul 29 18:25:19.887446 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:19.887397    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/693f4250-a184-4e82-85fa-7463cf5b74ae-istio-token\") pod \"e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm\" (UID: \"693f4250-a184-4e82-85fa-7463cf5b74ae\") " pod="openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm"
Jul 29 18:25:19.887592 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:19.887453    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/693f4250-a184-4e82-85fa-7463cf5b74ae-istio-envoy\") pod \"e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm\" (UID: \"693f4250-a184-4e82-85fa-7463cf5b74ae\") " pod="openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm"
Jul 29 18:25:19.887592 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:19.887521    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/693f4250-a184-4e82-85fa-7463cf5b74ae-workload-socket\") pod \"e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm\" (UID: \"693f4250-a184-4e82-85fa-7463cf5b74ae\") " pod="openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm"
Jul 29 18:25:19.887592 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:19.887571    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/693f4250-a184-4e82-85fa-7463cf5b74ae-workload-certs\") pod \"e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm\" (UID: \"693f4250-a184-4e82-85fa-7463cf5b74ae\") " pod="openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm"
Jul 29 18:25:19.989062 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:19.989032    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/693f4250-a184-4e82-85fa-7463cf5b74ae-istio-podinfo\") pod \"e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm\" (UID: \"693f4250-a184-4e82-85fa-7463cf5b74ae\") " pod="openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm"
Jul 29 18:25:19.989062 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:19.989067    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/693f4250-a184-4e82-85fa-7463cf5b74ae-istiod-ca-cert\") pod \"e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm\" (UID: \"693f4250-a184-4e82-85fa-7463cf5b74ae\") " pod="openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm"
Jul 29 18:25:19.989300 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:19.989085    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6ftc\" (UniqueName: \"kubernetes.io/projected/693f4250-a184-4e82-85fa-7463cf5b74ae-kube-api-access-x6ftc\") pod \"e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm\" (UID: \"693f4250-a184-4e82-85fa-7463cf5b74ae\") " pod="openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm"
Jul 29 18:25:19.989300 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:19.989113    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/693f4250-a184-4e82-85fa-7463cf5b74ae-istio-data\") pod \"e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm\" (UID: \"693f4250-a184-4e82-85fa-7463cf5b74ae\") " pod="openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm"
Jul 29 18:25:19.989300 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:19.989149    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/693f4250-a184-4e82-85fa-7463cf5b74ae-credential-socket\") pod \"e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm\" (UID: \"693f4250-a184-4e82-85fa-7463cf5b74ae\") " pod="openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm"
Jul 29 18:25:19.989300 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:19.989203    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/693f4250-a184-4e82-85fa-7463cf5b74ae-istio-token\") pod \"e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm\" (UID: \"693f4250-a184-4e82-85fa-7463cf5b74ae\") " pod="openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm"
Jul 29 18:25:19.989300 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:19.989249    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/693f4250-a184-4e82-85fa-7463cf5b74ae-istio-envoy\") pod \"e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm\" (UID: \"693f4250-a184-4e82-85fa-7463cf5b74ae\") " pod="openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm"
Jul 29 18:25:19.989300 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:19.989283    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/693f4250-a184-4e82-85fa-7463cf5b74ae-workload-socket\") pod \"e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm\" (UID: \"693f4250-a184-4e82-85fa-7463cf5b74ae\") " pod="openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm"
Jul 29 18:25:19.989607 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:19.989343    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/693f4250-a184-4e82-85fa-7463cf5b74ae-workload-certs\") pod \"e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm\" (UID: \"693f4250-a184-4e82-85fa-7463cf5b74ae\") " pod="openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm"
Jul 29 18:25:19.989607 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:19.989546    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/693f4250-a184-4e82-85fa-7463cf5b74ae-credential-socket\") pod \"e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm\" (UID: \"693f4250-a184-4e82-85fa-7463cf5b74ae\") " pod="openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm"
Jul 29 18:25:19.989771 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:19.989747    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/693f4250-a184-4e82-85fa-7463cf5b74ae-istio-data\") pod \"e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm\" (UID: \"693f4250-a184-4e82-85fa-7463cf5b74ae\") " pod="openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm"
Jul 29 18:25:19.989875 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:19.989775    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/693f4250-a184-4e82-85fa-7463cf5b74ae-workload-socket\") pod \"e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm\" (UID: \"693f4250-a184-4e82-85fa-7463cf5b74ae\") " pod="openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm"
Jul 29 18:25:19.989875 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:19.989792    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/693f4250-a184-4e82-85fa-7463cf5b74ae-workload-certs\") pod \"e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm\" (UID: \"693f4250-a184-4e82-85fa-7463cf5b74ae\") " pod="openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm"
Jul 29 18:25:19.989875 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:19.989849    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/693f4250-a184-4e82-85fa-7463cf5b74ae-istiod-ca-cert\") pod \"e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm\" (UID: \"693f4250-a184-4e82-85fa-7463cf5b74ae\") " pod="openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm"
Jul 29 18:25:19.991706 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:19.991685    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/693f4250-a184-4e82-85fa-7463cf5b74ae-istio-envoy\") pod \"e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm\" (UID: \"693f4250-a184-4e82-85fa-7463cf5b74ae\") " pod="openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm"
Jul 29 18:25:19.991936 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:19.991921    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/693f4250-a184-4e82-85fa-7463cf5b74ae-istio-podinfo\") pod \"e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm\" (UID: \"693f4250-a184-4e82-85fa-7463cf5b74ae\") " pod="openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm"
Jul 29 18:25:19.997307 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:19.997256    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6ftc\" (UniqueName: \"kubernetes.io/projected/693f4250-a184-4e82-85fa-7463cf5b74ae-kube-api-access-x6ftc\") pod \"e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm\" (UID: \"693f4250-a184-4e82-85fa-7463cf5b74ae\") " pod="openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm"
Jul 29 18:25:19.997307 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:19.997264    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/693f4250-a184-4e82-85fa-7463cf5b74ae-istio-token\") pod \"e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm\" (UID: \"693f4250-a184-4e82-85fa-7463cf5b74ae\") " pod="openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm"
Jul 29 18:25:20.060169 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:20.060128    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm"
Jul 29 18:25:20.206121 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:20.206089    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm"]
Jul 29 18:25:20.207219 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:25:20.207192    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod693f4250_a184_4e82_85fa_7463cf5b74ae.slice/crio-de8ea5ebcc34f82eedf2f6dfeb736881a25701235bec60e5fef4f7d2d558eb5f WatchSource:0}: Error finding container de8ea5ebcc34f82eedf2f6dfeb736881a25701235bec60e5fef4f7d2d558eb5f: Status 404 returned error can't find the container with id de8ea5ebcc34f82eedf2f6dfeb736881a25701235bec60e5fef4f7d2d558eb5f
Jul 29 18:25:20.209459 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:20.209429    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:25:20.209519 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:20.209492    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:25:20.209566 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:20.209524    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:25:20.307421 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:20.307387    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm" event={"ID":"693f4250-a184-4e82-85fa-7463cf5b74ae","Type":"ContainerStarted","Data":"6ec62ff108a65a7fbae2f8cbb10c9cf1fc51e52c17188beabac02e8e8f1335fa"}
Jul 29 18:25:20.307421 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:20.307439    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm" event={"ID":"693f4250-a184-4e82-85fa-7463cf5b74ae","Type":"ContainerStarted","Data":"de8ea5ebcc34f82eedf2f6dfeb736881a25701235bec60e5fef4f7d2d558eb5f"}
Jul 29 18:25:20.357262 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:20.357186    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm" podStartSLOduration=1.357164139 podStartE2EDuration="1.357164139s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:25:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:25:20.345541074 +0000 UTC m=+2000.214763905" watchObservedRunningTime="2026-07-29 18:25:20.357164139 +0000 UTC m=+2000.226386950"
Jul 29 18:25:20.655553 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:20.655509    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8c21693-4de2-43e5-a3e5-640d68a2d1f1" path="/var/lib/kubelet/pods/f8c21693-4de2-43e5-a3e5-640d68a2d1f1/volumes"
Jul 29 18:25:21.060799 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:21.060766    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm"
Jul 29 18:25:21.065997 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:21.065960    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm"
Jul 29 18:25:21.316245 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:21.316152    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm"
Jul 29 18:25:21.317106 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:21.317084    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm"
Jul 29 18:25:22.001744 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:22.001695    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-fa2af080-85cbdc5df7-sxk2z"]
Jul 29 18:25:22.006373 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:22.006346    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fa2af080-85cbdc5df7-sxk2z"
Jul 29 18:25:22.009167 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:22.009142    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-fa2af080-serving-cert\""
Jul 29 18:25:22.014657 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:22.014631    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-fa2af080-85cbdc5df7-sxk2z"]
Jul 29 18:25:22.109489 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:22.109454    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f059c35f-caca-4a1d-94e3-ce72ceedbf5e-maas-api-tls\") pod \"maas-api-e2e-migrate-fa2af080-85cbdc5df7-sxk2z\" (UID: \"f059c35f-caca-4a1d-94e3-ce72ceedbf5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fa2af080-85cbdc5df7-sxk2z"
Jul 29 18:25:22.109884 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:22.109517    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f059c35f-caca-4a1d-94e3-ce72ceedbf5e-openshift-service-ca\") pod \"maas-api-e2e-migrate-fa2af080-85cbdc5df7-sxk2z\" (UID: \"f059c35f-caca-4a1d-94e3-ce72ceedbf5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fa2af080-85cbdc5df7-sxk2z"
Jul 29 18:25:22.109884 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:22.109627    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5v94v\" (UniqueName: \"kubernetes.io/projected/f059c35f-caca-4a1d-94e3-ce72ceedbf5e-kube-api-access-5v94v\") pod \"maas-api-e2e-migrate-fa2af080-85cbdc5df7-sxk2z\" (UID: \"f059c35f-caca-4a1d-94e3-ce72ceedbf5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fa2af080-85cbdc5df7-sxk2z"
Jul 29 18:25:22.210986 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:22.210956    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5v94v\" (UniqueName: \"kubernetes.io/projected/f059c35f-caca-4a1d-94e3-ce72ceedbf5e-kube-api-access-5v94v\") pod \"maas-api-e2e-migrate-fa2af080-85cbdc5df7-sxk2z\" (UID: \"f059c35f-caca-4a1d-94e3-ce72ceedbf5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fa2af080-85cbdc5df7-sxk2z"
Jul 29 18:25:22.211166 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:22.210999    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f059c35f-caca-4a1d-94e3-ce72ceedbf5e-maas-api-tls\") pod \"maas-api-e2e-migrate-fa2af080-85cbdc5df7-sxk2z\" (UID: \"f059c35f-caca-4a1d-94e3-ce72ceedbf5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fa2af080-85cbdc5df7-sxk2z"
Jul 29 18:25:22.211166 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:22.211036    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f059c35f-caca-4a1d-94e3-ce72ceedbf5e-openshift-service-ca\") pod \"maas-api-e2e-migrate-fa2af080-85cbdc5df7-sxk2z\" (UID: \"f059c35f-caca-4a1d-94e3-ce72ceedbf5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fa2af080-85cbdc5df7-sxk2z"
Jul 29 18:25:22.211166 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:25:22.211120    2557 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-fa2af080-serving-cert: secret "maas-api-e2e-migrate-fa2af080-serving-cert" not found
Jul 29 18:25:22.211299 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:25:22.211194    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f059c35f-caca-4a1d-94e3-ce72ceedbf5e-maas-api-tls podName:f059c35f-caca-4a1d-94e3-ce72ceedbf5e nodeName:}" failed. No retries permitted until 2026-07-29 18:25:22.71117345 +0000 UTC m=+2002.580396243 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f059c35f-caca-4a1d-94e3-ce72ceedbf5e-maas-api-tls") pod "maas-api-e2e-migrate-fa2af080-85cbdc5df7-sxk2z" (UID: "f059c35f-caca-4a1d-94e3-ce72ceedbf5e") : secret "maas-api-e2e-migrate-fa2af080-serving-cert" not found
Jul 29 18:25:22.211749 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:22.211708    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f059c35f-caca-4a1d-94e3-ce72ceedbf5e-openshift-service-ca\") pod \"maas-api-e2e-migrate-fa2af080-85cbdc5df7-sxk2z\" (UID: \"f059c35f-caca-4a1d-94e3-ce72ceedbf5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fa2af080-85cbdc5df7-sxk2z"
Jul 29 18:25:22.220256 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:22.220233    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5v94v\" (UniqueName: \"kubernetes.io/projected/f059c35f-caca-4a1d-94e3-ce72ceedbf5e-kube-api-access-5v94v\") pod \"maas-api-e2e-migrate-fa2af080-85cbdc5df7-sxk2z\" (UID: \"f059c35f-caca-4a1d-94e3-ce72ceedbf5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fa2af080-85cbdc5df7-sxk2z"
Jul 29 18:25:22.716090 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:22.716061    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f059c35f-caca-4a1d-94e3-ce72ceedbf5e-maas-api-tls\") pod \"maas-api-e2e-migrate-fa2af080-85cbdc5df7-sxk2z\" (UID: \"f059c35f-caca-4a1d-94e3-ce72ceedbf5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fa2af080-85cbdc5df7-sxk2z"
Jul 29 18:25:22.718517 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:22.718494    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f059c35f-caca-4a1d-94e3-ce72ceedbf5e-maas-api-tls\") pod \"maas-api-e2e-migrate-fa2af080-85cbdc5df7-sxk2z\" (UID: \"f059c35f-caca-4a1d-94e3-ce72ceedbf5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fa2af080-85cbdc5df7-sxk2z"
Jul 29 18:25:22.920563 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:22.920518    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fa2af080-85cbdc5df7-sxk2z"
Jul 29 18:25:23.067092 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:23.067059    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-fa2af080-85cbdc5df7-sxk2z"]
Jul 29 18:25:23.068569 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:25:23.068541    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf059c35f_caca_4a1d_94e3_ce72ceedbf5e.slice/crio-b4cabd625e93204fe4b1f4b7217ffd43d7976bc7a55f6541ab9e46099ab81afc WatchSource:0}: Error finding container b4cabd625e93204fe4b1f4b7217ffd43d7976bc7a55f6541ab9e46099ab81afc: Status 404 returned error can't find the container with id b4cabd625e93204fe4b1f4b7217ffd43d7976bc7a55f6541ab9e46099ab81afc
Jul 29 18:25:23.329089 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:23.329060    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fa2af080-85cbdc5df7-sxk2z" event={"ID":"f059c35f-caca-4a1d-94e3-ce72ceedbf5e","Type":"ContainerStarted","Data":"b4cabd625e93204fe4b1f4b7217ffd43d7976bc7a55f6541ab9e46099ab81afc"}
Jul 29 18:25:23.997049 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:23.996992    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6"
Jul 29 18:25:24.130595 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:24.130567    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0407fda-71a1-435e-8904-b80c41e2a466-workload-socket\") pod \"c0407fda-71a1-435e-8904-b80c41e2a466\" (UID: \"c0407fda-71a1-435e-8904-b80c41e2a466\") "
Jul 29 18:25:24.130810 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:24.130607    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0407fda-71a1-435e-8904-b80c41e2a466-istio-envoy\") pod \"c0407fda-71a1-435e-8904-b80c41e2a466\" (UID: \"c0407fda-71a1-435e-8904-b80c41e2a466\") "
Jul 29 18:25:24.130810 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:24.130636    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0407fda-71a1-435e-8904-b80c41e2a466-istio-data\") pod \"c0407fda-71a1-435e-8904-b80c41e2a466\" (UID: \"c0407fda-71a1-435e-8904-b80c41e2a466\") "
Jul 29 18:25:24.130810 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:24.130663    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0407fda-71a1-435e-8904-b80c41e2a466-credential-socket\") pod \"c0407fda-71a1-435e-8904-b80c41e2a466\" (UID: \"c0407fda-71a1-435e-8904-b80c41e2a466\") "
Jul 29 18:25:24.130810 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:24.130679    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vzvq7\" (UniqueName: \"kubernetes.io/projected/c0407fda-71a1-435e-8904-b80c41e2a466-kube-api-access-vzvq7\") pod \"c0407fda-71a1-435e-8904-b80c41e2a466\" (UID: \"c0407fda-71a1-435e-8904-b80c41e2a466\") "
Jul 29 18:25:24.130810 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:24.130706    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0407fda-71a1-435e-8904-b80c41e2a466-istio-podinfo\") pod \"c0407fda-71a1-435e-8904-b80c41e2a466\" (UID: \"c0407fda-71a1-435e-8904-b80c41e2a466\") "
Jul 29 18:25:24.131088 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:24.130879    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0407fda-71a1-435e-8904-b80c41e2a466-istio-token\") pod \"c0407fda-71a1-435e-8904-b80c41e2a466\" (UID: \"c0407fda-71a1-435e-8904-b80c41e2a466\") "
Jul 29 18:25:24.131088 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:24.130935    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0407fda-71a1-435e-8904-b80c41e2a466-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c0407fda-71a1-435e-8904-b80c41e2a466" (UID: "c0407fda-71a1-435e-8904-b80c41e2a466"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:25:24.131088 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:24.130963    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0407fda-71a1-435e-8904-b80c41e2a466-workload-certs\") pod \"c0407fda-71a1-435e-8904-b80c41e2a466\" (UID: \"c0407fda-71a1-435e-8904-b80c41e2a466\") "
Jul 29 18:25:24.131088 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:24.131045    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0407fda-71a1-435e-8904-b80c41e2a466-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c0407fda-71a1-435e-8904-b80c41e2a466" (UID: "c0407fda-71a1-435e-8904-b80c41e2a466"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:25:24.131088 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:24.131058    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0407fda-71a1-435e-8904-b80c41e2a466-istiod-ca-cert\") pod \"c0407fda-71a1-435e-8904-b80c41e2a466\" (UID: \"c0407fda-71a1-435e-8904-b80c41e2a466\") "
Jul 29 18:25:24.131418 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:24.131250    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0407fda-71a1-435e-8904-b80c41e2a466-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c0407fda-71a1-435e-8904-b80c41e2a466" (UID: "c0407fda-71a1-435e-8904-b80c41e2a466"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:25:24.131643 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:24.131568    2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0407fda-71a1-435e-8904-b80c41e2a466-workload-socket\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:25:24.131643 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:24.131598    2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0407fda-71a1-435e-8904-b80c41e2a466-credential-socket\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:25:24.131643 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:24.131614    2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0407fda-71a1-435e-8904-b80c41e2a466-workload-certs\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:25:24.131643 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:24.131549    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0407fda-71a1-435e-8904-b80c41e2a466-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c0407fda-71a1-435e-8904-b80c41e2a466" (UID: "c0407fda-71a1-435e-8904-b80c41e2a466"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:25:24.132033 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:24.131940    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0407fda-71a1-435e-8904-b80c41e2a466-istio-data" (OuterVolumeSpecName: "istio-data") pod "c0407fda-71a1-435e-8904-b80c41e2a466" (UID: "c0407fda-71a1-435e-8904-b80c41e2a466"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:25:24.133176 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:24.133143    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0407fda-71a1-435e-8904-b80c41e2a466-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c0407fda-71a1-435e-8904-b80c41e2a466" (UID: "c0407fda-71a1-435e-8904-b80c41e2a466"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:25:24.133176 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:24.133156    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0407fda-71a1-435e-8904-b80c41e2a466-kube-api-access-vzvq7" (OuterVolumeSpecName: "kube-api-access-vzvq7") pod "c0407fda-71a1-435e-8904-b80c41e2a466" (UID: "c0407fda-71a1-435e-8904-b80c41e2a466"). InnerVolumeSpecName "kube-api-access-vzvq7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:25:24.133297 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:24.133228    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c0407fda-71a1-435e-8904-b80c41e2a466-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c0407fda-71a1-435e-8904-b80c41e2a466" (UID: "c0407fda-71a1-435e-8904-b80c41e2a466"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 18:25:24.133297 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:24.133263    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0407fda-71a1-435e-8904-b80c41e2a466-istio-token" (OuterVolumeSpecName: "istio-token") pod "c0407fda-71a1-435e-8904-b80c41e2a466" (UID: "c0407fda-71a1-435e-8904-b80c41e2a466"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:25:24.232292 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:24.232265    2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0407fda-71a1-435e-8904-b80c41e2a466-istio-data\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:25:24.232292 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:24.232292    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vzvq7\" (UniqueName: \"kubernetes.io/projected/c0407fda-71a1-435e-8904-b80c41e2a466-kube-api-access-vzvq7\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:25:24.232472 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:24.232302    2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0407fda-71a1-435e-8904-b80c41e2a466-istio-podinfo\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:25:24.232472 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:24.232310    2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0407fda-71a1-435e-8904-b80c41e2a466-istio-token\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:25:24.232472 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:24.232318    2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0407fda-71a1-435e-8904-b80c41e2a466-istiod-ca-cert\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:25:24.232472 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:24.232326    2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0407fda-71a1-435e-8904-b80c41e2a466-istio-envoy\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:25:24.336891 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:24.336859    2557 generic.go:356] "Generic (PLEG): container finished" podID="c0407fda-71a1-435e-8904-b80c41e2a466" containerID="ebe2acce81a7cca2b74b83d49a14962a312c9305e2c883f17b8b3f20141942ea" exitCode=0
Jul 29 18:25:24.337292 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:24.336925    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6"
Jul 29 18:25:24.337292 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:24.336975    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6" event={"ID":"c0407fda-71a1-435e-8904-b80c41e2a466","Type":"ContainerDied","Data":"ebe2acce81a7cca2b74b83d49a14962a312c9305e2c883f17b8b3f20141942ea"}
Jul 29 18:25:24.337292 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:24.337025    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6" event={"ID":"c0407fda-71a1-435e-8904-b80c41e2a466","Type":"ContainerDied","Data":"a93f5b9b2df4d8890d88f498de0694404a7f6cd230ba9e68745285e7e2bd1087"}
Jul 29 18:25:24.337292 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:24.337057    2557 scope.go:117] "RemoveContainer" containerID="ebe2acce81a7cca2b74b83d49a14962a312c9305e2c883f17b8b3f20141942ea"
Jul 29 18:25:24.339613 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:24.339584    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fa2af080-85cbdc5df7-sxk2z" event={"ID":"f059c35f-caca-4a1d-94e3-ce72ceedbf5e","Type":"ContainerStarted","Data":"7fd6691cbd4afd24d991270c79171afaa93084e3e6513b65f2b7cca327ce7992"}
Jul 29 18:25:24.340442 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:24.340175    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fa2af080-85cbdc5df7-sxk2z"
Jul 29 18:25:24.348997 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:24.348978    2557 scope.go:117] "RemoveContainer" containerID="b8d6912aca6f0324802276ae30c3c9229f0537646b8e6de8441e239ad1c91abb"
Jul 29 18:25:24.357869 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:24.357825    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fa2af080-85cbdc5df7-sxk2z" podStartSLOduration=3.034674853 podStartE2EDuration="3.357810135s" totalImagesPullingTime="323.135282ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:25:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:25:24.356968676 +0000 UTC m=+2004.226191488" watchObservedRunningTime="2026-07-29 18:25:24.357810135 +0000 UTC m=+2004.227032980"
Jul 29 18:25:24.358277 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:24.358247    2557 scope.go:117] "RemoveContainer" containerID="ebe2acce81a7cca2b74b83d49a14962a312c9305e2c883f17b8b3f20141942ea"
Jul 29 18:25:24.358516 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:25:24.358497    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ebe2acce81a7cca2b74b83d49a14962a312c9305e2c883f17b8b3f20141942ea\": container with ID starting with ebe2acce81a7cca2b74b83d49a14962a312c9305e2c883f17b8b3f20141942ea not found: ID does not exist" containerID="ebe2acce81a7cca2b74b83d49a14962a312c9305e2c883f17b8b3f20141942ea"
Jul 29 18:25:24.358571 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:24.358524    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebe2acce81a7cca2b74b83d49a14962a312c9305e2c883f17b8b3f20141942ea"} err="failed to get container status \"ebe2acce81a7cca2b74b83d49a14962a312c9305e2c883f17b8b3f20141942ea\": rpc error: code = NotFound desc = could not find container \"ebe2acce81a7cca2b74b83d49a14962a312c9305e2c883f17b8b3f20141942ea\": container with ID starting with ebe2acce81a7cca2b74b83d49a14962a312c9305e2c883f17b8b3f20141942ea not found: ID does not exist"
Jul 29 18:25:24.358571 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:24.358542    2557 scope.go:117] "RemoveContainer" containerID="b8d6912aca6f0324802276ae30c3c9229f0537646b8e6de8441e239ad1c91abb"
Jul 29 18:25:24.358824 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:25:24.358804    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8d6912aca6f0324802276ae30c3c9229f0537646b8e6de8441e239ad1c91abb\": container with ID starting with b8d6912aca6f0324802276ae30c3c9229f0537646b8e6de8441e239ad1c91abb not found: ID does not exist" containerID="b8d6912aca6f0324802276ae30c3c9229f0537646b8e6de8441e239ad1c91abb"
Jul 29 18:25:24.358881 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:24.358830    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8d6912aca6f0324802276ae30c3c9229f0537646b8e6de8441e239ad1c91abb"} err="failed to get container status \"b8d6912aca6f0324802276ae30c3c9229f0537646b8e6de8441e239ad1c91abb\": rpc error: code = NotFound desc = could not find container \"b8d6912aca6f0324802276ae30c3c9229f0537646b8e6de8441e239ad1c91abb\": container with ID starting with b8d6912aca6f0324802276ae30c3c9229f0537646b8e6de8441e239ad1c91abb not found: ID does not exist"
Jul 29 18:25:24.371623 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:24.371601    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6"]
Jul 29 18:25:24.375753 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:24.375710    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-0174a53b-openshift-default-5c6465f846-hm5t6"]
Jul 29 18:25:24.652773 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:24.652671    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0407fda-71a1-435e-8904-b80c41e2a466" path="/var/lib/kubelet/pods/c0407fda-71a1-435e-8904-b80c41e2a466/volumes"
Jul 29 18:25:31.361107 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:31.361066    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fa2af080-85cbdc5df7-sxk2z"
Jul 29 18:25:36.318018 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:36.317984    2557 patch_prober.go:28] interesting pod/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Jul 29 18:25:36.318389 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:36.318041    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm" podUID="693f4250-a184-4e82-85fa-7463cf5b74ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Jul 29 18:25:36.410808 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:36.410776    2557 generic.go:356] "Generic (PLEG): container finished" podID="693f4250-a184-4e82-85fa-7463cf5b74ae" containerID="6ec62ff108a65a7fbae2f8cbb10c9cf1fc51e52c17188beabac02e8e8f1335fa" exitCode=0
Jul 29 18:25:36.411004 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:36.410862    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm" event={"ID":"693f4250-a184-4e82-85fa-7463cf5b74ae","Type":"ContainerDied","Data":"6ec62ff108a65a7fbae2f8cbb10c9cf1fc51e52c17188beabac02e8e8f1335fa"}
Jul 29 18:25:36.411313 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:36.411296    2557 scope.go:117] "RemoveContainer" containerID="6ec62ff108a65a7fbae2f8cbb10c9cf1fc51e52c17188beabac02e8e8f1335fa"
Jul 29 18:25:36.412316 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:36.412291    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:25:36.412373 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:36.412358    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:25:36.412409 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:36.412391    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:25:37.417788 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:37.417754    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm" event={"ID":"693f4250-a184-4e82-85fa-7463cf5b74ae","Type":"ContainerStarted","Data":"daf0ae1c1c1c42ef00531c720288b36becec29686b16b604a0bbc7557e713b8f"}
Jul 29 18:25:38.060365 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:38.060321    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm"
Jul 29 18:25:38.060583 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:38.060530    2557 patch_prober.go:28] interesting pod/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Jul 29 18:25:38.060657 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:38.060570    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm" podUID="693f4250-a184-4e82-85fa-7463cf5b74ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Jul 29 18:25:39.061309 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:39.061265    2557 patch_prober.go:28] interesting pod/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Jul 29 18:25:39.061782 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:39.061361    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm" podUID="693f4250-a184-4e82-85fa-7463cf5b74ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Jul 29 18:25:40.061465 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:40.061424    2557 patch_prober.go:28] interesting pod/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Jul 29 18:25:40.061900 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:40.061506    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm" podUID="693f4250-a184-4e82-85fa-7463cf5b74ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Jul 29 18:25:41.061028 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:41.060986    2557 patch_prober.go:28] interesting pod/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Jul 29 18:25:41.061340 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:41.061310    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm" podUID="693f4250-a184-4e82-85fa-7463cf5b74ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Jul 29 18:25:42.064298 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:42.064262    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm"
Jul 29 18:25:42.064767 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:42.064660    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm"
Jul 29 18:25:42.065352 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:42.065336    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm"
Jul 29 18:25:52.551667 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:52.551560    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-fa2af080-85cbdc5df7-sxk2z"]
Jul 29 18:25:52.553133 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:52.552989    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fa2af080-85cbdc5df7-sxk2z" podUID="f059c35f-caca-4a1d-94e3-ce72ceedbf5e" containerName="maas-api" containerID="cri-o://7fd6691cbd4afd24d991270c79171afaa93084e3e6513b65f2b7cca327ce7992" gracePeriod=30
Jul 29 18:25:52.819354 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:52.819281    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fa2af080-85cbdc5df7-sxk2z"
Jul 29 18:25:52.899081 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:52.899044    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm"]
Jul 29 18:25:52.899321 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:52.899292    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm" podUID="693f4250-a184-4e82-85fa-7463cf5b74ae" containerName="istio-proxy" containerID="cri-o://daf0ae1c1c1c42ef00531c720288b36becec29686b16b604a0bbc7557e713b8f" gracePeriod=30
Jul 29 18:25:52.906289 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:52.906263    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f059c35f-caca-4a1d-94e3-ce72ceedbf5e-maas-api-tls\") pod \"f059c35f-caca-4a1d-94e3-ce72ceedbf5e\" (UID: \"f059c35f-caca-4a1d-94e3-ce72ceedbf5e\") "
Jul 29 18:25:52.906443 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:52.906424    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f059c35f-caca-4a1d-94e3-ce72ceedbf5e-openshift-service-ca\") pod \"f059c35f-caca-4a1d-94e3-ce72ceedbf5e\" (UID: \"f059c35f-caca-4a1d-94e3-ce72ceedbf5e\") "
Jul 29 18:25:52.906528 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:52.906507    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5v94v\" (UniqueName: \"kubernetes.io/projected/f059c35f-caca-4a1d-94e3-ce72ceedbf5e-kube-api-access-5v94v\") pod \"f059c35f-caca-4a1d-94e3-ce72ceedbf5e\" (UID: \"f059c35f-caca-4a1d-94e3-ce72ceedbf5e\") "
Jul 29 18:25:52.906796 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:52.906762    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f059c35f-caca-4a1d-94e3-ce72ceedbf5e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f059c35f-caca-4a1d-94e3-ce72ceedbf5e" (UID: "f059c35f-caca-4a1d-94e3-ce72ceedbf5e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:25:52.906893 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:52.906874    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f059c35f-caca-4a1d-94e3-ce72ceedbf5e-openshift-service-ca\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:25:52.909356 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:52.909333    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f059c35f-caca-4a1d-94e3-ce72ceedbf5e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f059c35f-caca-4a1d-94e3-ce72ceedbf5e" (UID: "f059c35f-caca-4a1d-94e3-ce72ceedbf5e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:25:52.909460 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:52.909441    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f059c35f-caca-4a1d-94e3-ce72ceedbf5e-kube-api-access-5v94v" (OuterVolumeSpecName: "kube-api-access-5v94v") pod "f059c35f-caca-4a1d-94e3-ce72ceedbf5e" (UID: "f059c35f-caca-4a1d-94e3-ce72ceedbf5e"). InnerVolumeSpecName "kube-api-access-5v94v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:25:53.008425 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:53.008381    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5v94v\" (UniqueName: \"kubernetes.io/projected/f059c35f-caca-4a1d-94e3-ce72ceedbf5e-kube-api-access-5v94v\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:25:53.008425 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:53.008425    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f059c35f-caca-4a1d-94e3-ce72ceedbf5e-maas-api-tls\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:25:53.519410 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:53.519373    2557 generic.go:356] "Generic (PLEG): container finished" podID="f059c35f-caca-4a1d-94e3-ce72ceedbf5e" containerID="7fd6691cbd4afd24d991270c79171afaa93084e3e6513b65f2b7cca327ce7992" exitCode=0
Jul 29 18:25:53.519612 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:53.519423    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fa2af080-85cbdc5df7-sxk2z"
Jul 29 18:25:53.519612 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:53.519458    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fa2af080-85cbdc5df7-sxk2z" event={"ID":"f059c35f-caca-4a1d-94e3-ce72ceedbf5e","Type":"ContainerDied","Data":"7fd6691cbd4afd24d991270c79171afaa93084e3e6513b65f2b7cca327ce7992"}
Jul 29 18:25:53.519612 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:53.519511    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fa2af080-85cbdc5df7-sxk2z" event={"ID":"f059c35f-caca-4a1d-94e3-ce72ceedbf5e","Type":"ContainerDied","Data":"b4cabd625e93204fe4b1f4b7217ffd43d7976bc7a55f6541ab9e46099ab81afc"}
Jul 29 18:25:53.519612 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:53.519535    2557 scope.go:117] "RemoveContainer" containerID="7fd6691cbd4afd24d991270c79171afaa93084e3e6513b65f2b7cca327ce7992"
Jul 29 18:25:53.529957 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:53.529936    2557 scope.go:117] "RemoveContainer" containerID="7fd6691cbd4afd24d991270c79171afaa93084e3e6513b65f2b7cca327ce7992"
Jul 29 18:25:53.530216 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:25:53.530198    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fd6691cbd4afd24d991270c79171afaa93084e3e6513b65f2b7cca327ce7992\": container with ID starting with 7fd6691cbd4afd24d991270c79171afaa93084e3e6513b65f2b7cca327ce7992 not found: ID does not exist" containerID="7fd6691cbd4afd24d991270c79171afaa93084e3e6513b65f2b7cca327ce7992"
Jul 29 18:25:53.530270 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:53.530225    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fd6691cbd4afd24d991270c79171afaa93084e3e6513b65f2b7cca327ce7992"} err="failed to get container status \"7fd6691cbd4afd24d991270c79171afaa93084e3e6513b65f2b7cca327ce7992\": rpc error: code = NotFound desc = could not find container \"7fd6691cbd4afd24d991270c79171afaa93084e3e6513b65f2b7cca327ce7992\": container with ID starting with 7fd6691cbd4afd24d991270c79171afaa93084e3e6513b65f2b7cca327ce7992 not found: ID does not exist"
Jul 29 18:25:53.543573 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:53.543548    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-fa2af080-85cbdc5df7-sxk2z"]
Jul 29 18:25:53.548089 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:53.548067    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-fa2af080-85cbdc5df7-sxk2z"]
Jul 29 18:25:54.653785 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:54.653748    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f059c35f-caca-4a1d-94e3-ce72ceedbf5e" path="/var/lib/kubelet/pods/f059c35f-caca-4a1d-94e3-ce72ceedbf5e/volumes"
Jul 29 18:25:57.066815 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:57.066769    2557 patch_prober.go:28] interesting pod/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Jul 29 18:25:57.067189 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:57.066840    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm" podUID="693f4250-a184-4e82-85fa-7463cf5b74ae" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 29 18:25:58.156931 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.156906    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm"
Jul 29 18:25:58.257689 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.257605    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/693f4250-a184-4e82-85fa-7463cf5b74ae-workload-socket\") pod \"693f4250-a184-4e82-85fa-7463cf5b74ae\" (UID: \"693f4250-a184-4e82-85fa-7463cf5b74ae\") "
Jul 29 18:25:58.257689 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.257653    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x6ftc\" (UniqueName: \"kubernetes.io/projected/693f4250-a184-4e82-85fa-7463cf5b74ae-kube-api-access-x6ftc\") pod \"693f4250-a184-4e82-85fa-7463cf5b74ae\" (UID: \"693f4250-a184-4e82-85fa-7463cf5b74ae\") "
Jul 29 18:25:58.257689 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.257677    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/693f4250-a184-4e82-85fa-7463cf5b74ae-credential-socket\") pod \"693f4250-a184-4e82-85fa-7463cf5b74ae\" (UID: \"693f4250-a184-4e82-85fa-7463cf5b74ae\") "
Jul 29 18:25:58.257992 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.257708    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/693f4250-a184-4e82-85fa-7463cf5b74ae-istio-envoy\") pod \"693f4250-a184-4e82-85fa-7463cf5b74ae\" (UID: \"693f4250-a184-4e82-85fa-7463cf5b74ae\") "
Jul 29 18:25:58.257992 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.257774    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/693f4250-a184-4e82-85fa-7463cf5b74ae-istiod-ca-cert\") pod \"693f4250-a184-4e82-85fa-7463cf5b74ae\" (UID: \"693f4250-a184-4e82-85fa-7463cf5b74ae\") "
Jul 29 18:25:58.257992 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.257802    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/693f4250-a184-4e82-85fa-7463cf5b74ae-istio-data\") pod \"693f4250-a184-4e82-85fa-7463cf5b74ae\" (UID: \"693f4250-a184-4e82-85fa-7463cf5b74ae\") "
Jul 29 18:25:58.257992 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.257827    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/693f4250-a184-4e82-85fa-7463cf5b74ae-istio-token\") pod \"693f4250-a184-4e82-85fa-7463cf5b74ae\" (UID: \"693f4250-a184-4e82-85fa-7463cf5b74ae\") "
Jul 29 18:25:58.257992 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.257897    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/693f4250-a184-4e82-85fa-7463cf5b74ae-istio-podinfo\") pod \"693f4250-a184-4e82-85fa-7463cf5b74ae\" (UID: \"693f4250-a184-4e82-85fa-7463cf5b74ae\") "
Jul 29 18:25:58.257992 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.257911    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/693f4250-a184-4e82-85fa-7463cf5b74ae-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "693f4250-a184-4e82-85fa-7463cf5b74ae" (UID: "693f4250-a184-4e82-85fa-7463cf5b74ae"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:25:58.257992 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.257931    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/693f4250-a184-4e82-85fa-7463cf5b74ae-workload-certs\") pod \"693f4250-a184-4e82-85fa-7463cf5b74ae\" (UID: \"693f4250-a184-4e82-85fa-7463cf5b74ae\") "
Jul 29 18:25:58.258532 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.258081    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/693f4250-a184-4e82-85fa-7463cf5b74ae-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "693f4250-a184-4e82-85fa-7463cf5b74ae" (UID: "693f4250-a184-4e82-85fa-7463cf5b74ae"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:25:58.258532 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.258306    2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/693f4250-a184-4e82-85fa-7463cf5b74ae-credential-socket\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:25:58.258532 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.258318    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/693f4250-a184-4e82-85fa-7463cf5b74ae-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "693f4250-a184-4e82-85fa-7463cf5b74ae" (UID: "693f4250-a184-4e82-85fa-7463cf5b74ae"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:25:58.258532 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.258328    2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/693f4250-a184-4e82-85fa-7463cf5b74ae-workload-socket\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:25:58.258532 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.258432    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/693f4250-a184-4e82-85fa-7463cf5b74ae-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "693f4250-a184-4e82-85fa-7463cf5b74ae" (UID: "693f4250-a184-4e82-85fa-7463cf5b74ae"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:25:58.258998 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.258714    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/693f4250-a184-4e82-85fa-7463cf5b74ae-istio-data" (OuterVolumeSpecName: "istio-data") pod "693f4250-a184-4e82-85fa-7463cf5b74ae" (UID: "693f4250-a184-4e82-85fa-7463cf5b74ae"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:25:58.260105 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.260079    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/693f4250-a184-4e82-85fa-7463cf5b74ae-kube-api-access-x6ftc" (OuterVolumeSpecName: "kube-api-access-x6ftc") pod "693f4250-a184-4e82-85fa-7463cf5b74ae" (UID: "693f4250-a184-4e82-85fa-7463cf5b74ae"). InnerVolumeSpecName "kube-api-access-x6ftc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:25:58.260258 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.260236    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/693f4250-a184-4e82-85fa-7463cf5b74ae-istio-token" (OuterVolumeSpecName: "istio-token") pod "693f4250-a184-4e82-85fa-7463cf5b74ae" (UID: "693f4250-a184-4e82-85fa-7463cf5b74ae"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:25:58.260304 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.260245    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/693f4250-a184-4e82-85fa-7463cf5b74ae-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "693f4250-a184-4e82-85fa-7463cf5b74ae" (UID: "693f4250-a184-4e82-85fa-7463cf5b74ae"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 18:25:58.260509 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.260482    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/693f4250-a184-4e82-85fa-7463cf5b74ae-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "693f4250-a184-4e82-85fa-7463cf5b74ae" (UID: "693f4250-a184-4e82-85fa-7463cf5b74ae"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:25:58.359305 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.359267    2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/693f4250-a184-4e82-85fa-7463cf5b74ae-istio-envoy\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:25:58.359305 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.359299    2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/693f4250-a184-4e82-85fa-7463cf5b74ae-istiod-ca-cert\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:25:58.359305 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.359308    2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/693f4250-a184-4e82-85fa-7463cf5b74ae-istio-data\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:25:58.359532 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.359316    2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/693f4250-a184-4e82-85fa-7463cf5b74ae-istio-token\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:25:58.359532 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.359324    2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/693f4250-a184-4e82-85fa-7463cf5b74ae-istio-podinfo\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:25:58.359532 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.359332    2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/693f4250-a184-4e82-85fa-7463cf5b74ae-workload-certs\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:25:58.359532 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.359340    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6ftc\" (UniqueName: \"kubernetes.io/projected/693f4250-a184-4e82-85fa-7463cf5b74ae-kube-api-access-x6ftc\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:25:58.551268 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.551234    2557 generic.go:356] "Generic (PLEG): container finished" podID="693f4250-a184-4e82-85fa-7463cf5b74ae" containerID="daf0ae1c1c1c42ef00531c720288b36becec29686b16b604a0bbc7557e713b8f" exitCode=0
Jul 29 18:25:58.551440 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.551287    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm"
Jul 29 18:25:58.551440 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.551324    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm" event={"ID":"693f4250-a184-4e82-85fa-7463cf5b74ae","Type":"ContainerDied","Data":"daf0ae1c1c1c42ef00531c720288b36becec29686b16b604a0bbc7557e713b8f"}
Jul 29 18:25:58.551440 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.551390    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm" event={"ID":"693f4250-a184-4e82-85fa-7463cf5b74ae","Type":"ContainerDied","Data":"de8ea5ebcc34f82eedf2f6dfeb736881a25701235bec60e5fef4f7d2d558eb5f"}
Jul 29 18:25:58.551440 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.551423    2557 scope.go:117] "RemoveContainer" containerID="daf0ae1c1c1c42ef00531c720288b36becec29686b16b604a0bbc7557e713b8f"
Jul 29 18:25:58.555832 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.555802    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f861467d-7bff777d78-cj9n4"]
Jul 29 18:25:58.556387 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.556362    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="693f4250-a184-4e82-85fa-7463cf5b74ae" containerName="istio-proxy"
Jul 29 18:25:58.556387 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.556387    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="693f4250-a184-4e82-85fa-7463cf5b74ae" containerName="istio-proxy"
Jul 29 18:25:58.556574 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.556414    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0407fda-71a1-435e-8904-b80c41e2a466" containerName="istio-proxy"
Jul 29 18:25:58.556574 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.556422    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0407fda-71a1-435e-8904-b80c41e2a466" containerName="istio-proxy"
Jul 29 18:25:58.556574 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.556433    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f059c35f-caca-4a1d-94e3-ce72ceedbf5e" containerName="maas-api"
Jul 29 18:25:58.556574 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.556441    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="f059c35f-caca-4a1d-94e3-ce72ceedbf5e" containerName="maas-api"
Jul 29 18:25:58.556574 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.556453    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0407fda-71a1-435e-8904-b80c41e2a466" containerName="istio-proxy"
Jul 29 18:25:58.556574 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.556461    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0407fda-71a1-435e-8904-b80c41e2a466" containerName="istio-proxy"
Jul 29 18:25:58.556574 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.556473    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="693f4250-a184-4e82-85fa-7463cf5b74ae" containerName="istio-proxy"
Jul 29 18:25:58.556574 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.556481    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="693f4250-a184-4e82-85fa-7463cf5b74ae" containerName="istio-proxy"
Jul 29 18:25:58.556574 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.556575    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0407fda-71a1-435e-8904-b80c41e2a466" containerName="istio-proxy"
Jul 29 18:25:58.557073 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.556590    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="693f4250-a184-4e82-85fa-7463cf5b74ae" containerName="istio-proxy"
Jul 29 18:25:58.557073 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.556598    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="f059c35f-caca-4a1d-94e3-ce72ceedbf5e" containerName="maas-api"
Jul 29 18:25:58.557073 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.556606    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="693f4250-a184-4e82-85fa-7463cf5b74ae" containerName="istio-proxy"
Jul 29 18:25:58.561139 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.561120    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f861467d-7bff777d78-cj9n4"
Jul 29 18:25:58.563867 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.563846    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-f861467d-serving-cert\""
Jul 29 18:25:58.564967 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.564859    2557 scope.go:117] "RemoveContainer" containerID="6ec62ff108a65a7fbae2f8cbb10c9cf1fc51e52c17188beabac02e8e8f1335fa"
Jul 29 18:25:58.569640 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.569623    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f861467d-7bff777d78-cj9n4"]
Jul 29 18:25:58.578965 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.578946    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm"]
Jul 29 18:25:58.579579 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.579556    2557 scope.go:117] "RemoveContainer" containerID="daf0ae1c1c1c42ef00531c720288b36becec29686b16b604a0bbc7557e713b8f"
Jul 29 18:25:58.579971 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:25:58.579912    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"daf0ae1c1c1c42ef00531c720288b36becec29686b16b604a0bbc7557e713b8f\": container with ID starting with daf0ae1c1c1c42ef00531c720288b36becec29686b16b604a0bbc7557e713b8f not found: ID does not exist" containerID="daf0ae1c1c1c42ef00531c720288b36becec29686b16b604a0bbc7557e713b8f"
Jul 29 18:25:58.579971 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.579941    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"daf0ae1c1c1c42ef00531c720288b36becec29686b16b604a0bbc7557e713b8f"} err="failed to get container status \"daf0ae1c1c1c42ef00531c720288b36becec29686b16b604a0bbc7557e713b8f\": rpc error: code = NotFound desc = could not find container \"daf0ae1c1c1c42ef00531c720288b36becec29686b16b604a0bbc7557e713b8f\": container with ID starting with daf0ae1c1c1c42ef00531c720288b36becec29686b16b604a0bbc7557e713b8f not found: ID does not exist"
Jul 29 18:25:58.579971 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.579965    2557 scope.go:117] "RemoveContainer" containerID="6ec62ff108a65a7fbae2f8cbb10c9cf1fc51e52c17188beabac02e8e8f1335fa"
Jul 29 18:25:58.580296 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:25:58.580276    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ec62ff108a65a7fbae2f8cbb10c9cf1fc51e52c17188beabac02e8e8f1335fa\": container with ID starting with 6ec62ff108a65a7fbae2f8cbb10c9cf1fc51e52c17188beabac02e8e8f1335fa not found: ID does not exist" containerID="6ec62ff108a65a7fbae2f8cbb10c9cf1fc51e52c17188beabac02e8e8f1335fa"
Jul 29 18:25:58.580390 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.580306    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ec62ff108a65a7fbae2f8cbb10c9cf1fc51e52c17188beabac02e8e8f1335fa"} err="failed to get container status \"6ec62ff108a65a7fbae2f8cbb10c9cf1fc51e52c17188beabac02e8e8f1335fa\": rpc error: code = NotFound desc = could not find container \"6ec62ff108a65a7fbae2f8cbb10c9cf1fc51e52c17188beabac02e8e8f1335fa\": container with ID starting with 6ec62ff108a65a7fbae2f8cbb10c9cf1fc51e52c17188beabac02e8e8f1335fa not found: ID does not exist"
Jul 29 18:25:58.584233 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.584212    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-fa2af080-gw-openshift-default-78d9d47964-lmhqm"]
Jul 29 18:25:58.655093 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.655062    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="693f4250-a184-4e82-85fa-7463cf5b74ae" path="/var/lib/kubelet/pods/693f4250-a184-4e82-85fa-7463cf5b74ae/volumes"
Jul 29 18:25:58.662077 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.662053    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1877b54-aa07-434e-82fe-86f1077ca2c4-maas-api-tls\") pod \"maas-api-e2e-ait-f861467d-7bff777d78-cj9n4\" (UID: \"f1877b54-aa07-434e-82fe-86f1077ca2c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f861467d-7bff777d78-cj9n4"
Jul 29 18:25:58.662217 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.662097    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1877b54-aa07-434e-82fe-86f1077ca2c4-openshift-service-ca\") pod \"maas-api-e2e-ait-f861467d-7bff777d78-cj9n4\" (UID: \"f1877b54-aa07-434e-82fe-86f1077ca2c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f861467d-7bff777d78-cj9n4"
Jul 29 18:25:58.662259 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.662215    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hm8ld\" (UniqueName: \"kubernetes.io/projected/f1877b54-aa07-434e-82fe-86f1077ca2c4-kube-api-access-hm8ld\") pod \"maas-api-e2e-ait-f861467d-7bff777d78-cj9n4\" (UID: \"f1877b54-aa07-434e-82fe-86f1077ca2c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f861467d-7bff777d78-cj9n4"
Jul 29 18:25:58.763337 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.763301    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1877b54-aa07-434e-82fe-86f1077ca2c4-openshift-service-ca\") pod \"maas-api-e2e-ait-f861467d-7bff777d78-cj9n4\" (UID: \"f1877b54-aa07-434e-82fe-86f1077ca2c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f861467d-7bff777d78-cj9n4"
Jul 29 18:25:58.763500 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.763448    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hm8ld\" (UniqueName: \"kubernetes.io/projected/f1877b54-aa07-434e-82fe-86f1077ca2c4-kube-api-access-hm8ld\") pod \"maas-api-e2e-ait-f861467d-7bff777d78-cj9n4\" (UID: \"f1877b54-aa07-434e-82fe-86f1077ca2c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f861467d-7bff777d78-cj9n4"
Jul 29 18:25:58.763561 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.763535    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1877b54-aa07-434e-82fe-86f1077ca2c4-maas-api-tls\") pod \"maas-api-e2e-ait-f861467d-7bff777d78-cj9n4\" (UID: \"f1877b54-aa07-434e-82fe-86f1077ca2c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f861467d-7bff777d78-cj9n4"
Jul 29 18:25:58.764140 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.764119    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1877b54-aa07-434e-82fe-86f1077ca2c4-openshift-service-ca\") pod \"maas-api-e2e-ait-f861467d-7bff777d78-cj9n4\" (UID: \"f1877b54-aa07-434e-82fe-86f1077ca2c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f861467d-7bff777d78-cj9n4"
Jul 29 18:25:58.766042 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.766022    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1877b54-aa07-434e-82fe-86f1077ca2c4-maas-api-tls\") pod \"maas-api-e2e-ait-f861467d-7bff777d78-cj9n4\" (UID: \"f1877b54-aa07-434e-82fe-86f1077ca2c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f861467d-7bff777d78-cj9n4"
Jul 29 18:25:58.774227 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.774208    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hm8ld\" (UniqueName: \"kubernetes.io/projected/f1877b54-aa07-434e-82fe-86f1077ca2c4-kube-api-access-hm8ld\") pod \"maas-api-e2e-ait-f861467d-7bff777d78-cj9n4\" (UID: \"f1877b54-aa07-434e-82fe-86f1077ca2c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f861467d-7bff777d78-cj9n4"
Jul 29 18:25:58.874617 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:58.874529    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f861467d-7bff777d78-cj9n4"
Jul 29 18:25:59.035749 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:25:59.035701    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1877b54_aa07_434e_82fe_86f1077ca2c4.slice/crio-c4649aaed41675e3e9b271185642c24d07ac0a399f0a04d0ebeec2cf5de1a2bc WatchSource:0}: Error finding container c4649aaed41675e3e9b271185642c24d07ac0a399f0a04d0ebeec2cf5de1a2bc: Status 404 returned error can't find the container with id c4649aaed41675e3e9b271185642c24d07ac0a399f0a04d0ebeec2cf5de1a2bc
Jul 29 18:25:59.035916 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:59.035891    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f861467d-7bff777d78-cj9n4"]
Jul 29 18:25:59.391080 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:59.391047    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f861467d-76f8894956-mz66d"]
Jul 29 18:25:59.391499 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:59.391486    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0407fda-71a1-435e-8904-b80c41e2a466" containerName="istio-proxy"
Jul 29 18:25:59.395830 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:59.395811    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-f861467d-76f8894956-mz66d"
Jul 29 18:25:59.399117 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:59.399098    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-f861467d-dockercfg-nhrcg\""
Jul 29 18:25:59.399517 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:59.399503    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-f861467d\""
Jul 29 18:25:59.409374 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:59.409354    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f861467d-76f8894956-mz66d"]
Jul 29 18:25:59.560157 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:59.560130    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f861467d-7bff777d78-cj9n4" event={"ID":"f1877b54-aa07-434e-82fe-86f1077ca2c4","Type":"ContainerStarted","Data":"cff30894eb92fe58ef54ec24280fc6949539aa55500fb53ed7ceba0fa95a766e"}
Jul 29 18:25:59.560298 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:59.560177    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f861467d-7bff777d78-cj9n4" event={"ID":"f1877b54-aa07-434e-82fe-86f1077ca2c4","Type":"ContainerStarted","Data":"c4649aaed41675e3e9b271185642c24d07ac0a399f0a04d0ebeec2cf5de1a2bc"}
Jul 29 18:25:59.560298 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:59.560268    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f861467d-7bff777d78-cj9n4"
Jul 29 18:25:59.570884 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:59.570862    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgdxt\" (UniqueName: \"kubernetes.io/projected/97b4b693-df1e-4c0b-8e5d-54b0e78c7b15-kube-api-access-wgdxt\") pod \"payload-pre-processing-e2e-ait-f861467d-76f8894956-mz66d\" (UID: \"97b4b693-df1e-4c0b-8e5d-54b0e78c7b15\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f861467d-76f8894956-mz66d"
Jul 29 18:25:59.570963 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:59.570900    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97b4b693-df1e-4c0b-8e5d-54b0e78c7b15-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-f861467d-76f8894956-mz66d\" (UID: \"97b4b693-df1e-4c0b-8e5d-54b0e78c7b15\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f861467d-76f8894956-mz66d"
Jul 29 18:25:59.585828 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:59.585770    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f861467d-7bff777d78-cj9n4" podStartSLOduration=1.157148805 podStartE2EDuration="1.585751914s" totalImagesPullingTime="428.603109ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:25:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:25:59.58150427 +0000 UTC m=+2039.450727080" watchObservedRunningTime="2026-07-29 18:25:59.585751914 +0000 UTC m=+2039.454974724"
Jul 29 18:25:59.672178 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:59.672081    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgdxt\" (UniqueName: \"kubernetes.io/projected/97b4b693-df1e-4c0b-8e5d-54b0e78c7b15-kube-api-access-wgdxt\") pod \"payload-pre-processing-e2e-ait-f861467d-76f8894956-mz66d\" (UID: \"97b4b693-df1e-4c0b-8e5d-54b0e78c7b15\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f861467d-76f8894956-mz66d"
Jul 29 18:25:59.672178 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:59.672132    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97b4b693-df1e-4c0b-8e5d-54b0e78c7b15-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-f861467d-76f8894956-mz66d\" (UID: \"97b4b693-df1e-4c0b-8e5d-54b0e78c7b15\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f861467d-76f8894956-mz66d"
Jul 29 18:25:59.672924 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:59.672898    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97b4b693-df1e-4c0b-8e5d-54b0e78c7b15-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-f861467d-76f8894956-mz66d\" (UID: \"97b4b693-df1e-4c0b-8e5d-54b0e78c7b15\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f861467d-76f8894956-mz66d"
Jul 29 18:25:59.680834 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:59.680812    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgdxt\" (UniqueName: \"kubernetes.io/projected/97b4b693-df1e-4c0b-8e5d-54b0e78c7b15-kube-api-access-wgdxt\") pod \"payload-pre-processing-e2e-ait-f861467d-76f8894956-mz66d\" (UID: \"97b4b693-df1e-4c0b-8e5d-54b0e78c7b15\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f861467d-76f8894956-mz66d"
Jul 29 18:25:59.707422 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:59.707390    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-f861467d-76f8894956-mz66d"
Jul 29 18:25:59.846224 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:25:59.846195    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f861467d-76f8894956-mz66d"]
Jul 29 18:25:59.847979 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:25:59.847938    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97b4b693_df1e_4c0b_8e5d_54b0e78c7b15.slice/crio-d596ebd61d4f0b2d7a51c0c192170915bcb97152c292dd742dcf8c83a1de7fda WatchSource:0}: Error finding container d596ebd61d4f0b2d7a51c0c192170915bcb97152c292dd742dcf8c83a1de7fda: Status 404 returned error can't find the container with id d596ebd61d4f0b2d7a51c0c192170915bcb97152c292dd742dcf8c83a1de7fda
Jul 29 18:26:00.567424 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:00.567392    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-f861467d-76f8894956-mz66d" event={"ID":"97b4b693-df1e-4c0b-8e5d-54b0e78c7b15","Type":"ContainerStarted","Data":"d20b5bf406ca399068a3088d4a60fd1d3b3ed35a8eefc7a94f2655ab2dd511bf"}
Jul 29 18:26:00.567863 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:00.567475    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-f861467d-76f8894956-mz66d" event={"ID":"97b4b693-df1e-4c0b-8e5d-54b0e78c7b15","Type":"ContainerStarted","Data":"d596ebd61d4f0b2d7a51c0c192170915bcb97152c292dd742dcf8c83a1de7fda"}
Jul 29 18:26:00.567863 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:00.567648    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-f861467d-76f8894956-mz66d"
Jul 29 18:26:00.591636 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:00.591586    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-f861467d-76f8894956-mz66d" podStartSLOduration=1.591569609 podStartE2EDuration="1.591569609s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:25:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:26:00.588908026 +0000 UTC m=+2040.458130835" watchObservedRunningTime="2026-07-29 18:26:00.591569609 +0000 UTC m=+2040.460792444"
Jul 29 18:26:05.572962 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:05.572932    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f861467d-7bff777d78-cj9n4"
Jul 29 18:26:12.580237 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:12.580201    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-f861467d-76f8894956-mz66d"
Jul 29 18:26:24.285842 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:24.285762    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f861467d-rg4rg"]
Jul 29 18:26:24.289575 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:24.289558    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f861467d-rg4rg"
Jul 29 18:26:24.292173 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:24.292148    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qrbhl\""
Jul 29 18:26:24.298244 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:24.296874    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f861467d-rg4rg"]
Jul 29 18:26:24.398237 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:24.398200    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e1550e9-2b0a-443d-9597-824bac89b0bc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f861467d-rg4rg\" (UID: \"0e1550e9-2b0a-443d-9597-824bac89b0bc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f861467d-rg4rg"
Jul 29 18:26:24.498781 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:24.498751    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e1550e9-2b0a-443d-9597-824bac89b0bc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f861467d-rg4rg\" (UID: \"0e1550e9-2b0a-443d-9597-824bac89b0bc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f861467d-rg4rg"
Jul 29 18:26:24.499389 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:24.499364    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e1550e9-2b0a-443d-9597-824bac89b0bc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f861467d-rg4rg\" (UID: \"0e1550e9-2b0a-443d-9597-824bac89b0bc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f861467d-rg4rg"
Jul 29 18:26:24.608247 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:24.608214    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f861467d-rg4rg"
Jul 29 18:26:24.746919 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:26:24.746878    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e1550e9_2b0a_443d_9597_824bac89b0bc.slice/crio-d80ee1cd5684097031f03c3c71562877f6be55922e6d8fba79e1391596c0a8a2 WatchSource:0}: Error finding container d80ee1cd5684097031f03c3c71562877f6be55922e6d8fba79e1391596c0a8a2: Status 404 returned error can't find the container with id d80ee1cd5684097031f03c3c71562877f6be55922e6d8fba79e1391596c0a8a2
Jul 29 18:26:24.747239 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:24.747215    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f861467d-rg4rg"]
Jul 29 18:26:25.710634 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:25.710535    2557 generic.go:356] "Generic (PLEG): container finished" podID="0e1550e9-2b0a-443d-9597-824bac89b0bc" containerID="bd4b818bedebd3d5415a627ca90d631c31da27979e13e562ca736c86589bfdf0" exitCode=0
Jul 29 18:26:25.710634 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:25.710619    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f861467d-rg4rg" event={"ID":"0e1550e9-2b0a-443d-9597-824bac89b0bc","Type":"ContainerDied","Data":"bd4b818bedebd3d5415a627ca90d631c31da27979e13e562ca736c86589bfdf0"}
Jul 29 18:26:25.711162 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:25.710672    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f861467d-rg4rg" event={"ID":"0e1550e9-2b0a-443d-9597-824bac89b0bc","Type":"ContainerStarted","Data":"d80ee1cd5684097031f03c3c71562877f6be55922e6d8fba79e1391596c0a8a2"}
Jul 29 18:26:26.849316 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:26.849287    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f861467d-rg4rg"
Jul 29 18:26:26.921177 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:26.921146    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e1550e9-2b0a-443d-9597-824bac89b0bc-maas-api-service-ca\") pod \"0e1550e9-2b0a-443d-9597-824bac89b0bc\" (UID: \"0e1550e9-2b0a-443d-9597-824bac89b0bc\") "
Jul 29 18:26:26.921470 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:26.921448    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e1550e9-2b0a-443d-9597-824bac89b0bc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0e1550e9-2b0a-443d-9597-824bac89b0bc" (UID: "0e1550e9-2b0a-443d-9597-824bac89b0bc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:26:27.022312 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:27.022242    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e1550e9-2b0a-443d-9597-824bac89b0bc-maas-api-service-ca\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:26:27.723010 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:27.722980    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f861467d-rg4rg"
Jul 29 18:26:27.723180 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:27.723014    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f861467d-rg4rg" event={"ID":"0e1550e9-2b0a-443d-9597-824bac89b0bc","Type":"ContainerDied","Data":"d80ee1cd5684097031f03c3c71562877f6be55922e6d8fba79e1391596c0a8a2"}
Jul 29 18:26:27.723180 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:27.723058    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d80ee1cd5684097031f03c3c71562877f6be55922e6d8fba79e1391596c0a8a2"
Jul 29 18:26:29.215247 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:29.215218    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f861467d-7bff777d78-cj9n4"]
Jul 29 18:26:29.215690 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:29.215414    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f861467d-7bff777d78-cj9n4" podUID="f1877b54-aa07-434e-82fe-86f1077ca2c4" containerName="maas-api" containerID="cri-o://cff30894eb92fe58ef54ec24280fc6949539aa55500fb53ed7ceba0fa95a766e" gracePeriod=30
Jul 29 18:26:29.276221 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:29.276189    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f861467d-76f8894956-mz66d"]
Jul 29 18:26:29.276466 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:29.276437    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-f861467d-76f8894956-mz66d" podUID="97b4b693-df1e-4c0b-8e5d-54b0e78c7b15" containerName="payload-pre-processing" containerID="cri-o://d20b5bf406ca399068a3088d4a60fd1d3b3ed35a8eefc7a94f2655ab2dd511bf" gracePeriod=30
Jul 29 18:26:29.410746 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:29.410700    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f861467d-rg4rg"]
Jul 29 18:26:29.415341 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:29.415313    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f861467d-rg4rg"]
Jul 29 18:26:29.544167 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:29.544138    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f861467d-7bff777d78-cj9n4"
Jul 29 18:26:29.561522 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:29.561499    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-f861467d-76f8894956-mz66d"
Jul 29 18:26:29.644490 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:29.644457    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wgdxt\" (UniqueName: \"kubernetes.io/projected/97b4b693-df1e-4c0b-8e5d-54b0e78c7b15-kube-api-access-wgdxt\") pod \"97b4b693-df1e-4c0b-8e5d-54b0e78c7b15\" (UID: \"97b4b693-df1e-4c0b-8e5d-54b0e78c7b15\") "
Jul 29 18:26:29.644647 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:29.644507    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1877b54-aa07-434e-82fe-86f1077ca2c4-openshift-service-ca\") pod \"f1877b54-aa07-434e-82fe-86f1077ca2c4\" (UID: \"f1877b54-aa07-434e-82fe-86f1077ca2c4\") "
Jul 29 18:26:29.644647 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:29.644537    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1877b54-aa07-434e-82fe-86f1077ca2c4-maas-api-tls\") pod \"f1877b54-aa07-434e-82fe-86f1077ca2c4\" (UID: \"f1877b54-aa07-434e-82fe-86f1077ca2c4\") "
Jul 29 18:26:29.644647 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:29.644587    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hm8ld\" (UniqueName: \"kubernetes.io/projected/f1877b54-aa07-434e-82fe-86f1077ca2c4-kube-api-access-hm8ld\") pod \"f1877b54-aa07-434e-82fe-86f1077ca2c4\" (UID: \"f1877b54-aa07-434e-82fe-86f1077ca2c4\") "
Jul 29 18:26:29.644647 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:29.644606    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97b4b693-df1e-4c0b-8e5d-54b0e78c7b15-plugins-config-volume\") pod \"97b4b693-df1e-4c0b-8e5d-54b0e78c7b15\" (UID: \"97b4b693-df1e-4c0b-8e5d-54b0e78c7b15\") "
Jul 29 18:26:29.645005 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:29.644975    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1877b54-aa07-434e-82fe-86f1077ca2c4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f1877b54-aa07-434e-82fe-86f1077ca2c4" (UID: "f1877b54-aa07-434e-82fe-86f1077ca2c4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:26:29.645126 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:29.645071    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97b4b693-df1e-4c0b-8e5d-54b0e78c7b15-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "97b4b693-df1e-4c0b-8e5d-54b0e78c7b15" (UID: "97b4b693-df1e-4c0b-8e5d-54b0e78c7b15"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:26:29.646825 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:29.646799    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1877b54-aa07-434e-82fe-86f1077ca2c4-kube-api-access-hm8ld" (OuterVolumeSpecName: "kube-api-access-hm8ld") pod "f1877b54-aa07-434e-82fe-86f1077ca2c4" (UID: "f1877b54-aa07-434e-82fe-86f1077ca2c4"). InnerVolumeSpecName "kube-api-access-hm8ld". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:26:29.646904 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:29.646840    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97b4b693-df1e-4c0b-8e5d-54b0e78c7b15-kube-api-access-wgdxt" (OuterVolumeSpecName: "kube-api-access-wgdxt") pod "97b4b693-df1e-4c0b-8e5d-54b0e78c7b15" (UID: "97b4b693-df1e-4c0b-8e5d-54b0e78c7b15"). InnerVolumeSpecName "kube-api-access-wgdxt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:26:29.647050 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:29.647031    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1877b54-aa07-434e-82fe-86f1077ca2c4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f1877b54-aa07-434e-82fe-86f1077ca2c4" (UID: "f1877b54-aa07-434e-82fe-86f1077ca2c4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:26:29.734648 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:29.734615    2557 generic.go:356] "Generic (PLEG): container finished" podID="f1877b54-aa07-434e-82fe-86f1077ca2c4" containerID="cff30894eb92fe58ef54ec24280fc6949539aa55500fb53ed7ceba0fa95a766e" exitCode=0
Jul 29 18:26:29.734809 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:29.734676    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f861467d-7bff777d78-cj9n4"
Jul 29 18:26:29.734809 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:29.734698    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f861467d-7bff777d78-cj9n4" event={"ID":"f1877b54-aa07-434e-82fe-86f1077ca2c4","Type":"ContainerDied","Data":"cff30894eb92fe58ef54ec24280fc6949539aa55500fb53ed7ceba0fa95a766e"}
Jul 29 18:26:29.734809 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:29.734784    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f861467d-7bff777d78-cj9n4" event={"ID":"f1877b54-aa07-434e-82fe-86f1077ca2c4","Type":"ContainerDied","Data":"c4649aaed41675e3e9b271185642c24d07ac0a399f0a04d0ebeec2cf5de1a2bc"}
Jul 29 18:26:29.734978 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:29.734816    2557 scope.go:117] "RemoveContainer" containerID="cff30894eb92fe58ef54ec24280fc6949539aa55500fb53ed7ceba0fa95a766e"
Jul 29 18:26:29.736436 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:29.736415    2557 generic.go:356] "Generic (PLEG): container finished" podID="97b4b693-df1e-4c0b-8e5d-54b0e78c7b15" containerID="d20b5bf406ca399068a3088d4a60fd1d3b3ed35a8eefc7a94f2655ab2dd511bf" exitCode=0
Jul 29 18:26:29.736579 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:29.736477    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-f861467d-76f8894956-mz66d"
Jul 29 18:26:29.736848 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:29.736494    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-f861467d-76f8894956-mz66d" event={"ID":"97b4b693-df1e-4c0b-8e5d-54b0e78c7b15","Type":"ContainerDied","Data":"d20b5bf406ca399068a3088d4a60fd1d3b3ed35a8eefc7a94f2655ab2dd511bf"}
Jul 29 18:26:29.736848 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:29.736797    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-f861467d-76f8894956-mz66d" event={"ID":"97b4b693-df1e-4c0b-8e5d-54b0e78c7b15","Type":"ContainerDied","Data":"d596ebd61d4f0b2d7a51c0c192170915bcb97152c292dd742dcf8c83a1de7fda"}
Jul 29 18:26:29.745442 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:29.745420    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1877b54-aa07-434e-82fe-86f1077ca2c4-openshift-service-ca\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:26:29.745442 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:29.745443    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1877b54-aa07-434e-82fe-86f1077ca2c4-maas-api-tls\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:26:29.745585 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:29.745452    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hm8ld\" (UniqueName: \"kubernetes.io/projected/f1877b54-aa07-434e-82fe-86f1077ca2c4-kube-api-access-hm8ld\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:26:29.745585 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:29.745460    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97b4b693-df1e-4c0b-8e5d-54b0e78c7b15-plugins-config-volume\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:26:29.745585 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:29.745469    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wgdxt\" (UniqueName: \"kubernetes.io/projected/97b4b693-df1e-4c0b-8e5d-54b0e78c7b15-kube-api-access-wgdxt\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:26:29.746238 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:29.746225    2557 scope.go:117] "RemoveContainer" containerID="cff30894eb92fe58ef54ec24280fc6949539aa55500fb53ed7ceba0fa95a766e"
Jul 29 18:26:29.746507 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:26:29.746486    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cff30894eb92fe58ef54ec24280fc6949539aa55500fb53ed7ceba0fa95a766e\": container with ID starting with cff30894eb92fe58ef54ec24280fc6949539aa55500fb53ed7ceba0fa95a766e not found: ID does not exist" containerID="cff30894eb92fe58ef54ec24280fc6949539aa55500fb53ed7ceba0fa95a766e"
Jul 29 18:26:29.746564 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:29.746516    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cff30894eb92fe58ef54ec24280fc6949539aa55500fb53ed7ceba0fa95a766e"} err="failed to get container status \"cff30894eb92fe58ef54ec24280fc6949539aa55500fb53ed7ceba0fa95a766e\": rpc error: code = NotFound desc = could not find container \"cff30894eb92fe58ef54ec24280fc6949539aa55500fb53ed7ceba0fa95a766e\": container with ID starting with cff30894eb92fe58ef54ec24280fc6949539aa55500fb53ed7ceba0fa95a766e not found: ID does not exist"
Jul 29 18:26:29.746564 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:29.746537    2557 scope.go:117] "RemoveContainer" containerID="d20b5bf406ca399068a3088d4a60fd1d3b3ed35a8eefc7a94f2655ab2dd511bf"
Jul 29 18:26:29.754916 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:29.754899    2557 scope.go:117] "RemoveContainer" containerID="d20b5bf406ca399068a3088d4a60fd1d3b3ed35a8eefc7a94f2655ab2dd511bf"
Jul 29 18:26:29.755150 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:26:29.755130    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d20b5bf406ca399068a3088d4a60fd1d3b3ed35a8eefc7a94f2655ab2dd511bf\": container with ID starting with d20b5bf406ca399068a3088d4a60fd1d3b3ed35a8eefc7a94f2655ab2dd511bf not found: ID does not exist" containerID="d20b5bf406ca399068a3088d4a60fd1d3b3ed35a8eefc7a94f2655ab2dd511bf"
Jul 29 18:26:29.755240 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:29.755154    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d20b5bf406ca399068a3088d4a60fd1d3b3ed35a8eefc7a94f2655ab2dd511bf"} err="failed to get container status \"d20b5bf406ca399068a3088d4a60fd1d3b3ed35a8eefc7a94f2655ab2dd511bf\": rpc error: code = NotFound desc = could not find container \"d20b5bf406ca399068a3088d4a60fd1d3b3ed35a8eefc7a94f2655ab2dd511bf\": container with ID starting with d20b5bf406ca399068a3088d4a60fd1d3b3ed35a8eefc7a94f2655ab2dd511bf not found: ID does not exist"
Jul 29 18:26:29.758657 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:29.758634    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f861467d-7bff777d78-cj9n4"]
Jul 29 18:26:29.762008 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:29.761988    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f861467d-7bff777d78-cj9n4"]
Jul 29 18:26:29.772160 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:29.772141    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f861467d-76f8894956-mz66d"]
Jul 29 18:26:29.775333 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:29.775312    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f861467d-76f8894956-mz66d"]
Jul 29 18:26:30.655878 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:30.655832    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e1550e9-2b0a-443d-9597-824bac89b0bc" path="/var/lib/kubelet/pods/0e1550e9-2b0a-443d-9597-824bac89b0bc/volumes"
Jul 29 18:26:30.656355 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:30.656336    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97b4b693-df1e-4c0b-8e5d-54b0e78c7b15" path="/var/lib/kubelet/pods/97b4b693-df1e-4c0b-8e5d-54b0e78c7b15/volumes"
Jul 29 18:26:30.656879 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:30.656861    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1877b54-aa07-434e-82fe-86f1077ca2c4" path="/var/lib/kubelet/pods/f1877b54-aa07-434e-82fe-86f1077ca2c4/volumes"
Jul 29 18:26:33.908667 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:33.908629    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-eb3353f1-57f8bdb897-bdqq8"]
Jul 29 18:26:33.910070 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:33.910044    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97b4b693-df1e-4c0b-8e5d-54b0e78c7b15" containerName="payload-pre-processing"
Jul 29 18:26:33.910070 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:33.910070    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="97b4b693-df1e-4c0b-8e5d-54b0e78c7b15" containerName="payload-pre-processing"
Jul 29 18:26:33.910282 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:33.910097    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1877b54-aa07-434e-82fe-86f1077ca2c4" containerName="maas-api"
Jul 29 18:26:33.910282 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:33.910107    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1877b54-aa07-434e-82fe-86f1077ca2c4" containerName="maas-api"
Jul 29 18:26:33.910282 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:33.910136    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e1550e9-2b0a-443d-9597-824bac89b0bc" containerName="revoke-keys"
Jul 29 18:26:33.910282 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:33.910144    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e1550e9-2b0a-443d-9597-824bac89b0bc" containerName="revoke-keys"
Jul 29 18:26:33.910486 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:33.910314    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="97b4b693-df1e-4c0b-8e5d-54b0e78c7b15" containerName="payload-pre-processing"
Jul 29 18:26:33.910486 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:33.910329    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e1550e9-2b0a-443d-9597-824bac89b0bc" containerName="revoke-keys"
Jul 29 18:26:33.910486 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:33.910348    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1877b54-aa07-434e-82fe-86f1077ca2c4" containerName="maas-api"
Jul 29 18:26:33.914374 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:33.914350    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-eb3353f1-57f8bdb897-bdqq8"
Jul 29 18:26:33.918869 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:33.918801    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-eb3353f1-serving-cert\""
Jul 29 18:26:33.923131 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:33.922593    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-eb3353f1-57f8bdb897-bdqq8"]
Jul 29 18:26:33.984469 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:33.984444    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3fd8a97c-0a72-4113-a209-352e0610f7f7-openshift-service-ca\") pod \"maas-api-e2e-derive-eb3353f1-57f8bdb897-bdqq8\" (UID: \"3fd8a97c-0a72-4113-a209-352e0610f7f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-eb3353f1-57f8bdb897-bdqq8"
Jul 29 18:26:33.984605 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:33.984513    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-244r7\" (UniqueName: \"kubernetes.io/projected/3fd8a97c-0a72-4113-a209-352e0610f7f7-kube-api-access-244r7\") pod \"maas-api-e2e-derive-eb3353f1-57f8bdb897-bdqq8\" (UID: \"3fd8a97c-0a72-4113-a209-352e0610f7f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-eb3353f1-57f8bdb897-bdqq8"
Jul 29 18:26:33.984605 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:33.984584    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3fd8a97c-0a72-4113-a209-352e0610f7f7-maas-api-tls\") pod \"maas-api-e2e-derive-eb3353f1-57f8bdb897-bdqq8\" (UID: \"3fd8a97c-0a72-4113-a209-352e0610f7f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-eb3353f1-57f8bdb897-bdqq8"
Jul 29 18:26:34.085585 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:34.085553    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3fd8a97c-0a72-4113-a209-352e0610f7f7-maas-api-tls\") pod \"maas-api-e2e-derive-eb3353f1-57f8bdb897-bdqq8\" (UID: \"3fd8a97c-0a72-4113-a209-352e0610f7f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-eb3353f1-57f8bdb897-bdqq8"
Jul 29 18:26:34.085771 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:34.085598    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3fd8a97c-0a72-4113-a209-352e0610f7f7-openshift-service-ca\") pod \"maas-api-e2e-derive-eb3353f1-57f8bdb897-bdqq8\" (UID: \"3fd8a97c-0a72-4113-a209-352e0610f7f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-eb3353f1-57f8bdb897-bdqq8"
Jul 29 18:26:34.085771 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:34.085661    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-244r7\" (UniqueName: \"kubernetes.io/projected/3fd8a97c-0a72-4113-a209-352e0610f7f7-kube-api-access-244r7\") pod \"maas-api-e2e-derive-eb3353f1-57f8bdb897-bdqq8\" (UID: \"3fd8a97c-0a72-4113-a209-352e0610f7f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-eb3353f1-57f8bdb897-bdqq8"
Jul 29 18:26:34.086420 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:34.086344    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3fd8a97c-0a72-4113-a209-352e0610f7f7-openshift-service-ca\") pod \"maas-api-e2e-derive-eb3353f1-57f8bdb897-bdqq8\" (UID: \"3fd8a97c-0a72-4113-a209-352e0610f7f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-eb3353f1-57f8bdb897-bdqq8"
Jul 29 18:26:34.088203 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:34.088181    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3fd8a97c-0a72-4113-a209-352e0610f7f7-maas-api-tls\") pod \"maas-api-e2e-derive-eb3353f1-57f8bdb897-bdqq8\" (UID: \"3fd8a97c-0a72-4113-a209-352e0610f7f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-eb3353f1-57f8bdb897-bdqq8"
Jul 29 18:26:34.096311 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:34.096286    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-244r7\" (UniqueName: \"kubernetes.io/projected/3fd8a97c-0a72-4113-a209-352e0610f7f7-kube-api-access-244r7\") pod \"maas-api-e2e-derive-eb3353f1-57f8bdb897-bdqq8\" (UID: \"3fd8a97c-0a72-4113-a209-352e0610f7f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-eb3353f1-57f8bdb897-bdqq8"
Jul 29 18:26:34.227284 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:34.227208    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-eb3353f1-57f8bdb897-bdqq8"
Jul 29 18:26:34.385352 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:34.385324    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-eb3353f1-57f8bdb897-bdqq8"]
Jul 29 18:26:34.386183 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:26:34.386150    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fd8a97c_0a72_4113_a209_352e0610f7f7.slice/crio-6c43c1b8fcbd7512ea385d9c09622ee6d07c052c192c3cf99f2ed2214ecec645 WatchSource:0}: Error finding container 6c43c1b8fcbd7512ea385d9c09622ee6d07c052c192c3cf99f2ed2214ecec645: Status 404 returned error can't find the container with id 6c43c1b8fcbd7512ea385d9c09622ee6d07c052c192c3cf99f2ed2214ecec645
Jul 29 18:26:34.644752 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:34.644704    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-eb3353f1-7dcb8f78c4-thq54"]
Jul 29 18:26:34.649507 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:34.649482    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-eb3353f1-7dcb8f78c4-thq54"
Jul 29 18:26:34.652460 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:34.652432    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-eb3353f1\""
Jul 29 18:26:34.652460 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:34.652456    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-eb3353f1-dockercfg-sxfnf\""
Jul 29 18:26:34.659389 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:34.659366    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-eb3353f1-7dcb8f78c4-thq54"]
Jul 29 18:26:34.693408 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:34.693385    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-eb3353f1-5f449c7d8-8qzrz"]
Jul 29 18:26:34.696838 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:34.696810    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-eb3353f1-5f449c7d8-8qzrz"
Jul 29 18:26:34.707971 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:34.707950    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-eb3353f1-5f449c7d8-8qzrz"]
Jul 29 18:26:34.770592 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:34.770560    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-eb3353f1-57f8bdb897-bdqq8" event={"ID":"3fd8a97c-0a72-4113-a209-352e0610f7f7","Type":"ContainerStarted","Data":"6c43c1b8fcbd7512ea385d9c09622ee6d07c052c192c3cf99f2ed2214ecec645"}
Jul 29 18:26:34.794223 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:34.794183    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnfgp\" (UniqueName: \"kubernetes.io/projected/9df5fe00-0767-4f61-b2f0-2db33cc956ac-kube-api-access-wnfgp\") pod \"payload-pre-processing-e2e-derive-eb3353f1-5f449c7d8-8qzrz\" (UID: \"9df5fe00-0767-4f61-b2f0-2db33cc956ac\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-eb3353f1-5f449c7d8-8qzrz"
Jul 29 18:26:34.794370 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:34.794321    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9df5fe00-0767-4f61-b2f0-2db33cc956ac-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-eb3353f1-5f449c7d8-8qzrz\" (UID: \"9df5fe00-0767-4f61-b2f0-2db33cc956ac\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-eb3353f1-5f449c7d8-8qzrz"
Jul 29 18:26:34.794429 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:34.794401    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/030487b5-3d02-43d7-bff3-d2527b2d43b9-plugins-config-volume\") pod \"payload-processing-e2e-derive-eb3353f1-7dcb8f78c4-thq54\" (UID: \"030487b5-3d02-43d7-bff3-d2527b2d43b9\") " pod="openshift-ingress/payload-processing-e2e-derive-eb3353f1-7dcb8f78c4-thq54"
Jul 29 18:26:34.794487 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:34.794443    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skprv\" (UniqueName: \"kubernetes.io/projected/030487b5-3d02-43d7-bff3-d2527b2d43b9-kube-api-access-skprv\") pod \"payload-processing-e2e-derive-eb3353f1-7dcb8f78c4-thq54\" (UID: \"030487b5-3d02-43d7-bff3-d2527b2d43b9\") " pod="openshift-ingress/payload-processing-e2e-derive-eb3353f1-7dcb8f78c4-thq54"
Jul 29 18:26:34.895315 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:34.895255    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9df5fe00-0767-4f61-b2f0-2db33cc956ac-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-eb3353f1-5f449c7d8-8qzrz\" (UID: \"9df5fe00-0767-4f61-b2f0-2db33cc956ac\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-eb3353f1-5f449c7d8-8qzrz"
Jul 29 18:26:34.895315 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:34.895297    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/030487b5-3d02-43d7-bff3-d2527b2d43b9-plugins-config-volume\") pod \"payload-processing-e2e-derive-eb3353f1-7dcb8f78c4-thq54\" (UID: \"030487b5-3d02-43d7-bff3-d2527b2d43b9\") " pod="openshift-ingress/payload-processing-e2e-derive-eb3353f1-7dcb8f78c4-thq54"
Jul 29 18:26:34.895464 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:34.895321    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skprv\" (UniqueName: \"kubernetes.io/projected/030487b5-3d02-43d7-bff3-d2527b2d43b9-kube-api-access-skprv\") pod \"payload-processing-e2e-derive-eb3353f1-7dcb8f78c4-thq54\" (UID: \"030487b5-3d02-43d7-bff3-d2527b2d43b9\") " pod="openshift-ingress/payload-processing-e2e-derive-eb3353f1-7dcb8f78c4-thq54"
Jul 29 18:26:34.895464 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:34.895367    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnfgp\" (UniqueName: \"kubernetes.io/projected/9df5fe00-0767-4f61-b2f0-2db33cc956ac-kube-api-access-wnfgp\") pod \"payload-pre-processing-e2e-derive-eb3353f1-5f449c7d8-8qzrz\" (UID: \"9df5fe00-0767-4f61-b2f0-2db33cc956ac\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-eb3353f1-5f449c7d8-8qzrz"
Jul 29 18:26:34.896061 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:34.896039    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9df5fe00-0767-4f61-b2f0-2db33cc956ac-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-eb3353f1-5f449c7d8-8qzrz\" (UID: \"9df5fe00-0767-4f61-b2f0-2db33cc956ac\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-eb3353f1-5f449c7d8-8qzrz"
Jul 29 18:26:34.896118 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:34.896046    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/030487b5-3d02-43d7-bff3-d2527b2d43b9-plugins-config-volume\") pod \"payload-processing-e2e-derive-eb3353f1-7dcb8f78c4-thq54\" (UID: \"030487b5-3d02-43d7-bff3-d2527b2d43b9\") " pod="openshift-ingress/payload-processing-e2e-derive-eb3353f1-7dcb8f78c4-thq54"
Jul 29 18:26:34.903676 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:34.903655    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnfgp\" (UniqueName: \"kubernetes.io/projected/9df5fe00-0767-4f61-b2f0-2db33cc956ac-kube-api-access-wnfgp\") pod \"payload-pre-processing-e2e-derive-eb3353f1-5f449c7d8-8qzrz\" (UID: \"9df5fe00-0767-4f61-b2f0-2db33cc956ac\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-eb3353f1-5f449c7d8-8qzrz"
Jul 29 18:26:34.904807 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:34.904788    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-skprv\" (UniqueName: \"kubernetes.io/projected/030487b5-3d02-43d7-bff3-d2527b2d43b9-kube-api-access-skprv\") pod \"payload-processing-e2e-derive-eb3353f1-7dcb8f78c4-thq54\" (UID: \"030487b5-3d02-43d7-bff3-d2527b2d43b9\") " pod="openshift-ingress/payload-processing-e2e-derive-eb3353f1-7dcb8f78c4-thq54"
Jul 29 18:26:34.962264 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:34.962241    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-eb3353f1-7dcb8f78c4-thq54"
Jul 29 18:26:35.007409 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:35.007369    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-eb3353f1-5f449c7d8-8qzrz"
Jul 29 18:26:35.146234 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:35.146153    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-eb3353f1-7dcb8f78c4-thq54"]
Jul 29 18:26:35.147565 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:26:35.147527    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod030487b5_3d02_43d7_bff3_d2527b2d43b9.slice/crio-312c4a7545e0ccf54e27bb4446cd3ef92541ee18caaddac289b23934a523bed0 WatchSource:0}: Error finding container 312c4a7545e0ccf54e27bb4446cd3ef92541ee18caaddac289b23934a523bed0: Status 404 returned error can't find the container with id 312c4a7545e0ccf54e27bb4446cd3ef92541ee18caaddac289b23934a523bed0
Jul 29 18:26:35.189312 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:35.189282    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-eb3353f1-5f449c7d8-8qzrz"]
Jul 29 18:26:35.193259 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:26:35.193224    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9df5fe00_0767_4f61_b2f0_2db33cc956ac.slice/crio-d1cba0814e64dde88ec06f74eb31fec180e3b8f8d670759d343653a831da4605 WatchSource:0}: Error finding container d1cba0814e64dde88ec06f74eb31fec180e3b8f8d670759d343653a831da4605: Status 404 returned error can't find the container with id d1cba0814e64dde88ec06f74eb31fec180e3b8f8d670759d343653a831da4605
Jul 29 18:26:35.777453 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:35.777416    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-eb3353f1-5f449c7d8-8qzrz" event={"ID":"9df5fe00-0767-4f61-b2f0-2db33cc956ac","Type":"ContainerStarted","Data":"84df92e02f204bc514aa4652e5869e2a4f810cc3a97a6aab3eccb1a04d1c37e8"}
Jul 29 18:26:35.777670 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:35.777478    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-eb3353f1-5f449c7d8-8qzrz" event={"ID":"9df5fe00-0767-4f61-b2f0-2db33cc956ac","Type":"ContainerStarted","Data":"d1cba0814e64dde88ec06f74eb31fec180e3b8f8d670759d343653a831da4605"}
Jul 29 18:26:35.777670 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:35.777506    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-eb3353f1-5f449c7d8-8qzrz"
Jul 29 18:26:35.779115 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:35.779083    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-eb3353f1-57f8bdb897-bdqq8" event={"ID":"3fd8a97c-0a72-4113-a209-352e0610f7f7","Type":"ContainerStarted","Data":"95862c1210be8a3dedcb246920b1fbebd0355100089944cc0c2e2ac4b96c7950"}
Jul 29 18:26:35.779244 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:35.779146    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-eb3353f1-57f8bdb897-bdqq8"
Jul 29 18:26:35.780629 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:35.780603    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-eb3353f1-7dcb8f78c4-thq54" event={"ID":"030487b5-3d02-43d7-bff3-d2527b2d43b9","Type":"ContainerStarted","Data":"212ce7e28e069c3eba4dffe1ddd07e61e6c88a7cbecd6b7c6055e450e5d68d0e"}
Jul 29 18:26:35.780760 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:35.780651    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-eb3353f1-7dcb8f78c4-thq54" event={"ID":"030487b5-3d02-43d7-bff3-d2527b2d43b9","Type":"ContainerStarted","Data":"312c4a7545e0ccf54e27bb4446cd3ef92541ee18caaddac289b23934a523bed0"}
Jul 29 18:26:35.780760 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:35.780702    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-eb3353f1-7dcb8f78c4-thq54"
Jul 29 18:26:35.793831 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:35.793779    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-eb3353f1-5f449c7d8-8qzrz" podStartSLOduration=1.7937642889999998 podStartE2EDuration="1.793764289s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:26:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:26:35.793413667 +0000 UTC m=+2075.662636470" watchObservedRunningTime="2026-07-29 18:26:35.793764289 +0000 UTC m=+2075.662987097"
Jul 29 18:26:35.812621 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:35.812580    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-eb3353f1-7dcb8f78c4-thq54" podStartSLOduration=1.812569216 podStartE2EDuration="1.812569216s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:26:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:26:35.811009196 +0000 UTC m=+2075.680232004" watchObservedRunningTime="2026-07-29 18:26:35.812569216 +0000 UTC m=+2075.681792062"
Jul 29 18:26:35.830715 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:35.830677    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-eb3353f1-57f8bdb897-bdqq8" podStartSLOduration=2.343550733 podStartE2EDuration="2.830666814s" totalImagesPullingTime="487.116081ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:26:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:26:35.828206293 +0000 UTC m=+2075.697429103" watchObservedRunningTime="2026-07-29 18:26:35.830666814 +0000 UTC m=+2075.699889620"
Jul 29 18:26:41.793000 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:41.792970    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-eb3353f1-57f8bdb897-bdqq8"
Jul 29 18:26:46.789951 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:46.789921    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-eb3353f1-7dcb8f78c4-thq54"
Jul 29 18:26:46.790383 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:26:46.789999    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-eb3353f1-5f449c7d8-8qzrz"
Jul 29 18:27:00.798854 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:00.798824    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gg2wj_2bf0bb71-4047-45a7-bd8e-5d6efabd26bd/console-operator/2.log"
Jul 29 18:27:00.803801 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:00.803767    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gg2wj_2bf0bb71-4047-45a7-bd8e-5d6efabd26bd/console-operator/2.log"
Jul 29 18:27:04.295979 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:04.295947    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-eb3353f1-57f8bdb897-bdqq8"]
Jul 29 18:27:04.296417 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:04.296161    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-eb3353f1-57f8bdb897-bdqq8" podUID="3fd8a97c-0a72-4113-a209-352e0610f7f7" containerName="maas-api" containerID="cri-o://95862c1210be8a3dedcb246920b1fbebd0355100089944cc0c2e2ac4b96c7950" gracePeriod=30
Jul 29 18:27:04.355878 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:04.355844    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-eb3353f1-7dcb8f78c4-thq54"]
Jul 29 18:27:04.356146 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:04.356091    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-eb3353f1-7dcb8f78c4-thq54" podUID="030487b5-3d02-43d7-bff3-d2527b2d43b9" containerName="payload-processing" containerID="cri-o://212ce7e28e069c3eba4dffe1ddd07e61e6c88a7cbecd6b7c6055e450e5d68d0e" gracePeriod=30
Jul 29 18:27:04.360534 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:04.360509    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-eb3353f1-5f449c7d8-8qzrz"]
Jul 29 18:27:04.361158 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:04.361120    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-eb3353f1-5f449c7d8-8qzrz" podUID="9df5fe00-0767-4f61-b2f0-2db33cc956ac" containerName="payload-pre-processing" containerID="cri-o://84df92e02f204bc514aa4652e5869e2a4f810cc3a97a6aab3eccb1a04d1c37e8" gracePeriod=30
Jul 29 18:27:04.632605 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:04.632564    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-eb3353f1-57f8bdb897-bdqq8"
Jul 29 18:27:04.663698 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:04.663674    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3fd8a97c-0a72-4113-a209-352e0610f7f7-maas-api-tls\") pod \"3fd8a97c-0a72-4113-a209-352e0610f7f7\" (UID: \"3fd8a97c-0a72-4113-a209-352e0610f7f7\") "
Jul 29 18:27:04.663886 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:04.663766    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3fd8a97c-0a72-4113-a209-352e0610f7f7-openshift-service-ca\") pod \"3fd8a97c-0a72-4113-a209-352e0610f7f7\" (UID: \"3fd8a97c-0a72-4113-a209-352e0610f7f7\") "
Jul 29 18:27:04.663886 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:04.663797    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-244r7\" (UniqueName: \"kubernetes.io/projected/3fd8a97c-0a72-4113-a209-352e0610f7f7-kube-api-access-244r7\") pod \"3fd8a97c-0a72-4113-a209-352e0610f7f7\" (UID: \"3fd8a97c-0a72-4113-a209-352e0610f7f7\") "
Jul 29 18:27:04.664166 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:04.664142    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3fd8a97c-0a72-4113-a209-352e0610f7f7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3fd8a97c-0a72-4113-a209-352e0610f7f7" (UID: "3fd8a97c-0a72-4113-a209-352e0610f7f7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:27:04.664452 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:04.664426    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3fd8a97c-0a72-4113-a209-352e0610f7f7-openshift-service-ca\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:27:04.666183 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:04.666160    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3fd8a97c-0a72-4113-a209-352e0610f7f7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3fd8a97c-0a72-4113-a209-352e0610f7f7" (UID: "3fd8a97c-0a72-4113-a209-352e0610f7f7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:27:04.666499 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:04.666475    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fd8a97c-0a72-4113-a209-352e0610f7f7-kube-api-access-244r7" (OuterVolumeSpecName: "kube-api-access-244r7") pod "3fd8a97c-0a72-4113-a209-352e0610f7f7" (UID: "3fd8a97c-0a72-4113-a209-352e0610f7f7"). InnerVolumeSpecName "kube-api-access-244r7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:27:04.668091 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:04.667584    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-eb3353f1-7dcb8f78c4-thq54"
Jul 29 18:27:04.683692 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:04.683672    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-eb3353f1-5f449c7d8-8qzrz"
Jul 29 18:27:04.765384 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:04.765344    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9df5fe00-0767-4f61-b2f0-2db33cc956ac-plugins-config-volume\") pod \"9df5fe00-0767-4f61-b2f0-2db33cc956ac\" (UID: \"9df5fe00-0767-4f61-b2f0-2db33cc956ac\") "
Jul 29 18:27:04.765591 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:04.765428    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-skprv\" (UniqueName: \"kubernetes.io/projected/030487b5-3d02-43d7-bff3-d2527b2d43b9-kube-api-access-skprv\") pod \"030487b5-3d02-43d7-bff3-d2527b2d43b9\" (UID: \"030487b5-3d02-43d7-bff3-d2527b2d43b9\") "
Jul 29 18:27:04.765591 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:04.765473    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/030487b5-3d02-43d7-bff3-d2527b2d43b9-plugins-config-volume\") pod \"030487b5-3d02-43d7-bff3-d2527b2d43b9\" (UID: \"030487b5-3d02-43d7-bff3-d2527b2d43b9\") "
Jul 29 18:27:04.765591 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:04.765546    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wnfgp\" (UniqueName: \"kubernetes.io/projected/9df5fe00-0767-4f61-b2f0-2db33cc956ac-kube-api-access-wnfgp\") pod \"9df5fe00-0767-4f61-b2f0-2db33cc956ac\" (UID: \"9df5fe00-0767-4f61-b2f0-2db33cc956ac\") "
Jul 29 18:27:04.765783 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:04.765756    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9df5fe00-0767-4f61-b2f0-2db33cc956ac-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9df5fe00-0767-4f61-b2f0-2db33cc956ac" (UID: "9df5fe00-0767-4f61-b2f0-2db33cc956ac"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:27:04.766204 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:04.766169    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/030487b5-3d02-43d7-bff3-d2527b2d43b9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "030487b5-3d02-43d7-bff3-d2527b2d43b9" (UID: "030487b5-3d02-43d7-bff3-d2527b2d43b9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:27:04.766340 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:04.766212    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3fd8a97c-0a72-4113-a209-352e0610f7f7-maas-api-tls\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:27:04.766340 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:04.766235    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9df5fe00-0767-4f61-b2f0-2db33cc956ac-plugins-config-volume\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:27:04.766340 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:04.766250    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-244r7\" (UniqueName: \"kubernetes.io/projected/3fd8a97c-0a72-4113-a209-352e0610f7f7-kube-api-access-244r7\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:27:04.767892 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:04.767868    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9df5fe00-0767-4f61-b2f0-2db33cc956ac-kube-api-access-wnfgp" (OuterVolumeSpecName: "kube-api-access-wnfgp") pod "9df5fe00-0767-4f61-b2f0-2db33cc956ac" (UID: "9df5fe00-0767-4f61-b2f0-2db33cc956ac"). InnerVolumeSpecName "kube-api-access-wnfgp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:27:04.767892 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:04.767880    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/030487b5-3d02-43d7-bff3-d2527b2d43b9-kube-api-access-skprv" (OuterVolumeSpecName: "kube-api-access-skprv") pod "030487b5-3d02-43d7-bff3-d2527b2d43b9" (UID: "030487b5-3d02-43d7-bff3-d2527b2d43b9"). InnerVolumeSpecName "kube-api-access-skprv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:27:04.866879 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:04.866819    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-skprv\" (UniqueName: \"kubernetes.io/projected/030487b5-3d02-43d7-bff3-d2527b2d43b9-kube-api-access-skprv\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:27:04.866879 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:04.866841    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/030487b5-3d02-43d7-bff3-d2527b2d43b9-plugins-config-volume\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:27:04.866879 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:04.866850    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wnfgp\" (UniqueName: \"kubernetes.io/projected/9df5fe00-0767-4f61-b2f0-2db33cc956ac-kube-api-access-wnfgp\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:27:04.945990 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:04.945962    2557 generic.go:356] "Generic (PLEG): container finished" podID="030487b5-3d02-43d7-bff3-d2527b2d43b9" containerID="212ce7e28e069c3eba4dffe1ddd07e61e6c88a7cbecd6b7c6055e450e5d68d0e" exitCode=0
Jul 29 18:27:04.946127 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:04.946016    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-eb3353f1-7dcb8f78c4-thq54"
Jul 29 18:27:04.946127 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:04.946053    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-eb3353f1-7dcb8f78c4-thq54" event={"ID":"030487b5-3d02-43d7-bff3-d2527b2d43b9","Type":"ContainerDied","Data":"212ce7e28e069c3eba4dffe1ddd07e61e6c88a7cbecd6b7c6055e450e5d68d0e"}
Jul 29 18:27:04.946253 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:04.946147    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-eb3353f1-7dcb8f78c4-thq54" event={"ID":"030487b5-3d02-43d7-bff3-d2527b2d43b9","Type":"ContainerDied","Data":"312c4a7545e0ccf54e27bb4446cd3ef92541ee18caaddac289b23934a523bed0"}
Jul 29 18:27:04.946253 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:04.946181    2557 scope.go:117] "RemoveContainer" containerID="212ce7e28e069c3eba4dffe1ddd07e61e6c88a7cbecd6b7c6055e450e5d68d0e"
Jul 29 18:27:04.947533 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:04.947508    2557 generic.go:356] "Generic (PLEG): container finished" podID="9df5fe00-0767-4f61-b2f0-2db33cc956ac" containerID="84df92e02f204bc514aa4652e5869e2a4f810cc3a97a6aab3eccb1a04d1c37e8" exitCode=0
Jul 29 18:27:04.947662 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:04.947565    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-eb3353f1-5f449c7d8-8qzrz"
Jul 29 18:27:04.947662 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:04.947600    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-eb3353f1-5f449c7d8-8qzrz" event={"ID":"9df5fe00-0767-4f61-b2f0-2db33cc956ac","Type":"ContainerDied","Data":"84df92e02f204bc514aa4652e5869e2a4f810cc3a97a6aab3eccb1a04d1c37e8"}
Jul 29 18:27:04.947662 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:04.947640    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-eb3353f1-5f449c7d8-8qzrz" event={"ID":"9df5fe00-0767-4f61-b2f0-2db33cc956ac","Type":"ContainerDied","Data":"d1cba0814e64dde88ec06f74eb31fec180e3b8f8d670759d343653a831da4605"}
Jul 29 18:27:04.949168 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:04.949144    2557 generic.go:356] "Generic (PLEG): container finished" podID="3fd8a97c-0a72-4113-a209-352e0610f7f7" containerID="95862c1210be8a3dedcb246920b1fbebd0355100089944cc0c2e2ac4b96c7950" exitCode=0
Jul 29 18:27:04.949292 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:04.949178    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-eb3353f1-57f8bdb897-bdqq8" event={"ID":"3fd8a97c-0a72-4113-a209-352e0610f7f7","Type":"ContainerDied","Data":"95862c1210be8a3dedcb246920b1fbebd0355100089944cc0c2e2ac4b96c7950"}
Jul 29 18:27:04.949292 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:04.949204    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-eb3353f1-57f8bdb897-bdqq8"
Jul 29 18:27:04.949292 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:04.949214    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-eb3353f1-57f8bdb897-bdqq8" event={"ID":"3fd8a97c-0a72-4113-a209-352e0610f7f7","Type":"ContainerDied","Data":"6c43c1b8fcbd7512ea385d9c09622ee6d07c052c192c3cf99f2ed2214ecec645"}
Jul 29 18:27:04.957770 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:04.957754    2557 scope.go:117] "RemoveContainer" containerID="212ce7e28e069c3eba4dffe1ddd07e61e6c88a7cbecd6b7c6055e450e5d68d0e"
Jul 29 18:27:04.958027 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:27:04.958008    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"212ce7e28e069c3eba4dffe1ddd07e61e6c88a7cbecd6b7c6055e450e5d68d0e\": container with ID starting with 212ce7e28e069c3eba4dffe1ddd07e61e6c88a7cbecd6b7c6055e450e5d68d0e not found: ID does not exist" containerID="212ce7e28e069c3eba4dffe1ddd07e61e6c88a7cbecd6b7c6055e450e5d68d0e"
Jul 29 18:27:04.958111 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:04.958039    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"212ce7e28e069c3eba4dffe1ddd07e61e6c88a7cbecd6b7c6055e450e5d68d0e"} err="failed to get container status \"212ce7e28e069c3eba4dffe1ddd07e61e6c88a7cbecd6b7c6055e450e5d68d0e\": rpc error: code = NotFound desc = could not find container \"212ce7e28e069c3eba4dffe1ddd07e61e6c88a7cbecd6b7c6055e450e5d68d0e\": container with ID starting with 212ce7e28e069c3eba4dffe1ddd07e61e6c88a7cbecd6b7c6055e450e5d68d0e not found: ID does not exist"
Jul 29 18:27:04.958111 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:04.958060    2557 scope.go:117] "RemoveContainer" containerID="84df92e02f204bc514aa4652e5869e2a4f810cc3a97a6aab3eccb1a04d1c37e8"
Jul 29 18:27:04.966674 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:04.966658    2557 scope.go:117] "RemoveContainer" containerID="84df92e02f204bc514aa4652e5869e2a4f810cc3a97a6aab3eccb1a04d1c37e8"
Jul 29 18:27:04.966944 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:27:04.966921    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84df92e02f204bc514aa4652e5869e2a4f810cc3a97a6aab3eccb1a04d1c37e8\": container with ID starting with 84df92e02f204bc514aa4652e5869e2a4f810cc3a97a6aab3eccb1a04d1c37e8 not found: ID does not exist" containerID="84df92e02f204bc514aa4652e5869e2a4f810cc3a97a6aab3eccb1a04d1c37e8"
Jul 29 18:27:04.967053 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:04.966947    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84df92e02f204bc514aa4652e5869e2a4f810cc3a97a6aab3eccb1a04d1c37e8"} err="failed to get container status \"84df92e02f204bc514aa4652e5869e2a4f810cc3a97a6aab3eccb1a04d1c37e8\": rpc error: code = NotFound desc = could not find container \"84df92e02f204bc514aa4652e5869e2a4f810cc3a97a6aab3eccb1a04d1c37e8\": container with ID starting with 84df92e02f204bc514aa4652e5869e2a4f810cc3a97a6aab3eccb1a04d1c37e8 not found: ID does not exist"
Jul 29 18:27:04.967053 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:04.966962    2557 scope.go:117] "RemoveContainer" containerID="95862c1210be8a3dedcb246920b1fbebd0355100089944cc0c2e2ac4b96c7950"
Jul 29 18:27:04.974049 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:04.974029    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-eb3353f1-7dcb8f78c4-thq54"]
Jul 29 18:27:04.975909 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:04.975893    2557 scope.go:117] "RemoveContainer" containerID="95862c1210be8a3dedcb246920b1fbebd0355100089944cc0c2e2ac4b96c7950"
Jul 29 18:27:04.976135 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:27:04.976114    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95862c1210be8a3dedcb246920b1fbebd0355100089944cc0c2e2ac4b96c7950\": container with ID starting with 95862c1210be8a3dedcb246920b1fbebd0355100089944cc0c2e2ac4b96c7950 not found: ID does not exist" containerID="95862c1210be8a3dedcb246920b1fbebd0355100089944cc0c2e2ac4b96c7950"
Jul 29 18:27:04.976188 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:04.976141    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95862c1210be8a3dedcb246920b1fbebd0355100089944cc0c2e2ac4b96c7950"} err="failed to get container status \"95862c1210be8a3dedcb246920b1fbebd0355100089944cc0c2e2ac4b96c7950\": rpc error: code = NotFound desc = could not find container \"95862c1210be8a3dedcb246920b1fbebd0355100089944cc0c2e2ac4b96c7950\": container with ID starting with 95862c1210be8a3dedcb246920b1fbebd0355100089944cc0c2e2ac4b96c7950 not found: ID does not exist"
Jul 29 18:27:04.977910 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:04.977893    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-eb3353f1-7dcb8f78c4-thq54"]
Jul 29 18:27:04.988673 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:04.988649    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-eb3353f1-5f449c7d8-8qzrz"]
Jul 29 18:27:04.991984 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:04.991965    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-eb3353f1-5f449c7d8-8qzrz"]
Jul 29 18:27:05.001991 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:05.001972    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-eb3353f1-57f8bdb897-bdqq8"]
Jul 29 18:27:05.005808 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:05.005787    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-eb3353f1-57f8bdb897-bdqq8"]
Jul 29 18:27:06.653383 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:06.653349    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="030487b5-3d02-43d7-bff3-d2527b2d43b9" path="/var/lib/kubelet/pods/030487b5-3d02-43d7-bff3-d2527b2d43b9/volumes"
Jul 29 18:27:06.653779 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:06.653751    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3fd8a97c-0a72-4113-a209-352e0610f7f7" path="/var/lib/kubelet/pods/3fd8a97c-0a72-4113-a209-352e0610f7f7/volumes"
Jul 29 18:27:06.654076 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:27:06.654062    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9df5fe00-0767-4f61-b2f0-2db33cc956ac" path="/var/lib/kubelet/pods/9df5fe00-0767-4f61-b2f0-2db33cc956ac/volumes"
Jul 29 18:30:00.143060 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:30:00.143019    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755830-fcch4"]
Jul 29 18:30:00.143565 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:30:00.143413    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="030487b5-3d02-43d7-bff3-d2527b2d43b9" containerName="payload-processing"
Jul 29 18:30:00.143565 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:30:00.143424    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="030487b5-3d02-43d7-bff3-d2527b2d43b9" containerName="payload-processing"
Jul 29 18:30:00.143565 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:30:00.143440    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9df5fe00-0767-4f61-b2f0-2db33cc956ac" containerName="payload-pre-processing"
Jul 29 18:30:00.143565 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:30:00.143446    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="9df5fe00-0767-4f61-b2f0-2db33cc956ac" containerName="payload-pre-processing"
Jul 29 18:30:00.143565 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:30:00.143459    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fd8a97c-0a72-4113-a209-352e0610f7f7" containerName="maas-api"
Jul 29 18:30:00.143565 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:30:00.143465    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fd8a97c-0a72-4113-a209-352e0610f7f7" containerName="maas-api"
Jul 29 18:30:00.143565 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:30:00.143524    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="030487b5-3d02-43d7-bff3-d2527b2d43b9" containerName="payload-processing"
Jul 29 18:30:00.143565 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:30:00.143533    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fd8a97c-0a72-4113-a209-352e0610f7f7" containerName="maas-api"
Jul 29 18:30:00.143565 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:30:00.143549    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="9df5fe00-0767-4f61-b2f0-2db33cc956ac" containerName="payload-pre-processing"
Jul 29 18:30:00.146637 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:30:00.146620    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755830-fcch4"
Jul 29 18:30:00.149382 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:30:00.149362    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qrbhl\""
Jul 29 18:30:00.156498 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:30:00.156480    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755830-fcch4"
Jul 29 18:30:00.166673 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:30:00.166651    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755830-fcch4"]
Jul 29 18:30:00.296921 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:30:00.296893    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755830-fcch4"]
Jul 29 18:30:00.298498 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:30:00.298473    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e8a09f2_7bd4_4fcf_a517_7819546f9fd4.slice/crio-c253e49a8f139457ecae289f871adabb9097d101dc029d45e5345055b27b62af WatchSource:0}: Error finding container c253e49a8f139457ecae289f871adabb9097d101dc029d45e5345055b27b62af: Status 404 returned error can't find the container with id c253e49a8f139457ecae289f871adabb9097d101dc029d45e5345055b27b62af
Jul 29 18:30:00.934860 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:30:00.934767    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755830-fcch4" event={"ID":"7e8a09f2-7bd4-4fcf-a517-7819546f9fd4","Type":"ContainerStarted","Data":"c6a529fa1d5fcb087f9e514f7a9b963ae400c101603d9c757515d1fe1201a8cd"}
Jul 29 18:30:00.934860 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:30:00.934844    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755830-fcch4" event={"ID":"7e8a09f2-7bd4-4fcf-a517-7819546f9fd4","Type":"ContainerStarted","Data":"c253e49a8f139457ecae289f871adabb9097d101dc029d45e5345055b27b62af"}
Jul 29 18:30:00.950625 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:30:00.950567    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755830-fcch4" podStartSLOduration=0.950520021 podStartE2EDuration="950.520021ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:30:00.94974324 +0000 UTC m=+2280.818966039" watchObservedRunningTime="2026-07-29 18:30:00.950520021 +0000 UTC m=+2280.819742828"
Jul 29 18:30:01.944062 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:30:01.944022    2557 generic.go:356] "Generic (PLEG): container finished" podID="7e8a09f2-7bd4-4fcf-a517-7819546f9fd4" containerID="c6a529fa1d5fcb087f9e514f7a9b963ae400c101603d9c757515d1fe1201a8cd" exitCode=0
Jul 29 18:30:01.944443 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:30:01.944111    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755830-fcch4" event={"ID":"7e8a09f2-7bd4-4fcf-a517-7819546f9fd4","Type":"ContainerDied","Data":"c6a529fa1d5fcb087f9e514f7a9b963ae400c101603d9c757515d1fe1201a8cd"}
Jul 29 18:30:03.091811 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:30:03.091784    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755830-fcch4"
Jul 29 18:30:03.958087 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:30:03.958054    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755830-fcch4" event={"ID":"7e8a09f2-7bd4-4fcf-a517-7819546f9fd4","Type":"ContainerDied","Data":"c253e49a8f139457ecae289f871adabb9097d101dc029d45e5345055b27b62af"}
Jul 29 18:30:03.958087 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:30:03.958086    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755830-fcch4"
Jul 29 18:30:03.958297 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:30:03.958101    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c253e49a8f139457ecae289f871adabb9097d101dc029d45e5345055b27b62af"
Jul 29 18:31:10.790682 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:10.790649    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-96b81602-7cf476d879-z9zgh"]
Jul 29 18:31:10.793198 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:10.791074    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e8a09f2-7bd4-4fcf-a517-7819546f9fd4" containerName="cleanup"
Jul 29 18:31:10.793198 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:10.791087    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e8a09f2-7bd4-4fcf-a517-7819546f9fd4" containerName="cleanup"
Jul 29 18:31:10.793198 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:10.791181    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e8a09f2-7bd4-4fcf-a517-7819546f9fd4" containerName="cleanup"
Jul 29 18:31:10.794176 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:10.794156    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-96b81602-7cf476d879-z9zgh"
Jul 29 18:31:10.796973 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:10.796951    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-96b81602-serving-cert\""
Jul 29 18:31:10.805676 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:10.805652    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-96b81602-7cf476d879-z9zgh"]
Jul 29 18:31:10.957059 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:10.957019    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dqt6\" (UniqueName: \"kubernetes.io/projected/1f0886a8-b41d-4cf7-8f64-d28b3dd14094-kube-api-access-9dqt6\") pod \"maas-api-e2e-mt-96b81602-7cf476d879-z9zgh\" (UID: \"1f0886a8-b41d-4cf7-8f64-d28b3dd14094\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-96b81602-7cf476d879-z9zgh"
Jul 29 18:31:10.957226 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:10.957071    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f0886a8-b41d-4cf7-8f64-d28b3dd14094-openshift-service-ca\") pod \"maas-api-e2e-mt-96b81602-7cf476d879-z9zgh\" (UID: \"1f0886a8-b41d-4cf7-8f64-d28b3dd14094\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-96b81602-7cf476d879-z9zgh"
Jul 29 18:31:10.957226 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:10.957182    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f0886a8-b41d-4cf7-8f64-d28b3dd14094-maas-api-tls\") pod \"maas-api-e2e-mt-96b81602-7cf476d879-z9zgh\" (UID: \"1f0886a8-b41d-4cf7-8f64-d28b3dd14094\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-96b81602-7cf476d879-z9zgh"
Jul 29 18:31:11.058170 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:11.058076    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dqt6\" (UniqueName: \"kubernetes.io/projected/1f0886a8-b41d-4cf7-8f64-d28b3dd14094-kube-api-access-9dqt6\") pod \"maas-api-e2e-mt-96b81602-7cf476d879-z9zgh\" (UID: \"1f0886a8-b41d-4cf7-8f64-d28b3dd14094\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-96b81602-7cf476d879-z9zgh"
Jul 29 18:31:11.058170 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:11.058123    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f0886a8-b41d-4cf7-8f64-d28b3dd14094-openshift-service-ca\") pod \"maas-api-e2e-mt-96b81602-7cf476d879-z9zgh\" (UID: \"1f0886a8-b41d-4cf7-8f64-d28b3dd14094\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-96b81602-7cf476d879-z9zgh"
Jul 29 18:31:11.058414 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:11.058205    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f0886a8-b41d-4cf7-8f64-d28b3dd14094-maas-api-tls\") pod \"maas-api-e2e-mt-96b81602-7cf476d879-z9zgh\" (UID: \"1f0886a8-b41d-4cf7-8f64-d28b3dd14094\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-96b81602-7cf476d879-z9zgh"
Jul 29 18:31:11.058414 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:31:11.058319    2557 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-96b81602-serving-cert: secret "maas-api-e2e-mt-96b81602-serving-cert" not found
Jul 29 18:31:11.058414 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:31:11.058382    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f0886a8-b41d-4cf7-8f64-d28b3dd14094-maas-api-tls podName:1f0886a8-b41d-4cf7-8f64-d28b3dd14094 nodeName:}" failed. No retries permitted until 2026-07-29 18:31:11.558362507 +0000 UTC m=+2351.427585315 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1f0886a8-b41d-4cf7-8f64-d28b3dd14094-maas-api-tls") pod "maas-api-e2e-mt-96b81602-7cf476d879-z9zgh" (UID: "1f0886a8-b41d-4cf7-8f64-d28b3dd14094") : secret "maas-api-e2e-mt-96b81602-serving-cert" not found
Jul 29 18:31:11.058900 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:11.058876    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f0886a8-b41d-4cf7-8f64-d28b3dd14094-openshift-service-ca\") pod \"maas-api-e2e-mt-96b81602-7cf476d879-z9zgh\" (UID: \"1f0886a8-b41d-4cf7-8f64-d28b3dd14094\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-96b81602-7cf476d879-z9zgh"
Jul 29 18:31:11.067285 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:11.067261    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dqt6\" (UniqueName: \"kubernetes.io/projected/1f0886a8-b41d-4cf7-8f64-d28b3dd14094-kube-api-access-9dqt6\") pod \"maas-api-e2e-mt-96b81602-7cf476d879-z9zgh\" (UID: \"1f0886a8-b41d-4cf7-8f64-d28b3dd14094\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-96b81602-7cf476d879-z9zgh"
Jul 29 18:31:11.562476 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:11.562440    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f0886a8-b41d-4cf7-8f64-d28b3dd14094-maas-api-tls\") pod \"maas-api-e2e-mt-96b81602-7cf476d879-z9zgh\" (UID: \"1f0886a8-b41d-4cf7-8f64-d28b3dd14094\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-96b81602-7cf476d879-z9zgh"
Jul 29 18:31:11.565049 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:11.565015    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f0886a8-b41d-4cf7-8f64-d28b3dd14094-maas-api-tls\") pod \"maas-api-e2e-mt-96b81602-7cf476d879-z9zgh\" (UID: \"1f0886a8-b41d-4cf7-8f64-d28b3dd14094\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-96b81602-7cf476d879-z9zgh"
Jul 29 18:31:11.600298 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:11.600268    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-96b81602-766d5f94d8-zspp5"]
Jul 29 18:31:11.603938 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:11.603921    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-96b81602-766d5f94d8-zspp5"
Jul 29 18:31:11.606570 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:11.606553    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-96b81602-dockercfg-zhf96\""
Jul 29 18:31:11.606624 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:11.606562    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-96b81602\""
Jul 29 18:31:11.612481 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:11.612457    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-96b81602-766d5f94d8-zspp5"]
Jul 29 18:31:11.706507 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:11.706471    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-96b81602-7cf476d879-z9zgh"
Jul 29 18:31:11.764482 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:11.764435    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/06fdf683-8e42-43a2-bd2a-87ece8662c90-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-96b81602-766d5f94d8-zspp5\" (UID: \"06fdf683-8e42-43a2-bd2a-87ece8662c90\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-96b81602-766d5f94d8-zspp5"
Jul 29 18:31:11.765116 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:11.765091    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gv4mm\" (UniqueName: \"kubernetes.io/projected/06fdf683-8e42-43a2-bd2a-87ece8662c90-kube-api-access-gv4mm\") pod \"payload-pre-processing-e2e-mt-96b81602-766d5f94d8-zspp5\" (UID: \"06fdf683-8e42-43a2-bd2a-87ece8662c90\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-96b81602-766d5f94d8-zspp5"
Jul 29 18:31:11.866190 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:11.866096    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/06fdf683-8e42-43a2-bd2a-87ece8662c90-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-96b81602-766d5f94d8-zspp5\" (UID: \"06fdf683-8e42-43a2-bd2a-87ece8662c90\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-96b81602-766d5f94d8-zspp5"
Jul 29 18:31:11.866576 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:11.866185    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gv4mm\" (UniqueName: \"kubernetes.io/projected/06fdf683-8e42-43a2-bd2a-87ece8662c90-kube-api-access-gv4mm\") pod \"payload-pre-processing-e2e-mt-96b81602-766d5f94d8-zspp5\" (UID: \"06fdf683-8e42-43a2-bd2a-87ece8662c90\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-96b81602-766d5f94d8-zspp5"
Jul 29 18:31:11.866940 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:11.866914    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/06fdf683-8e42-43a2-bd2a-87ece8662c90-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-96b81602-766d5f94d8-zspp5\" (UID: \"06fdf683-8e42-43a2-bd2a-87ece8662c90\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-96b81602-766d5f94d8-zspp5"
Jul 29 18:31:11.875352 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:11.875320    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gv4mm\" (UniqueName: \"kubernetes.io/projected/06fdf683-8e42-43a2-bd2a-87ece8662c90-kube-api-access-gv4mm\") pod \"payload-pre-processing-e2e-mt-96b81602-766d5f94d8-zspp5\" (UID: \"06fdf683-8e42-43a2-bd2a-87ece8662c90\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-96b81602-766d5f94d8-zspp5"
Jul 29 18:31:11.916230 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:11.916191    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-96b81602-766d5f94d8-zspp5"
Jul 29 18:31:12.058127 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:31:12.058100    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f0886a8_b41d_4cf7_8f64_d28b3dd14094.slice/crio-28ee43da0220ad66a3778694bb677674e7149de23e00da7ccf30009bed73d8f5 WatchSource:0}: Error finding container 28ee43da0220ad66a3778694bb677674e7149de23e00da7ccf30009bed73d8f5: Status 404 returned error can't find the container with id 28ee43da0220ad66a3778694bb677674e7149de23e00da7ccf30009bed73d8f5
Jul 29 18:31:12.058553 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:12.058531    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-96b81602-7cf476d879-z9zgh"]
Jul 29 18:31:12.059766 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:12.059746    2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 18:31:12.060778 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:31:12.060750    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06fdf683_8e42_43a2_bd2a_87ece8662c90.slice/crio-5b5d0c4a25affcc82c971374cb8729dd6167f87e87d24a0940f38620c3a561c4 WatchSource:0}: Error finding container 5b5d0c4a25affcc82c971374cb8729dd6167f87e87d24a0940f38620c3a561c4: Status 404 returned error can't find the container with id 5b5d0c4a25affcc82c971374cb8729dd6167f87e87d24a0940f38620c3a561c4
Jul 29 18:31:12.062570 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:12.062542    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-96b81602-766d5f94d8-zspp5"]
Jul 29 18:31:12.369411 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:12.369361    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-96b81602-7cf476d879-z9zgh" event={"ID":"1f0886a8-b41d-4cf7-8f64-d28b3dd14094","Type":"ContainerStarted","Data":"28ee43da0220ad66a3778694bb677674e7149de23e00da7ccf30009bed73d8f5"}
Jul 29 18:31:12.371365 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:12.371331    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-96b81602-766d5f94d8-zspp5" event={"ID":"06fdf683-8e42-43a2-bd2a-87ece8662c90","Type":"ContainerStarted","Data":"9de516e5db8d2ffe50254accf3415a3109b7a4948b7e39639a71f1cf759203c1"}
Jul 29 18:31:12.371497 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:12.371375    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-96b81602-766d5f94d8-zspp5" event={"ID":"06fdf683-8e42-43a2-bd2a-87ece8662c90","Type":"ContainerStarted","Data":"5b5d0c4a25affcc82c971374cb8729dd6167f87e87d24a0940f38620c3a561c4"}
Jul 29 18:31:12.371497 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:12.371409    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-96b81602-766d5f94d8-zspp5"
Jul 29 18:31:12.389580 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:12.389479    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-96b81602-766d5f94d8-zspp5" podStartSLOduration=1.389464273 podStartE2EDuration="1.389464273s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:31:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:31:12.387077068 +0000 UTC m=+2352.256299876" watchObservedRunningTime="2026-07-29 18:31:12.389464273 +0000 UTC m=+2352.258687081"
Jul 29 18:31:13.379537 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:13.379484    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-96b81602-7cf476d879-z9zgh" event={"ID":"1f0886a8-b41d-4cf7-8f64-d28b3dd14094","Type":"ContainerStarted","Data":"0c2b746b7eadbf99761db9dff031a788ddacd2473a57aca2f9aac662766ed2ef"}
Jul 29 18:31:13.379986 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:13.379691    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-96b81602-7cf476d879-z9zgh"
Jul 29 18:31:13.397703 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:13.397649    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-96b81602-7cf476d879-z9zgh" podStartSLOduration=2.938228441 podStartE2EDuration="3.397633231s" totalImagesPullingTime="459.40479ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:31:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:31:13.394780259 +0000 UTC m=+2353.264003067" watchObservedRunningTime="2026-07-29 18:31:13.397633231 +0000 UTC m=+2353.266856037"
Jul 29 18:31:19.390566 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:19.390539    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-96b81602-7cf476d879-z9zgh"
Jul 29 18:31:23.381911 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:23.381882    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-96b81602-766d5f94d8-zspp5"
Jul 29 18:31:42.999093 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:42.999063    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-96b81602-h5tz4"]
Jul 29 18:31:43.002296 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:43.002280    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-96b81602-h5tz4"
Jul 29 18:31:43.005083 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:43.005062    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qrbhl\""
Jul 29 18:31:43.011769 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:43.011746    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-96b81602-h5tz4"]
Jul 29 18:31:43.045874 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:43.045847    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5224098d-6c96-43ec-a6d8-1327643e10af-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-96b81602-h5tz4\" (UID: \"5224098d-6c96-43ec-a6d8-1327643e10af\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-96b81602-h5tz4"
Jul 29 18:31:43.146981 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:43.146945    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5224098d-6c96-43ec-a6d8-1327643e10af-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-96b81602-h5tz4\" (UID: \"5224098d-6c96-43ec-a6d8-1327643e10af\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-96b81602-h5tz4"
Jul 29 18:31:43.147553 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:43.147533    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5224098d-6c96-43ec-a6d8-1327643e10af-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-96b81602-h5tz4\" (UID: \"5224098d-6c96-43ec-a6d8-1327643e10af\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-96b81602-h5tz4"
Jul 29 18:31:43.313372 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:43.313347    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-96b81602-h5tz4"
Jul 29 18:31:43.452640 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:43.452612    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-96b81602-h5tz4"]
Jul 29 18:31:43.454655 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:31:43.454627    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5224098d_6c96_43ec_a6d8_1327643e10af.slice/crio-60bafb4ab61bb58b687ead72e656f862e3811a1b14e33e60a58064c321910ed2 WatchSource:0}: Error finding container 60bafb4ab61bb58b687ead72e656f862e3811a1b14e33e60a58064c321910ed2: Status 404 returned error can't find the container with id 60bafb4ab61bb58b687ead72e656f862e3811a1b14e33e60a58064c321910ed2
Jul 29 18:31:43.556006 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:43.555973    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-96b81602-h5tz4" event={"ID":"5224098d-6c96-43ec-a6d8-1327643e10af","Type":"ContainerStarted","Data":"60bafb4ab61bb58b687ead72e656f862e3811a1b14e33e60a58064c321910ed2"}
Jul 29 18:31:44.563507 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:44.563468    2557 generic.go:356] "Generic (PLEG): container finished" podID="5224098d-6c96-43ec-a6d8-1327643e10af" containerID="eefba6d8d4844dc13847c6ff002021f1bffca020ae83f60b13d5063f76b8f4fc" exitCode=0
Jul 29 18:31:44.563940 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:44.563552    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-96b81602-h5tz4" event={"ID":"5224098d-6c96-43ec-a6d8-1327643e10af","Type":"ContainerDied","Data":"eefba6d8d4844dc13847c6ff002021f1bffca020ae83f60b13d5063f76b8f4fc"}
Jul 29 18:31:45.702742 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:45.702698    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-96b81602-h5tz4"
Jul 29 18:31:45.773776 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:45.773741    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5224098d-6c96-43ec-a6d8-1327643e10af-maas-api-service-ca\") pod \"5224098d-6c96-43ec-a6d8-1327643e10af\" (UID: \"5224098d-6c96-43ec-a6d8-1327643e10af\") "
Jul 29 18:31:45.774032 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:45.774008    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5224098d-6c96-43ec-a6d8-1327643e10af-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5224098d-6c96-43ec-a6d8-1327643e10af" (UID: "5224098d-6c96-43ec-a6d8-1327643e10af"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:31:45.874710 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:45.874650    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5224098d-6c96-43ec-a6d8-1327643e10af-maas-api-service-ca\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:31:46.576081 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:46.576049    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-96b81602-h5tz4"
Jul 29 18:31:46.576256 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:46.576100    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-96b81602-h5tz4" event={"ID":"5224098d-6c96-43ec-a6d8-1327643e10af","Type":"ContainerDied","Data":"60bafb4ab61bb58b687ead72e656f862e3811a1b14e33e60a58064c321910ed2"}
Jul 29 18:31:46.576256 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:46.576143    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="60bafb4ab61bb58b687ead72e656f862e3811a1b14e33e60a58064c321910ed2"
Jul 29 18:31:53.429636 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:53.429597    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-96b81602-7cf476d879-z9zgh"]
Jul 29 18:31:53.430105 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:53.429924    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-96b81602-7cf476d879-z9zgh" podUID="1f0886a8-b41d-4cf7-8f64-d28b3dd14094" containerName="maas-api" containerID="cri-o://0c2b746b7eadbf99761db9dff031a788ddacd2473a57aca2f9aac662766ed2ef" gracePeriod=30
Jul 29 18:31:53.498899 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:53.498857    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-96b81602-766d5f94d8-zspp5"]
Jul 29 18:31:53.499110 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:53.499083    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-96b81602-766d5f94d8-zspp5" podUID="06fdf683-8e42-43a2-bd2a-87ece8662c90" containerName="payload-pre-processing" containerID="cri-o://9de516e5db8d2ffe50254accf3415a3109b7a4948b7e39639a71f1cf759203c1" gracePeriod=30
Jul 29 18:31:53.619422 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:53.619390    2557 generic.go:356] "Generic (PLEG): container finished" podID="1f0886a8-b41d-4cf7-8f64-d28b3dd14094" containerID="0c2b746b7eadbf99761db9dff031a788ddacd2473a57aca2f9aac662766ed2ef" exitCode=0
Jul 29 18:31:53.619527 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:53.619464    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-96b81602-7cf476d879-z9zgh" event={"ID":"1f0886a8-b41d-4cf7-8f64-d28b3dd14094","Type":"ContainerDied","Data":"0c2b746b7eadbf99761db9dff031a788ddacd2473a57aca2f9aac662766ed2ef"}
Jul 29 18:31:53.621632 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:53.621600    2557 generic.go:356] "Generic (PLEG): container finished" podID="06fdf683-8e42-43a2-bd2a-87ece8662c90" containerID="9de516e5db8d2ffe50254accf3415a3109b7a4948b7e39639a71f1cf759203c1" exitCode=0
Jul 29 18:31:53.621763 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:53.621636    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-96b81602-766d5f94d8-zspp5" event={"ID":"06fdf683-8e42-43a2-bd2a-87ece8662c90","Type":"ContainerDied","Data":"9de516e5db8d2ffe50254accf3415a3109b7a4948b7e39639a71f1cf759203c1"}
Jul 29 18:31:53.646446 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:53.646374    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-96b81602-h5tz4"]
Jul 29 18:31:53.650608 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:53.650573    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-96b81602-h5tz4"]
Jul 29 18:31:53.766468 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:53.766444    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-96b81602-7cf476d879-z9zgh"
Jul 29 18:31:53.795085 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:53.795058    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-96b81602-766d5f94d8-zspp5"
Jul 29 18:31:53.848132 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:53.848098    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f0886a8-b41d-4cf7-8f64-d28b3dd14094-maas-api-tls\") pod \"1f0886a8-b41d-4cf7-8f64-d28b3dd14094\" (UID: \"1f0886a8-b41d-4cf7-8f64-d28b3dd14094\") "
Jul 29 18:31:53.848343 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:53.848173    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f0886a8-b41d-4cf7-8f64-d28b3dd14094-openshift-service-ca\") pod \"1f0886a8-b41d-4cf7-8f64-d28b3dd14094\" (UID: \"1f0886a8-b41d-4cf7-8f64-d28b3dd14094\") "
Jul 29 18:31:53.848343 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:53.848251    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9dqt6\" (UniqueName: \"kubernetes.io/projected/1f0886a8-b41d-4cf7-8f64-d28b3dd14094-kube-api-access-9dqt6\") pod \"1f0886a8-b41d-4cf7-8f64-d28b3dd14094\" (UID: \"1f0886a8-b41d-4cf7-8f64-d28b3dd14094\") "
Jul 29 18:31:53.848574 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:53.848536    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f0886a8-b41d-4cf7-8f64-d28b3dd14094-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1f0886a8-b41d-4cf7-8f64-d28b3dd14094" (UID: "1f0886a8-b41d-4cf7-8f64-d28b3dd14094"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:31:53.850518 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:53.850493    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f0886a8-b41d-4cf7-8f64-d28b3dd14094-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1f0886a8-b41d-4cf7-8f64-d28b3dd14094" (UID: "1f0886a8-b41d-4cf7-8f64-d28b3dd14094"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:31:53.850611 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:53.850509    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f0886a8-b41d-4cf7-8f64-d28b3dd14094-kube-api-access-9dqt6" (OuterVolumeSpecName: "kube-api-access-9dqt6") pod "1f0886a8-b41d-4cf7-8f64-d28b3dd14094" (UID: "1f0886a8-b41d-4cf7-8f64-d28b3dd14094"). InnerVolumeSpecName "kube-api-access-9dqt6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:31:53.949491 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:53.949406    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gv4mm\" (UniqueName: \"kubernetes.io/projected/06fdf683-8e42-43a2-bd2a-87ece8662c90-kube-api-access-gv4mm\") pod \"06fdf683-8e42-43a2-bd2a-87ece8662c90\" (UID: \"06fdf683-8e42-43a2-bd2a-87ece8662c90\") "
Jul 29 18:31:53.949491 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:53.949476    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/06fdf683-8e42-43a2-bd2a-87ece8662c90-plugins-config-volume\") pod \"06fdf683-8e42-43a2-bd2a-87ece8662c90\" (UID: \"06fdf683-8e42-43a2-bd2a-87ece8662c90\") "
Jul 29 18:31:53.949809 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:53.949788    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9dqt6\" (UniqueName: \"kubernetes.io/projected/1f0886a8-b41d-4cf7-8f64-d28b3dd14094-kube-api-access-9dqt6\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:31:53.949900 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:53.949815    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f0886a8-b41d-4cf7-8f64-d28b3dd14094-maas-api-tls\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:31:53.949900 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:53.949831    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f0886a8-b41d-4cf7-8f64-d28b3dd14094-openshift-service-ca\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:31:53.950003 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:53.949944    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06fdf683-8e42-43a2-bd2a-87ece8662c90-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "06fdf683-8e42-43a2-bd2a-87ece8662c90" (UID: "06fdf683-8e42-43a2-bd2a-87ece8662c90"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:31:53.951985 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:53.951957    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06fdf683-8e42-43a2-bd2a-87ece8662c90-kube-api-access-gv4mm" (OuterVolumeSpecName: "kube-api-access-gv4mm") pod "06fdf683-8e42-43a2-bd2a-87ece8662c90" (UID: "06fdf683-8e42-43a2-bd2a-87ece8662c90"). InnerVolumeSpecName "kube-api-access-gv4mm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:31:54.050900 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:54.050862    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/06fdf683-8e42-43a2-bd2a-87ece8662c90-plugins-config-volume\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:31:54.050900 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:54.050894    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gv4mm\" (UniqueName: \"kubernetes.io/projected/06fdf683-8e42-43a2-bd2a-87ece8662c90-kube-api-access-gv4mm\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:31:54.628262 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:54.628231    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-96b81602-766d5f94d8-zspp5"
Jul 29 18:31:54.628770 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:54.628265    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-96b81602-766d5f94d8-zspp5" event={"ID":"06fdf683-8e42-43a2-bd2a-87ece8662c90","Type":"ContainerDied","Data":"5b5d0c4a25affcc82c971374cb8729dd6167f87e87d24a0940f38620c3a561c4"}
Jul 29 18:31:54.628770 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:54.628322    2557 scope.go:117] "RemoveContainer" containerID="9de516e5db8d2ffe50254accf3415a3109b7a4948b7e39639a71f1cf759203c1"
Jul 29 18:31:54.629613 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:54.629586    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-96b81602-7cf476d879-z9zgh" event={"ID":"1f0886a8-b41d-4cf7-8f64-d28b3dd14094","Type":"ContainerDied","Data":"28ee43da0220ad66a3778694bb677674e7149de23e00da7ccf30009bed73d8f5"}
Jul 29 18:31:54.629691 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:54.629627    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-96b81602-7cf476d879-z9zgh"
Jul 29 18:31:54.641033 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:54.641014    2557 scope.go:117] "RemoveContainer" containerID="0c2b746b7eadbf99761db9dff031a788ddacd2473a57aca2f9aac662766ed2ef"
Jul 29 18:31:54.654958 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:54.654930    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5224098d-6c96-43ec-a6d8-1327643e10af" path="/var/lib/kubelet/pods/5224098d-6c96-43ec-a6d8-1327643e10af/volumes"
Jul 29 18:31:54.657210 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:54.657189    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-96b81602-7cf476d879-z9zgh"]
Jul 29 18:31:54.663230 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:54.663209    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-96b81602-7cf476d879-z9zgh"]
Jul 29 18:31:54.680293 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:54.680270    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-96b81602-766d5f94d8-zspp5"]
Jul 29 18:31:54.683508 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:54.683480    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-96b81602-766d5f94d8-zspp5"]
Jul 29 18:31:56.653118 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:56.653088    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06fdf683-8e42-43a2-bd2a-87ece8662c90" path="/var/lib/kubelet/pods/06fdf683-8e42-43a2-bd2a-87ece8662c90/volumes"
Jul 29 18:31:56.653481 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:31:56.653465    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f0886a8-b41d-4cf7-8f64-d28b3dd14094" path="/var/lib/kubelet/pods/1f0886a8-b41d-4cf7-8f64-d28b3dd14094/volumes"
Jul 29 18:32:00.836315 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:32:00.836287    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gg2wj_2bf0bb71-4047-45a7-bd8e-5d6efabd26bd/console-operator/2.log"
Jul 29 18:32:00.846019 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:32:00.845981    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gg2wj_2bf0bb71-4047-45a7-bd8e-5d6efabd26bd/console-operator/2.log"
Jul 29 18:32:41.871117 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:32:41.871016    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4p58n"]
Jul 29 18:32:41.871627 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:32:41.871244    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4p58n" podUID="6c0ee61e-c00b-4e2c-964d-9460da93537a" containerName="manager" containerID="cri-o://b19711cd09b1f8bce7d37b84cf3d2926a1b4ec84d62327e2f14d38a68ca8d61f" gracePeriod=10
Jul 29 18:32:43.531564 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:32:43.531541    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4p58n"
Jul 29 18:32:43.727241 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:32:43.727151    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6c0ee61e-c00b-4e2c-964d-9460da93537a-extensions-socket-volume\") pod \"6c0ee61e-c00b-4e2c-964d-9460da93537a\" (UID: \"6c0ee61e-c00b-4e2c-964d-9460da93537a\") "
Jul 29 18:32:43.727241 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:32:43.727219    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6lv6w\" (UniqueName: \"kubernetes.io/projected/6c0ee61e-c00b-4e2c-964d-9460da93537a-kube-api-access-6lv6w\") pod \"6c0ee61e-c00b-4e2c-964d-9460da93537a\" (UID: \"6c0ee61e-c00b-4e2c-964d-9460da93537a\") "
Jul 29 18:32:43.727510 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:32:43.727478    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c0ee61e-c00b-4e2c-964d-9460da93537a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "6c0ee61e-c00b-4e2c-964d-9460da93537a" (UID: "6c0ee61e-c00b-4e2c-964d-9460da93537a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:32:43.729383 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:32:43.729363    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c0ee61e-c00b-4e2c-964d-9460da93537a-kube-api-access-6lv6w" (OuterVolumeSpecName: "kube-api-access-6lv6w") pod "6c0ee61e-c00b-4e2c-964d-9460da93537a" (UID: "6c0ee61e-c00b-4e2c-964d-9460da93537a"). InnerVolumeSpecName "kube-api-access-6lv6w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:32:43.828466 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:32:43.828423    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6lv6w\" (UniqueName: \"kubernetes.io/projected/6c0ee61e-c00b-4e2c-964d-9460da93537a-kube-api-access-6lv6w\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:32:43.828466 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:32:43.828461    2557 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6c0ee61e-c00b-4e2c-964d-9460da93537a-extensions-socket-volume\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:32:43.913814 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:32:43.913785    2557 generic.go:356] "Generic (PLEG): container finished" podID="6c0ee61e-c00b-4e2c-964d-9460da93537a" containerID="b19711cd09b1f8bce7d37b84cf3d2926a1b4ec84d62327e2f14d38a68ca8d61f" exitCode=0
Jul 29 18:32:43.913974 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:32:43.913831    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4p58n"
Jul 29 18:32:43.913974 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:32:43.913881    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4p58n" event={"ID":"6c0ee61e-c00b-4e2c-964d-9460da93537a","Type":"ContainerDied","Data":"b19711cd09b1f8bce7d37b84cf3d2926a1b4ec84d62327e2f14d38a68ca8d61f"}
Jul 29 18:32:43.913974 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:32:43.913929    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4p58n" event={"ID":"6c0ee61e-c00b-4e2c-964d-9460da93537a","Type":"ContainerDied","Data":"95ebe873b913e7b93069a74d416d60417bf4a4a00e605cbbb7c8f3ec7bfd9f48"}
Jul 29 18:32:43.913974 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:32:43.913950    2557 scope.go:117] "RemoveContainer" containerID="b19711cd09b1f8bce7d37b84cf3d2926a1b4ec84d62327e2f14d38a68ca8d61f"
Jul 29 18:32:43.924839 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:32:43.924824    2557 scope.go:117] "RemoveContainer" containerID="b19711cd09b1f8bce7d37b84cf3d2926a1b4ec84d62327e2f14d38a68ca8d61f"
Jul 29 18:32:43.925069 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:32:43.925050    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b19711cd09b1f8bce7d37b84cf3d2926a1b4ec84d62327e2f14d38a68ca8d61f\": container with ID starting with b19711cd09b1f8bce7d37b84cf3d2926a1b4ec84d62327e2f14d38a68ca8d61f not found: ID does not exist" containerID="b19711cd09b1f8bce7d37b84cf3d2926a1b4ec84d62327e2f14d38a68ca8d61f"
Jul 29 18:32:43.925110 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:32:43.925077    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b19711cd09b1f8bce7d37b84cf3d2926a1b4ec84d62327e2f14d38a68ca8d61f"} err="failed to get container status \"b19711cd09b1f8bce7d37b84cf3d2926a1b4ec84d62327e2f14d38a68ca8d61f\": rpc error: code = NotFound desc = could not find container \"b19711cd09b1f8bce7d37b84cf3d2926a1b4ec84d62327e2f14d38a68ca8d61f\": container with ID starting with b19711cd09b1f8bce7d37b84cf3d2926a1b4ec84d62327e2f14d38a68ca8d61f not found: ID does not exist"
Jul 29 18:32:43.939601 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:32:43.939576    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4p58n"]
Jul 29 18:32:43.946517 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:32:43.946489    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4p58n"]
Jul 29 18:32:44.652838 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:32:44.652809    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c0ee61e-c00b-4e2c-964d-9460da93537a" path="/var/lib/kubelet/pods/6c0ee61e-c00b-4e2c-964d-9460da93537a/volumes"
Jul 29 18:33:00.958052 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:33:00.958017    2557 scope.go:117] "RemoveContainer" containerID="bd4b818bedebd3d5415a627ca90d631c31da27979e13e562ca736c86589bfdf0"
Jul 29 18:33:57.549363 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:33:57.549327    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xvr6r"]
Jul 29 18:33:57.549970 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:33:57.549771    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f0886a8-b41d-4cf7-8f64-d28b3dd14094" containerName="maas-api"
Jul 29 18:33:57.549970 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:33:57.549784    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f0886a8-b41d-4cf7-8f64-d28b3dd14094" containerName="maas-api"
Jul 29 18:33:57.549970 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:33:57.549800    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06fdf683-8e42-43a2-bd2a-87ece8662c90" containerName="payload-pre-processing"
Jul 29 18:33:57.549970 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:33:57.549805    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="06fdf683-8e42-43a2-bd2a-87ece8662c90" containerName="payload-pre-processing"
Jul 29 18:33:57.549970 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:33:57.549821    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5224098d-6c96-43ec-a6d8-1327643e10af" containerName="revoke-keys"
Jul 29 18:33:57.549970 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:33:57.549832    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="5224098d-6c96-43ec-a6d8-1327643e10af" containerName="revoke-keys"
Jul 29 18:33:57.549970 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:33:57.549857    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c0ee61e-c00b-4e2c-964d-9460da93537a" containerName="manager"
Jul 29 18:33:57.549970 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:33:57.549862    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c0ee61e-c00b-4e2c-964d-9460da93537a" containerName="manager"
Jul 29 18:33:57.549970 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:33:57.549936    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="5224098d-6c96-43ec-a6d8-1327643e10af" containerName="revoke-keys"
Jul 29 18:33:57.549970 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:33:57.549946    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="06fdf683-8e42-43a2-bd2a-87ece8662c90" containerName="payload-pre-processing"
Jul 29 18:33:57.549970 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:33:57.549955    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c0ee61e-c00b-4e2c-964d-9460da93537a" containerName="manager"
Jul 29 18:33:57.549970 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:33:57.549965    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f0886a8-b41d-4cf7-8f64-d28b3dd14094" containerName="maas-api"
Jul 29 18:33:57.552969 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:33:57.552951    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xvr6r"
Jul 29 18:33:57.555806 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:33:57.555786    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-nnb89\""
Jul 29 18:33:57.564445 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:33:57.564425    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xvr6r"]
Jul 29 18:33:57.689145 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:33:57.689104    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fa3433e1-e499-4bb0-803d-97906bc639f2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-xvr6r\" (UID: \"fa3433e1-e499-4bb0-803d-97906bc639f2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xvr6r"
Jul 29 18:33:57.689145 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:33:57.689145    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ch6dk\" (UniqueName: \"kubernetes.io/projected/fa3433e1-e499-4bb0-803d-97906bc639f2-kube-api-access-ch6dk\") pod \"kuadrant-operator-controller-manager-86878c769f-xvr6r\" (UID: \"fa3433e1-e499-4bb0-803d-97906bc639f2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xvr6r"
Jul 29 18:33:57.790131 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:33:57.790086    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ch6dk\" (UniqueName: \"kubernetes.io/projected/fa3433e1-e499-4bb0-803d-97906bc639f2-kube-api-access-ch6dk\") pod \"kuadrant-operator-controller-manager-86878c769f-xvr6r\" (UID: \"fa3433e1-e499-4bb0-803d-97906bc639f2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xvr6r"
Jul 29 18:33:57.790298 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:33:57.790263    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fa3433e1-e499-4bb0-803d-97906bc639f2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-xvr6r\" (UID: \"fa3433e1-e499-4bb0-803d-97906bc639f2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xvr6r"
Jul 29 18:33:57.790616 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:33:57.790599    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fa3433e1-e499-4bb0-803d-97906bc639f2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-xvr6r\" (UID: \"fa3433e1-e499-4bb0-803d-97906bc639f2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xvr6r"
Jul 29 18:33:57.800134 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:33:57.800067    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ch6dk\" (UniqueName: \"kubernetes.io/projected/fa3433e1-e499-4bb0-803d-97906bc639f2-kube-api-access-ch6dk\") pod \"kuadrant-operator-controller-manager-86878c769f-xvr6r\" (UID: \"fa3433e1-e499-4bb0-803d-97906bc639f2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xvr6r"
Jul 29 18:33:57.863946 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:33:57.863909    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xvr6r"
Jul 29 18:33:58.005477 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:33:58.005438    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa3433e1_e499_4bb0_803d_97906bc639f2.slice/crio-760340a421b9ce1b0fcbb7ce315d11487fc994a4f294893acb4d3bff2c81d12a WatchSource:0}: Error finding container 760340a421b9ce1b0fcbb7ce315d11487fc994a4f294893acb4d3bff2c81d12a: Status 404 returned error can't find the container with id 760340a421b9ce1b0fcbb7ce315d11487fc994a4f294893acb4d3bff2c81d12a
Jul 29 18:33:58.005798 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:33:58.005775    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xvr6r"]
Jul 29 18:33:58.324359 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:33:58.324262    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xvr6r" event={"ID":"fa3433e1-e499-4bb0-803d-97906bc639f2","Type":"ContainerStarted","Data":"3564fec3851407aac8b962e8bfadaa3ca2a7765b0bc6f0d4f9e37b1420a8ad2c"}
Jul 29 18:33:58.324359 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:33:58.324314    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xvr6r" event={"ID":"fa3433e1-e499-4bb0-803d-97906bc639f2","Type":"ContainerStarted","Data":"760340a421b9ce1b0fcbb7ce315d11487fc994a4f294893acb4d3bff2c81d12a"}
Jul 29 18:33:58.324593 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:33:58.324405    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xvr6r"
Jul 29 18:33:58.353222 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:33:58.353164    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xvr6r" podStartSLOduration=1.35314697 podStartE2EDuration="1.35314697s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:33:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:33:58.35177973 +0000 UTC m=+2518.221002538" watchObservedRunningTime="2026-07-29 18:33:58.35314697 +0000 UTC m=+2518.222369776"
Jul 29 18:34:09.332488 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:34:09.332447    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xvr6r"
Jul 29 18:34:19.772757 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:34:19.772697    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8be27fe2-7c45dd856-mcbdf"]
Jul 29 18:34:19.779251 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:34:19.779223    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8be27fe2-7c45dd856-mcbdf"
Jul 29 18:34:19.781975 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:34:19.781948    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-8be27fe2-serving-cert\""
Jul 29 18:34:19.784027 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:34:19.783996    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e36c42d0-1a99-4869-866f-0be38435f1da-openshift-service-ca\") pod \"maas-api-e2e-mt-8be27fe2-7c45dd856-mcbdf\" (UID: \"e36c42d0-1a99-4869-866f-0be38435f1da\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8be27fe2-7c45dd856-mcbdf"
Jul 29 18:34:19.784214 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:34:19.784065    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e36c42d0-1a99-4869-866f-0be38435f1da-maas-api-tls\") pod \"maas-api-e2e-mt-8be27fe2-7c45dd856-mcbdf\" (UID: \"e36c42d0-1a99-4869-866f-0be38435f1da\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8be27fe2-7c45dd856-mcbdf"
Jul 29 18:34:19.784214 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:34:19.784124    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lg8q5\" (UniqueName: \"kubernetes.io/projected/e36c42d0-1a99-4869-866f-0be38435f1da-kube-api-access-lg8q5\") pod \"maas-api-e2e-mt-8be27fe2-7c45dd856-mcbdf\" (UID: \"e36c42d0-1a99-4869-866f-0be38435f1da\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8be27fe2-7c45dd856-mcbdf"
Jul 29 18:34:19.784639 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:34:19.784613    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8be27fe2-7c45dd856-mcbdf"]
Jul 29 18:34:19.884758 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:34:19.884687    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e36c42d0-1a99-4869-866f-0be38435f1da-maas-api-tls\") pod \"maas-api-e2e-mt-8be27fe2-7c45dd856-mcbdf\" (UID: \"e36c42d0-1a99-4869-866f-0be38435f1da\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8be27fe2-7c45dd856-mcbdf"
Jul 29 18:34:19.884953 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:34:19.884770    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lg8q5\" (UniqueName: \"kubernetes.io/projected/e36c42d0-1a99-4869-866f-0be38435f1da-kube-api-access-lg8q5\") pod \"maas-api-e2e-mt-8be27fe2-7c45dd856-mcbdf\" (UID: \"e36c42d0-1a99-4869-866f-0be38435f1da\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8be27fe2-7c45dd856-mcbdf"
Jul 29 18:34:19.884953 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:34:19.884846    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e36c42d0-1a99-4869-866f-0be38435f1da-openshift-service-ca\") pod \"maas-api-e2e-mt-8be27fe2-7c45dd856-mcbdf\" (UID: \"e36c42d0-1a99-4869-866f-0be38435f1da\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8be27fe2-7c45dd856-mcbdf"
Jul 29 18:34:19.884953 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:34:19.884857    2557 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-8be27fe2-serving-cert: secret "maas-api-e2e-mt-8be27fe2-serving-cert" not found
Jul 29 18:34:19.884953 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:34:19.884927    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e36c42d0-1a99-4869-866f-0be38435f1da-maas-api-tls podName:e36c42d0-1a99-4869-866f-0be38435f1da nodeName:}" failed. No retries permitted until 2026-07-29 18:34:20.384906184 +0000 UTC m=+2540.254128982 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e36c42d0-1a99-4869-866f-0be38435f1da-maas-api-tls") pod "maas-api-e2e-mt-8be27fe2-7c45dd856-mcbdf" (UID: "e36c42d0-1a99-4869-866f-0be38435f1da") : secret "maas-api-e2e-mt-8be27fe2-serving-cert" not found
Jul 29 18:34:19.885550 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:34:19.885531    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e36c42d0-1a99-4869-866f-0be38435f1da-openshift-service-ca\") pod \"maas-api-e2e-mt-8be27fe2-7c45dd856-mcbdf\" (UID: \"e36c42d0-1a99-4869-866f-0be38435f1da\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8be27fe2-7c45dd856-mcbdf"
Jul 29 18:34:19.894465 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:34:19.894440    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lg8q5\" (UniqueName: \"kubernetes.io/projected/e36c42d0-1a99-4869-866f-0be38435f1da-kube-api-access-lg8q5\") pod \"maas-api-e2e-mt-8be27fe2-7c45dd856-mcbdf\" (UID: \"e36c42d0-1a99-4869-866f-0be38435f1da\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8be27fe2-7c45dd856-mcbdf"
Jul 29 18:34:20.388067 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:34:20.388020    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e36c42d0-1a99-4869-866f-0be38435f1da-maas-api-tls\") pod \"maas-api-e2e-mt-8be27fe2-7c45dd856-mcbdf\" (UID: \"e36c42d0-1a99-4869-866f-0be38435f1da\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8be27fe2-7c45dd856-mcbdf"
Jul 29 18:34:20.390610 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:34:20.390584    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e36c42d0-1a99-4869-866f-0be38435f1da-maas-api-tls\") pod \"maas-api-e2e-mt-8be27fe2-7c45dd856-mcbdf\" (UID: \"e36c42d0-1a99-4869-866f-0be38435f1da\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8be27fe2-7c45dd856-mcbdf"
Jul 29 18:34:20.393505 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:34:20.393478    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8be27fe2-7c45dd856-mcbdf"
Jul 29 18:34:20.582876 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:34:20.582843    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-8be27fe2-8556685cc4-4nffr"]
Jul 29 18:34:20.586630 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:34:20.586611    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-8be27fe2-8556685cc4-4nffr"
Jul 29 18:34:20.589481 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:34:20.589452    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c23e03f3-1656-463d-836e-cad710aae9ba-plugins-config-volume\") pod \"payload-processing-e2e-mt-8be27fe2-8556685cc4-4nffr\" (UID: \"c23e03f3-1656-463d-836e-cad710aae9ba\") " pod="openshift-ingress/payload-processing-e2e-mt-8be27fe2-8556685cc4-4nffr"
Jul 29 18:34:20.589646 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:34:20.589564    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-8be27fe2\""
Jul 29 18:34:20.589646 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:34:20.589607    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7llz\" (UniqueName: \"kubernetes.io/projected/c23e03f3-1656-463d-836e-cad710aae9ba-kube-api-access-w7llz\") pod \"payload-processing-e2e-mt-8be27fe2-8556685cc4-4nffr\" (UID: \"c23e03f3-1656-463d-836e-cad710aae9ba\") " pod="openshift-ingress/payload-processing-e2e-mt-8be27fe2-8556685cc4-4nffr"
Jul 29 18:34:20.589646 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:34:20.589615    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-8be27fe2-dockercfg-jpngc\""
Jul 29 18:34:20.596854 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:34:20.596825    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-8be27fe2-8556685cc4-4nffr"]
Jul 29 18:34:20.693116 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:34:20.693009    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7llz\" (UniqueName: \"kubernetes.io/projected/c23e03f3-1656-463d-836e-cad710aae9ba-kube-api-access-w7llz\") pod \"payload-processing-e2e-mt-8be27fe2-8556685cc4-4nffr\" (UID: \"c23e03f3-1656-463d-836e-cad710aae9ba\") " pod="openshift-ingress/payload-processing-e2e-mt-8be27fe2-8556685cc4-4nffr"
Jul 29 18:34:20.693116 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:34:20.693101    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c23e03f3-1656-463d-836e-cad710aae9ba-plugins-config-volume\") pod \"payload-processing-e2e-mt-8be27fe2-8556685cc4-4nffr\" (UID: \"c23e03f3-1656-463d-836e-cad710aae9ba\") " pod="openshift-ingress/payload-processing-e2e-mt-8be27fe2-8556685cc4-4nffr"
Jul 29 18:34:20.694469 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:34:20.694362    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c23e03f3-1656-463d-836e-cad710aae9ba-plugins-config-volume\") pod \"payload-processing-e2e-mt-8be27fe2-8556685cc4-4nffr\" (UID: \"c23e03f3-1656-463d-836e-cad710aae9ba\") " pod="openshift-ingress/payload-processing-e2e-mt-8be27fe2-8556685cc4-4nffr"
Jul 29 18:34:20.704335 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:34:20.704293    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7llz\" (UniqueName: \"kubernetes.io/projected/c23e03f3-1656-463d-836e-cad710aae9ba-kube-api-access-w7llz\") pod \"payload-processing-e2e-mt-8be27fe2-8556685cc4-4nffr\" (UID: \"c23e03f3-1656-463d-836e-cad710aae9ba\") " pod="openshift-ingress/payload-processing-e2e-mt-8be27fe2-8556685cc4-4nffr"
Jul 29 18:34:20.758475 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:34:20.758439    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8be27fe2-7c45dd856-mcbdf"]
Jul 29 18:34:20.760500 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:34:20.760470    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode36c42d0_1a99_4869_866f_0be38435f1da.slice/crio-aecac75dea740857474ab02f4c1fa0130389071580bacce8e13dc4c757a0a1a0 WatchSource:0}: Error finding container aecac75dea740857474ab02f4c1fa0130389071580bacce8e13dc4c757a0a1a0: Status 404 returned error can't find the container with id aecac75dea740857474ab02f4c1fa0130389071580bacce8e13dc4c757a0a1a0
Jul 29 18:34:20.899073 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:34:20.899038    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-8be27fe2-8556685cc4-4nffr"
Jul 29 18:34:21.051141 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:34:21.051108    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-8be27fe2-8556685cc4-4nffr"]
Jul 29 18:34:21.054518 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:34:21.054474    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc23e03f3_1656_463d_836e_cad710aae9ba.slice/crio-891378cab3222f1939186b40c69738a1115c8283426998c2c6412ff7c839c543 WatchSource:0}: Error finding container 891378cab3222f1939186b40c69738a1115c8283426998c2c6412ff7c839c543: Status 404 returned error can't find the container with id 891378cab3222f1939186b40c69738a1115c8283426998c2c6412ff7c839c543
Jul 29 18:34:21.457174 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:34:21.457138    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-8be27fe2-8556685cc4-4nffr" event={"ID":"c23e03f3-1656-463d-836e-cad710aae9ba","Type":"ContainerStarted","Data":"668714fe2929f91a75470c46362ec7065a68c18276ff7207cc45ba0db6c5ee86"}
Jul 29 18:34:21.457382 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:34:21.457196    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-8be27fe2-8556685cc4-4nffr" event={"ID":"c23e03f3-1656-463d-836e-cad710aae9ba","Type":"ContainerStarted","Data":"891378cab3222f1939186b40c69738a1115c8283426998c2c6412ff7c839c543"}
Jul 29 18:34:21.457382 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:34:21.457225    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-8be27fe2-8556685cc4-4nffr"
Jul 29 18:34:21.458451 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:34:21.458430    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8be27fe2-7c45dd856-mcbdf" event={"ID":"e36c42d0-1a99-4869-866f-0be38435f1da","Type":"ContainerStarted","Data":"aecac75dea740857474ab02f4c1fa0130389071580bacce8e13dc4c757a0a1a0"}
Jul 29 18:34:21.498214 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:34:21.498159    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-8be27fe2-8556685cc4-4nffr" podStartSLOduration=1.498142306 podStartE2EDuration="1.498142306s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:34:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:34:21.494808918 +0000 UTC m=+2541.364031724" watchObservedRunningTime="2026-07-29 18:34:21.498142306 +0000 UTC m=+2541.367365125"
Jul 29 18:34:22.468022 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:34:22.467983    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8be27fe2-7c45dd856-mcbdf" event={"ID":"e36c42d0-1a99-4869-866f-0be38435f1da","Type":"ContainerStarted","Data":"1e1b85fe83e0ed8ab235c45eac52b46ef8207f02714ae6e710e33a5f35a28bfb"}
Jul 29 18:34:22.487438 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:34:22.487395    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8be27fe2-7c45dd856-mcbdf" podStartSLOduration=1.986457557 podStartE2EDuration="3.487378299s" totalImagesPullingTime="1.500920742s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:34:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:34:22.487057695 +0000 UTC m=+2542.356280504" watchObservedRunningTime="2026-07-29 18:34:22.487378299 +0000 UTC m=+2542.356601105"
Jul 29 18:34:23.473763 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:34:23.473733    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8be27fe2-7c45dd856-mcbdf"
Jul 29 18:34:29.485795 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:34:29.485766    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8be27fe2-7c45dd856-mcbdf"
Jul 29 18:34:32.469952 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:34:32.469915    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-8be27fe2-8556685cc4-4nffr"
Jul 29 18:35:24.502493 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:24.502405    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8be27fe2-6vtwm"]
Jul 29 18:35:24.506066 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:24.506038    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8be27fe2-6vtwm"
Jul 29 18:35:24.508772 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:24.508751    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qrbhl\""
Jul 29 18:35:24.514351 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:24.514306    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8be27fe2-6vtwm"]
Jul 29 18:35:24.573580 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:24.573544    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b78a49b-66f7-41a3-9bf4-644867a65577-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-8be27fe2-6vtwm\" (UID: \"7b78a49b-66f7-41a3-9bf4-644867a65577\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8be27fe2-6vtwm"
Jul 29 18:35:24.674852 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:24.674823    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b78a49b-66f7-41a3-9bf4-644867a65577-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-8be27fe2-6vtwm\" (UID: \"7b78a49b-66f7-41a3-9bf4-644867a65577\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8be27fe2-6vtwm"
Jul 29 18:35:24.675499 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:24.675459    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b78a49b-66f7-41a3-9bf4-644867a65577-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-8be27fe2-6vtwm\" (UID: \"7b78a49b-66f7-41a3-9bf4-644867a65577\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8be27fe2-6vtwm"
Jul 29 18:35:24.817564 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:24.817534    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8be27fe2-6vtwm"
Jul 29 18:35:24.955893 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:24.955857    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8be27fe2-6vtwm"]
Jul 29 18:35:24.956060 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:35:24.955929    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b78a49b_66f7_41a3_9bf4_644867a65577.slice/crio-7d0581c7704b49e5eb50388a0d5e7cc48345f2cc7813868170dba06ae7aaf47b WatchSource:0}: Error finding container 7d0581c7704b49e5eb50388a0d5e7cc48345f2cc7813868170dba06ae7aaf47b: Status 404 returned error can't find the container with id 7d0581c7704b49e5eb50388a0d5e7cc48345f2cc7813868170dba06ae7aaf47b
Jul 29 18:35:25.834511 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:25.834479    2557 generic.go:356] "Generic (PLEG): container finished" podID="7b78a49b-66f7-41a3-9bf4-644867a65577" containerID="4f87b6b76a8bc0dd027237331a023c00f18943faa9c5bc279d9d3a041a3bdd6b" exitCode=0
Jul 29 18:35:25.834891 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:25.834557    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8be27fe2-6vtwm" event={"ID":"7b78a49b-66f7-41a3-9bf4-644867a65577","Type":"ContainerDied","Data":"4f87b6b76a8bc0dd027237331a023c00f18943faa9c5bc279d9d3a041a3bdd6b"}
Jul 29 18:35:25.834891 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:25.834599    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8be27fe2-6vtwm" event={"ID":"7b78a49b-66f7-41a3-9bf4-644867a65577","Type":"ContainerStarted","Data":"7d0581c7704b49e5eb50388a0d5e7cc48345f2cc7813868170dba06ae7aaf47b"}
Jul 29 18:35:26.995438 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:26.995411    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8be27fe2-6vtwm"
Jul 29 18:35:27.096236 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:27.096161    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b78a49b-66f7-41a3-9bf4-644867a65577-maas-api-service-ca\") pod \"7b78a49b-66f7-41a3-9bf4-644867a65577\" (UID: \"7b78a49b-66f7-41a3-9bf4-644867a65577\") "
Jul 29 18:35:27.096534 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:27.096513    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b78a49b-66f7-41a3-9bf4-644867a65577-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7b78a49b-66f7-41a3-9bf4-644867a65577" (UID: "7b78a49b-66f7-41a3-9bf4-644867a65577"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:35:27.197577 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:27.197532    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b78a49b-66f7-41a3-9bf4-644867a65577-maas-api-service-ca\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:35:27.847309 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:27.847282    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8be27fe2-6vtwm"
Jul 29 18:35:27.847309 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:27.847292    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8be27fe2-6vtwm" event={"ID":"7b78a49b-66f7-41a3-9bf4-644867a65577","Type":"ContainerDied","Data":"7d0581c7704b49e5eb50388a0d5e7cc48345f2cc7813868170dba06ae7aaf47b"}
Jul 29 18:35:27.847518 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:27.847339    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7d0581c7704b49e5eb50388a0d5e7cc48345f2cc7813868170dba06ae7aaf47b"
Jul 29 18:35:40.014146 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:40.014101    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8be27fe2-7c45dd856-mcbdf"]
Jul 29 18:35:40.014901 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:40.014845    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8be27fe2-7c45dd856-mcbdf" podUID="e36c42d0-1a99-4869-866f-0be38435f1da" containerName="maas-api" containerID="cri-o://1e1b85fe83e0ed8ab235c45eac52b46ef8207f02714ae6e710e33a5f35a28bfb" gracePeriod=30
Jul 29 18:35:40.077567 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:40.077526    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-8be27fe2-8556685cc4-4nffr"]
Jul 29 18:35:40.077854 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:40.077825    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-8be27fe2-8556685cc4-4nffr" podUID="c23e03f3-1656-463d-836e-cad710aae9ba" containerName="payload-processing" containerID="cri-o://668714fe2929f91a75470c46362ec7065a68c18276ff7207cc45ba0db6c5ee86" gracePeriod=30
Jul 29 18:35:40.204864 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:40.204807    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8be27fe2-6vtwm"]
Jul 29 18:35:40.209019 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:40.208990    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8be27fe2-6vtwm"]
Jul 29 18:35:40.341662 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:40.341636    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8be27fe2-7c45dd856-mcbdf"
Jul 29 18:35:40.366716 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:40.366687    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-8be27fe2-8556685cc4-4nffr"
Jul 29 18:35:40.417589 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:40.417556    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e36c42d0-1a99-4869-866f-0be38435f1da-maas-api-tls\") pod \"e36c42d0-1a99-4869-866f-0be38435f1da\" (UID: \"e36c42d0-1a99-4869-866f-0be38435f1da\") "
Jul 29 18:35:40.417793 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:40.417626    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lg8q5\" (UniqueName: \"kubernetes.io/projected/e36c42d0-1a99-4869-866f-0be38435f1da-kube-api-access-lg8q5\") pod \"e36c42d0-1a99-4869-866f-0be38435f1da\" (UID: \"e36c42d0-1a99-4869-866f-0be38435f1da\") "
Jul 29 18:35:40.417793 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:40.417667    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e36c42d0-1a99-4869-866f-0be38435f1da-openshift-service-ca\") pod \"e36c42d0-1a99-4869-866f-0be38435f1da\" (UID: \"e36c42d0-1a99-4869-866f-0be38435f1da\") "
Jul 29 18:35:40.418081 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:40.418047    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e36c42d0-1a99-4869-866f-0be38435f1da-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e36c42d0-1a99-4869-866f-0be38435f1da" (UID: "e36c42d0-1a99-4869-866f-0be38435f1da"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:35:40.419813 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:40.419783    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e36c42d0-1a99-4869-866f-0be38435f1da-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e36c42d0-1a99-4869-866f-0be38435f1da" (UID: "e36c42d0-1a99-4869-866f-0be38435f1da"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:35:40.419899 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:40.419812    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e36c42d0-1a99-4869-866f-0be38435f1da-kube-api-access-lg8q5" (OuterVolumeSpecName: "kube-api-access-lg8q5") pod "e36c42d0-1a99-4869-866f-0be38435f1da" (UID: "e36c42d0-1a99-4869-866f-0be38435f1da"). InnerVolumeSpecName "kube-api-access-lg8q5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:35:40.518563 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:40.518535    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w7llz\" (UniqueName: \"kubernetes.io/projected/c23e03f3-1656-463d-836e-cad710aae9ba-kube-api-access-w7llz\") pod \"c23e03f3-1656-463d-836e-cad710aae9ba\" (UID: \"c23e03f3-1656-463d-836e-cad710aae9ba\") "
Jul 29 18:35:40.518713 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:40.518642    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c23e03f3-1656-463d-836e-cad710aae9ba-plugins-config-volume\") pod \"c23e03f3-1656-463d-836e-cad710aae9ba\" (UID: \"c23e03f3-1656-463d-836e-cad710aae9ba\") "
Jul 29 18:35:40.518871 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:40.518859    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e36c42d0-1a99-4869-866f-0be38435f1da-maas-api-tls\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:35:40.518911 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:40.518873    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lg8q5\" (UniqueName: \"kubernetes.io/projected/e36c42d0-1a99-4869-866f-0be38435f1da-kube-api-access-lg8q5\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:35:40.518911 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:40.518883    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e36c42d0-1a99-4869-866f-0be38435f1da-openshift-service-ca\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:35:40.518992 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:40.518972    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c23e03f3-1656-463d-836e-cad710aae9ba-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c23e03f3-1656-463d-836e-cad710aae9ba" (UID: "c23e03f3-1656-463d-836e-cad710aae9ba"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:35:40.520659 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:40.520637    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c23e03f3-1656-463d-836e-cad710aae9ba-kube-api-access-w7llz" (OuterVolumeSpecName: "kube-api-access-w7llz") pod "c23e03f3-1656-463d-836e-cad710aae9ba" (UID: "c23e03f3-1656-463d-836e-cad710aae9ba"). InnerVolumeSpecName "kube-api-access-w7llz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:35:40.619917 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:40.619839    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w7llz\" (UniqueName: \"kubernetes.io/projected/c23e03f3-1656-463d-836e-cad710aae9ba-kube-api-access-w7llz\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:35:40.619917 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:40.619863    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c23e03f3-1656-463d-836e-cad710aae9ba-plugins-config-volume\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:35:40.653401 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:40.653370    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b78a49b-66f7-41a3-9bf4-644867a65577" path="/var/lib/kubelet/pods/7b78a49b-66f7-41a3-9bf4-644867a65577/volumes"
Jul 29 18:35:40.922409 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:40.922318    2557 generic.go:356] "Generic (PLEG): container finished" podID="e36c42d0-1a99-4869-866f-0be38435f1da" containerID="1e1b85fe83e0ed8ab235c45eac52b46ef8207f02714ae6e710e33a5f35a28bfb" exitCode=0
Jul 29 18:35:40.922409 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:40.922360    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8be27fe2-7c45dd856-mcbdf"
Jul 29 18:35:40.922631 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:40.922408    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8be27fe2-7c45dd856-mcbdf" event={"ID":"e36c42d0-1a99-4869-866f-0be38435f1da","Type":"ContainerDied","Data":"1e1b85fe83e0ed8ab235c45eac52b46ef8207f02714ae6e710e33a5f35a28bfb"}
Jul 29 18:35:40.922631 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:40.922465    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8be27fe2-7c45dd856-mcbdf" event={"ID":"e36c42d0-1a99-4869-866f-0be38435f1da","Type":"ContainerDied","Data":"aecac75dea740857474ab02f4c1fa0130389071580bacce8e13dc4c757a0a1a0"}
Jul 29 18:35:40.922631 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:40.922490    2557 scope.go:117] "RemoveContainer" containerID="1e1b85fe83e0ed8ab235c45eac52b46ef8207f02714ae6e710e33a5f35a28bfb"
Jul 29 18:35:40.924151 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:40.924128    2557 generic.go:356] "Generic (PLEG): container finished" podID="c23e03f3-1656-463d-836e-cad710aae9ba" containerID="668714fe2929f91a75470c46362ec7065a68c18276ff7207cc45ba0db6c5ee86" exitCode=0
Jul 29 18:35:40.924236 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:40.924164    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-8be27fe2-8556685cc4-4nffr" event={"ID":"c23e03f3-1656-463d-836e-cad710aae9ba","Type":"ContainerDied","Data":"668714fe2929f91a75470c46362ec7065a68c18276ff7207cc45ba0db6c5ee86"}
Jul 29 18:35:40.924236 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:40.924173    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-8be27fe2-8556685cc4-4nffr"
Jul 29 18:35:40.924236 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:40.924200    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-8be27fe2-8556685cc4-4nffr" event={"ID":"c23e03f3-1656-463d-836e-cad710aae9ba","Type":"ContainerDied","Data":"891378cab3222f1939186b40c69738a1115c8283426998c2c6412ff7c839c543"}
Jul 29 18:35:40.932554 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:40.932533    2557 scope.go:117] "RemoveContainer" containerID="1e1b85fe83e0ed8ab235c45eac52b46ef8207f02714ae6e710e33a5f35a28bfb"
Jul 29 18:35:40.932872 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:35:40.932847    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e1b85fe83e0ed8ab235c45eac52b46ef8207f02714ae6e710e33a5f35a28bfb\": container with ID starting with 1e1b85fe83e0ed8ab235c45eac52b46ef8207f02714ae6e710e33a5f35a28bfb not found: ID does not exist" containerID="1e1b85fe83e0ed8ab235c45eac52b46ef8207f02714ae6e710e33a5f35a28bfb"
Jul 29 18:35:40.932922 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:40.932875    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e1b85fe83e0ed8ab235c45eac52b46ef8207f02714ae6e710e33a5f35a28bfb"} err="failed to get container status \"1e1b85fe83e0ed8ab235c45eac52b46ef8207f02714ae6e710e33a5f35a28bfb\": rpc error: code = NotFound desc = could not find container \"1e1b85fe83e0ed8ab235c45eac52b46ef8207f02714ae6e710e33a5f35a28bfb\": container with ID starting with 1e1b85fe83e0ed8ab235c45eac52b46ef8207f02714ae6e710e33a5f35a28bfb not found: ID does not exist"
Jul 29 18:35:40.932922 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:40.932898    2557 scope.go:117] "RemoveContainer" containerID="668714fe2929f91a75470c46362ec7065a68c18276ff7207cc45ba0db6c5ee86"
Jul 29 18:35:40.943221 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:40.942974    2557 scope.go:117] "RemoveContainer" containerID="668714fe2929f91a75470c46362ec7065a68c18276ff7207cc45ba0db6c5ee86"
Jul 29 18:35:40.945516 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:35:40.945486    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"668714fe2929f91a75470c46362ec7065a68c18276ff7207cc45ba0db6c5ee86\": container with ID starting with 668714fe2929f91a75470c46362ec7065a68c18276ff7207cc45ba0db6c5ee86 not found: ID does not exist" containerID="668714fe2929f91a75470c46362ec7065a68c18276ff7207cc45ba0db6c5ee86"
Jul 29 18:35:40.945629 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:40.945520    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"668714fe2929f91a75470c46362ec7065a68c18276ff7207cc45ba0db6c5ee86"} err="failed to get container status \"668714fe2929f91a75470c46362ec7065a68c18276ff7207cc45ba0db6c5ee86\": rpc error: code = NotFound desc = could not find container \"668714fe2929f91a75470c46362ec7065a68c18276ff7207cc45ba0db6c5ee86\": container with ID starting with 668714fe2929f91a75470c46362ec7065a68c18276ff7207cc45ba0db6c5ee86 not found: ID does not exist"
Jul 29 18:35:40.964482 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:40.964451    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-8be27fe2-8556685cc4-4nffr"]
Jul 29 18:35:40.982893 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:40.982871    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-8be27fe2-8556685cc4-4nffr"]
Jul 29 18:35:41.003503 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:41.003470    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8be27fe2-7c45dd856-mcbdf"]
Jul 29 18:35:41.010661 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:41.010638    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8be27fe2-7c45dd856-mcbdf"]
Jul 29 18:35:42.652633 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:42.652599    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c23e03f3-1656-463d-836e-cad710aae9ba" path="/var/lib/kubelet/pods/c23e03f3-1656-463d-836e-cad710aae9ba/volumes"
Jul 29 18:35:42.653023 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:35:42.652989    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e36c42d0-1a99-4869-866f-0be38435f1da" path="/var/lib/kubelet/pods/e36c42d0-1a99-4869-866f-0be38435f1da/volumes"
Jul 29 18:37:00.875514 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:37:00.875477    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gg2wj_2bf0bb71-4047-45a7-bd8e-5d6efabd26bd/console-operator/2.log"
Jul 29 18:37:00.885674 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:37:00.885647    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gg2wj_2bf0bb71-4047-45a7-bd8e-5d6efabd26bd/console-operator/2.log"
Jul 29 18:38:01.024864 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:38:01.024832    2557 scope.go:117] "RemoveContainer" containerID="eefba6d8d4844dc13847c6ff002021f1bffca020ae83f60b13d5063f76b8f4fc"
Jul 29 18:39:59.490798 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:39:59.490764    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-54bd5a-864d768664-cdv78"]
Jul 29 18:39:59.491274 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:39:59.491169    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b78a49b-66f7-41a3-9bf4-644867a65577" containerName="revoke-keys"
Jul 29 18:39:59.491274 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:39:59.491182    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b78a49b-66f7-41a3-9bf4-644867a65577" containerName="revoke-keys"
Jul 29 18:39:59.491274 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:39:59.491191    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c23e03f3-1656-463d-836e-cad710aae9ba" containerName="payload-processing"
Jul 29 18:39:59.491274 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:39:59.491197    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="c23e03f3-1656-463d-836e-cad710aae9ba" containerName="payload-processing"
Jul 29 18:39:59.491274 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:39:59.491206    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e36c42d0-1a99-4869-866f-0be38435f1da" containerName="maas-api"
Jul 29 18:39:59.491274 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:39:59.491212    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="e36c42d0-1a99-4869-866f-0be38435f1da" containerName="maas-api"
Jul 29 18:39:59.491467 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:39:59.491283    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b78a49b-66f7-41a3-9bf4-644867a65577" containerName="revoke-keys"
Jul 29 18:39:59.491467 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:39:59.491290    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="c23e03f3-1656-463d-836e-cad710aae9ba" containerName="payload-processing"
Jul 29 18:39:59.491467 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:39:59.491296    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="e36c42d0-1a99-4869-866f-0be38435f1da" containerName="maas-api"
Jul 29 18:39:59.494685 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:39:59.494664    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-54bd5a-864d768664-cdv78"
Jul 29 18:39:59.497461 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:39:59.497441    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-54bd5a-serving-cert\""
Jul 29 18:39:59.504923 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:39:59.504899    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-54bd5a-864d768664-cdv78"]
Jul 29 18:39:59.569512 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:39:59.569483    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aebb0d8f-114d-4c62-a489-79e69176404d-maas-api-tls\") pod \"maas-api-e2e-inf-a-54bd5a-864d768664-cdv78\" (UID: \"aebb0d8f-114d-4c62-a489-79e69176404d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-54bd5a-864d768664-cdv78"
Jul 29 18:39:59.569512 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:39:59.569517    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aebb0d8f-114d-4c62-a489-79e69176404d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-54bd5a-864d768664-cdv78\" (UID: \"aebb0d8f-114d-4c62-a489-79e69176404d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-54bd5a-864d768664-cdv78"
Jul 29 18:39:59.569737 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:39:59.569541    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mnq4\" (UniqueName: \"kubernetes.io/projected/aebb0d8f-114d-4c62-a489-79e69176404d-kube-api-access-8mnq4\") pod \"maas-api-e2e-inf-a-54bd5a-864d768664-cdv78\" (UID: \"aebb0d8f-114d-4c62-a489-79e69176404d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-54bd5a-864d768664-cdv78"
Jul 29 18:39:59.670273 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:39:59.670237    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aebb0d8f-114d-4c62-a489-79e69176404d-maas-api-tls\") pod \"maas-api-e2e-inf-a-54bd5a-864d768664-cdv78\" (UID: \"aebb0d8f-114d-4c62-a489-79e69176404d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-54bd5a-864d768664-cdv78"
Jul 29 18:39:59.670273 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:39:59.670279    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aebb0d8f-114d-4c62-a489-79e69176404d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-54bd5a-864d768664-cdv78\" (UID: \"aebb0d8f-114d-4c62-a489-79e69176404d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-54bd5a-864d768664-cdv78"
Jul 29 18:39:59.670490 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:39:59.670307    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mnq4\" (UniqueName: \"kubernetes.io/projected/aebb0d8f-114d-4c62-a489-79e69176404d-kube-api-access-8mnq4\") pod \"maas-api-e2e-inf-a-54bd5a-864d768664-cdv78\" (UID: \"aebb0d8f-114d-4c62-a489-79e69176404d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-54bd5a-864d768664-cdv78"
Jul 29 18:39:59.670490 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:39:59.670403    2557 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-54bd5a-serving-cert: secret "maas-api-e2e-inf-a-54bd5a-serving-cert" not found
Jul 29 18:39:59.670490 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:39:59.670485    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aebb0d8f-114d-4c62-a489-79e69176404d-maas-api-tls podName:aebb0d8f-114d-4c62-a489-79e69176404d nodeName:}" failed. No retries permitted until 2026-07-29 18:40:00.17046399 +0000 UTC m=+2880.039686784 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/aebb0d8f-114d-4c62-a489-79e69176404d-maas-api-tls") pod "maas-api-e2e-inf-a-54bd5a-864d768664-cdv78" (UID: "aebb0d8f-114d-4c62-a489-79e69176404d") : secret "maas-api-e2e-inf-a-54bd5a-serving-cert" not found
Jul 29 18:39:59.670999 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:39:59.670979    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aebb0d8f-114d-4c62-a489-79e69176404d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-54bd5a-864d768664-cdv78\" (UID: \"aebb0d8f-114d-4c62-a489-79e69176404d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-54bd5a-864d768664-cdv78"
Jul 29 18:39:59.678974 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:39:59.678954    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mnq4\" (UniqueName: \"kubernetes.io/projected/aebb0d8f-114d-4c62-a489-79e69176404d-kube-api-access-8mnq4\") pod \"maas-api-e2e-inf-a-54bd5a-864d768664-cdv78\" (UID: \"aebb0d8f-114d-4c62-a489-79e69176404d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-54bd5a-864d768664-cdv78"
Jul 29 18:40:00.168783 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:00.168750    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-54bd5a-7ddfc687fb-6fbcs"]
Jul 29 18:40:00.172363 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:00.172346    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-54bd5a-7ddfc687fb-6fbcs"
Jul 29 18:40:00.173663 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:00.173639    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aebb0d8f-114d-4c62-a489-79e69176404d-maas-api-tls\") pod \"maas-api-e2e-inf-a-54bd5a-864d768664-cdv78\" (UID: \"aebb0d8f-114d-4c62-a489-79e69176404d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-54bd5a-864d768664-cdv78"
Jul 29 18:40:00.175159 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:00.175136    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-54bd5a\""
Jul 29 18:40:00.176238 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:00.176220    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aebb0d8f-114d-4c62-a489-79e69176404d-maas-api-tls\") pod \"maas-api-e2e-inf-a-54bd5a-864d768664-cdv78\" (UID: \"aebb0d8f-114d-4c62-a489-79e69176404d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-54bd5a-864d768664-cdv78"
Jul 29 18:40:00.186181 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:00.186152    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-54bd5a-7ddfc687fb-6fbcs"]
Jul 29 18:40:00.274695 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:00.274662    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/876ee961-1d59-4b6e-91fa-d9c0b248f5f4-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-54bd5a-7ddfc687fb-6fbcs\" (UID: \"876ee961-1d59-4b6e-91fa-d9c0b248f5f4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-54bd5a-7ddfc687fb-6fbcs"
Jul 29 18:40:00.274887 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:00.274778    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qn5pg\" (UniqueName: \"kubernetes.io/projected/876ee961-1d59-4b6e-91fa-d9c0b248f5f4-kube-api-access-qn5pg\") pod \"payload-pre-processing-e2e-inf-a-54bd5a-7ddfc687fb-6fbcs\" (UID: \"876ee961-1d59-4b6e-91fa-d9c0b248f5f4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-54bd5a-7ddfc687fb-6fbcs"
Jul 29 18:40:00.375977 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:00.375936    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qn5pg\" (UniqueName: \"kubernetes.io/projected/876ee961-1d59-4b6e-91fa-d9c0b248f5f4-kube-api-access-qn5pg\") pod \"payload-pre-processing-e2e-inf-a-54bd5a-7ddfc687fb-6fbcs\" (UID: \"876ee961-1d59-4b6e-91fa-d9c0b248f5f4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-54bd5a-7ddfc687fb-6fbcs"
Jul 29 18:40:00.376151 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:00.376000    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/876ee961-1d59-4b6e-91fa-d9c0b248f5f4-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-54bd5a-7ddfc687fb-6fbcs\" (UID: \"876ee961-1d59-4b6e-91fa-d9c0b248f5f4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-54bd5a-7ddfc687fb-6fbcs"
Jul 29 18:40:00.376737 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:00.376698    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/876ee961-1d59-4b6e-91fa-d9c0b248f5f4-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-54bd5a-7ddfc687fb-6fbcs\" (UID: \"876ee961-1d59-4b6e-91fa-d9c0b248f5f4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-54bd5a-7ddfc687fb-6fbcs"
Jul 29 18:40:00.389291 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:00.389266    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qn5pg\" (UniqueName: \"kubernetes.io/projected/876ee961-1d59-4b6e-91fa-d9c0b248f5f4-kube-api-access-qn5pg\") pod \"payload-pre-processing-e2e-inf-a-54bd5a-7ddfc687fb-6fbcs\" (UID: \"876ee961-1d59-4b6e-91fa-d9c0b248f5f4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-54bd5a-7ddfc687fb-6fbcs"
Jul 29 18:40:00.407138 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:00.407113    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-54bd5a-864d768664-cdv78"
Jul 29 18:40:00.493097 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:00.493070    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-54bd5a-7ddfc687fb-6fbcs"
Jul 29 18:40:00.554717 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:00.554681    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-54bd5a-864d768664-cdv78"]
Jul 29 18:40:00.556919 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:40:00.556885    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaebb0d8f_114d_4c62_a489_79e69176404d.slice/crio-138acb6bfa1100368e47c832353bc383be61524c644e44bac3140013b2ff5326 WatchSource:0}: Error finding container 138acb6bfa1100368e47c832353bc383be61524c644e44bac3140013b2ff5326: Status 404 returned error can't find the container with id 138acb6bfa1100368e47c832353bc383be61524c644e44bac3140013b2ff5326
Jul 29 18:40:00.559239 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:00.558842    2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 18:40:00.641581 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:00.641554    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-54bd5a-7ddfc687fb-6fbcs"]
Jul 29 18:40:00.643080 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:40:00.643052    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod876ee961_1d59_4b6e_91fa_d9c0b248f5f4.slice/crio-8e748643e88e0e17805624ff840f111217bb24244c27d825f28391a4b60bea9a WatchSource:0}: Error finding container 8e748643e88e0e17805624ff840f111217bb24244c27d825f28391a4b60bea9a: Status 404 returned error can't find the container with id 8e748643e88e0e17805624ff840f111217bb24244c27d825f28391a4b60bea9a
Jul 29 18:40:01.392409 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:01.391940    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-54bd5a-864d768664-cdv78" event={"ID":"aebb0d8f-114d-4c62-a489-79e69176404d","Type":"ContainerStarted","Data":"5a9ce449c9587496cd0769effc4a6e12d13677a70f61d58455bdcb542869fa0a"}
Jul 29 18:40:01.392409 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:01.392011    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-54bd5a-864d768664-cdv78" event={"ID":"aebb0d8f-114d-4c62-a489-79e69176404d","Type":"ContainerStarted","Data":"138acb6bfa1100368e47c832353bc383be61524c644e44bac3140013b2ff5326"}
Jul 29 18:40:01.392409 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:01.392162    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-54bd5a-864d768664-cdv78"
Jul 29 18:40:01.393799 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:01.393775    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-54bd5a-7ddfc687fb-6fbcs" event={"ID":"876ee961-1d59-4b6e-91fa-d9c0b248f5f4","Type":"ContainerStarted","Data":"c55e6eb4e77afa93a880e9b3b0443d6a64955334b020e4a3c8814e395770a8b2"}
Jul 29 18:40:01.393942 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:01.393812    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-54bd5a-7ddfc687fb-6fbcs" event={"ID":"876ee961-1d59-4b6e-91fa-d9c0b248f5f4","Type":"ContainerStarted","Data":"8e748643e88e0e17805624ff840f111217bb24244c27d825f28391a4b60bea9a"}
Jul 29 18:40:01.393942 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:01.393905    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-54bd5a-7ddfc687fb-6fbcs"
Jul 29 18:40:01.409935 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:01.409868    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-54bd5a-864d768664-cdv78" podStartSLOduration=1.9104251890000001 podStartE2EDuration="2.409854355s" totalImagesPullingTime="499.429166ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:39:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:40:01.407359387 +0000 UTC m=+2881.276582197" watchObservedRunningTime="2026-07-29 18:40:01.409854355 +0000 UTC m=+2881.279077162"
Jul 29 18:40:01.421761 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:01.421694    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-54bd5a-7ddfc687fb-6fbcs" podStartSLOduration=1.4216829770000001 podStartE2EDuration="1.421682977s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:40:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:40:01.421294265 +0000 UTC m=+2881.290517075" watchObservedRunningTime="2026-07-29 18:40:01.421682977 +0000 UTC m=+2881.290905783"
Jul 29 18:40:07.405127 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:07.405095    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-54bd5a-864d768664-cdv78"
Jul 29 18:40:12.401787 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:12.401754    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-54bd5a-7ddfc687fb-6fbcs"
Jul 29 18:40:27.595489 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:27.595446    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-084953-66f7c689d9-q7v9n"]
Jul 29 18:40:27.599228 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:27.599204    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-084953-66f7c689d9-q7v9n"
Jul 29 18:40:27.601715 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:27.601694    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-084953-serving-cert\""
Jul 29 18:40:27.610949 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:27.610926    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-084953-66f7c689d9-q7v9n"]
Jul 29 18:40:27.725277 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:27.725241    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2eafeab0-8797-4592-9a8f-2f8390e7f67c-openshift-service-ca\") pod \"maas-api-e2e-inf-b-084953-66f7c689d9-q7v9n\" (UID: \"2eafeab0-8797-4592-9a8f-2f8390e7f67c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-084953-66f7c689d9-q7v9n"
Jul 29 18:40:27.725450 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:27.725288    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2eafeab0-8797-4592-9a8f-2f8390e7f67c-maas-api-tls\") pod \"maas-api-e2e-inf-b-084953-66f7c689d9-q7v9n\" (UID: \"2eafeab0-8797-4592-9a8f-2f8390e7f67c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-084953-66f7c689d9-q7v9n"
Jul 29 18:40:27.725450 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:27.725398    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8r2hp\" (UniqueName: \"kubernetes.io/projected/2eafeab0-8797-4592-9a8f-2f8390e7f67c-kube-api-access-8r2hp\") pod \"maas-api-e2e-inf-b-084953-66f7c689d9-q7v9n\" (UID: \"2eafeab0-8797-4592-9a8f-2f8390e7f67c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-084953-66f7c689d9-q7v9n"
Jul 29 18:40:27.826353 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:27.826318    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2eafeab0-8797-4592-9a8f-2f8390e7f67c-openshift-service-ca\") pod \"maas-api-e2e-inf-b-084953-66f7c689d9-q7v9n\" (UID: \"2eafeab0-8797-4592-9a8f-2f8390e7f67c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-084953-66f7c689d9-q7v9n"
Jul 29 18:40:27.826513 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:27.826368    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2eafeab0-8797-4592-9a8f-2f8390e7f67c-maas-api-tls\") pod \"maas-api-e2e-inf-b-084953-66f7c689d9-q7v9n\" (UID: \"2eafeab0-8797-4592-9a8f-2f8390e7f67c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-084953-66f7c689d9-q7v9n"
Jul 29 18:40:27.826513 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:27.826458    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8r2hp\" (UniqueName: \"kubernetes.io/projected/2eafeab0-8797-4592-9a8f-2f8390e7f67c-kube-api-access-8r2hp\") pod \"maas-api-e2e-inf-b-084953-66f7c689d9-q7v9n\" (UID: \"2eafeab0-8797-4592-9a8f-2f8390e7f67c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-084953-66f7c689d9-q7v9n"
Jul 29 18:40:27.826597 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:40:27.826547    2557 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-084953-serving-cert: secret "maas-api-e2e-inf-b-084953-serving-cert" not found
Jul 29 18:40:27.826645 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:40:27.826633    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2eafeab0-8797-4592-9a8f-2f8390e7f67c-maas-api-tls podName:2eafeab0-8797-4592-9a8f-2f8390e7f67c nodeName:}" failed. No retries permitted until 2026-07-29 18:40:28.326608998 +0000 UTC m=+2908.195831785 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2eafeab0-8797-4592-9a8f-2f8390e7f67c-maas-api-tls") pod "maas-api-e2e-inf-b-084953-66f7c689d9-q7v9n" (UID: "2eafeab0-8797-4592-9a8f-2f8390e7f67c") : secret "maas-api-e2e-inf-b-084953-serving-cert" not found
Jul 29 18:40:27.826998 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:27.826974    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2eafeab0-8797-4592-9a8f-2f8390e7f67c-openshift-service-ca\") pod \"maas-api-e2e-inf-b-084953-66f7c689d9-q7v9n\" (UID: \"2eafeab0-8797-4592-9a8f-2f8390e7f67c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-084953-66f7c689d9-q7v9n"
Jul 29 18:40:27.841444 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:27.841424    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8r2hp\" (UniqueName: \"kubernetes.io/projected/2eafeab0-8797-4592-9a8f-2f8390e7f67c-kube-api-access-8r2hp\") pod \"maas-api-e2e-inf-b-084953-66f7c689d9-q7v9n\" (UID: \"2eafeab0-8797-4592-9a8f-2f8390e7f67c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-084953-66f7c689d9-q7v9n"
Jul 29 18:40:28.226575 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:28.226542    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-084953-79cdf9c48b-6cbdh"]
Jul 29 18:40:28.230202 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:28.230186    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-084953-79cdf9c48b-6cbdh"
Jul 29 18:40:28.233150 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:28.233124    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-084953\""
Jul 29 18:40:28.233270 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:28.233162    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-084953-dockercfg-j85dq\""
Jul 29 18:40:28.240747 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:28.240706    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-084953-79cdf9c48b-6cbdh"]
Jul 29 18:40:28.331996 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:28.331965    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2eafeab0-8797-4592-9a8f-2f8390e7f67c-maas-api-tls\") pod \"maas-api-e2e-inf-b-084953-66f7c689d9-q7v9n\" (UID: \"2eafeab0-8797-4592-9a8f-2f8390e7f67c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-084953-66f7c689d9-q7v9n"
Jul 29 18:40:28.332157 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:28.332077    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d7f17559-ab00-4ae6-bf6b-2b50d84d1d24-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-084953-79cdf9c48b-6cbdh\" (UID: \"d7f17559-ab00-4ae6-bf6b-2b50d84d1d24\") " pod="openshift-ingress/payload-processing-e2e-inf-b-084953-79cdf9c48b-6cbdh"
Jul 29 18:40:28.332157 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:28.332110    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k592k\" (UniqueName: \"kubernetes.io/projected/d7f17559-ab00-4ae6-bf6b-2b50d84d1d24-kube-api-access-k592k\") pod \"payload-processing-e2e-inf-b-084953-79cdf9c48b-6cbdh\" (UID: \"d7f17559-ab00-4ae6-bf6b-2b50d84d1d24\") " pod="openshift-ingress/payload-processing-e2e-inf-b-084953-79cdf9c48b-6cbdh"
Jul 29 18:40:28.334821 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:28.334796    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2eafeab0-8797-4592-9a8f-2f8390e7f67c-maas-api-tls\") pod \"maas-api-e2e-inf-b-084953-66f7c689d9-q7v9n\" (UID: \"2eafeab0-8797-4592-9a8f-2f8390e7f67c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-084953-66f7c689d9-q7v9n"
Jul 29 18:40:28.433062 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:28.433028    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d7f17559-ab00-4ae6-bf6b-2b50d84d1d24-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-084953-79cdf9c48b-6cbdh\" (UID: \"d7f17559-ab00-4ae6-bf6b-2b50d84d1d24\") " pod="openshift-ingress/payload-processing-e2e-inf-b-084953-79cdf9c48b-6cbdh"
Jul 29 18:40:28.433062 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:28.433061    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k592k\" (UniqueName: \"kubernetes.io/projected/d7f17559-ab00-4ae6-bf6b-2b50d84d1d24-kube-api-access-k592k\") pod \"payload-processing-e2e-inf-b-084953-79cdf9c48b-6cbdh\" (UID: \"d7f17559-ab00-4ae6-bf6b-2b50d84d1d24\") " pod="openshift-ingress/payload-processing-e2e-inf-b-084953-79cdf9c48b-6cbdh"
Jul 29 18:40:28.433783 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:28.433758    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d7f17559-ab00-4ae6-bf6b-2b50d84d1d24-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-084953-79cdf9c48b-6cbdh\" (UID: \"d7f17559-ab00-4ae6-bf6b-2b50d84d1d24\") " pod="openshift-ingress/payload-processing-e2e-inf-b-084953-79cdf9c48b-6cbdh"
Jul 29 18:40:28.441389 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:28.441363    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k592k\" (UniqueName: \"kubernetes.io/projected/d7f17559-ab00-4ae6-bf6b-2b50d84d1d24-kube-api-access-k592k\") pod \"payload-processing-e2e-inf-b-084953-79cdf9c48b-6cbdh\" (UID: \"d7f17559-ab00-4ae6-bf6b-2b50d84d1d24\") " pod="openshift-ingress/payload-processing-e2e-inf-b-084953-79cdf9c48b-6cbdh"
Jul 29 18:40:28.510906 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:28.510845    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-084953-66f7c689d9-q7v9n"
Jul 29 18:40:28.543473 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:28.543451    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-084953-79cdf9c48b-6cbdh"
Jul 29 18:40:28.673342 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:28.673261    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-084953-66f7c689d9-q7v9n"]
Jul 29 18:40:28.674456 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:40:28.674427    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2eafeab0_8797_4592_9a8f_2f8390e7f67c.slice/crio-174ecf381a2138ac26ee3c2d5e765ec3550a7d0fcded42f785fc0c4a68ad5861 WatchSource:0}: Error finding container 174ecf381a2138ac26ee3c2d5e765ec3550a7d0fcded42f785fc0c4a68ad5861: Status 404 returned error can't find the container with id 174ecf381a2138ac26ee3c2d5e765ec3550a7d0fcded42f785fc0c4a68ad5861
Jul 29 18:40:28.702803 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:28.702779    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-084953-79cdf9c48b-6cbdh"]
Jul 29 18:40:28.703579 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:40:28.703554    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7f17559_ab00_4ae6_bf6b_2b50d84d1d24.slice/crio-3436b652724a9d44939fe3ccd673328f9e00b01c499dace887acf5d4bcd6bd61 WatchSource:0}: Error finding container 3436b652724a9d44939fe3ccd673328f9e00b01c499dace887acf5d4bcd6bd61: Status 404 returned error can't find the container with id 3436b652724a9d44939fe3ccd673328f9e00b01c499dace887acf5d4bcd6bd61
Jul 29 18:40:29.574945 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:29.574895    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-084953-66f7c689d9-q7v9n" event={"ID":"2eafeab0-8797-4592-9a8f-2f8390e7f67c","Type":"ContainerStarted","Data":"70b415ce5f94b524221988093705a06ea4afa49d550320b911e35008aec1057e"}
Jul 29 18:40:29.575162 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:29.574966    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-084953-66f7c689d9-q7v9n"
Jul 29 18:40:29.575162 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:29.574990    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-084953-66f7c689d9-q7v9n" event={"ID":"2eafeab0-8797-4592-9a8f-2f8390e7f67c","Type":"ContainerStarted","Data":"174ecf381a2138ac26ee3c2d5e765ec3550a7d0fcded42f785fc0c4a68ad5861"}
Jul 29 18:40:29.576584 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:29.576559    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-084953-79cdf9c48b-6cbdh" event={"ID":"d7f17559-ab00-4ae6-bf6b-2b50d84d1d24","Type":"ContainerStarted","Data":"20cff3c430ad445f0690d1b0c468ae304fbfd0f2bb6e5da23483e379c9761bb2"}
Jul 29 18:40:29.576718 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:29.576602    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-084953-79cdf9c48b-6cbdh" event={"ID":"d7f17559-ab00-4ae6-bf6b-2b50d84d1d24","Type":"ContainerStarted","Data":"3436b652724a9d44939fe3ccd673328f9e00b01c499dace887acf5d4bcd6bd61"}
Jul 29 18:40:29.576718 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:29.576652    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-084953-79cdf9c48b-6cbdh"
Jul 29 18:40:29.589928 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:29.589866    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-084953-66f7c689d9-q7v9n" podStartSLOduration=2.104193538 podStartE2EDuration="2.58985172s" totalImagesPullingTime="485.658182ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:40:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:40:29.589623064 +0000 UTC m=+2909.458845871" watchObservedRunningTime="2026-07-29 18:40:29.58985172 +0000 UTC m=+2909.459074543"
Jul 29 18:40:29.606278 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:29.606237    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-084953-79cdf9c48b-6cbdh" podStartSLOduration=1.606224599 podStartE2EDuration="1.606224599s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:40:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:40:29.60577664 +0000 UTC m=+2909.474999448" watchObservedRunningTime="2026-07-29 18:40:29.606224599 +0000 UTC m=+2909.475447406"
Jul 29 18:40:35.587467 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:35.587432    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-084953-66f7c689d9-q7v9n"
Jul 29 18:40:40.584911 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:40:40.584877    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-084953-79cdf9c48b-6cbdh"
Jul 29 18:42:00.913397 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:00.913367    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gg2wj_2bf0bb71-4047-45a7-bd8e-5d6efabd26bd/console-operator/2.log"
Jul 29 18:42:00.924497 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:00.924479    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gg2wj_2bf0bb71-4047-45a7-bd8e-5d6efabd26bd/console-operator/2.log"
Jul 29 18:42:01.072886 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:01.072853    2557 scope.go:117] "RemoveContainer" containerID="4f87b6b76a8bc0dd027237331a023c00f18943faa9c5bc279d9d3a041a3bdd6b"
Jul 29 18:42:18.038272 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:18.038234    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-54bd5a-l6lk2"]
Jul 29 18:42:18.042191 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:18.042169    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-54bd5a-l6lk2"
Jul 29 18:42:18.044986 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:18.044940    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qrbhl\""
Jul 29 18:42:18.049945 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:18.049871    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-54bd5a-l6lk2"]
Jul 29 18:42:18.130041 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:18.130002    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0be752cb-b18a-407d-a1f6-97f0d8ad967d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-54bd5a-l6lk2\" (UID: \"0be752cb-b18a-407d-a1f6-97f0d8ad967d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-54bd5a-l6lk2"
Jul 29 18:42:18.231156 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:18.231120    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0be752cb-b18a-407d-a1f6-97f0d8ad967d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-54bd5a-l6lk2\" (UID: \"0be752cb-b18a-407d-a1f6-97f0d8ad967d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-54bd5a-l6lk2"
Jul 29 18:42:18.231765 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:18.231711    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0be752cb-b18a-407d-a1f6-97f0d8ad967d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-54bd5a-l6lk2\" (UID: \"0be752cb-b18a-407d-a1f6-97f0d8ad967d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-54bd5a-l6lk2"
Jul 29 18:42:18.355787 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:18.355703    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-54bd5a-l6lk2"
Jul 29 18:42:18.495760 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:18.495708    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-54bd5a-l6lk2"]
Jul 29 18:42:18.497037 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:42:18.497012    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0be752cb_b18a_407d_a1f6_97f0d8ad967d.slice/crio-66cd2c12b53e77549cdf705187cec6ad74faa86836a42bcd2a8cca2c6610b5f2 WatchSource:0}: Error finding container 66cd2c12b53e77549cdf705187cec6ad74faa86836a42bcd2a8cca2c6610b5f2: Status 404 returned error can't find the container with id 66cd2c12b53e77549cdf705187cec6ad74faa86836a42bcd2a8cca2c6610b5f2
Jul 29 18:42:19.202000 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:19.201907    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-54bd5a-l6lk2" event={"ID":"0be752cb-b18a-407d-a1f6-97f0d8ad967d","Type":"ContainerStarted","Data":"fe783fe8c680ecb6b9bcd51d861272b3f9cae1976141c468bdaa33a3255a5910"}
Jul 29 18:42:19.202000 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:19.201981    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-54bd5a-l6lk2" event={"ID":"0be752cb-b18a-407d-a1f6-97f0d8ad967d","Type":"ContainerStarted","Data":"66cd2c12b53e77549cdf705187cec6ad74faa86836a42bcd2a8cca2c6610b5f2"}
Jul 29 18:42:19.216923 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:19.216876    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-54bd5a-l6lk2" podStartSLOduration=1.216861141 podStartE2EDuration="1.216861141s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:42:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:42:19.21510384 +0000 UTC m=+3019.084326650" watchObservedRunningTime="2026-07-29 18:42:19.216861141 +0000 UTC m=+3019.086083949"
Jul 29 18:42:20.209014 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:20.208978    2557 generic.go:356] "Generic (PLEG): container finished" podID="0be752cb-b18a-407d-a1f6-97f0d8ad967d" containerID="fe783fe8c680ecb6b9bcd51d861272b3f9cae1976141c468bdaa33a3255a5910" exitCode=0
Jul 29 18:42:20.209434 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:20.209045    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-54bd5a-l6lk2" event={"ID":"0be752cb-b18a-407d-a1f6-97f0d8ad967d","Type":"ContainerDied","Data":"fe783fe8c680ecb6b9bcd51d861272b3f9cae1976141c468bdaa33a3255a5910"}
Jul 29 18:42:21.353814 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:21.353786    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-54bd5a-l6lk2"
Jul 29 18:42:21.462995 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:21.462961    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0be752cb-b18a-407d-a1f6-97f0d8ad967d-maas-api-service-ca\") pod \"0be752cb-b18a-407d-a1f6-97f0d8ad967d\" (UID: \"0be752cb-b18a-407d-a1f6-97f0d8ad967d\") "
Jul 29 18:42:21.463340 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:21.463316    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0be752cb-b18a-407d-a1f6-97f0d8ad967d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0be752cb-b18a-407d-a1f6-97f0d8ad967d" (UID: "0be752cb-b18a-407d-a1f6-97f0d8ad967d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:42:21.564121 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:21.564094    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0be752cb-b18a-407d-a1f6-97f0d8ad967d-maas-api-service-ca\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:42:22.221664 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:22.221637    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-54bd5a-l6lk2"
Jul 29 18:42:22.221857 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:22.221639    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-54bd5a-l6lk2" event={"ID":"0be752cb-b18a-407d-a1f6-97f0d8ad967d","Type":"ContainerDied","Data":"66cd2c12b53e77549cdf705187cec6ad74faa86836a42bcd2a8cca2c6610b5f2"}
Jul 29 18:42:22.221857 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:22.221772    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="66cd2c12b53e77549cdf705187cec6ad74faa86836a42bcd2a8cca2c6610b5f2"
Jul 29 18:42:28.601913 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:28.601881    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-54bd5a-864d768664-cdv78"]
Jul 29 18:42:28.602361 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:28.602098    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-54bd5a-864d768664-cdv78" podUID="aebb0d8f-114d-4c62-a489-79e69176404d" containerName="maas-api" containerID="cri-o://5a9ce449c9587496cd0769effc4a6e12d13677a70f61d58455bdcb542869fa0a" gracePeriod=30
Jul 29 18:42:28.659154 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:28.659117    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-54bd5a-7ddfc687fb-6fbcs"]
Jul 29 18:42:28.659410 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:28.659359    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-54bd5a-7ddfc687fb-6fbcs" podUID="876ee961-1d59-4b6e-91fa-d9c0b248f5f4" containerName="payload-pre-processing" containerID="cri-o://c55e6eb4e77afa93a880e9b3b0443d6a64955334b020e4a3c8814e395770a8b2" gracePeriod=30
Jul 29 18:42:28.801383 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:28.801287    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-54bd5a-l6lk2"]
Jul 29 18:42:28.804397 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:28.804368    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-54bd5a-l6lk2"]
Jul 29 18:42:28.925707 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:28.925675    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-54bd5a-864d768664-cdv78"
Jul 29 18:42:28.961846 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:28.961818    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-54bd5a-7ddfc687fb-6fbcs"
Jul 29 18:42:29.032252 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:29.032225    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8mnq4\" (UniqueName: \"kubernetes.io/projected/aebb0d8f-114d-4c62-a489-79e69176404d-kube-api-access-8mnq4\") pod \"aebb0d8f-114d-4c62-a489-79e69176404d\" (UID: \"aebb0d8f-114d-4c62-a489-79e69176404d\") "
Jul 29 18:42:29.032445 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:29.032271    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/876ee961-1d59-4b6e-91fa-d9c0b248f5f4-plugins-config-volume\") pod \"876ee961-1d59-4b6e-91fa-d9c0b248f5f4\" (UID: \"876ee961-1d59-4b6e-91fa-d9c0b248f5f4\") "
Jul 29 18:42:29.032445 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:29.032319    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aebb0d8f-114d-4c62-a489-79e69176404d-maas-api-tls\") pod \"aebb0d8f-114d-4c62-a489-79e69176404d\" (UID: \"aebb0d8f-114d-4c62-a489-79e69176404d\") "
Jul 29 18:42:29.032445 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:29.032420    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aebb0d8f-114d-4c62-a489-79e69176404d-openshift-service-ca\") pod \"aebb0d8f-114d-4c62-a489-79e69176404d\" (UID: \"aebb0d8f-114d-4c62-a489-79e69176404d\") "
Jul 29 18:42:29.032609 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:29.032513    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qn5pg\" (UniqueName: \"kubernetes.io/projected/876ee961-1d59-4b6e-91fa-d9c0b248f5f4-kube-api-access-qn5pg\") pod \"876ee961-1d59-4b6e-91fa-d9c0b248f5f4\" (UID: \"876ee961-1d59-4b6e-91fa-d9c0b248f5f4\") "
Jul 29 18:42:29.032771 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:29.032715    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/876ee961-1d59-4b6e-91fa-d9c0b248f5f4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "876ee961-1d59-4b6e-91fa-d9c0b248f5f4" (UID: "876ee961-1d59-4b6e-91fa-d9c0b248f5f4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:42:29.032877 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:29.032772    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aebb0d8f-114d-4c62-a489-79e69176404d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "aebb0d8f-114d-4c62-a489-79e69176404d" (UID: "aebb0d8f-114d-4c62-a489-79e69176404d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:42:29.032933 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:29.032924    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/876ee961-1d59-4b6e-91fa-d9c0b248f5f4-plugins-config-volume\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:42:29.032988 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:29.032936    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aebb0d8f-114d-4c62-a489-79e69176404d-openshift-service-ca\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:42:29.034624 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:29.034595    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/876ee961-1d59-4b6e-91fa-d9c0b248f5f4-kube-api-access-qn5pg" (OuterVolumeSpecName: "kube-api-access-qn5pg") pod "876ee961-1d59-4b6e-91fa-d9c0b248f5f4" (UID: "876ee961-1d59-4b6e-91fa-d9c0b248f5f4"). InnerVolumeSpecName "kube-api-access-qn5pg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:42:29.034624 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:29.034606    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aebb0d8f-114d-4c62-a489-79e69176404d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "aebb0d8f-114d-4c62-a489-79e69176404d" (UID: "aebb0d8f-114d-4c62-a489-79e69176404d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:42:29.034816 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:29.034646    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aebb0d8f-114d-4c62-a489-79e69176404d-kube-api-access-8mnq4" (OuterVolumeSpecName: "kube-api-access-8mnq4") pod "aebb0d8f-114d-4c62-a489-79e69176404d" (UID: "aebb0d8f-114d-4c62-a489-79e69176404d"). InnerVolumeSpecName "kube-api-access-8mnq4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:42:29.134072 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:29.133986    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8mnq4\" (UniqueName: \"kubernetes.io/projected/aebb0d8f-114d-4c62-a489-79e69176404d-kube-api-access-8mnq4\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:42:29.134072 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:29.134026    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aebb0d8f-114d-4c62-a489-79e69176404d-maas-api-tls\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:42:29.134072 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:29.134036    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qn5pg\" (UniqueName: \"kubernetes.io/projected/876ee961-1d59-4b6e-91fa-d9c0b248f5f4-kube-api-access-qn5pg\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:42:29.263047 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:29.263011    2557 generic.go:356] "Generic (PLEG): container finished" podID="aebb0d8f-114d-4c62-a489-79e69176404d" containerID="5a9ce449c9587496cd0769effc4a6e12d13677a70f61d58455bdcb542869fa0a" exitCode=0
Jul 29 18:42:29.263242 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:29.263053    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-54bd5a-864d768664-cdv78"
Jul 29 18:42:29.263242 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:29.263097    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-54bd5a-864d768664-cdv78" event={"ID":"aebb0d8f-114d-4c62-a489-79e69176404d","Type":"ContainerDied","Data":"5a9ce449c9587496cd0769effc4a6e12d13677a70f61d58455bdcb542869fa0a"}
Jul 29 18:42:29.263242 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:29.263143    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-54bd5a-864d768664-cdv78" event={"ID":"aebb0d8f-114d-4c62-a489-79e69176404d","Type":"ContainerDied","Data":"138acb6bfa1100368e47c832353bc383be61524c644e44bac3140013b2ff5326"}
Jul 29 18:42:29.263242 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:29.263163    2557 scope.go:117] "RemoveContainer" containerID="5a9ce449c9587496cd0769effc4a6e12d13677a70f61d58455bdcb542869fa0a"
Jul 29 18:42:29.264567 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:29.264539    2557 generic.go:356] "Generic (PLEG): container finished" podID="876ee961-1d59-4b6e-91fa-d9c0b248f5f4" containerID="c55e6eb4e77afa93a880e9b3b0443d6a64955334b020e4a3c8814e395770a8b2" exitCode=0
Jul 29 18:42:29.264671 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:29.264593    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-54bd5a-7ddfc687fb-6fbcs"
Jul 29 18:42:29.264671 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:29.264633    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-54bd5a-7ddfc687fb-6fbcs" event={"ID":"876ee961-1d59-4b6e-91fa-d9c0b248f5f4","Type":"ContainerDied","Data":"c55e6eb4e77afa93a880e9b3b0443d6a64955334b020e4a3c8814e395770a8b2"}
Jul 29 18:42:29.264942 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:29.264676    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-54bd5a-7ddfc687fb-6fbcs" event={"ID":"876ee961-1d59-4b6e-91fa-d9c0b248f5f4","Type":"ContainerDied","Data":"8e748643e88e0e17805624ff840f111217bb24244c27d825f28391a4b60bea9a"}
Jul 29 18:42:29.274343 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:29.274327    2557 scope.go:117] "RemoveContainer" containerID="5a9ce449c9587496cd0769effc4a6e12d13677a70f61d58455bdcb542869fa0a"
Jul 29 18:42:29.274629 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:42:29.274608    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a9ce449c9587496cd0769effc4a6e12d13677a70f61d58455bdcb542869fa0a\": container with ID starting with 5a9ce449c9587496cd0769effc4a6e12d13677a70f61d58455bdcb542869fa0a not found: ID does not exist" containerID="5a9ce449c9587496cd0769effc4a6e12d13677a70f61d58455bdcb542869fa0a"
Jul 29 18:42:29.274783 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:29.274634    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a9ce449c9587496cd0769effc4a6e12d13677a70f61d58455bdcb542869fa0a"} err="failed to get container status \"5a9ce449c9587496cd0769effc4a6e12d13677a70f61d58455bdcb542869fa0a\": rpc error: code = NotFound desc = could not find container \"5a9ce449c9587496cd0769effc4a6e12d13677a70f61d58455bdcb542869fa0a\": container with ID starting with 5a9ce449c9587496cd0769effc4a6e12d13677a70f61d58455bdcb542869fa0a not found: ID does not exist"
Jul 29 18:42:29.274783 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:29.274653    2557 scope.go:117] "RemoveContainer" containerID="c55e6eb4e77afa93a880e9b3b0443d6a64955334b020e4a3c8814e395770a8b2"
Jul 29 18:42:29.282976 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:29.282961    2557 scope.go:117] "RemoveContainer" containerID="c55e6eb4e77afa93a880e9b3b0443d6a64955334b020e4a3c8814e395770a8b2"
Jul 29 18:42:29.283219 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:42:29.283199    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c55e6eb4e77afa93a880e9b3b0443d6a64955334b020e4a3c8814e395770a8b2\": container with ID starting with c55e6eb4e77afa93a880e9b3b0443d6a64955334b020e4a3c8814e395770a8b2 not found: ID does not exist" containerID="c55e6eb4e77afa93a880e9b3b0443d6a64955334b020e4a3c8814e395770a8b2"
Jul 29 18:42:29.283281 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:29.283229    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c55e6eb4e77afa93a880e9b3b0443d6a64955334b020e4a3c8814e395770a8b2"} err="failed to get container status \"c55e6eb4e77afa93a880e9b3b0443d6a64955334b020e4a3c8814e395770a8b2\": rpc error: code = NotFound desc = could not find container \"c55e6eb4e77afa93a880e9b3b0443d6a64955334b020e4a3c8814e395770a8b2\": container with ID starting with c55e6eb4e77afa93a880e9b3b0443d6a64955334b020e4a3c8814e395770a8b2 not found: ID does not exist"
Jul 29 18:42:29.289938 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:29.289918    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-54bd5a-864d768664-cdv78"]
Jul 29 18:42:29.293911 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:29.293892    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-54bd5a-864d768664-cdv78"]
Jul 29 18:42:29.304673 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:29.304654    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-54bd5a-7ddfc687fb-6fbcs"]
Jul 29 18:42:29.306963 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:29.306945    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-54bd5a-7ddfc687fb-6fbcs"]
Jul 29 18:42:30.652575 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:30.652541    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0be752cb-b18a-407d-a1f6-97f0d8ad967d" path="/var/lib/kubelet/pods/0be752cb-b18a-407d-a1f6-97f0d8ad967d/volumes"
Jul 29 18:42:30.652973 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:30.652900    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="876ee961-1d59-4b6e-91fa-d9c0b248f5f4" path="/var/lib/kubelet/pods/876ee961-1d59-4b6e-91fa-d9c0b248f5f4/volumes"
Jul 29 18:42:30.653236 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:30.653223    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aebb0d8f-114d-4c62-a489-79e69176404d" path="/var/lib/kubelet/pods/aebb0d8f-114d-4c62-a489-79e69176404d/volumes"
Jul 29 18:42:35.653902 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:35.653864    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-084953-lknpj"]
Jul 29 18:42:35.654489 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:35.654468    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aebb0d8f-114d-4c62-a489-79e69176404d" containerName="maas-api"
Jul 29 18:42:35.654544 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:35.654496    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="aebb0d8f-114d-4c62-a489-79e69176404d" containerName="maas-api"
Jul 29 18:42:35.654544 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:35.654507    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0be752cb-b18a-407d-a1f6-97f0d8ad967d" containerName="revoke-keys"
Jul 29 18:42:35.654544 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:35.654516    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="0be752cb-b18a-407d-a1f6-97f0d8ad967d" containerName="revoke-keys"
Jul 29 18:42:35.654656 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:35.654558    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="876ee961-1d59-4b6e-91fa-d9c0b248f5f4" containerName="payload-pre-processing"
Jul 29 18:42:35.654656 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:35.654568    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="876ee961-1d59-4b6e-91fa-d9c0b248f5f4" containerName="payload-pre-processing"
Jul 29 18:42:35.654656 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:35.654643    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="876ee961-1d59-4b6e-91fa-d9c0b248f5f4" containerName="payload-pre-processing"
Jul 29 18:42:35.654656 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:35.654651    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="aebb0d8f-114d-4c62-a489-79e69176404d" containerName="maas-api"
Jul 29 18:42:35.654656 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:35.654659    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="0be752cb-b18a-407d-a1f6-97f0d8ad967d" containerName="revoke-keys"
Jul 29 18:42:35.659395 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:35.659371    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-084953-lknpj"
Jul 29 18:42:35.662309 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:35.662283    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qrbhl\""
Jul 29 18:42:35.662889 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:35.662866    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-084953-lknpj"]
Jul 29 18:42:35.794673 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:35.794635    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1424cbca-8a6a-44b3-9eb2-9a1319a70a9a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-084953-lknpj\" (UID: \"1424cbca-8a6a-44b3-9eb2-9a1319a70a9a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-084953-lknpj"
Jul 29 18:42:35.896091 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:35.896052    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1424cbca-8a6a-44b3-9eb2-9a1319a70a9a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-084953-lknpj\" (UID: \"1424cbca-8a6a-44b3-9eb2-9a1319a70a9a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-084953-lknpj"
Jul 29 18:42:35.896791 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:35.896771    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1424cbca-8a6a-44b3-9eb2-9a1319a70a9a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-084953-lknpj\" (UID: \"1424cbca-8a6a-44b3-9eb2-9a1319a70a9a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-084953-lknpj"
Jul 29 18:42:35.972295 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:35.972213    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-084953-lknpj"
Jul 29 18:42:36.131933 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:36.131886    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-084953-lknpj"]
Jul 29 18:42:36.133024 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:42:36.132994    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1424cbca_8a6a_44b3_9eb2_9a1319a70a9a.slice/crio-6444e59abbabdea1451425b5044c72fc1ca564f55b3c8c3a1457c8550cb39c1b WatchSource:0}: Error finding container 6444e59abbabdea1451425b5044c72fc1ca564f55b3c8c3a1457c8550cb39c1b: Status 404 returned error can't find the container with id 6444e59abbabdea1451425b5044c72fc1ca564f55b3c8c3a1457c8550cb39c1b
Jul 29 18:42:36.307963 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:36.307920    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-084953-lknpj" event={"ID":"1424cbca-8a6a-44b3-9eb2-9a1319a70a9a","Type":"ContainerStarted","Data":"6444e59abbabdea1451425b5044c72fc1ca564f55b3c8c3a1457c8550cb39c1b"}
Jul 29 18:42:37.322184 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:37.322151    2557 generic.go:356] "Generic (PLEG): container finished" podID="1424cbca-8a6a-44b3-9eb2-9a1319a70a9a" containerID="7f6220ea4ace30a1db0a317f7c69400ebe8e074d8dc652565f2bc88870e5a06d" exitCode=0
Jul 29 18:42:37.322557 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:37.322230    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-084953-lknpj" event={"ID":"1424cbca-8a6a-44b3-9eb2-9a1319a70a9a","Type":"ContainerDied","Data":"7f6220ea4ace30a1db0a317f7c69400ebe8e074d8dc652565f2bc88870e5a06d"}
Jul 29 18:42:38.469369 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:38.469343    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-084953-lknpj"
Jul 29 18:42:38.621432 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:38.621357    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1424cbca-8a6a-44b3-9eb2-9a1319a70a9a-maas-api-service-ca\") pod \"1424cbca-8a6a-44b3-9eb2-9a1319a70a9a\" (UID: \"1424cbca-8a6a-44b3-9eb2-9a1319a70a9a\") "
Jul 29 18:42:38.621685 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:38.621662    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1424cbca-8a6a-44b3-9eb2-9a1319a70a9a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1424cbca-8a6a-44b3-9eb2-9a1319a70a9a" (UID: "1424cbca-8a6a-44b3-9eb2-9a1319a70a9a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:42:38.723143 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:38.723109    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1424cbca-8a6a-44b3-9eb2-9a1319a70a9a-maas-api-service-ca\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:42:39.336441 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:39.336393    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-084953-lknpj"
Jul 29 18:42:39.336441 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:39.336423    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-084953-lknpj" event={"ID":"1424cbca-8a6a-44b3-9eb2-9a1319a70a9a","Type":"ContainerDied","Data":"6444e59abbabdea1451425b5044c72fc1ca564f55b3c8c3a1457c8550cb39c1b"}
Jul 29 18:42:39.336707 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:39.336477    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6444e59abbabdea1451425b5044c72fc1ca564f55b3c8c3a1457c8550cb39c1b"
Jul 29 18:42:46.377130 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:46.377095    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-084953-66f7c689d9-q7v9n"]
Jul 29 18:42:46.377658 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:46.377616    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-084953-66f7c689d9-q7v9n" podUID="2eafeab0-8797-4592-9a8f-2f8390e7f67c" containerName="maas-api" containerID="cri-o://70b415ce5f94b524221988093705a06ea4afa49d550320b911e35008aec1057e" gracePeriod=30
Jul 29 18:42:46.432776 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:46.432692    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-084953-79cdf9c48b-6cbdh"]
Jul 29 18:42:46.432996 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:46.432963    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-084953-79cdf9c48b-6cbdh" podUID="d7f17559-ab00-4ae6-bf6b-2b50d84d1d24" containerName="payload-processing" containerID="cri-o://20cff3c430ad445f0690d1b0c468ae304fbfd0f2bb6e5da23483e379c9761bb2" gracePeriod=30
Jul 29 18:42:46.568431 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:46.568398    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-084953-lknpj"]
Jul 29 18:42:46.572512 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:46.572483    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-084953-lknpj"]
Jul 29 18:42:46.656377 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:46.656344    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1424cbca-8a6a-44b3-9eb2-9a1319a70a9a" path="/var/lib/kubelet/pods/1424cbca-8a6a-44b3-9eb2-9a1319a70a9a/volumes"
Jul 29 18:42:46.701280 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:46.701254    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-084953-66f7c689d9-q7v9n"
Jul 29 18:42:46.735888 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:46.735860    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-084953-79cdf9c48b-6cbdh"
Jul 29 18:42:46.801356 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:46.801330    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2eafeab0-8797-4592-9a8f-2f8390e7f67c-openshift-service-ca\") pod \"2eafeab0-8797-4592-9a8f-2f8390e7f67c\" (UID: \"2eafeab0-8797-4592-9a8f-2f8390e7f67c\") "
Jul 29 18:42:46.801495 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:46.801426    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2eafeab0-8797-4592-9a8f-2f8390e7f67c-maas-api-tls\") pod \"2eafeab0-8797-4592-9a8f-2f8390e7f67c\" (UID: \"2eafeab0-8797-4592-9a8f-2f8390e7f67c\") "
Jul 29 18:42:46.801495 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:46.801472    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8r2hp\" (UniqueName: \"kubernetes.io/projected/2eafeab0-8797-4592-9a8f-2f8390e7f67c-kube-api-access-8r2hp\") pod \"2eafeab0-8797-4592-9a8f-2f8390e7f67c\" (UID: \"2eafeab0-8797-4592-9a8f-2f8390e7f67c\") "
Jul 29 18:42:46.801707 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:46.801686    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2eafeab0-8797-4592-9a8f-2f8390e7f67c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2eafeab0-8797-4592-9a8f-2f8390e7f67c" (UID: "2eafeab0-8797-4592-9a8f-2f8390e7f67c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:42:46.802149 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:46.802122    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2eafeab0-8797-4592-9a8f-2f8390e7f67c-openshift-service-ca\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:42:46.803653 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:46.803626    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2eafeab0-8797-4592-9a8f-2f8390e7f67c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2eafeab0-8797-4592-9a8f-2f8390e7f67c" (UID: "2eafeab0-8797-4592-9a8f-2f8390e7f67c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:42:46.803653 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:46.803635    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2eafeab0-8797-4592-9a8f-2f8390e7f67c-kube-api-access-8r2hp" (OuterVolumeSpecName: "kube-api-access-8r2hp") pod "2eafeab0-8797-4592-9a8f-2f8390e7f67c" (UID: "2eafeab0-8797-4592-9a8f-2f8390e7f67c"). InnerVolumeSpecName "kube-api-access-8r2hp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:42:46.903435 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:46.903363    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k592k\" (UniqueName: \"kubernetes.io/projected/d7f17559-ab00-4ae6-bf6b-2b50d84d1d24-kube-api-access-k592k\") pod \"d7f17559-ab00-4ae6-bf6b-2b50d84d1d24\" (UID: \"d7f17559-ab00-4ae6-bf6b-2b50d84d1d24\") "
Jul 29 18:42:46.903435 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:46.903425    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d7f17559-ab00-4ae6-bf6b-2b50d84d1d24-plugins-config-volume\") pod \"d7f17559-ab00-4ae6-bf6b-2b50d84d1d24\" (UID: \"d7f17559-ab00-4ae6-bf6b-2b50d84d1d24\") "
Jul 29 18:42:46.903645 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:46.903585    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8r2hp\" (UniqueName: \"kubernetes.io/projected/2eafeab0-8797-4592-9a8f-2f8390e7f67c-kube-api-access-8r2hp\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:42:46.903645 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:46.903597    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2eafeab0-8797-4592-9a8f-2f8390e7f67c-maas-api-tls\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:42:46.903822 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:46.903800    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7f17559-ab00-4ae6-bf6b-2b50d84d1d24-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d7f17559-ab00-4ae6-bf6b-2b50d84d1d24" (UID: "d7f17559-ab00-4ae6-bf6b-2b50d84d1d24"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:42:46.905609 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:46.905578    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7f17559-ab00-4ae6-bf6b-2b50d84d1d24-kube-api-access-k592k" (OuterVolumeSpecName: "kube-api-access-k592k") pod "d7f17559-ab00-4ae6-bf6b-2b50d84d1d24" (UID: "d7f17559-ab00-4ae6-bf6b-2b50d84d1d24"). InnerVolumeSpecName "kube-api-access-k592k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:42:47.005058 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:47.005026    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k592k\" (UniqueName: \"kubernetes.io/projected/d7f17559-ab00-4ae6-bf6b-2b50d84d1d24-kube-api-access-k592k\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:42:47.005058 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:47.005053    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d7f17559-ab00-4ae6-bf6b-2b50d84d1d24-plugins-config-volume\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:42:47.391569 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:47.391527    2557 generic.go:356] "Generic (PLEG): container finished" podID="2eafeab0-8797-4592-9a8f-2f8390e7f67c" containerID="70b415ce5f94b524221988093705a06ea4afa49d550320b911e35008aec1057e" exitCode=0
Jul 29 18:42:47.391569 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:47.391568    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-084953-66f7c689d9-q7v9n"
Jul 29 18:42:47.392106 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:47.391610    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-084953-66f7c689d9-q7v9n" event={"ID":"2eafeab0-8797-4592-9a8f-2f8390e7f67c","Type":"ContainerDied","Data":"70b415ce5f94b524221988093705a06ea4afa49d550320b911e35008aec1057e"}
Jul 29 18:42:47.392106 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:47.391661    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-084953-66f7c689d9-q7v9n" event={"ID":"2eafeab0-8797-4592-9a8f-2f8390e7f67c","Type":"ContainerDied","Data":"174ecf381a2138ac26ee3c2d5e765ec3550a7d0fcded42f785fc0c4a68ad5861"}
Jul 29 18:42:47.392106 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:47.391684    2557 scope.go:117] "RemoveContainer" containerID="70b415ce5f94b524221988093705a06ea4afa49d550320b911e35008aec1057e"
Jul 29 18:42:47.393320 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:47.393295    2557 generic.go:356] "Generic (PLEG): container finished" podID="d7f17559-ab00-4ae6-bf6b-2b50d84d1d24" containerID="20cff3c430ad445f0690d1b0c468ae304fbfd0f2bb6e5da23483e379c9761bb2" exitCode=0
Jul 29 18:42:47.393461 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:47.393345    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-084953-79cdf9c48b-6cbdh"
Jul 29 18:42:47.393461 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:47.393428    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-084953-79cdf9c48b-6cbdh" event={"ID":"d7f17559-ab00-4ae6-bf6b-2b50d84d1d24","Type":"ContainerDied","Data":"20cff3c430ad445f0690d1b0c468ae304fbfd0f2bb6e5da23483e379c9761bb2"}
Jul 29 18:42:47.393572 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:47.393471    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-084953-79cdf9c48b-6cbdh" event={"ID":"d7f17559-ab00-4ae6-bf6b-2b50d84d1d24","Type":"ContainerDied","Data":"3436b652724a9d44939fe3ccd673328f9e00b01c499dace887acf5d4bcd6bd61"}
Jul 29 18:42:47.409182 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:47.409165    2557 scope.go:117] "RemoveContainer" containerID="70b415ce5f94b524221988093705a06ea4afa49d550320b911e35008aec1057e"
Jul 29 18:42:47.409422 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:42:47.409403    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70b415ce5f94b524221988093705a06ea4afa49d550320b911e35008aec1057e\": container with ID starting with 70b415ce5f94b524221988093705a06ea4afa49d550320b911e35008aec1057e not found: ID does not exist" containerID="70b415ce5f94b524221988093705a06ea4afa49d550320b911e35008aec1057e"
Jul 29 18:42:47.409469 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:47.409430    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70b415ce5f94b524221988093705a06ea4afa49d550320b911e35008aec1057e"} err="failed to get container status \"70b415ce5f94b524221988093705a06ea4afa49d550320b911e35008aec1057e\": rpc error: code = NotFound desc = could not find container \"70b415ce5f94b524221988093705a06ea4afa49d550320b911e35008aec1057e\": container with ID starting with 70b415ce5f94b524221988093705a06ea4afa49d550320b911e35008aec1057e not found: ID does not exist"
Jul 29 18:42:47.409469 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:47.409447    2557 scope.go:117] "RemoveContainer" containerID="20cff3c430ad445f0690d1b0c468ae304fbfd0f2bb6e5da23483e379c9761bb2"
Jul 29 18:42:47.417670 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:47.417654    2557 scope.go:117] "RemoveContainer" containerID="20cff3c430ad445f0690d1b0c468ae304fbfd0f2bb6e5da23483e379c9761bb2"
Jul 29 18:42:47.417971 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:42:47.417941    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20cff3c430ad445f0690d1b0c468ae304fbfd0f2bb6e5da23483e379c9761bb2\": container with ID starting with 20cff3c430ad445f0690d1b0c468ae304fbfd0f2bb6e5da23483e379c9761bb2 not found: ID does not exist" containerID="20cff3c430ad445f0690d1b0c468ae304fbfd0f2bb6e5da23483e379c9761bb2"
Jul 29 18:42:47.418014 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:47.417978    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20cff3c430ad445f0690d1b0c468ae304fbfd0f2bb6e5da23483e379c9761bb2"} err="failed to get container status \"20cff3c430ad445f0690d1b0c468ae304fbfd0f2bb6e5da23483e379c9761bb2\": rpc error: code = NotFound desc = could not find container \"20cff3c430ad445f0690d1b0c468ae304fbfd0f2bb6e5da23483e379c9761bb2\": container with ID starting with 20cff3c430ad445f0690d1b0c468ae304fbfd0f2bb6e5da23483e379c9761bb2 not found: ID does not exist"
Jul 29 18:42:47.424383 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:47.424355    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-084953-79cdf9c48b-6cbdh"]
Jul 29 18:42:47.427250 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:47.427231    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-084953-79cdf9c48b-6cbdh"]
Jul 29 18:42:47.437620 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:47.437601    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-084953-66f7c689d9-q7v9n"]
Jul 29 18:42:47.440964 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:47.440942    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-084953-66f7c689d9-q7v9n"]
Jul 29 18:42:48.652989 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:48.652949    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2eafeab0-8797-4592-9a8f-2f8390e7f67c" path="/var/lib/kubelet/pods/2eafeab0-8797-4592-9a8f-2f8390e7f67c/volumes"
Jul 29 18:42:48.653517 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:48.653494    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7f17559-ab00-4ae6-bf6b-2b50d84d1d24" path="/var/lib/kubelet/pods/d7f17559-ab00-4ae6-bf6b-2b50d84d1d24/volumes"
Jul 29 18:42:57.406989 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:57.406909    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-109296-c898cdd69-5x2m6"]
Jul 29 18:42:57.407495 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:57.407475    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7f17559-ab00-4ae6-bf6b-2b50d84d1d24" containerName="payload-processing"
Jul 29 18:42:57.407594 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:57.407500    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7f17559-ab00-4ae6-bf6b-2b50d84d1d24" containerName="payload-processing"
Jul 29 18:42:57.407594 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:57.407524    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2eafeab0-8797-4592-9a8f-2f8390e7f67c" containerName="maas-api"
Jul 29 18:42:57.407594 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:57.407533    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="2eafeab0-8797-4592-9a8f-2f8390e7f67c" containerName="maas-api"
Jul 29 18:42:57.407594 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:57.407559    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1424cbca-8a6a-44b3-9eb2-9a1319a70a9a" containerName="revoke-keys"
Jul 29 18:42:57.407594 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:57.407568    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="1424cbca-8a6a-44b3-9eb2-9a1319a70a9a" containerName="revoke-keys"
Jul 29 18:42:57.407856 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:57.407671    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7f17559-ab00-4ae6-bf6b-2b50d84d1d24" containerName="payload-processing"
Jul 29 18:42:57.407856 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:57.407687    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="2eafeab0-8797-4592-9a8f-2f8390e7f67c" containerName="maas-api"
Jul 29 18:42:57.407856 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:57.407696    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="1424cbca-8a6a-44b3-9eb2-9a1319a70a9a" containerName="revoke-keys"
Jul 29 18:42:57.412578 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:57.412553    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-109296-c898cdd69-5x2m6"
Jul 29 18:42:57.415039 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:57.415020    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-109296-serving-cert\""
Jul 29 18:42:57.422446 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:57.422424    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-109296-c898cdd69-5x2m6"]
Jul 29 18:42:57.499765 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:57.499705    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbhtx\" (UniqueName: \"kubernetes.io/projected/79565189-d94e-4e43-9dd3-0053eec5654a-kube-api-access-wbhtx\") pod \"maas-api-e2e-api-a-109296-c898cdd69-5x2m6\" (UID: \"79565189-d94e-4e43-9dd3-0053eec5654a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-109296-c898cdd69-5x2m6"
Jul 29 18:42:57.499943 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:57.499778    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79565189-d94e-4e43-9dd3-0053eec5654a-openshift-service-ca\") pod \"maas-api-e2e-api-a-109296-c898cdd69-5x2m6\" (UID: \"79565189-d94e-4e43-9dd3-0053eec5654a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-109296-c898cdd69-5x2m6"
Jul 29 18:42:57.499943 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:57.499822    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79565189-d94e-4e43-9dd3-0053eec5654a-maas-api-tls\") pod \"maas-api-e2e-api-a-109296-c898cdd69-5x2m6\" (UID: \"79565189-d94e-4e43-9dd3-0053eec5654a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-109296-c898cdd69-5x2m6"
Jul 29 18:42:57.600544 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:57.600504    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbhtx\" (UniqueName: \"kubernetes.io/projected/79565189-d94e-4e43-9dd3-0053eec5654a-kube-api-access-wbhtx\") pod \"maas-api-e2e-api-a-109296-c898cdd69-5x2m6\" (UID: \"79565189-d94e-4e43-9dd3-0053eec5654a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-109296-c898cdd69-5x2m6"
Jul 29 18:42:57.600777 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:57.600563    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79565189-d94e-4e43-9dd3-0053eec5654a-openshift-service-ca\") pod \"maas-api-e2e-api-a-109296-c898cdd69-5x2m6\" (UID: \"79565189-d94e-4e43-9dd3-0053eec5654a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-109296-c898cdd69-5x2m6"
Jul 29 18:42:57.600777 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:57.600613    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79565189-d94e-4e43-9dd3-0053eec5654a-maas-api-tls\") pod \"maas-api-e2e-api-a-109296-c898cdd69-5x2m6\" (UID: \"79565189-d94e-4e43-9dd3-0053eec5654a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-109296-c898cdd69-5x2m6"
Jul 29 18:42:57.601239 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:57.601216    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79565189-d94e-4e43-9dd3-0053eec5654a-openshift-service-ca\") pod \"maas-api-e2e-api-a-109296-c898cdd69-5x2m6\" (UID: \"79565189-d94e-4e43-9dd3-0053eec5654a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-109296-c898cdd69-5x2m6"
Jul 29 18:42:57.603233 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:57.603212    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79565189-d94e-4e43-9dd3-0053eec5654a-maas-api-tls\") pod \"maas-api-e2e-api-a-109296-c898cdd69-5x2m6\" (UID: \"79565189-d94e-4e43-9dd3-0053eec5654a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-109296-c898cdd69-5x2m6"
Jul 29 18:42:57.608554 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:57.608532    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbhtx\" (UniqueName: \"kubernetes.io/projected/79565189-d94e-4e43-9dd3-0053eec5654a-kube-api-access-wbhtx\") pod \"maas-api-e2e-api-a-109296-c898cdd69-5x2m6\" (UID: \"79565189-d94e-4e43-9dd3-0053eec5654a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-109296-c898cdd69-5x2m6"
Jul 29 18:42:57.726519 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:57.726419    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-109296-c898cdd69-5x2m6"
Jul 29 18:42:57.878633 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:57.878607    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-109296-c898cdd69-5x2m6"]
Jul 29 18:42:57.880268 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:42:57.880241    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79565189_d94e_4e43_9dd3_0053eec5654a.slice/crio-b9cc066a5f7379ec9c990ee236c1b46c4621ff977d0c95c6d3b743a99c14bf05 WatchSource:0}: Error finding container b9cc066a5f7379ec9c990ee236c1b46c4621ff977d0c95c6d3b743a99c14bf05: Status 404 returned error can't find the container with id b9cc066a5f7379ec9c990ee236c1b46c4621ff977d0c95c6d3b743a99c14bf05
Jul 29 18:42:58.463554 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:58.463516    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-109296-c898cdd69-5x2m6" event={"ID":"79565189-d94e-4e43-9dd3-0053eec5654a","Type":"ContainerStarted","Data":"c284a9992281e2a181b57be6bc115cc0f592d91c732584549b933e96179c5293"}
Jul 29 18:42:58.463987 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:58.463576    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-109296-c898cdd69-5x2m6" event={"ID":"79565189-d94e-4e43-9dd3-0053eec5654a","Type":"ContainerStarted","Data":"b9cc066a5f7379ec9c990ee236c1b46c4621ff977d0c95c6d3b743a99c14bf05"}
Jul 29 18:42:58.463987 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:58.463752    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-109296-c898cdd69-5x2m6"
Jul 29 18:42:58.481245 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:42:58.481197    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-109296-c898cdd69-5x2m6" podStartSLOduration=1.006125026 podStartE2EDuration="1.481182165s" totalImagesPullingTime="475.057139ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:42:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:42:58.47938031 +0000 UTC m=+3058.348603118" watchObservedRunningTime="2026-07-29 18:42:58.481182165 +0000 UTC m=+3058.350405155"
Jul 29 18:43:04.475410 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:43:04.475381    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-109296-c898cdd69-5x2m6"
Jul 29 18:43:26.428307 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:43:26.428265    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-95c5f7-79f6c57b54-x5n4v"]
Jul 29 18:43:26.431931 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:43:26.431909    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-95c5f7-79f6c57b54-x5n4v"
Jul 29 18:43:26.435052 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:43:26.435027    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-95c5f7-serving-cert\""
Jul 29 18:43:26.450326 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:43:26.450300    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-95c5f7-79f6c57b54-x5n4v"]
Jul 29 18:43:26.566838 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:43:26.566807    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f02a8d6-e137-4e64-899c-6ff50e7194f1-maas-api-tls\") pod \"maas-api-e2e-api-b-95c5f7-79f6c57b54-x5n4v\" (UID: \"2f02a8d6-e137-4e64-899c-6ff50e7194f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-95c5f7-79f6c57b54-x5n4v"
Jul 29 18:43:26.567005 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:43:26.566874    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgghf\" (UniqueName: \"kubernetes.io/projected/2f02a8d6-e137-4e64-899c-6ff50e7194f1-kube-api-access-fgghf\") pod \"maas-api-e2e-api-b-95c5f7-79f6c57b54-x5n4v\" (UID: \"2f02a8d6-e137-4e64-899c-6ff50e7194f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-95c5f7-79f6c57b54-x5n4v"
Jul 29 18:43:26.567054 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:43:26.567016    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f02a8d6-e137-4e64-899c-6ff50e7194f1-openshift-service-ca\") pod \"maas-api-e2e-api-b-95c5f7-79f6c57b54-x5n4v\" (UID: \"2f02a8d6-e137-4e64-899c-6ff50e7194f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-95c5f7-79f6c57b54-x5n4v"
Jul 29 18:43:26.668459 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:43:26.668431    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgghf\" (UniqueName: \"kubernetes.io/projected/2f02a8d6-e137-4e64-899c-6ff50e7194f1-kube-api-access-fgghf\") pod \"maas-api-e2e-api-b-95c5f7-79f6c57b54-x5n4v\" (UID: \"2f02a8d6-e137-4e64-899c-6ff50e7194f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-95c5f7-79f6c57b54-x5n4v"
Jul 29 18:43:26.668622 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:43:26.668489    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f02a8d6-e137-4e64-899c-6ff50e7194f1-openshift-service-ca\") pod \"maas-api-e2e-api-b-95c5f7-79f6c57b54-x5n4v\" (UID: \"2f02a8d6-e137-4e64-899c-6ff50e7194f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-95c5f7-79f6c57b54-x5n4v"
Jul 29 18:43:26.668622 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:43:26.668530    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f02a8d6-e137-4e64-899c-6ff50e7194f1-maas-api-tls\") pod \"maas-api-e2e-api-b-95c5f7-79f6c57b54-x5n4v\" (UID: \"2f02a8d6-e137-4e64-899c-6ff50e7194f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-95c5f7-79f6c57b54-x5n4v"
Jul 29 18:43:26.668706 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:43:26.668648    2557 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-95c5f7-serving-cert: secret "maas-api-e2e-api-b-95c5f7-serving-cert" not found
Jul 29 18:43:26.668706 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:43:26.668697    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f02a8d6-e137-4e64-899c-6ff50e7194f1-maas-api-tls podName:2f02a8d6-e137-4e64-899c-6ff50e7194f1 nodeName:}" failed. No retries permitted until 2026-07-29 18:43:27.168683388 +0000 UTC m=+3087.037906174 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2f02a8d6-e137-4e64-899c-6ff50e7194f1-maas-api-tls") pod "maas-api-e2e-api-b-95c5f7-79f6c57b54-x5n4v" (UID: "2f02a8d6-e137-4e64-899c-6ff50e7194f1") : secret "maas-api-e2e-api-b-95c5f7-serving-cert" not found
Jul 29 18:43:26.669175 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:43:26.669155    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f02a8d6-e137-4e64-899c-6ff50e7194f1-openshift-service-ca\") pod \"maas-api-e2e-api-b-95c5f7-79f6c57b54-x5n4v\" (UID: \"2f02a8d6-e137-4e64-899c-6ff50e7194f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-95c5f7-79f6c57b54-x5n4v"
Jul 29 18:43:26.677031 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:43:26.677005    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgghf\" (UniqueName: \"kubernetes.io/projected/2f02a8d6-e137-4e64-899c-6ff50e7194f1-kube-api-access-fgghf\") pod \"maas-api-e2e-api-b-95c5f7-79f6c57b54-x5n4v\" (UID: \"2f02a8d6-e137-4e64-899c-6ff50e7194f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-95c5f7-79f6c57b54-x5n4v"
Jul 29 18:43:27.172976 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:43:27.172942    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f02a8d6-e137-4e64-899c-6ff50e7194f1-maas-api-tls\") pod \"maas-api-e2e-api-b-95c5f7-79f6c57b54-x5n4v\" (UID: \"2f02a8d6-e137-4e64-899c-6ff50e7194f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-95c5f7-79f6c57b54-x5n4v"
Jul 29 18:43:27.175489 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:43:27.175460    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f02a8d6-e137-4e64-899c-6ff50e7194f1-maas-api-tls\") pod \"maas-api-e2e-api-b-95c5f7-79f6c57b54-x5n4v\" (UID: \"2f02a8d6-e137-4e64-899c-6ff50e7194f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-95c5f7-79f6c57b54-x5n4v"
Jul 29 18:43:27.345999 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:43:27.345970    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-95c5f7-79f6c57b54-x5n4v"
Jul 29 18:43:27.494897 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:43:27.494869    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-95c5f7-79f6c57b54-x5n4v"]
Jul 29 18:43:27.496189 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:43:27.496159    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f02a8d6_e137_4e64_899c_6ff50e7194f1.slice/crio-71e32ad527bfb6f012d8245d451cced62e2a1621ca8a45aa38589043062f142d WatchSource:0}: Error finding container 71e32ad527bfb6f012d8245d451cced62e2a1621ca8a45aa38589043062f142d: Status 404 returned error can't find the container with id 71e32ad527bfb6f012d8245d451cced62e2a1621ca8a45aa38589043062f142d
Jul 29 18:43:27.630382 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:43:27.630347    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-95c5f7-79f6c57b54-x5n4v" event={"ID":"2f02a8d6-e137-4e64-899c-6ff50e7194f1","Type":"ContainerStarted","Data":"71e32ad527bfb6f012d8245d451cced62e2a1621ca8a45aa38589043062f142d"}
Jul 29 18:43:28.639036 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:43:28.639000    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-95c5f7-79f6c57b54-x5n4v" event={"ID":"2f02a8d6-e137-4e64-899c-6ff50e7194f1","Type":"ContainerStarted","Data":"2fab693e7746e564bb4e210f1a3bc8722ca24659b20db9ec4dff6cdef46e31dc"}
Jul 29 18:43:28.639428 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:43:28.639057    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-95c5f7-79f6c57b54-x5n4v"
Jul 29 18:43:28.656518 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:43:28.656470    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-95c5f7-79f6c57b54-x5n4v" podStartSLOduration=2.205471896 podStartE2EDuration="2.6564564s" totalImagesPullingTime="450.984504ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:43:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:43:28.654207416 +0000 UTC m=+3088.523430223" watchObservedRunningTime="2026-07-29 18:43:28.6564564 +0000 UTC m=+3088.525679207"
Jul 29 18:43:34.653581 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:43:34.653549    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-95c5f7-79f6c57b54-x5n4v"
Jul 29 18:43:54.224764 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:43:54.224708    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-109296-7p69t"]
Jul 29 18:43:54.228517 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:43:54.228497    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-109296-7p69t"
Jul 29 18:43:54.231363 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:43:54.231324    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qrbhl\""
Jul 29 18:43:54.234704 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:43:54.234677    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-109296-7p69t"]
Jul 29 18:43:54.324002 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:43:54.323966    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/196c472b-d6ec-4614-8875-62b9c17dcb88-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-109296-7p69t\" (UID: \"196c472b-d6ec-4614-8875-62b9c17dcb88\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-109296-7p69t"
Jul 29 18:43:54.424819 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:43:54.424780    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/196c472b-d6ec-4614-8875-62b9c17dcb88-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-109296-7p69t\" (UID: \"196c472b-d6ec-4614-8875-62b9c17dcb88\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-109296-7p69t"
Jul 29 18:43:54.425386 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:43:54.425366    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/196c472b-d6ec-4614-8875-62b9c17dcb88-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-109296-7p69t\" (UID: \"196c472b-d6ec-4614-8875-62b9c17dcb88\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-109296-7p69t"
Jul 29 18:43:54.541451 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:43:54.541418    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-109296-7p69t"
Jul 29 18:43:54.682450 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:43:54.682418    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-109296-7p69t"]
Jul 29 18:43:54.683770 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:43:54.683717    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod196c472b_d6ec_4614_8875_62b9c17dcb88.slice/crio-d7c5514622d42f8ad243c9a71b776b854eb7740d6565b349fb150a44396a52a3 WatchSource:0}: Error finding container d7c5514622d42f8ad243c9a71b776b854eb7740d6565b349fb150a44396a52a3: Status 404 returned error can't find the container with id d7c5514622d42f8ad243c9a71b776b854eb7740d6565b349fb150a44396a52a3
Jul 29 18:43:54.788389 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:43:54.788359    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-109296-7p69t" event={"ID":"196c472b-d6ec-4614-8875-62b9c17dcb88","Type":"ContainerStarted","Data":"d7c5514622d42f8ad243c9a71b776b854eb7740d6565b349fb150a44396a52a3"}
Jul 29 18:43:55.795490 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:43:55.795453    2557 generic.go:356] "Generic (PLEG): container finished" podID="196c472b-d6ec-4614-8875-62b9c17dcb88" containerID="cfd9c8d0038d29fe8a4f4ed2bf93414c0c7cffcf7cdfa3dc8b7eee7f8ff2f51e" exitCode=0
Jul 29 18:43:55.795927 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:43:55.795545    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-109296-7p69t" event={"ID":"196c472b-d6ec-4614-8875-62b9c17dcb88","Type":"ContainerDied","Data":"cfd9c8d0038d29fe8a4f4ed2bf93414c0c7cffcf7cdfa3dc8b7eee7f8ff2f51e"}
Jul 29 18:43:56.941584 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:43:56.941552    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-109296-7p69t"
Jul 29 18:43:57.050100 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:43:57.050064    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/196c472b-d6ec-4614-8875-62b9c17dcb88-maas-api-service-ca\") pod \"196c472b-d6ec-4614-8875-62b9c17dcb88\" (UID: \"196c472b-d6ec-4614-8875-62b9c17dcb88\") "
Jul 29 18:43:57.050439 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:43:57.050416    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/196c472b-d6ec-4614-8875-62b9c17dcb88-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "196c472b-d6ec-4614-8875-62b9c17dcb88" (UID: "196c472b-d6ec-4614-8875-62b9c17dcb88"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:43:57.150879 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:43:57.150796    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/196c472b-d6ec-4614-8875-62b9c17dcb88-maas-api-service-ca\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:43:57.808212 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:43:57.808182    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-109296-7p69t"
Jul 29 18:43:57.808212 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:43:57.808199    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-109296-7p69t" event={"ID":"196c472b-d6ec-4614-8875-62b9c17dcb88","Type":"ContainerDied","Data":"d7c5514622d42f8ad243c9a71b776b854eb7740d6565b349fb150a44396a52a3"}
Jul 29 18:43:57.808426 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:43:57.808255    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d7c5514622d42f8ad243c9a71b776b854eb7740d6565b349fb150a44396a52a3"
Jul 29 18:44:09.606623 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:09.606582    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-109296-c898cdd69-5x2m6"]
Jul 29 18:44:09.607181 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:09.606818    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-109296-c898cdd69-5x2m6" podUID="79565189-d94e-4e43-9dd3-0053eec5654a" containerName="maas-api" containerID="cri-o://c284a9992281e2a181b57be6bc115cc0f592d91c732584549b933e96179c5293" gracePeriod=30
Jul 29 18:44:09.824305 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:09.823115    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-109296-7p69t"]
Jul 29 18:44:09.824305 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:09.823171    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-109296-7p69t"]
Jul 29 18:44:09.875523 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:09.875478    2557 generic.go:356] "Generic (PLEG): container finished" podID="79565189-d94e-4e43-9dd3-0053eec5654a" containerID="c284a9992281e2a181b57be6bc115cc0f592d91c732584549b933e96179c5293" exitCode=0
Jul 29 18:44:09.875662 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:09.875532    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-109296-c898cdd69-5x2m6" event={"ID":"79565189-d94e-4e43-9dd3-0053eec5654a","Type":"ContainerDied","Data":"c284a9992281e2a181b57be6bc115cc0f592d91c732584549b933e96179c5293"}
Jul 29 18:44:09.875662 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:09.875572    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-109296-c898cdd69-5x2m6" event={"ID":"79565189-d94e-4e43-9dd3-0053eec5654a","Type":"ContainerDied","Data":"b9cc066a5f7379ec9c990ee236c1b46c4621ff977d0c95c6d3b743a99c14bf05"}
Jul 29 18:44:09.875662 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:09.875587    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b9cc066a5f7379ec9c990ee236c1b46c4621ff977d0c95c6d3b743a99c14bf05"
Jul 29 18:44:09.884878 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:09.884860    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-109296-c898cdd69-5x2m6"
Jul 29 18:44:09.972390 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:09.972362    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79565189-d94e-4e43-9dd3-0053eec5654a-maas-api-tls\") pod \"79565189-d94e-4e43-9dd3-0053eec5654a\" (UID: \"79565189-d94e-4e43-9dd3-0053eec5654a\") "
Jul 29 18:44:09.972549 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:09.972442    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79565189-d94e-4e43-9dd3-0053eec5654a-openshift-service-ca\") pod \"79565189-d94e-4e43-9dd3-0053eec5654a\" (UID: \"79565189-d94e-4e43-9dd3-0053eec5654a\") "
Jul 29 18:44:09.972549 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:09.972479    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wbhtx\" (UniqueName: \"kubernetes.io/projected/79565189-d94e-4e43-9dd3-0053eec5654a-kube-api-access-wbhtx\") pod \"79565189-d94e-4e43-9dd3-0053eec5654a\" (UID: \"79565189-d94e-4e43-9dd3-0053eec5654a\") "
Jul 29 18:44:09.972896 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:09.972865    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79565189-d94e-4e43-9dd3-0053eec5654a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "79565189-d94e-4e43-9dd3-0053eec5654a" (UID: "79565189-d94e-4e43-9dd3-0053eec5654a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:44:09.974659 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:09.974629    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79565189-d94e-4e43-9dd3-0053eec5654a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "79565189-d94e-4e43-9dd3-0053eec5654a" (UID: "79565189-d94e-4e43-9dd3-0053eec5654a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:44:09.974788 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:09.974692    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79565189-d94e-4e43-9dd3-0053eec5654a-kube-api-access-wbhtx" (OuterVolumeSpecName: "kube-api-access-wbhtx") pod "79565189-d94e-4e43-9dd3-0053eec5654a" (UID: "79565189-d94e-4e43-9dd3-0053eec5654a"). InnerVolumeSpecName "kube-api-access-wbhtx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:44:10.073335 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:10.073296    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wbhtx\" (UniqueName: \"kubernetes.io/projected/79565189-d94e-4e43-9dd3-0053eec5654a-kube-api-access-wbhtx\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:44:10.073335 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:10.073324    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79565189-d94e-4e43-9dd3-0053eec5654a-maas-api-tls\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:44:10.073335 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:10.073337    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79565189-d94e-4e43-9dd3-0053eec5654a-openshift-service-ca\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:44:10.652809 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:10.652772    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="196c472b-d6ec-4614-8875-62b9c17dcb88" path="/var/lib/kubelet/pods/196c472b-d6ec-4614-8875-62b9c17dcb88/volumes"
Jul 29 18:44:10.881172 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:10.881137    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-109296-c898cdd69-5x2m6"
Jul 29 18:44:10.897918 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:10.897883    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-109296-c898cdd69-5x2m6"]
Jul 29 18:44:10.900202 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:10.900181    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-109296-c898cdd69-5x2m6"]
Jul 29 18:44:12.653054 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:12.653018    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79565189-d94e-4e43-9dd3-0053eec5654a" path="/var/lib/kubelet/pods/79565189-d94e-4e43-9dd3-0053eec5654a/volumes"
Jul 29 18:44:16.387254 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:16.387224    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-95c5f7-fzjwj"]
Jul 29 18:44:16.387644 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:16.387630    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="196c472b-d6ec-4614-8875-62b9c17dcb88" containerName="revoke-keys"
Jul 29 18:44:16.387644 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:16.387641    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="196c472b-d6ec-4614-8875-62b9c17dcb88" containerName="revoke-keys"
Jul 29 18:44:16.387715 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:16.387651    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79565189-d94e-4e43-9dd3-0053eec5654a" containerName="maas-api"
Jul 29 18:44:16.387715 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:16.387656    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="79565189-d94e-4e43-9dd3-0053eec5654a" containerName="maas-api"
Jul 29 18:44:16.387812 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:16.387736    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="79565189-d94e-4e43-9dd3-0053eec5654a" containerName="maas-api"
Jul 29 18:44:16.387812 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:16.387749    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="196c472b-d6ec-4614-8875-62b9c17dcb88" containerName="revoke-keys"
Jul 29 18:44:16.391043 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:16.391026    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-95c5f7-fzjwj"
Jul 29 18:44:16.393596 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:16.393572    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qrbhl\""
Jul 29 18:44:16.400299 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:16.400205    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-95c5f7-fzjwj"]
Jul 29 18:44:16.532346 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:16.532308    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b43fb7db-bf4e-4a2d-9108-74befda6bb1e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-95c5f7-fzjwj\" (UID: \"b43fb7db-bf4e-4a2d-9108-74befda6bb1e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-95c5f7-fzjwj"
Jul 29 18:44:16.633566 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:16.633538    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b43fb7db-bf4e-4a2d-9108-74befda6bb1e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-95c5f7-fzjwj\" (UID: \"b43fb7db-bf4e-4a2d-9108-74befda6bb1e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-95c5f7-fzjwj"
Jul 29 18:44:16.634219 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:16.634191    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b43fb7db-bf4e-4a2d-9108-74befda6bb1e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-95c5f7-fzjwj\" (UID: \"b43fb7db-bf4e-4a2d-9108-74befda6bb1e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-95c5f7-fzjwj"
Jul 29 18:44:16.704090 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:16.704021    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-95c5f7-fzjwj"
Jul 29 18:44:16.843671 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:16.843639    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-95c5f7-fzjwj"]
Jul 29 18:44:16.845083 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:44:16.845052    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb43fb7db_bf4e_4a2d_9108_74befda6bb1e.slice/crio-7f273e131e1123858da65d19e8cf0b96d2b4eb8dd801a497f55a4694173bfff9 WatchSource:0}: Error finding container 7f273e131e1123858da65d19e8cf0b96d2b4eb8dd801a497f55a4694173bfff9: Status 404 returned error can't find the container with id 7f273e131e1123858da65d19e8cf0b96d2b4eb8dd801a497f55a4694173bfff9
Jul 29 18:44:16.918660 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:16.918626    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-95c5f7-fzjwj" event={"ID":"b43fb7db-bf4e-4a2d-9108-74befda6bb1e","Type":"ContainerStarted","Data":"7f273e131e1123858da65d19e8cf0b96d2b4eb8dd801a497f55a4694173bfff9"}
Jul 29 18:44:17.926661 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:17.926571    2557 generic.go:356] "Generic (PLEG): container finished" podID="b43fb7db-bf4e-4a2d-9108-74befda6bb1e" containerID="22ec6e8f09e4431ee436c6d4b6ec8b7ee6747d5f655c340c061c23e38e397f64" exitCode=0
Jul 29 18:44:17.927049 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:17.926654    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-95c5f7-fzjwj" event={"ID":"b43fb7db-bf4e-4a2d-9108-74befda6bb1e","Type":"ContainerDied","Data":"22ec6e8f09e4431ee436c6d4b6ec8b7ee6747d5f655c340c061c23e38e397f64"}
Jul 29 18:44:19.076581 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:19.076558    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-95c5f7-fzjwj"
Jul 29 18:44:19.156719 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:19.156692    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b43fb7db-bf4e-4a2d-9108-74befda6bb1e-maas-api-service-ca\") pod \"b43fb7db-bf4e-4a2d-9108-74befda6bb1e\" (UID: \"b43fb7db-bf4e-4a2d-9108-74befda6bb1e\") "
Jul 29 18:44:19.156987 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:19.156965    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b43fb7db-bf4e-4a2d-9108-74befda6bb1e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b43fb7db-bf4e-4a2d-9108-74befda6bb1e" (UID: "b43fb7db-bf4e-4a2d-9108-74befda6bb1e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:44:19.258096 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:19.258026    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b43fb7db-bf4e-4a2d-9108-74befda6bb1e-maas-api-service-ca\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:44:19.939703 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:19.939673    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-95c5f7-fzjwj"
Jul 29 18:44:19.939703 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:19.939681    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-95c5f7-fzjwj" event={"ID":"b43fb7db-bf4e-4a2d-9108-74befda6bb1e","Type":"ContainerDied","Data":"7f273e131e1123858da65d19e8cf0b96d2b4eb8dd801a497f55a4694173bfff9"}
Jul 29 18:44:19.939926 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:19.939755    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7f273e131e1123858da65d19e8cf0b96d2b4eb8dd801a497f55a4694173bfff9"
Jul 29 18:44:31.768767 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:31.768515    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-95c5f7-79f6c57b54-x5n4v"]
Jul 29 18:44:31.769452 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:31.769394    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-95c5f7-79f6c57b54-x5n4v" podUID="2f02a8d6-e137-4e64-899c-6ff50e7194f1" containerName="maas-api" containerID="cri-o://2fab693e7746e564bb4e210f1a3bc8722ca24659b20db9ec4dff6cdef46e31dc" gracePeriod=30
Jul 29 18:44:31.964694 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:31.964651    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-95c5f7-fzjwj"]
Jul 29 18:44:31.968599 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:31.968567    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-95c5f7-fzjwj"]
Jul 29 18:44:32.011240 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:32.011206    2557 generic.go:356] "Generic (PLEG): container finished" podID="2f02a8d6-e137-4e64-899c-6ff50e7194f1" containerID="2fab693e7746e564bb4e210f1a3bc8722ca24659b20db9ec4dff6cdef46e31dc" exitCode=0
Jul 29 18:44:32.011400 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:32.011239    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-95c5f7-79f6c57b54-x5n4v" event={"ID":"2f02a8d6-e137-4e64-899c-6ff50e7194f1","Type":"ContainerDied","Data":"2fab693e7746e564bb4e210f1a3bc8722ca24659b20db9ec4dff6cdef46e31dc"}
Jul 29 18:44:32.033433 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:32.033372    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-95c5f7-79f6c57b54-x5n4v"
Jul 29 18:44:32.063804 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:32.063780    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f02a8d6-e137-4e64-899c-6ff50e7194f1-openshift-service-ca\") pod \"2f02a8d6-e137-4e64-899c-6ff50e7194f1\" (UID: \"2f02a8d6-e137-4e64-899c-6ff50e7194f1\") "
Jul 29 18:44:32.063976 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:32.063816    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f02a8d6-e137-4e64-899c-6ff50e7194f1-maas-api-tls\") pod \"2f02a8d6-e137-4e64-899c-6ff50e7194f1\" (UID: \"2f02a8d6-e137-4e64-899c-6ff50e7194f1\") "
Jul 29 18:44:32.063976 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:32.063903    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fgghf\" (UniqueName: \"kubernetes.io/projected/2f02a8d6-e137-4e64-899c-6ff50e7194f1-kube-api-access-fgghf\") pod \"2f02a8d6-e137-4e64-899c-6ff50e7194f1\" (UID: \"2f02a8d6-e137-4e64-899c-6ff50e7194f1\") "
Jul 29 18:44:32.064132 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:32.064110    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f02a8d6-e137-4e64-899c-6ff50e7194f1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2f02a8d6-e137-4e64-899c-6ff50e7194f1" (UID: "2f02a8d6-e137-4e64-899c-6ff50e7194f1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:44:32.066130 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:32.066105    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f02a8d6-e137-4e64-899c-6ff50e7194f1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2f02a8d6-e137-4e64-899c-6ff50e7194f1" (UID: "2f02a8d6-e137-4e64-899c-6ff50e7194f1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:44:32.066130 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:32.066112    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f02a8d6-e137-4e64-899c-6ff50e7194f1-kube-api-access-fgghf" (OuterVolumeSpecName: "kube-api-access-fgghf") pod "2f02a8d6-e137-4e64-899c-6ff50e7194f1" (UID: "2f02a8d6-e137-4e64-899c-6ff50e7194f1"). InnerVolumeSpecName "kube-api-access-fgghf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:44:32.164656 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:32.164605    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f02a8d6-e137-4e64-899c-6ff50e7194f1-maas-api-tls\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:44:32.164656 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:32.164650    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fgghf\" (UniqueName: \"kubernetes.io/projected/2f02a8d6-e137-4e64-899c-6ff50e7194f1-kube-api-access-fgghf\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:44:32.164656 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:32.164664    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f02a8d6-e137-4e64-899c-6ff50e7194f1-openshift-service-ca\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:44:32.652964 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:32.652926    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b43fb7db-bf4e-4a2d-9108-74befda6bb1e" path="/var/lib/kubelet/pods/b43fb7db-bf4e-4a2d-9108-74befda6bb1e/volumes"
Jul 29 18:44:33.018125 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:33.018037    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-95c5f7-79f6c57b54-x5n4v" event={"ID":"2f02a8d6-e137-4e64-899c-6ff50e7194f1","Type":"ContainerDied","Data":"71e32ad527bfb6f012d8245d451cced62e2a1621ca8a45aa38589043062f142d"}
Jul 29 18:44:33.018125 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:33.018068    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-95c5f7-79f6c57b54-x5n4v"
Jul 29 18:44:33.018125 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:33.018095    2557 scope.go:117] "RemoveContainer" containerID="2fab693e7746e564bb4e210f1a3bc8722ca24659b20db9ec4dff6cdef46e31dc"
Jul 29 18:44:33.035225 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:33.035202    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-95c5f7-79f6c57b54-x5n4v"]
Jul 29 18:44:33.039112 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:33.039086    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-95c5f7-79f6c57b54-x5n4v"]
Jul 29 18:44:34.653104 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:34.653074    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f02a8d6-e137-4e64-899c-6ff50e7194f1" path="/var/lib/kubelet/pods/2f02a8d6-e137-4e64-899c-6ff50e7194f1/volumes"
Jul 29 18:44:42.983190 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:42.983159    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-6e9884-5bc8f8d7fc-qp5js"]
Jul 29 18:44:42.983695 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:42.983546    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b43fb7db-bf4e-4a2d-9108-74befda6bb1e" containerName="revoke-keys"
Jul 29 18:44:42.983695 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:42.983558    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="b43fb7db-bf4e-4a2d-9108-74befda6bb1e" containerName="revoke-keys"
Jul 29 18:44:42.983695 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:42.983571    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f02a8d6-e137-4e64-899c-6ff50e7194f1" containerName="maas-api"
Jul 29 18:44:42.983695 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:42.983577    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f02a8d6-e137-4e64-899c-6ff50e7194f1" containerName="maas-api"
Jul 29 18:44:42.983695 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:42.983641    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f02a8d6-e137-4e64-899c-6ff50e7194f1" containerName="maas-api"
Jul 29 18:44:42.983695 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:42.983651    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="b43fb7db-bf4e-4a2d-9108-74befda6bb1e" containerName="revoke-keys"
Jul 29 18:44:42.986663 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:42.986644    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6e9884-5bc8f8d7fc-qp5js"
Jul 29 18:44:42.989101 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:42.989077    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-6e9884-serving-cert\""
Jul 29 18:44:42.998079 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:42.998057    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-6e9884-5bc8f8d7fc-qp5js"]
Jul 29 18:44:43.063390 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:43.063359    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/14f681f3-cbb6-423c-a25f-43b9bec512d3-openshift-service-ca\") pod \"maas-api-e2e-shared-a-6e9884-5bc8f8d7fc-qp5js\" (UID: \"14f681f3-cbb6-423c-a25f-43b9bec512d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6e9884-5bc8f8d7fc-qp5js"
Jul 29 18:44:43.063390 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:43.063393    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2q6gk\" (UniqueName: \"kubernetes.io/projected/14f681f3-cbb6-423c-a25f-43b9bec512d3-kube-api-access-2q6gk\") pod \"maas-api-e2e-shared-a-6e9884-5bc8f8d7fc-qp5js\" (UID: \"14f681f3-cbb6-423c-a25f-43b9bec512d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6e9884-5bc8f8d7fc-qp5js"
Jul 29 18:44:43.063592 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:43.063439    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/14f681f3-cbb6-423c-a25f-43b9bec512d3-maas-api-tls\") pod \"maas-api-e2e-shared-a-6e9884-5bc8f8d7fc-qp5js\" (UID: \"14f681f3-cbb6-423c-a25f-43b9bec512d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6e9884-5bc8f8d7fc-qp5js"
Jul 29 18:44:43.164708 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:43.164675    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/14f681f3-cbb6-423c-a25f-43b9bec512d3-openshift-service-ca\") pod \"maas-api-e2e-shared-a-6e9884-5bc8f8d7fc-qp5js\" (UID: \"14f681f3-cbb6-423c-a25f-43b9bec512d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6e9884-5bc8f8d7fc-qp5js"
Jul 29 18:44:43.164708 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:43.164710    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2q6gk\" (UniqueName: \"kubernetes.io/projected/14f681f3-cbb6-423c-a25f-43b9bec512d3-kube-api-access-2q6gk\") pod \"maas-api-e2e-shared-a-6e9884-5bc8f8d7fc-qp5js\" (UID: \"14f681f3-cbb6-423c-a25f-43b9bec512d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6e9884-5bc8f8d7fc-qp5js"
Jul 29 18:44:43.164986 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:43.164790    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/14f681f3-cbb6-423c-a25f-43b9bec512d3-maas-api-tls\") pod \"maas-api-e2e-shared-a-6e9884-5bc8f8d7fc-qp5js\" (UID: \"14f681f3-cbb6-423c-a25f-43b9bec512d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6e9884-5bc8f8d7fc-qp5js"
Jul 29 18:44:43.164986 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:44:43.164894    2557 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-6e9884-serving-cert: secret "maas-api-e2e-shared-a-6e9884-serving-cert" not found
Jul 29 18:44:43.164986 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:44:43.164957    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14f681f3-cbb6-423c-a25f-43b9bec512d3-maas-api-tls podName:14f681f3-cbb6-423c-a25f-43b9bec512d3 nodeName:}" failed. No retries permitted until 2026-07-29 18:44:43.664941971 +0000 UTC m=+3163.534164756 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/14f681f3-cbb6-423c-a25f-43b9bec512d3-maas-api-tls") pod "maas-api-e2e-shared-a-6e9884-5bc8f8d7fc-qp5js" (UID: "14f681f3-cbb6-423c-a25f-43b9bec512d3") : secret "maas-api-e2e-shared-a-6e9884-serving-cert" not found
Jul 29 18:44:43.165341 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:43.165322    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/14f681f3-cbb6-423c-a25f-43b9bec512d3-openshift-service-ca\") pod \"maas-api-e2e-shared-a-6e9884-5bc8f8d7fc-qp5js\" (UID: \"14f681f3-cbb6-423c-a25f-43b9bec512d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6e9884-5bc8f8d7fc-qp5js"
Jul 29 18:44:43.173666 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:43.173646    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2q6gk\" (UniqueName: \"kubernetes.io/projected/14f681f3-cbb6-423c-a25f-43b9bec512d3-kube-api-access-2q6gk\") pod \"maas-api-e2e-shared-a-6e9884-5bc8f8d7fc-qp5js\" (UID: \"14f681f3-cbb6-423c-a25f-43b9bec512d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6e9884-5bc8f8d7fc-qp5js"
Jul 29 18:44:43.669371 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:43.669329    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/14f681f3-cbb6-423c-a25f-43b9bec512d3-maas-api-tls\") pod \"maas-api-e2e-shared-a-6e9884-5bc8f8d7fc-qp5js\" (UID: \"14f681f3-cbb6-423c-a25f-43b9bec512d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6e9884-5bc8f8d7fc-qp5js"
Jul 29 18:44:43.671903 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:43.671880    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/14f681f3-cbb6-423c-a25f-43b9bec512d3-maas-api-tls\") pod \"maas-api-e2e-shared-a-6e9884-5bc8f8d7fc-qp5js\" (UID: \"14f681f3-cbb6-423c-a25f-43b9bec512d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6e9884-5bc8f8d7fc-qp5js"
Jul 29 18:44:43.898255 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:43.898221    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6e9884-5bc8f8d7fc-qp5js"
Jul 29 18:44:44.043240 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:44.043209    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-6e9884-5bc8f8d7fc-qp5js"]
Jul 29 18:44:44.045070 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:44:44.045039    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14f681f3_cbb6_423c_a25f_43b9bec512d3.slice/crio-e9581b6f0c46136f0c34e7c95dc1c0b7ba6bd0249a16f73e77605ea6f964fb61 WatchSource:0}: Error finding container e9581b6f0c46136f0c34e7c95dc1c0b7ba6bd0249a16f73e77605ea6f964fb61: Status 404 returned error can't find the container with id e9581b6f0c46136f0c34e7c95dc1c0b7ba6bd0249a16f73e77605ea6f964fb61
Jul 29 18:44:44.086421 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:44.086391    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6e9884-5bc8f8d7fc-qp5js" event={"ID":"14f681f3-cbb6-423c-a25f-43b9bec512d3","Type":"ContainerStarted","Data":"e9581b6f0c46136f0c34e7c95dc1c0b7ba6bd0249a16f73e77605ea6f964fb61"}
Jul 29 18:44:45.094223 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:45.094188    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6e9884-5bc8f8d7fc-qp5js" event={"ID":"14f681f3-cbb6-423c-a25f-43b9bec512d3","Type":"ContainerStarted","Data":"94a30aaa74ed0c65f1cec53afa7c32784b757853ef0f68615fba61ed77ccca00"}
Jul 29 18:44:45.094698 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:45.094258    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6e9884-5bc8f8d7fc-qp5js"
Jul 29 18:44:45.112743 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:45.112671    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6e9884-5bc8f8d7fc-qp5js" podStartSLOduration=2.758844078 podStartE2EDuration="3.112655301s" totalImagesPullingTime="353.811223ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:44:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:44:45.109570494 +0000 UTC m=+3164.978793301" watchObservedRunningTime="2026-07-29 18:44:45.112655301 +0000 UTC m=+3164.981878107"
Jul 29 18:44:48.184001 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:48.183967    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-zxrcp"]
Jul 29 18:44:48.187599 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:48.187582    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-zxrcp"
Jul 29 18:44:48.190210 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:48.190191    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-4xt2r\""
Jul 29 18:44:48.194377 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:48.194358    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-zxrcp"]
Jul 29 18:44:48.309330 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:48.309300    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqm22\" (UniqueName: \"kubernetes.io/projected/f23ae7b5-7cb7-4017-9a1b-27ab84a2aacd-kube-api-access-wqm22\") pod \"kuadrant-operator-catalog-zxrcp\" (UID: \"f23ae7b5-7cb7-4017-9a1b-27ab84a2aacd\") " pod="kuadrant-system/kuadrant-operator-catalog-zxrcp"
Jul 29 18:44:48.410119 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:48.410089    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqm22\" (UniqueName: \"kubernetes.io/projected/f23ae7b5-7cb7-4017-9a1b-27ab84a2aacd-kube-api-access-wqm22\") pod \"kuadrant-operator-catalog-zxrcp\" (UID: \"f23ae7b5-7cb7-4017-9a1b-27ab84a2aacd\") " pod="kuadrant-system/kuadrant-operator-catalog-zxrcp"
Jul 29 18:44:48.418553 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:48.418530    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqm22\" (UniqueName: \"kubernetes.io/projected/f23ae7b5-7cb7-4017-9a1b-27ab84a2aacd-kube-api-access-wqm22\") pod \"kuadrant-operator-catalog-zxrcp\" (UID: \"f23ae7b5-7cb7-4017-9a1b-27ab84a2aacd\") " pod="kuadrant-system/kuadrant-operator-catalog-zxrcp"
Jul 29 18:44:48.498394 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:48.498294    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-zxrcp"
Jul 29 18:44:48.630945 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:48.630920    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-zxrcp"]
Jul 29 18:44:48.632371 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:44:48.632342    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf23ae7b5_7cb7_4017_9a1b_27ab84a2aacd.slice/crio-fa33089ec74496210b6eff2cca9b5d03d863fef920e0a7b18fb9e0065009fd63 WatchSource:0}: Error finding container fa33089ec74496210b6eff2cca9b5d03d863fef920e0a7b18fb9e0065009fd63: Status 404 returned error can't find the container with id fa33089ec74496210b6eff2cca9b5d03d863fef920e0a7b18fb9e0065009fd63
Jul 29 18:44:49.118073 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:49.118041    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-zxrcp" event={"ID":"f23ae7b5-7cb7-4017-9a1b-27ab84a2aacd","Type":"ContainerStarted","Data":"fa33089ec74496210b6eff2cca9b5d03d863fef920e0a7b18fb9e0065009fd63"}
Jul 29 18:44:51.106022 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:51.105994    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6e9884-5bc8f8d7fc-qp5js"
Jul 29 18:44:51.131680 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:51.131649    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-zxrcp" event={"ID":"f23ae7b5-7cb7-4017-9a1b-27ab84a2aacd","Type":"ContainerStarted","Data":"7a2ee110d9b048e8be24c712317c97a3ab9bf71b631482a6f5f5a22d4588ceae"}
Jul 29 18:44:51.147769 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:51.147696    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-zxrcp" podStartSLOduration=1.23801591 podStartE2EDuration="3.147675111s" totalImagesPullingTime="1.909659201s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:44:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:44:51.145938326 +0000 UTC m=+3171.015161144" watchObservedRunningTime="2026-07-29 18:44:51.147675111 +0000 UTC m=+3171.016897919"
Jul 29 18:44:58.499082 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:58.499044    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-zxrcp"
Jul 29 18:44:58.499543 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:58.499104    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-zxrcp"
Jul 29 18:44:58.521390 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:58.521364    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-zxrcp"
Jul 29 18:44:59.200756 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:59.200710    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-zxrcp"
Jul 29 18:44:59.232084 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:44:59.232054    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-zxrcp"]
Jul 29 18:45:00.135303 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:00.135270    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6e9884-29755845-z4gws"]
Jul 29 18:45:00.139301 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:00.139280    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755845-9cbbh"]
Jul 29 18:45:00.139418 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:00.139399    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6e9884-29755845-z4gws"
Jul 29 18:45:00.142100 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:00.142082    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qrbhl\""
Jul 29 18:45:00.142756 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:00.142742    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755845-9cbbh"
Jul 29 18:45:00.146759 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:00.146714    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6e9884-29755845-z4gws"]
Jul 29 18:45:00.151189 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:00.151167    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755845-9cbbh"]
Jul 29 18:45:00.153609 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:00.153592    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6e9884-29755845-z4gws"
Jul 29 18:45:00.161216 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:00.161198    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755845-9cbbh"
Jul 29 18:45:00.304223 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:00.304195    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6e9884-29755845-z4gws"]
Jul 29 18:45:00.305540 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:45:00.305501    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbeeb4c2d_edaa_4153_bb1d_f1dfd95e5eb9.slice/crio-7e12a9d252122c56a5b42aee8f554386e4d9a573e048079947af939c27f908d1 WatchSource:0}: Error finding container 7e12a9d252122c56a5b42aee8f554386e4d9a573e048079947af939c27f908d1: Status 404 returned error can't find the container with id 7e12a9d252122c56a5b42aee8f554386e4d9a573e048079947af939c27f908d1
Jul 29 18:45:00.331956 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:00.331930    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755845-9cbbh"]
Jul 29 18:45:00.332460 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:45:00.332435    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod345eaead_a9fb_4a6f_8bbe_5451b03b4925.slice/crio-30440f284aa9870a721b4041e7a0aae4f6fc6ecc5230a6545f8018236348dc24 WatchSource:0}: Error finding container 30440f284aa9870a721b4041e7a0aae4f6fc6ecc5230a6545f8018236348dc24: Status 404 returned error can't find the container with id 30440f284aa9870a721b4041e7a0aae4f6fc6ecc5230a6545f8018236348dc24
Jul 29 18:45:01.193814 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:01.193765    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6e9884-29755845-z4gws" event={"ID":"beeb4c2d-edaa-4153-bb1d-f1dfd95e5eb9","Type":"ContainerStarted","Data":"a42f934467ed81f5958050806e8554d0367b8d12509166f856ed5f6ac9490979"}
Jul 29 18:45:01.194242 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:01.193835    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6e9884-29755845-z4gws" event={"ID":"beeb4c2d-edaa-4153-bb1d-f1dfd95e5eb9","Type":"ContainerStarted","Data":"7e12a9d252122c56a5b42aee8f554386e4d9a573e048079947af939c27f908d1"}
Jul 29 18:45:01.195283 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:01.195258    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755845-9cbbh" event={"ID":"345eaead-a9fb-4a6f-8bbe-5451b03b4925","Type":"ContainerStarted","Data":"f5e9d322a3c12c70c27bbd035906183d12ac341c50af7f85015418592e9e8197"}
Jul 29 18:45:01.195427 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:01.195310    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755845-9cbbh" event={"ID":"345eaead-a9fb-4a6f-8bbe-5451b03b4925","Type":"ContainerStarted","Data":"30440f284aa9870a721b4041e7a0aae4f6fc6ecc5230a6545f8018236348dc24"}
Jul 29 18:45:01.195427 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:01.195343    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-zxrcp" podUID="f23ae7b5-7cb7-4017-9a1b-27ab84a2aacd" containerName="registry-server" containerID="cri-o://7a2ee110d9b048e8be24c712317c97a3ab9bf71b631482a6f5f5a22d4588ceae" gracePeriod=2
Jul 29 18:45:01.211950 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:01.211906    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6e9884-29755845-z4gws" podStartSLOduration=1.211892075 podStartE2EDuration="1.211892075s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:45:01.209391764 +0000 UTC m=+3181.078614590" watchObservedRunningTime="2026-07-29 18:45:01.211892075 +0000 UTC m=+3181.081114881"
Jul 29 18:45:01.225527 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:01.225485    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755845-9cbbh" podStartSLOduration=1.225469921 podStartE2EDuration="1.225469921s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:45:01.223216724 +0000 UTC m=+3181.092439532" watchObservedRunningTime="2026-07-29 18:45:01.225469921 +0000 UTC m=+3181.094692728"
Jul 29 18:45:01.544375 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:01.544348    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-zxrcp"
Jul 29 18:45:01.644259 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:01.644233    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wqm22\" (UniqueName: \"kubernetes.io/projected/f23ae7b5-7cb7-4017-9a1b-27ab84a2aacd-kube-api-access-wqm22\") pod \"f23ae7b5-7cb7-4017-9a1b-27ab84a2aacd\" (UID: \"f23ae7b5-7cb7-4017-9a1b-27ab84a2aacd\") "
Jul 29 18:45:01.646376 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:01.646353    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f23ae7b5-7cb7-4017-9a1b-27ab84a2aacd-kube-api-access-wqm22" (OuterVolumeSpecName: "kube-api-access-wqm22") pod "f23ae7b5-7cb7-4017-9a1b-27ab84a2aacd" (UID: "f23ae7b5-7cb7-4017-9a1b-27ab84a2aacd"). InnerVolumeSpecName "kube-api-access-wqm22". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:45:01.745563 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:01.745485    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqm22\" (UniqueName: \"kubernetes.io/projected/f23ae7b5-7cb7-4017-9a1b-27ab84a2aacd-kube-api-access-wqm22\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:45:02.203305 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:02.203262    2557 generic.go:356] "Generic (PLEG): container finished" podID="beeb4c2d-edaa-4153-bb1d-f1dfd95e5eb9" containerID="a42f934467ed81f5958050806e8554d0367b8d12509166f856ed5f6ac9490979" exitCode=0
Jul 29 18:45:02.203769 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:02.203349    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6e9884-29755845-z4gws" event={"ID":"beeb4c2d-edaa-4153-bb1d-f1dfd95e5eb9","Type":"ContainerDied","Data":"a42f934467ed81f5958050806e8554d0367b8d12509166f856ed5f6ac9490979"}
Jul 29 18:45:02.204546 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:02.204527    2557 generic.go:356] "Generic (PLEG): container finished" podID="f23ae7b5-7cb7-4017-9a1b-27ab84a2aacd" containerID="7a2ee110d9b048e8be24c712317c97a3ab9bf71b631482a6f5f5a22d4588ceae" exitCode=0
Jul 29 18:45:02.204593 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:02.204564    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-zxrcp"
Jul 29 18:45:02.204628 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:02.204608    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-zxrcp" event={"ID":"f23ae7b5-7cb7-4017-9a1b-27ab84a2aacd","Type":"ContainerDied","Data":"7a2ee110d9b048e8be24c712317c97a3ab9bf71b631482a6f5f5a22d4588ceae"}
Jul 29 18:45:02.204668 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:02.204650    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-zxrcp" event={"ID":"f23ae7b5-7cb7-4017-9a1b-27ab84a2aacd","Type":"ContainerDied","Data":"fa33089ec74496210b6eff2cca9b5d03d863fef920e0a7b18fb9e0065009fd63"}
Jul 29 18:45:02.204706 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:02.204673    2557 scope.go:117] "RemoveContainer" containerID="7a2ee110d9b048e8be24c712317c97a3ab9bf71b631482a6f5f5a22d4588ceae"
Jul 29 18:45:02.206278 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:02.206246    2557 generic.go:356] "Generic (PLEG): container finished" podID="345eaead-a9fb-4a6f-8bbe-5451b03b4925" containerID="f5e9d322a3c12c70c27bbd035906183d12ac341c50af7f85015418592e9e8197" exitCode=0
Jul 29 18:45:02.206387 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:02.206272    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755845-9cbbh" event={"ID":"345eaead-a9fb-4a6f-8bbe-5451b03b4925","Type":"ContainerDied","Data":"f5e9d322a3c12c70c27bbd035906183d12ac341c50af7f85015418592e9e8197"}
Jul 29 18:45:02.214579 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:02.214561    2557 scope.go:117] "RemoveContainer" containerID="7a2ee110d9b048e8be24c712317c97a3ab9bf71b631482a6f5f5a22d4588ceae"
Jul 29 18:45:02.214866 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:45:02.214848    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a2ee110d9b048e8be24c712317c97a3ab9bf71b631482a6f5f5a22d4588ceae\": container with ID starting with 7a2ee110d9b048e8be24c712317c97a3ab9bf71b631482a6f5f5a22d4588ceae not found: ID does not exist" containerID="7a2ee110d9b048e8be24c712317c97a3ab9bf71b631482a6f5f5a22d4588ceae"
Jul 29 18:45:02.214919 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:02.214874    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a2ee110d9b048e8be24c712317c97a3ab9bf71b631482a6f5f5a22d4588ceae"} err="failed to get container status \"7a2ee110d9b048e8be24c712317c97a3ab9bf71b631482a6f5f5a22d4588ceae\": rpc error: code = NotFound desc = could not find container \"7a2ee110d9b048e8be24c712317c97a3ab9bf71b631482a6f5f5a22d4588ceae\": container with ID starting with 7a2ee110d9b048e8be24c712317c97a3ab9bf71b631482a6f5f5a22d4588ceae not found: ID does not exist"
Jul 29 18:45:02.244677 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:02.244649    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-zxrcp"]
Jul 29 18:45:02.248678 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:02.248654    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-zxrcp"]
Jul 29 18:45:02.653255 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:02.653223    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f23ae7b5-7cb7-4017-9a1b-27ab84a2aacd" path="/var/lib/kubelet/pods/f23ae7b5-7cb7-4017-9a1b-27ab84a2aacd/volumes"
Jul 29 18:45:03.364110 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:03.364088    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755845-9cbbh"
Jul 29 18:45:03.373434 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:03.373412    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6e9884-29755845-z4gws"
Jul 29 18:45:04.221689 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:04.221660    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755845-9cbbh"
Jul 29 18:45:04.221911 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:04.221713    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755845-9cbbh" event={"ID":"345eaead-a9fb-4a6f-8bbe-5451b03b4925","Type":"ContainerDied","Data":"30440f284aa9870a721b4041e7a0aae4f6fc6ecc5230a6545f8018236348dc24"}
Jul 29 18:45:04.221911 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:04.221823    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="30440f284aa9870a721b4041e7a0aae4f6fc6ecc5230a6545f8018236348dc24"
Jul 29 18:45:04.223206 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:04.223185    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6e9884-29755845-z4gws" event={"ID":"beeb4c2d-edaa-4153-bb1d-f1dfd95e5eb9","Type":"ContainerDied","Data":"7e12a9d252122c56a5b42aee8f554386e4d9a573e048079947af939c27f908d1"}
Jul 29 18:45:04.223332 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:04.223221    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6e9884-29755845-z4gws"
Jul 29 18:45:04.223332 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:04.223226    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7e12a9d252122c56a5b42aee8f554386e4d9a573e048079947af939c27f908d1"
Jul 29 18:45:11.092418 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:11.092387    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-750f0d-6f5bdf6cfb-sn7sj"]
Jul 29 18:45:11.092908 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:11.092835    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="beeb4c2d-edaa-4153-bb1d-f1dfd95e5eb9" containerName="cleanup"
Jul 29 18:45:11.092908 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:11.092848    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="beeb4c2d-edaa-4153-bb1d-f1dfd95e5eb9" containerName="cleanup"
Jul 29 18:45:11.092908 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:11.092863    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="345eaead-a9fb-4a6f-8bbe-5451b03b4925" containerName="cleanup"
Jul 29 18:45:11.092908 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:11.092869    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="345eaead-a9fb-4a6f-8bbe-5451b03b4925" containerName="cleanup"
Jul 29 18:45:11.092908 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:11.092883    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f23ae7b5-7cb7-4017-9a1b-27ab84a2aacd" containerName="registry-server"
Jul 29 18:45:11.092908 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:11.092889    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="f23ae7b5-7cb7-4017-9a1b-27ab84a2aacd" containerName="registry-server"
Jul 29 18:45:11.093236 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:11.092958    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="f23ae7b5-7cb7-4017-9a1b-27ab84a2aacd" containerName="registry-server"
Jul 29 18:45:11.093236 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:11.092968    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="beeb4c2d-edaa-4153-bb1d-f1dfd95e5eb9" containerName="cleanup"
Jul 29 18:45:11.093236 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:11.092974    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="345eaead-a9fb-4a6f-8bbe-5451b03b4925" containerName="cleanup"
Jul 29 18:45:11.096034 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:11.096013    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-750f0d-6f5bdf6cfb-sn7sj"
Jul 29 18:45:11.098819 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:11.098797    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-750f0d-serving-cert\""
Jul 29 18:45:11.111140 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:11.108987    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-750f0d-6f5bdf6cfb-sn7sj"]
Jul 29 18:45:11.242549 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:11.242507    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63f41491-af00-4be7-ab6e-7b2d955d2f62-openshift-service-ca\") pod \"maas-api-e2e-shared-b-750f0d-6f5bdf6cfb-sn7sj\" (UID: \"63f41491-af00-4be7-ab6e-7b2d955d2f62\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-750f0d-6f5bdf6cfb-sn7sj"
Jul 29 18:45:11.242716 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:11.242560    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63f41491-af00-4be7-ab6e-7b2d955d2f62-maas-api-tls\") pod \"maas-api-e2e-shared-b-750f0d-6f5bdf6cfb-sn7sj\" (UID: \"63f41491-af00-4be7-ab6e-7b2d955d2f62\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-750f0d-6f5bdf6cfb-sn7sj"
Jul 29 18:45:11.242792 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:11.242711    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xhv7\" (UniqueName: \"kubernetes.io/projected/63f41491-af00-4be7-ab6e-7b2d955d2f62-kube-api-access-9xhv7\") pod \"maas-api-e2e-shared-b-750f0d-6f5bdf6cfb-sn7sj\" (UID: \"63f41491-af00-4be7-ab6e-7b2d955d2f62\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-750f0d-6f5bdf6cfb-sn7sj"
Jul 29 18:45:11.343941 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:11.343849    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xhv7\" (UniqueName: \"kubernetes.io/projected/63f41491-af00-4be7-ab6e-7b2d955d2f62-kube-api-access-9xhv7\") pod \"maas-api-e2e-shared-b-750f0d-6f5bdf6cfb-sn7sj\" (UID: \"63f41491-af00-4be7-ab6e-7b2d955d2f62\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-750f0d-6f5bdf6cfb-sn7sj"
Jul 29 18:45:11.343941 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:11.343904    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63f41491-af00-4be7-ab6e-7b2d955d2f62-openshift-service-ca\") pod \"maas-api-e2e-shared-b-750f0d-6f5bdf6cfb-sn7sj\" (UID: \"63f41491-af00-4be7-ab6e-7b2d955d2f62\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-750f0d-6f5bdf6cfb-sn7sj"
Jul 29 18:45:11.343941 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:11.343936    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63f41491-af00-4be7-ab6e-7b2d955d2f62-maas-api-tls\") pod \"maas-api-e2e-shared-b-750f0d-6f5bdf6cfb-sn7sj\" (UID: \"63f41491-af00-4be7-ab6e-7b2d955d2f62\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-750f0d-6f5bdf6cfb-sn7sj"
Jul 29 18:45:11.344227 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:45:11.344084    2557 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-750f0d-serving-cert: secret "maas-api-e2e-shared-b-750f0d-serving-cert" not found
Jul 29 18:45:11.344227 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:45:11.344155    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63f41491-af00-4be7-ab6e-7b2d955d2f62-maas-api-tls podName:63f41491-af00-4be7-ab6e-7b2d955d2f62 nodeName:}" failed. No retries permitted until 2026-07-29 18:45:11.844135582 +0000 UTC m=+3191.713358373 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/63f41491-af00-4be7-ab6e-7b2d955d2f62-maas-api-tls") pod "maas-api-e2e-shared-b-750f0d-6f5bdf6cfb-sn7sj" (UID: "63f41491-af00-4be7-ab6e-7b2d955d2f62") : secret "maas-api-e2e-shared-b-750f0d-serving-cert" not found
Jul 29 18:45:11.344664 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:11.344645    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63f41491-af00-4be7-ab6e-7b2d955d2f62-openshift-service-ca\") pod \"maas-api-e2e-shared-b-750f0d-6f5bdf6cfb-sn7sj\" (UID: \"63f41491-af00-4be7-ab6e-7b2d955d2f62\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-750f0d-6f5bdf6cfb-sn7sj"
Jul 29 18:45:11.352447 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:11.352417    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xhv7\" (UniqueName: \"kubernetes.io/projected/63f41491-af00-4be7-ab6e-7b2d955d2f62-kube-api-access-9xhv7\") pod \"maas-api-e2e-shared-b-750f0d-6f5bdf6cfb-sn7sj\" (UID: \"63f41491-af00-4be7-ab6e-7b2d955d2f62\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-750f0d-6f5bdf6cfb-sn7sj"
Jul 29 18:45:11.848454 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:11.848411    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63f41491-af00-4be7-ab6e-7b2d955d2f62-maas-api-tls\") pod \"maas-api-e2e-shared-b-750f0d-6f5bdf6cfb-sn7sj\" (UID: \"63f41491-af00-4be7-ab6e-7b2d955d2f62\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-750f0d-6f5bdf6cfb-sn7sj"
Jul 29 18:45:11.851043 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:11.851021    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63f41491-af00-4be7-ab6e-7b2d955d2f62-maas-api-tls\") pod \"maas-api-e2e-shared-b-750f0d-6f5bdf6cfb-sn7sj\" (UID: \"63f41491-af00-4be7-ab6e-7b2d955d2f62\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-750f0d-6f5bdf6cfb-sn7sj"
Jul 29 18:45:12.016346 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:12.016311    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-750f0d-6f5bdf6cfb-sn7sj"
Jul 29 18:45:12.157472 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:12.157447    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-750f0d-6f5bdf6cfb-sn7sj"]
Jul 29 18:45:12.158991 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:45:12.158948    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63f41491_af00_4be7_ab6e_7b2d955d2f62.slice/crio-8be1932d873b018d4a8dda6e1084f10a9f4d9c2abaf6c4aa86e65814daace638 WatchSource:0}: Error finding container 8be1932d873b018d4a8dda6e1084f10a9f4d9c2abaf6c4aa86e65814daace638: Status 404 returned error can't find the container with id 8be1932d873b018d4a8dda6e1084f10a9f4d9c2abaf6c4aa86e65814daace638
Jul 29 18:45:12.160447 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:12.160427    2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 18:45:12.270327 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:12.270295    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-750f0d-6f5bdf6cfb-sn7sj" event={"ID":"63f41491-af00-4be7-ab6e-7b2d955d2f62","Type":"ContainerStarted","Data":"8be1932d873b018d4a8dda6e1084f10a9f4d9c2abaf6c4aa86e65814daace638"}
Jul 29 18:45:13.277903 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:13.277867    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-750f0d-6f5bdf6cfb-sn7sj" event={"ID":"63f41491-af00-4be7-ab6e-7b2d955d2f62","Type":"ContainerStarted","Data":"a5635c164365da0745bb9233664965ca940be701b3178e47b2d341af82c69d23"}
Jul 29 18:45:13.278310 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:13.277932    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-750f0d-6f5bdf6cfb-sn7sj"
Jul 29 18:45:13.293167 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:13.293114    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-750f0d-6f5bdf6cfb-sn7sj" podStartSLOduration=1.892512903 podStartE2EDuration="2.293099114s" totalImagesPullingTime="400.586211ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:45:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:45:13.292023171 +0000 UTC m=+3193.161245979" watchObservedRunningTime="2026-07-29 18:45:13.293099114 +0000 UTC m=+3193.162321921"
Jul 29 18:45:19.289893 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:45:19.289865    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-750f0d-6f5bdf6cfb-sn7sj"
Jul 29 18:47:00.952392 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:47:00.952284    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gg2wj_2bf0bb71-4047-45a7-bd8e-5d6efabd26bd/console-operator/2.log"
Jul 29 18:47:00.965603 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:47:00.965581    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gg2wj_2bf0bb71-4047-45a7-bd8e-5d6efabd26bd/console-operator/2.log"
Jul 29 18:49:01.182068 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:49:01.182037    2557 scope.go:117] "RemoveContainer" containerID="c284a9992281e2a181b57be6bc115cc0f592d91c732584549b933e96179c5293"
Jul 29 18:49:01.191565 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:49:01.191542    2557 scope.go:117] "RemoveContainer" containerID="7f6220ea4ace30a1db0a317f7c69400ebe8e074d8dc652565f2bc88870e5a06d"
Jul 29 18:49:01.200376 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:49:01.200352    2557 scope.go:117] "RemoveContainer" containerID="fe783fe8c680ecb6b9bcd51d861272b3f9cae1976141c468bdaa33a3255a5910"
Jul 29 18:50:01.229433 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:50:01.229400    2557 scope.go:117] "RemoveContainer" containerID="cfd9c8d0038d29fe8a4f4ed2bf93414c0c7cffcf7cdfa3dc8b7eee7f8ff2f51e"
Jul 29 18:51:01.252154 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:51:01.252103    2557 scope.go:117] "RemoveContainer" containerID="22ec6e8f09e4431ee436c6d4b6ec8b7ee6747d5f655c340c061c23e38e397f64"
Jul 29 18:52:00.988886 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:52:00.988777    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gg2wj_2bf0bb71-4047-45a7-bd8e-5d6efabd26bd/console-operator/2.log"
Jul 29 18:52:01.009885 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:52:01.009859    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gg2wj_2bf0bb71-4047-45a7-bd8e-5d6efabd26bd/console-operator/2.log"
Jul 29 18:56:04.155140 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:04.155106    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7488-140611831847728"]
Jul 29 18:56:04.158688 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:04.158657    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7488-140611831847728"
Jul 29 18:56:04.161354 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:04.161327    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-zflq2\""
Jul 29 18:56:04.162188 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:04.162166    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7488-140611831847728"]
Jul 29 18:56:04.243643 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:04.243595    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twrss\" (UniqueName: \"kubernetes.io/projected/0edb0c5d-b1af-4ece-a211-e4036af8b2ae-kube-api-access-twrss\") pod \"test-curl-7488-140611831847728\" (UID: \"0edb0c5d-b1af-4ece-a211-e4036af8b2ae\") " pod="opendatahub/test-curl-7488-140611831847728"
Jul 29 18:56:04.344853 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:04.344811    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twrss\" (UniqueName: \"kubernetes.io/projected/0edb0c5d-b1af-4ece-a211-e4036af8b2ae-kube-api-access-twrss\") pod \"test-curl-7488-140611831847728\" (UID: \"0edb0c5d-b1af-4ece-a211-e4036af8b2ae\") " pod="opendatahub/test-curl-7488-140611831847728"
Jul 29 18:56:04.353885 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:04.353848    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twrss\" (UniqueName: \"kubernetes.io/projected/0edb0c5d-b1af-4ece-a211-e4036af8b2ae-kube-api-access-twrss\") pod \"test-curl-7488-140611831847728\" (UID: \"0edb0c5d-b1af-4ece-a211-e4036af8b2ae\") " pod="opendatahub/test-curl-7488-140611831847728"
Jul 29 18:56:04.470001 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:04.469928    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7488-140611831847728"
Jul 29 18:56:04.608143 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:04.608109    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7488-140611831847728"]
Jul 29 18:56:04.608795 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:56:04.608767    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0edb0c5d_b1af_4ece_a211_e4036af8b2ae.slice/crio-8918d62e49d6e94c8f9cc8a492a1670626a0712738d53fdaddcedd5b42e29b6a WatchSource:0}: Error finding container 8918d62e49d6e94c8f9cc8a492a1670626a0712738d53fdaddcedd5b42e29b6a: Status 404 returned error can't find the container with id 8918d62e49d6e94c8f9cc8a492a1670626a0712738d53fdaddcedd5b42e29b6a
Jul 29 18:56:04.610410 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:04.610393    2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 18:56:05.093338 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:05.093291    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7488-140611831847728" event={"ID":"0edb0c5d-b1af-4ece-a211-e4036af8b2ae","Type":"ContainerStarted","Data":"8918d62e49d6e94c8f9cc8a492a1670626a0712738d53fdaddcedd5b42e29b6a"}
Jul 29 18:56:06.101005 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:06.100963    2557 generic.go:356] "Generic (PLEG): container finished" podID="0edb0c5d-b1af-4ece-a211-e4036af8b2ae" containerID="5b7dd6aef00ca4aa938468f70c3afab5c72b061e7c256cabce5263a42f8f5d30" exitCode=0
Jul 29 18:56:06.101408 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:06.101143    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7488-140611831847728" event={"ID":"0edb0c5d-b1af-4ece-a211-e4036af8b2ae","Type":"ContainerDied","Data":"5b7dd6aef00ca4aa938468f70c3afab5c72b061e7c256cabce5263a42f8f5d30"}
Jul 29 18:56:07.254358 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:07.254331    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7488-140611831847728"
Jul 29 18:56:07.287886 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:07.287865    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7488-140611831847728_0edb0c5d-b1af-4ece-a211-e4036af8b2ae/test-curl-7488-140611831847728/0.log"
Jul 29 18:56:07.311772 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:07.311743    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7488-140611831847728"]
Jul 29 18:56:07.315149 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:07.315126    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7488-140611831847728"]
Jul 29 18:56:07.373112 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:07.373041    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-twrss\" (UniqueName: \"kubernetes.io/projected/0edb0c5d-b1af-4ece-a211-e4036af8b2ae-kube-api-access-twrss\") pod \"0edb0c5d-b1af-4ece-a211-e4036af8b2ae\" (UID: \"0edb0c5d-b1af-4ece-a211-e4036af8b2ae\") "
Jul 29 18:56:07.375270 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:07.375244    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0edb0c5d-b1af-4ece-a211-e4036af8b2ae-kube-api-access-twrss" (OuterVolumeSpecName: "kube-api-access-twrss") pod "0edb0c5d-b1af-4ece-a211-e4036af8b2ae" (UID: "0edb0c5d-b1af-4ece-a211-e4036af8b2ae"). InnerVolumeSpecName "kube-api-access-twrss". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:56:07.439676 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:07.439644    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7488-140611831847600"]
Jul 29 18:56:07.440148 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:07.440128    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0edb0c5d-b1af-4ece-a211-e4036af8b2ae" containerName="test-curl-7488-140611831847728"
Jul 29 18:56:07.440234 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:07.440151    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="0edb0c5d-b1af-4ece-a211-e4036af8b2ae" containerName="test-curl-7488-140611831847728"
Jul 29 18:56:07.440285 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:07.440240    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="0edb0c5d-b1af-4ece-a211-e4036af8b2ae" containerName="test-curl-7488-140611831847728"
Jul 29 18:56:07.443450 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:07.443421    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7488-140611831847600"
Jul 29 18:56:07.450402 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:07.450334    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7488-140611831847600"]
Jul 29 18:56:07.474045 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:07.474018    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-twrss\" (UniqueName: \"kubernetes.io/projected/0edb0c5d-b1af-4ece-a211-e4036af8b2ae-kube-api-access-twrss\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:56:07.574801 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:07.574769    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r46n2\" (UniqueName: \"kubernetes.io/projected/347fc7e3-173f-4edc-bf99-4c970ac38f89-kube-api-access-r46n2\") pod \"test-curl-7488-140611831847600\" (UID: \"347fc7e3-173f-4edc-bf99-4c970ac38f89\") " pod="opendatahub/test-curl-7488-140611831847600"
Jul 29 18:56:07.675253 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:07.675174    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r46n2\" (UniqueName: \"kubernetes.io/projected/347fc7e3-173f-4edc-bf99-4c970ac38f89-kube-api-access-r46n2\") pod \"test-curl-7488-140611831847600\" (UID: \"347fc7e3-173f-4edc-bf99-4c970ac38f89\") " pod="opendatahub/test-curl-7488-140611831847600"
Jul 29 18:56:07.683569 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:07.683544    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r46n2\" (UniqueName: \"kubernetes.io/projected/347fc7e3-173f-4edc-bf99-4c970ac38f89-kube-api-access-r46n2\") pod \"test-curl-7488-140611831847600\" (UID: \"347fc7e3-173f-4edc-bf99-4c970ac38f89\") " pod="opendatahub/test-curl-7488-140611831847600"
Jul 29 18:56:07.753686 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:07.753654    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7488-140611831847600"
Jul 29 18:56:07.894420 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:07.894389    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7488-140611831847600"]
Jul 29 18:56:07.895433 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:56:07.895406    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod347fc7e3_173f_4edc_bf99_4c970ac38f89.slice/crio-f3c14c0bc062383ecac9e69dd551f6008f0eef545843c2272a2a57e486d7867f WatchSource:0}: Error finding container f3c14c0bc062383ecac9e69dd551f6008f0eef545843c2272a2a57e486d7867f: Status 404 returned error can't find the container with id f3c14c0bc062383ecac9e69dd551f6008f0eef545843c2272a2a57e486d7867f
Jul 29 18:56:08.114545 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:08.114509    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7488-140611831847600" event={"ID":"347fc7e3-173f-4edc-bf99-4c970ac38f89","Type":"ContainerStarted","Data":"f3c14c0bc062383ecac9e69dd551f6008f0eef545843c2272a2a57e486d7867f"}
Jul 29 18:56:08.115840 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:08.115810    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8918d62e49d6e94c8f9cc8a492a1670626a0712738d53fdaddcedd5b42e29b6a"
Jul 29 18:56:08.115840 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:08.115819    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7488-140611831847728"
Jul 29 18:56:08.126502 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:56:08.126475    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7488-140611831847728\" is forbidden: User \"system:node:ip-10-0-142-156.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-156.ec2.internal' and this object" podUID="0edb0c5d-b1af-4ece-a211-e4036af8b2ae" pod="opendatahub/test-curl-7488-140611831847728"
Jul 29 18:56:08.652423 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:08.652390    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0edb0c5d-b1af-4ece-a211-e4036af8b2ae" path="/var/lib/kubelet/pods/0edb0c5d-b1af-4ece-a211-e4036af8b2ae/volumes"
Jul 29 18:56:09.122814 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:09.122784    2557 generic.go:356] "Generic (PLEG): container finished" podID="347fc7e3-173f-4edc-bf99-4c970ac38f89" containerID="f27e7c1d0e833e1bed1b4dbe35751cf065fb8854944bda2a997b8d00aca8efd9" exitCode=0
Jul 29 18:56:09.122995 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:09.122819    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7488-140611831847600" event={"ID":"347fc7e3-173f-4edc-bf99-4c970ac38f89","Type":"ContainerDied","Data":"f27e7c1d0e833e1bed1b4dbe35751cf065fb8854944bda2a997b8d00aca8efd9"}
Jul 29 18:56:10.263964 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:10.263939    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7488-140611831847600"
Jul 29 18:56:10.282844 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:10.282818    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7488-140611831847600_347fc7e3-173f-4edc-bf99-4c970ac38f89/test-curl-7488-140611831847600/0.log"
Jul 29 18:56:10.307511 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:10.307476    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7488-140611831847600"]
Jul 29 18:56:10.311891 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:10.311867    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7488-140611831847600"]
Jul 29 18:56:10.400318 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:10.400241    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r46n2\" (UniqueName: \"kubernetes.io/projected/347fc7e3-173f-4edc-bf99-4c970ac38f89-kube-api-access-r46n2\") pod \"347fc7e3-173f-4edc-bf99-4c970ac38f89\" (UID: \"347fc7e3-173f-4edc-bf99-4c970ac38f89\") "
Jul 29 18:56:10.402459 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:10.402434    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/347fc7e3-173f-4edc-bf99-4c970ac38f89-kube-api-access-r46n2" (OuterVolumeSpecName: "kube-api-access-r46n2") pod "347fc7e3-173f-4edc-bf99-4c970ac38f89" (UID: "347fc7e3-173f-4edc-bf99-4c970ac38f89"). InnerVolumeSpecName "kube-api-access-r46n2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:56:10.465566 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:10.465534    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7488-140611831789872"]
Jul 29 18:56:10.465959 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:10.465944    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="347fc7e3-173f-4edc-bf99-4c970ac38f89" containerName="test-curl-7488-140611831847600"
Jul 29 18:56:10.466015 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:10.465962    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="347fc7e3-173f-4edc-bf99-4c970ac38f89" containerName="test-curl-7488-140611831847600"
Jul 29 18:56:10.466048 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:10.466038    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="347fc7e3-173f-4edc-bf99-4c970ac38f89" containerName="test-curl-7488-140611831847600"
Jul 29 18:56:10.469066 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:10.469043    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7488-140611831789872"
Jul 29 18:56:10.472307 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:10.472279    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7488-140611831789872"]
Jul 29 18:56:10.501584 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:10.501556    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r46n2\" (UniqueName: \"kubernetes.io/projected/347fc7e3-173f-4edc-bf99-4c970ac38f89-kube-api-access-r46n2\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:56:10.602332 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:10.602298    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjbfm\" (UniqueName: \"kubernetes.io/projected/2bfe5bb0-8a48-4908-97c1-d4239686c367-kube-api-access-sjbfm\") pod \"test-curl-7488-140611831789872\" (UID: \"2bfe5bb0-8a48-4908-97c1-d4239686c367\") " pod="opendatahub/test-curl-7488-140611831789872"
Jul 29 18:56:10.653283 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:10.653213    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="347fc7e3-173f-4edc-bf99-4c970ac38f89" path="/var/lib/kubelet/pods/347fc7e3-173f-4edc-bf99-4c970ac38f89/volumes"
Jul 29 18:56:10.702846 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:10.702815    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjbfm\" (UniqueName: \"kubernetes.io/projected/2bfe5bb0-8a48-4908-97c1-d4239686c367-kube-api-access-sjbfm\") pod \"test-curl-7488-140611831789872\" (UID: \"2bfe5bb0-8a48-4908-97c1-d4239686c367\") " pod="opendatahub/test-curl-7488-140611831789872"
Jul 29 18:56:10.711842 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:10.711819    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjbfm\" (UniqueName: \"kubernetes.io/projected/2bfe5bb0-8a48-4908-97c1-d4239686c367-kube-api-access-sjbfm\") pod \"test-curl-7488-140611831789872\" (UID: \"2bfe5bb0-8a48-4908-97c1-d4239686c367\") " pod="opendatahub/test-curl-7488-140611831789872"
Jul 29 18:56:10.782004 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:10.781978    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7488-140611831789872"
Jul 29 18:56:10.918380 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:10.918351    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7488-140611831789872"]
Jul 29 18:56:10.919418 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:56:10.919398    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2bfe5bb0_8a48_4908_97c1_d4239686c367.slice/crio-08c11db351a40d9237699e67f434beec6147a91e14e7eff57f38404369d94bef WatchSource:0}: Error finding container 08c11db351a40d9237699e67f434beec6147a91e14e7eff57f38404369d94bef: Status 404 returned error can't find the container with id 08c11db351a40d9237699e67f434beec6147a91e14e7eff57f38404369d94bef
Jul 29 18:56:11.134243 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:11.134214    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7488-140611831789872" event={"ID":"2bfe5bb0-8a48-4908-97c1-d4239686c367","Type":"ContainerStarted","Data":"08c11db351a40d9237699e67f434beec6147a91e14e7eff57f38404369d94bef"}
Jul 29 18:56:11.135444 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:11.135423    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7488-140611831847600"
Jul 29 18:56:11.135573 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:11.135470    2557 scope.go:117] "RemoveContainer" containerID="f27e7c1d0e833e1bed1b4dbe35751cf065fb8854944bda2a997b8d00aca8efd9"
Jul 29 18:56:12.144180 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:12.144143    2557 generic.go:356] "Generic (PLEG): container finished" podID="2bfe5bb0-8a48-4908-97c1-d4239686c367" containerID="6852b7d52e57518e1005bb6afb37896eb5e2ce0943e8eb4f66944d85dde8f739" exitCode=0
Jul 29 18:56:12.144620 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:12.144219    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7488-140611831789872" event={"ID":"2bfe5bb0-8a48-4908-97c1-d4239686c367","Type":"ContainerDied","Data":"6852b7d52e57518e1005bb6afb37896eb5e2ce0943e8eb4f66944d85dde8f739"}
Jul 29 18:56:13.289998 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:13.289971    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7488-140611831789872"
Jul 29 18:56:13.307169 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:13.307135    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7488-140611831789872_2bfe5bb0-8a48-4908-97c1-d4239686c367/test-curl-7488-140611831789872/0.log"
Jul 29 18:56:13.329623 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:13.329598    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7488-140611831789872"]
Jul 29 18:56:13.333913 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:13.333877    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7488-140611831789872"]
Jul 29 18:56:13.425748 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:13.425664    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sjbfm\" (UniqueName: \"kubernetes.io/projected/2bfe5bb0-8a48-4908-97c1-d4239686c367-kube-api-access-sjbfm\") pod \"2bfe5bb0-8a48-4908-97c1-d4239686c367\" (UID: \"2bfe5bb0-8a48-4908-97c1-d4239686c367\") "
Jul 29 18:56:13.427964 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:13.427939    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2bfe5bb0-8a48-4908-97c1-d4239686c367-kube-api-access-sjbfm" (OuterVolumeSpecName: "kube-api-access-sjbfm") pod "2bfe5bb0-8a48-4908-97c1-d4239686c367" (UID: "2bfe5bb0-8a48-4908-97c1-d4239686c367"). InnerVolumeSpecName "kube-api-access-sjbfm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:56:13.444145 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:13.444121    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7488-140611831789872"]
Jul 29 18:56:13.444540 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:13.444526    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bfe5bb0-8a48-4908-97c1-d4239686c367" containerName="test-curl-7488-140611831789872"
Jul 29 18:56:13.444619 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:13.444540    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bfe5bb0-8a48-4908-97c1-d4239686c367" containerName="test-curl-7488-140611831789872"
Jul 29 18:56:13.444619 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:13.444618    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bfe5bb0-8a48-4908-97c1-d4239686c367" containerName="test-curl-7488-140611831789872"
Jul 29 18:56:13.447678 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:13.447664    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7488-140611831789872"
Jul 29 18:56:13.454151 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:13.454125    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7488-140611831789872"]
Jul 29 18:56:13.527022 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:13.526993    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ch7q4\" (UniqueName: \"kubernetes.io/projected/7a84813e-48e1-4061-9cf6-a6a6917cf6ea-kube-api-access-ch7q4\") pod \"test-curl-7488-140611831789872\" (UID: \"7a84813e-48e1-4061-9cf6-a6a6917cf6ea\") " pod="opendatahub/test-curl-7488-140611831789872"
Jul 29 18:56:13.527190 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:13.527064    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sjbfm\" (UniqueName: \"kubernetes.io/projected/2bfe5bb0-8a48-4908-97c1-d4239686c367-kube-api-access-sjbfm\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:56:13.627496 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:13.627454    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ch7q4\" (UniqueName: \"kubernetes.io/projected/7a84813e-48e1-4061-9cf6-a6a6917cf6ea-kube-api-access-ch7q4\") pod \"test-curl-7488-140611831789872\" (UID: \"7a84813e-48e1-4061-9cf6-a6a6917cf6ea\") " pod="opendatahub/test-curl-7488-140611831789872"
Jul 29 18:56:13.635353 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:13.635328    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ch7q4\" (UniqueName: \"kubernetes.io/projected/7a84813e-48e1-4061-9cf6-a6a6917cf6ea-kube-api-access-ch7q4\") pod \"test-curl-7488-140611831789872\" (UID: \"7a84813e-48e1-4061-9cf6-a6a6917cf6ea\") " pod="opendatahub/test-curl-7488-140611831789872"
Jul 29 18:56:13.759537 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:13.759432    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7488-140611831789872"
Jul 29 18:56:13.895252 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:13.895228    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7488-140611831789872"]
Jul 29 18:56:13.896201 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:56:13.896173    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7a84813e_48e1_4061_9cf6_a6a6917cf6ea.slice/crio-84f54376c5bd6a0cce127dfb9fda4f474a61b3d5e8f3befa5c51161f1593b674 WatchSource:0}: Error finding container 84f54376c5bd6a0cce127dfb9fda4f474a61b3d5e8f3befa5c51161f1593b674: Status 404 returned error can't find the container with id 84f54376c5bd6a0cce127dfb9fda4f474a61b3d5e8f3befa5c51161f1593b674
Jul 29 18:56:14.158401 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:14.158373    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="08c11db351a40d9237699e67f434beec6147a91e14e7eff57f38404369d94bef"
Jul 29 18:56:14.158401 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:14.158376    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7488-140611831789872"
Jul 29 18:56:14.159553 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:14.159528    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7488-140611831789872" event={"ID":"7a84813e-48e1-4061-9cf6-a6a6917cf6ea","Type":"ContainerStarted","Data":"84f54376c5bd6a0cce127dfb9fda4f474a61b3d5e8f3befa5c51161f1593b674"}
Jul 29 18:56:14.170345 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:14.170309    2557 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7488-140611831789872" oldPodUID="2bfe5bb0-8a48-4908-97c1-d4239686c367" podUID="7a84813e-48e1-4061-9cf6-a6a6917cf6ea"
Jul 29 18:56:14.654799 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:14.654766    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2bfe5bb0-8a48-4908-97c1-d4239686c367" path="/var/lib/kubelet/pods/2bfe5bb0-8a48-4908-97c1-d4239686c367/volumes"
Jul 29 18:56:15.166381 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:15.166342    2557 generic.go:356] "Generic (PLEG): container finished" podID="7a84813e-48e1-4061-9cf6-a6a6917cf6ea" containerID="8541312aaed3edd989c24f0c1f391267b07657301c53a55097fdee7c8ecf8277" exitCode=0
Jul 29 18:56:15.166593 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:15.166433    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7488-140611831789872" event={"ID":"7a84813e-48e1-4061-9cf6-a6a6917cf6ea","Type":"ContainerDied","Data":"8541312aaed3edd989c24f0c1f391267b07657301c53a55097fdee7c8ecf8277"}
Jul 29 18:56:16.311525 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:16.311494    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7488-140611831789872"
Jul 29 18:56:16.333201 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:16.333175    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7488-140611831789872_7a84813e-48e1-4061-9cf6-a6a6917cf6ea/test-curl-7488-140611831789872/0.log"
Jul 29 18:56:16.354446 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:16.354424    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7488-140611831789872"]
Jul 29 18:56:16.362183 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:16.361135    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7488-140611831789872"]
Jul 29 18:56:16.453865 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:16.453776    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ch7q4\" (UniqueName: \"kubernetes.io/projected/7a84813e-48e1-4061-9cf6-a6a6917cf6ea-kube-api-access-ch7q4\") pod \"7a84813e-48e1-4061-9cf6-a6a6917cf6ea\" (UID: \"7a84813e-48e1-4061-9cf6-a6a6917cf6ea\") "
Jul 29 18:56:16.456041 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:16.456015    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a84813e-48e1-4061-9cf6-a6a6917cf6ea-kube-api-access-ch7q4" (OuterVolumeSpecName: "kube-api-access-ch7q4") pod "7a84813e-48e1-4061-9cf6-a6a6917cf6ea" (UID: "7a84813e-48e1-4061-9cf6-a6a6917cf6ea"). InnerVolumeSpecName "kube-api-access-ch7q4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:56:16.473374 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:16.473351    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7488-140611831763856"]
Jul 29 18:56:16.473807 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:16.473793    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a84813e-48e1-4061-9cf6-a6a6917cf6ea" containerName="test-curl-7488-140611831789872"
Jul 29 18:56:16.473868 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:16.473808    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a84813e-48e1-4061-9cf6-a6a6917cf6ea" containerName="test-curl-7488-140611831789872"
Jul 29 18:56:16.473907 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:16.473878    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a84813e-48e1-4061-9cf6-a6a6917cf6ea" containerName="test-curl-7488-140611831789872"
Jul 29 18:56:16.476876 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:16.476862    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7488-140611831763856"
Jul 29 18:56:16.483575 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:16.483549    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7488-140611831763856"]
Jul 29 18:56:16.554484 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:16.554454    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqbg8\" (UniqueName: \"kubernetes.io/projected/cc8a926f-f37f-4934-979b-fdc61878fe47-kube-api-access-bqbg8\") pod \"test-curl-7488-140611831763856\" (UID: \"cc8a926f-f37f-4934-979b-fdc61878fe47\") " pod="opendatahub/test-curl-7488-140611831763856"
Jul 29 18:56:16.554628 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:16.554606    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ch7q4\" (UniqueName: \"kubernetes.io/projected/7a84813e-48e1-4061-9cf6-a6a6917cf6ea-kube-api-access-ch7q4\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:56:16.653540 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:16.653508    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a84813e-48e1-4061-9cf6-a6a6917cf6ea" path="/var/lib/kubelet/pods/7a84813e-48e1-4061-9cf6-a6a6917cf6ea/volumes"
Jul 29 18:56:16.655166 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:16.655148    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqbg8\" (UniqueName: \"kubernetes.io/projected/cc8a926f-f37f-4934-979b-fdc61878fe47-kube-api-access-bqbg8\") pod \"test-curl-7488-140611831763856\" (UID: \"cc8a926f-f37f-4934-979b-fdc61878fe47\") " pod="opendatahub/test-curl-7488-140611831763856"
Jul 29 18:56:16.663774 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:16.663752    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqbg8\" (UniqueName: \"kubernetes.io/projected/cc8a926f-f37f-4934-979b-fdc61878fe47-kube-api-access-bqbg8\") pod \"test-curl-7488-140611831763856\" (UID: \"cc8a926f-f37f-4934-979b-fdc61878fe47\") " pod="opendatahub/test-curl-7488-140611831763856"
Jul 29 18:56:16.787820 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:16.787798    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7488-140611831763856"
Jul 29 18:56:17.148228 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:17.148198    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7488-140611831763856"]
Jul 29 18:56:17.148306 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:56:17.148255    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcc8a926f_f37f_4934_979b_fdc61878fe47.slice/crio-0b9597478118aab6c6dda883dd8510dbe42d6d060ccc0809a98d05dfde111eff WatchSource:0}: Error finding container 0b9597478118aab6c6dda883dd8510dbe42d6d060ccc0809a98d05dfde111eff: Status 404 returned error can't find the container with id 0b9597478118aab6c6dda883dd8510dbe42d6d060ccc0809a98d05dfde111eff
Jul 29 18:56:17.179135 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:17.179113    2557 scope.go:117] "RemoveContainer" containerID="8541312aaed3edd989c24f0c1f391267b07657301c53a55097fdee7c8ecf8277"
Jul 29 18:56:17.179249 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:17.179141    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7488-140611831789872"
Jul 29 18:56:17.183412 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:17.181096    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7488-140611831763856" event={"ID":"cc8a926f-f37f-4934-979b-fdc61878fe47","Type":"ContainerStarted","Data":"0b9597478118aab6c6dda883dd8510dbe42d6d060ccc0809a98d05dfde111eff"}
Jul 29 18:56:18.189007 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:18.188974    2557 generic.go:356] "Generic (PLEG): container finished" podID="cc8a926f-f37f-4934-979b-fdc61878fe47" containerID="dd34f976b01c85be7ae47c0a407f0e9efbd76470b609e57621ae7c48cc0bdfc9" exitCode=0
Jul 29 18:56:18.189431 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:18.189063    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7488-140611831763856" event={"ID":"cc8a926f-f37f-4934-979b-fdc61878fe47","Type":"ContainerDied","Data":"dd34f976b01c85be7ae47c0a407f0e9efbd76470b609e57621ae7c48cc0bdfc9"}
Jul 29 18:56:19.335130 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:19.335110    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7488-140611831763856"
Jul 29 18:56:19.353657 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:19.353636    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7488-140611831763856_cc8a926f-f37f-4934-979b-fdc61878fe47/test-curl-7488-140611831763856/0.log"
Jul 29 18:56:19.376886 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:19.376859    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7488-140611831763856"]
Jul 29 18:56:19.381705 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:19.381678    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7488-140611831763856"]
Jul 29 18:56:19.481502 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:19.481438    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bqbg8\" (UniqueName: \"kubernetes.io/projected/cc8a926f-f37f-4934-979b-fdc61878fe47-kube-api-access-bqbg8\") pod \"cc8a926f-f37f-4934-979b-fdc61878fe47\" (UID: \"cc8a926f-f37f-4934-979b-fdc61878fe47\") "
Jul 29 18:56:19.483595 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:19.483568    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc8a926f-f37f-4934-979b-fdc61878fe47-kube-api-access-bqbg8" (OuterVolumeSpecName: "kube-api-access-bqbg8") pod "cc8a926f-f37f-4934-979b-fdc61878fe47" (UID: "cc8a926f-f37f-4934-979b-fdc61878fe47"). InnerVolumeSpecName "kube-api-access-bqbg8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:56:19.501160 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:19.501139    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7488-140611831763856"]
Jul 29 18:56:19.501547 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:19.501533    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc8a926f-f37f-4934-979b-fdc61878fe47" containerName="test-curl-7488-140611831763856"
Jul 29 18:56:19.501595 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:19.501552    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc8a926f-f37f-4934-979b-fdc61878fe47" containerName="test-curl-7488-140611831763856"
Jul 29 18:56:19.501650 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:19.501640    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc8a926f-f37f-4934-979b-fdc61878fe47" containerName="test-curl-7488-140611831763856"
Jul 29 18:56:19.504628 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:19.504611    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7488-140611831763856"
Jul 29 18:56:19.508537 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:19.508512    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7488-140611831763856"]
Jul 29 18:56:19.582902 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:19.582874    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8rsm\" (UniqueName: \"kubernetes.io/projected/365234a6-03d2-41df-b704-a0d33d53ec03-kube-api-access-j8rsm\") pod \"test-curl-7488-140611831763856\" (UID: \"365234a6-03d2-41df-b704-a0d33d53ec03\") " pod="opendatahub/test-curl-7488-140611831763856"
Jul 29 18:56:19.583040 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:19.582934    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bqbg8\" (UniqueName: \"kubernetes.io/projected/cc8a926f-f37f-4934-979b-fdc61878fe47-kube-api-access-bqbg8\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:56:19.683474 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:19.683435    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8rsm\" (UniqueName: \"kubernetes.io/projected/365234a6-03d2-41df-b704-a0d33d53ec03-kube-api-access-j8rsm\") pod \"test-curl-7488-140611831763856\" (UID: \"365234a6-03d2-41df-b704-a0d33d53ec03\") " pod="opendatahub/test-curl-7488-140611831763856"
Jul 29 18:56:19.691861 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:19.691835    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8rsm\" (UniqueName: \"kubernetes.io/projected/365234a6-03d2-41df-b704-a0d33d53ec03-kube-api-access-j8rsm\") pod \"test-curl-7488-140611831763856\" (UID: \"365234a6-03d2-41df-b704-a0d33d53ec03\") " pod="opendatahub/test-curl-7488-140611831763856"
Jul 29 18:56:19.815822 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:19.815796    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7488-140611831763856"
Jul 29 18:56:19.961279 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:19.961254    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7488-140611831763856"]
Jul 29 18:56:19.962759 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:56:19.962710    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod365234a6_03d2_41df_b704_a0d33d53ec03.slice/crio-214d1fc5dc9ca835f49cb460e914629bd380cbac633867200a76d4528d347ccc WatchSource:0}: Error finding container 214d1fc5dc9ca835f49cb460e914629bd380cbac633867200a76d4528d347ccc: Status 404 returned error can't find the container with id 214d1fc5dc9ca835f49cb460e914629bd380cbac633867200a76d4528d347ccc
Jul 29 18:56:20.202222 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:20.202140    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7488-140611831763856"
Jul 29 18:56:20.202222 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:20.202179    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0b9597478118aab6c6dda883dd8510dbe42d6d060ccc0809a98d05dfde111eff"
Jul 29 18:56:20.203397 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:20.203373    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7488-140611831763856" event={"ID":"365234a6-03d2-41df-b704-a0d33d53ec03","Type":"ContainerStarted","Data":"214d1fc5dc9ca835f49cb460e914629bd380cbac633867200a76d4528d347ccc"}
Jul 29 18:56:20.214305 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:20.214283    2557 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7488-140611831763856" oldPodUID="cc8a926f-f37f-4934-979b-fdc61878fe47" podUID="365234a6-03d2-41df-b704-a0d33d53ec03"
Jul 29 18:56:20.652502 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:20.652468    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc8a926f-f37f-4934-979b-fdc61878fe47" path="/var/lib/kubelet/pods/cc8a926f-f37f-4934-979b-fdc61878fe47/volumes"
Jul 29 18:56:21.210594 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:21.210560    2557 generic.go:356] "Generic (PLEG): container finished" podID="365234a6-03d2-41df-b704-a0d33d53ec03" containerID="1d422faf694676614a5c41e45dc0ef56a396050dc0ec9dca679a6e98955ec85e" exitCode=0
Jul 29 18:56:21.210781 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:21.210641    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7488-140611831763856" event={"ID":"365234a6-03d2-41df-b704-a0d33d53ec03","Type":"ContainerDied","Data":"1d422faf694676614a5c41e45dc0ef56a396050dc0ec9dca679a6e98955ec85e"}
Jul 29 18:56:22.358594 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:22.358570    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7488-140611831763856"
Jul 29 18:56:22.376356 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:22.376317    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7488-140611831763856_365234a6-03d2-41df-b704-a0d33d53ec03/test-curl-7488-140611831763856/0.log"
Jul 29 18:56:22.399419 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:22.399389    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7488-140611831763856"]
Jul 29 18:56:22.403659 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:22.403637    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7488-140611831763856"]
Jul 29 18:56:22.508953 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:22.508875    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j8rsm\" (UniqueName: \"kubernetes.io/projected/365234a6-03d2-41df-b704-a0d33d53ec03-kube-api-access-j8rsm\") pod \"365234a6-03d2-41df-b704-a0d33d53ec03\" (UID: \"365234a6-03d2-41df-b704-a0d33d53ec03\") "
Jul 29 18:56:22.511066 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:22.511045    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/365234a6-03d2-41df-b704-a0d33d53ec03-kube-api-access-j8rsm" (OuterVolumeSpecName: "kube-api-access-j8rsm") pod "365234a6-03d2-41df-b704-a0d33d53ec03" (UID: "365234a6-03d2-41df-b704-a0d33d53ec03"). InnerVolumeSpecName "kube-api-access-j8rsm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:56:22.610397 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:22.610364    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j8rsm\" (UniqueName: \"kubernetes.io/projected/365234a6-03d2-41df-b704-a0d33d53ec03-kube-api-access-j8rsm\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:56:22.652658 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:22.652622    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="365234a6-03d2-41df-b704-a0d33d53ec03" path="/var/lib/kubelet/pods/365234a6-03d2-41df-b704-a0d33d53ec03/volumes"
Jul 29 18:56:23.185864 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:23.185829    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw"]
Jul 29 18:56:23.186321 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:23.186301    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="365234a6-03d2-41df-b704-a0d33d53ec03" containerName="test-curl-7488-140611831763856"
Jul 29 18:56:23.186411 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:23.186324    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="365234a6-03d2-41df-b704-a0d33d53ec03" containerName="test-curl-7488-140611831763856"
Jul 29 18:56:23.186472 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:23.186449    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="365234a6-03d2-41df-b704-a0d33d53ec03" containerName="test-curl-7488-140611831763856"
Jul 29 18:56:23.189742 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:23.189708    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw"
Jul 29 18:56:23.192520 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:23.192500    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-3140cc-openshift-default-dockercfg-9mfrg\""
Jul 29 18:56:23.200300 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:23.200275    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw"]
Jul 29 18:56:23.228223 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:23.228198    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7488-140611831763856"
Jul 29 18:56:23.228223 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:23.228219    2557 scope.go:117] "RemoveContainer" containerID="1d422faf694676614a5c41e45dc0ef56a396050dc0ec9dca679a6e98955ec85e"
Jul 29 18:56:23.316127 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:23.316094    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9cf34b68-ba5b-49ad-bdc7-fb7bc133b863-workload-certs\") pod \"e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw\" (UID: \"9cf34b68-ba5b-49ad-bdc7-fb7bc133b863\") " pod="openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw"
Jul 29 18:56:23.316127 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:23.316140    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9cf34b68-ba5b-49ad-bdc7-fb7bc133b863-istiod-ca-cert\") pod \"e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw\" (UID: \"9cf34b68-ba5b-49ad-bdc7-fb7bc133b863\") " pod="openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw"
Jul 29 18:56:23.316376 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:23.316170    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9cf34b68-ba5b-49ad-bdc7-fb7bc133b863-istio-podinfo\") pod \"e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw\" (UID: \"9cf34b68-ba5b-49ad-bdc7-fb7bc133b863\") " pod="openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw"
Jul 29 18:56:23.316376 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:23.316242    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvzr7\" (UniqueName: \"kubernetes.io/projected/9cf34b68-ba5b-49ad-bdc7-fb7bc133b863-kube-api-access-qvzr7\") pod \"e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw\" (UID: \"9cf34b68-ba5b-49ad-bdc7-fb7bc133b863\") " pod="openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw"
Jul 29 18:56:23.316376 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:23.316276    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9cf34b68-ba5b-49ad-bdc7-fb7bc133b863-istio-envoy\") pod \"e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw\" (UID: \"9cf34b68-ba5b-49ad-bdc7-fb7bc133b863\") " pod="openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw"
Jul 29 18:56:23.316376 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:23.316299    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cf34b68-ba5b-49ad-bdc7-fb7bc133b863-credential-socket\") pod \"e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw\" (UID: \"9cf34b68-ba5b-49ad-bdc7-fb7bc133b863\") " pod="openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw"
Jul 29 18:56:23.316376 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:23.316313    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9cf34b68-ba5b-49ad-bdc7-fb7bc133b863-istio-token\") pod \"e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw\" (UID: \"9cf34b68-ba5b-49ad-bdc7-fb7bc133b863\") " pod="openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw"
Jul 29 18:56:23.316376 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:23.316331    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cf34b68-ba5b-49ad-bdc7-fb7bc133b863-workload-socket\") pod \"e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw\" (UID: \"9cf34b68-ba5b-49ad-bdc7-fb7bc133b863\") " pod="openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw"
Jul 29 18:56:23.316592 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:23.316402    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9cf34b68-ba5b-49ad-bdc7-fb7bc133b863-istio-data\") pod \"e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw\" (UID: \"9cf34b68-ba5b-49ad-bdc7-fb7bc133b863\") " pod="openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw"
Jul 29 18:56:23.417038 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:23.417008    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9cf34b68-ba5b-49ad-bdc7-fb7bc133b863-istio-data\") pod \"e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw\" (UID: \"9cf34b68-ba5b-49ad-bdc7-fb7bc133b863\") " pod="openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw"
Jul 29 18:56:23.417493 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:23.417046    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9cf34b68-ba5b-49ad-bdc7-fb7bc133b863-workload-certs\") pod \"e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw\" (UID: \"9cf34b68-ba5b-49ad-bdc7-fb7bc133b863\") " pod="openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw"
Jul 29 18:56:23.417493 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:23.417074    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9cf34b68-ba5b-49ad-bdc7-fb7bc133b863-istiod-ca-cert\") pod \"e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw\" (UID: \"9cf34b68-ba5b-49ad-bdc7-fb7bc133b863\") " pod="openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw"
Jul 29 18:56:23.417493 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:23.417192    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9cf34b68-ba5b-49ad-bdc7-fb7bc133b863-istio-podinfo\") pod \"e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw\" (UID: \"9cf34b68-ba5b-49ad-bdc7-fb7bc133b863\") " pod="openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw"
Jul 29 18:56:23.417493 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:23.417263    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvzr7\" (UniqueName: \"kubernetes.io/projected/9cf34b68-ba5b-49ad-bdc7-fb7bc133b863-kube-api-access-qvzr7\") pod \"e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw\" (UID: \"9cf34b68-ba5b-49ad-bdc7-fb7bc133b863\") " pod="openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw"
Jul 29 18:56:23.417493 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:23.417293    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9cf34b68-ba5b-49ad-bdc7-fb7bc133b863-istio-envoy\") pod \"e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw\" (UID: \"9cf34b68-ba5b-49ad-bdc7-fb7bc133b863\") " pod="openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw"
Jul 29 18:56:23.417493 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:23.417325    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cf34b68-ba5b-49ad-bdc7-fb7bc133b863-credential-socket\") pod \"e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw\" (UID: \"9cf34b68-ba5b-49ad-bdc7-fb7bc133b863\") " pod="openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw"
Jul 29 18:56:23.417493 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:23.417342    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9cf34b68-ba5b-49ad-bdc7-fb7bc133b863-istio-token\") pod \"e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw\" (UID: \"9cf34b68-ba5b-49ad-bdc7-fb7bc133b863\") " pod="openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw"
Jul 29 18:56:23.417493 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:23.417361    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cf34b68-ba5b-49ad-bdc7-fb7bc133b863-workload-socket\") pod \"e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw\" (UID: \"9cf34b68-ba5b-49ad-bdc7-fb7bc133b863\") " pod="openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw"
Jul 29 18:56:23.417493 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:23.417384    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9cf34b68-ba5b-49ad-bdc7-fb7bc133b863-istio-data\") pod \"e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw\" (UID: \"9cf34b68-ba5b-49ad-bdc7-fb7bc133b863\") " pod="openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw"
Jul 29 18:56:23.417947 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:23.417712    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cf34b68-ba5b-49ad-bdc7-fb7bc133b863-credential-socket\") pod \"e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw\" (UID: \"9cf34b68-ba5b-49ad-bdc7-fb7bc133b863\") " pod="openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw"
Jul 29 18:56:23.417947 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:23.417769    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9cf34b68-ba5b-49ad-bdc7-fb7bc133b863-workload-certs\") pod \"e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw\" (UID: \"9cf34b68-ba5b-49ad-bdc7-fb7bc133b863\") " pod="openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw"
Jul 29 18:56:23.417947 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:23.417808    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cf34b68-ba5b-49ad-bdc7-fb7bc133b863-workload-socket\") pod \"e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw\" (UID: \"9cf34b68-ba5b-49ad-bdc7-fb7bc133b863\") " pod="openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw"
Jul 29 18:56:23.417947 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:23.417855    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9cf34b68-ba5b-49ad-bdc7-fb7bc133b863-istiod-ca-cert\") pod \"e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw\" (UID: \"9cf34b68-ba5b-49ad-bdc7-fb7bc133b863\") " pod="openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw"
Jul 29 18:56:23.419739 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:23.419701    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9cf34b68-ba5b-49ad-bdc7-fb7bc133b863-istio-envoy\") pod \"e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw\" (UID: \"9cf34b68-ba5b-49ad-bdc7-fb7bc133b863\") " pod="openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw"
Jul 29 18:56:23.419858 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:23.419821    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9cf34b68-ba5b-49ad-bdc7-fb7bc133b863-istio-podinfo\") pod \"e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw\" (UID: \"9cf34b68-ba5b-49ad-bdc7-fb7bc133b863\") " pod="openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw"
Jul 29 18:56:23.425932 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:23.425907    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvzr7\" (UniqueName: \"kubernetes.io/projected/9cf34b68-ba5b-49ad-bdc7-fb7bc133b863-kube-api-access-qvzr7\") pod \"e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw\" (UID: \"9cf34b68-ba5b-49ad-bdc7-fb7bc133b863\") " pod="openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw"
Jul 29 18:56:23.426256 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:23.426240    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9cf34b68-ba5b-49ad-bdc7-fb7bc133b863-istio-token\") pod \"e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw\" (UID: \"9cf34b68-ba5b-49ad-bdc7-fb7bc133b863\") " pod="openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw"
Jul 29 18:56:23.501533 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:23.501468    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw"
Jul 29 18:56:23.648110 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:23.647484    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw"]
Jul 29 18:56:23.648800 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:56:23.648772    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cf34b68_ba5b_49ad_bdc7_fb7bc133b863.slice/crio-df7107124d5bcdadd7ca959b9c46d696ac2b27648bf59d3e9b1a01df2c53eea2 WatchSource:0}: Error finding container df7107124d5bcdadd7ca959b9c46d696ac2b27648bf59d3e9b1a01df2c53eea2: Status 404 returned error can't find the container with id df7107124d5bcdadd7ca959b9c46d696ac2b27648bf59d3e9b1a01df2c53eea2
Jul 29 18:56:23.651808 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:23.651711    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:56:23.651913 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:23.651845    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:56:23.651913 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:23.651883    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:56:24.236204 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:24.236161    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw" event={"ID":"9cf34b68-ba5b-49ad-bdc7-fb7bc133b863","Type":"ContainerStarted","Data":"cf0478470e1980752206530b4dd3ce6648ffacaf7266fd887f6de32ae872313b"}
Jul 29 18:56:24.236438 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:24.236232    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw" event={"ID":"9cf34b68-ba5b-49ad-bdc7-fb7bc133b863","Type":"ContainerStarted","Data":"df7107124d5bcdadd7ca959b9c46d696ac2b27648bf59d3e9b1a01df2c53eea2"}
Jul 29 18:56:24.257010 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:24.256959    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw" podStartSLOduration=1.256944543 podStartE2EDuration="1.256944543s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:56:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:56:24.255921185 +0000 UTC m=+3864.125143993" watchObservedRunningTime="2026-07-29 18:56:24.256944543 +0000 UTC m=+3864.126167350"
Jul 29 18:56:24.502084 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:24.501996    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw"
Jul 29 18:56:24.506876 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:24.506850    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw"
Jul 29 18:56:25.243573 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:25.243511    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw"
Jul 29 18:56:25.244388 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:25.244370    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw"
Jul 29 18:56:26.297592 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:26.297564    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-3140cc-6c4fc77759-6xgcn"]
Jul 29 18:56:26.301248 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:26.301232    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3140cc-6c4fc77759-6xgcn"
Jul 29 18:56:26.304503 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:26.304485    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-3140cc-serving-cert\""
Jul 29 18:56:26.312705 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:26.312677    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-3140cc-6c4fc77759-6xgcn"]
Jul 29 18:56:26.443888 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:26.443855    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d1b8d9b-4131-4713-b2e0-1c42247852d8-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-3140cc-6c4fc77759-6xgcn\" (UID: \"0d1b8d9b-4131-4713-b2e0-1c42247852d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3140cc-6c4fc77759-6xgcn"
Jul 29 18:56:26.444059 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:26.443901    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d1b8d9b-4131-4713-b2e0-1c42247852d8-maas-api-tls\") pod \"maas-api-e2e-ipp-a-3140cc-6c4fc77759-6xgcn\" (UID: \"0d1b8d9b-4131-4713-b2e0-1c42247852d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3140cc-6c4fc77759-6xgcn"
Jul 29 18:56:26.444059 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:26.443932    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dplhg\" (UniqueName: \"kubernetes.io/projected/0d1b8d9b-4131-4713-b2e0-1c42247852d8-kube-api-access-dplhg\") pod \"maas-api-e2e-ipp-a-3140cc-6c4fc77759-6xgcn\" (UID: \"0d1b8d9b-4131-4713-b2e0-1c42247852d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3140cc-6c4fc77759-6xgcn"
Jul 29 18:56:26.545282 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:26.545243    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d1b8d9b-4131-4713-b2e0-1c42247852d8-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-3140cc-6c4fc77759-6xgcn\" (UID: \"0d1b8d9b-4131-4713-b2e0-1c42247852d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3140cc-6c4fc77759-6xgcn"
Jul 29 18:56:26.545447 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:26.545317    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d1b8d9b-4131-4713-b2e0-1c42247852d8-maas-api-tls\") pod \"maas-api-e2e-ipp-a-3140cc-6c4fc77759-6xgcn\" (UID: \"0d1b8d9b-4131-4713-b2e0-1c42247852d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3140cc-6c4fc77759-6xgcn"
Jul 29 18:56:26.545447 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:26.545364    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dplhg\" (UniqueName: \"kubernetes.io/projected/0d1b8d9b-4131-4713-b2e0-1c42247852d8-kube-api-access-dplhg\") pod \"maas-api-e2e-ipp-a-3140cc-6c4fc77759-6xgcn\" (UID: \"0d1b8d9b-4131-4713-b2e0-1c42247852d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3140cc-6c4fc77759-6xgcn"
Jul 29 18:56:26.546000 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:26.545979    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d1b8d9b-4131-4713-b2e0-1c42247852d8-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-3140cc-6c4fc77759-6xgcn\" (UID: \"0d1b8d9b-4131-4713-b2e0-1c42247852d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3140cc-6c4fc77759-6xgcn"
Jul 29 18:56:26.548072 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:26.548022    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d1b8d9b-4131-4713-b2e0-1c42247852d8-maas-api-tls\") pod \"maas-api-e2e-ipp-a-3140cc-6c4fc77759-6xgcn\" (UID: \"0d1b8d9b-4131-4713-b2e0-1c42247852d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3140cc-6c4fc77759-6xgcn"
Jul 29 18:56:26.553888 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:26.553867    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dplhg\" (UniqueName: \"kubernetes.io/projected/0d1b8d9b-4131-4713-b2e0-1c42247852d8-kube-api-access-dplhg\") pod \"maas-api-e2e-ipp-a-3140cc-6c4fc77759-6xgcn\" (UID: \"0d1b8d9b-4131-4713-b2e0-1c42247852d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3140cc-6c4fc77759-6xgcn"
Jul 29 18:56:26.612679 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:26.612653    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3140cc-6c4fc77759-6xgcn"
Jul 29 18:56:26.757430 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:26.757406    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-3140cc-6c4fc77759-6xgcn"]
Jul 29 18:56:26.759460 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:56:26.759418    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d1b8d9b_4131_4713_b2e0_1c42247852d8.slice/crio-567f1879af06b0512680e0765a8dc3b41def0622805e4b3b6bc4cad85cbde60a WatchSource:0}: Error finding container 567f1879af06b0512680e0765a8dc3b41def0622805e4b3b6bc4cad85cbde60a: Status 404 returned error can't find the container with id 567f1879af06b0512680e0765a8dc3b41def0622805e4b3b6bc4cad85cbde60a
Jul 29 18:56:27.257579 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:27.257542    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3140cc-6c4fc77759-6xgcn" event={"ID":"0d1b8d9b-4131-4713-b2e0-1c42247852d8","Type":"ContainerStarted","Data":"567f1879af06b0512680e0765a8dc3b41def0622805e4b3b6bc4cad85cbde60a"}
Jul 29 18:56:28.265582 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:28.265546    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3140cc-6c4fc77759-6xgcn" event={"ID":"0d1b8d9b-4131-4713-b2e0-1c42247852d8","Type":"ContainerStarted","Data":"4160da590409dcba1a0506fe5eefbb469bc747c91c30ee2d90c3c3db4b19a06d"}
Jul 29 18:56:28.265957 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:28.265629    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3140cc-6c4fc77759-6xgcn"
Jul 29 18:56:28.282881 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:28.282832    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3140cc-6c4fc77759-6xgcn" podStartSLOduration=1.7520301 podStartE2EDuration="2.282817699s" totalImagesPullingTime="530.787599ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:56:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:56:28.281718381 +0000 UTC m=+3868.150941192" watchObservedRunningTime="2026-07-29 18:56:28.282817699 +0000 UTC m=+3868.152040505"
Jul 29 18:56:34.278183 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:34.278149    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3140cc-6c4fc77759-6xgcn"
Jul 29 18:56:41.285886 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:41.285844    2557 patch_prober.go:28] interesting pod/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 29 18:56:41.286365 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:41.285917    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw" podUID="9cf34b68-ba5b-49ad-bdc7-fb7bc133b863" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 29 18:56:43.361646 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:43.361609    2557 generic.go:356] "Generic (PLEG): container finished" podID="9cf34b68-ba5b-49ad-bdc7-fb7bc133b863" containerID="cf0478470e1980752206530b4dd3ce6648ffacaf7266fd887f6de32ae872313b" exitCode=0
Jul 29 18:56:43.362251 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:43.361685    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw" event={"ID":"9cf34b68-ba5b-49ad-bdc7-fb7bc133b863","Type":"ContainerDied","Data":"cf0478470e1980752206530b4dd3ce6648ffacaf7266fd887f6de32ae872313b"}
Jul 29 18:56:43.362251 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:43.362125    2557 scope.go:117] "RemoveContainer" containerID="cf0478470e1980752206530b4dd3ce6648ffacaf7266fd887f6de32ae872313b"
Jul 29 18:56:43.363330 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:43.363273    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:56:43.363403 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:43.363371    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:56:43.363481 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:43.363401    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:56:44.370945 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:44.370909    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw" event={"ID":"9cf34b68-ba5b-49ad-bdc7-fb7bc133b863","Type":"ContainerStarted","Data":"1b861e342c68da278abcacd6a45d443629eea5ffadc8f509154e91f632500e76"}
Jul 29 18:56:44.502145 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:44.502112    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw"
Jul 29 18:56:44.502313 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:44.502264    2557 patch_prober.go:28] interesting pod/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Jul 29 18:56:44.502370 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:44.502322    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw" podUID="9cf34b68-ba5b-49ad-bdc7-fb7bc133b863" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Jul 29 18:56:45.502790 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:45.502756    2557 patch_prober.go:28] interesting pod/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Jul 29 18:56:45.503166 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:45.502824    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw" podUID="9cf34b68-ba5b-49ad-bdc7-fb7bc133b863" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Jul 29 18:56:46.502550 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:46.502513    2557 patch_prober.go:28] interesting pod/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Jul 29 18:56:46.502744 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:46.502587    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw" podUID="9cf34b68-ba5b-49ad-bdc7-fb7bc133b863" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Jul 29 18:56:47.502461 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:47.502427    2557 patch_prober.go:28] interesting pod/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Jul 29 18:56:47.502871 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:47.502482    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw" podUID="9cf34b68-ba5b-49ad-bdc7-fb7bc133b863" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Jul 29 18:56:48.505500 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:48.505469    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw"
Jul 29 18:56:48.505907 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:48.505891    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw"
Jul 29 18:56:48.506804 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:48.506784    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw"
Jul 29 18:56:55.613465 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:55.613430    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-a10ee3-cb74b475f-qmbpd"]
Jul 29 18:56:55.618127 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:55.618104    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a10ee3-cb74b475f-qmbpd"
Jul 29 18:56:55.620916 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:55.620893    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-a10ee3-serving-cert\""
Jul 29 18:56:55.626592 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:55.626571    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-a10ee3-cb74b475f-qmbpd"]
Jul 29 18:56:55.733675 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:55.733615    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/265ebc75-4fd9-4f83-975f-24d18979fa82-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-a10ee3-cb74b475f-qmbpd\" (UID: \"265ebc75-4fd9-4f83-975f-24d18979fa82\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a10ee3-cb74b475f-qmbpd"
Jul 29 18:56:55.733891 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:55.733682    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5n7l\" (UniqueName: \"kubernetes.io/projected/265ebc75-4fd9-4f83-975f-24d18979fa82-kube-api-access-n5n7l\") pod \"maas-api-e2e-ipp-b-a10ee3-cb74b475f-qmbpd\" (UID: \"265ebc75-4fd9-4f83-975f-24d18979fa82\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a10ee3-cb74b475f-qmbpd"
Jul 29 18:56:55.733891 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:55.733808    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/265ebc75-4fd9-4f83-975f-24d18979fa82-maas-api-tls\") pod \"maas-api-e2e-ipp-b-a10ee3-cb74b475f-qmbpd\" (UID: \"265ebc75-4fd9-4f83-975f-24d18979fa82\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a10ee3-cb74b475f-qmbpd"
Jul 29 18:56:55.835411 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:55.835378    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/265ebc75-4fd9-4f83-975f-24d18979fa82-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-a10ee3-cb74b475f-qmbpd\" (UID: \"265ebc75-4fd9-4f83-975f-24d18979fa82\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a10ee3-cb74b475f-qmbpd"
Jul 29 18:56:55.835411 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:55.835424    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5n7l\" (UniqueName: \"kubernetes.io/projected/265ebc75-4fd9-4f83-975f-24d18979fa82-kube-api-access-n5n7l\") pod \"maas-api-e2e-ipp-b-a10ee3-cb74b475f-qmbpd\" (UID: \"265ebc75-4fd9-4f83-975f-24d18979fa82\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a10ee3-cb74b475f-qmbpd"
Jul 29 18:56:55.835638 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:55.835466    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/265ebc75-4fd9-4f83-975f-24d18979fa82-maas-api-tls\") pod \"maas-api-e2e-ipp-b-a10ee3-cb74b475f-qmbpd\" (UID: \"265ebc75-4fd9-4f83-975f-24d18979fa82\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a10ee3-cb74b475f-qmbpd"
Jul 29 18:56:55.835638 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:56:55.835606    2557 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-a10ee3-serving-cert: secret "maas-api-e2e-ipp-b-a10ee3-serving-cert" not found
Jul 29 18:56:55.835743 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:56:55.835666    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/265ebc75-4fd9-4f83-975f-24d18979fa82-maas-api-tls podName:265ebc75-4fd9-4f83-975f-24d18979fa82 nodeName:}" failed. No retries permitted until 2026-07-29 18:56:56.335646375 +0000 UTC m=+3896.204869163 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/265ebc75-4fd9-4f83-975f-24d18979fa82-maas-api-tls") pod "maas-api-e2e-ipp-b-a10ee3-cb74b475f-qmbpd" (UID: "265ebc75-4fd9-4f83-975f-24d18979fa82") : secret "maas-api-e2e-ipp-b-a10ee3-serving-cert" not found
Jul 29 18:56:55.836174 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:55.836153    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/265ebc75-4fd9-4f83-975f-24d18979fa82-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-a10ee3-cb74b475f-qmbpd\" (UID: \"265ebc75-4fd9-4f83-975f-24d18979fa82\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a10ee3-cb74b475f-qmbpd"
Jul 29 18:56:55.848365 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:55.848338    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5n7l\" (UniqueName: \"kubernetes.io/projected/265ebc75-4fd9-4f83-975f-24d18979fa82-kube-api-access-n5n7l\") pod \"maas-api-e2e-ipp-b-a10ee3-cb74b475f-qmbpd\" (UID: \"265ebc75-4fd9-4f83-975f-24d18979fa82\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a10ee3-cb74b475f-qmbpd"
Jul 29 18:56:56.341179 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:56.341137    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/265ebc75-4fd9-4f83-975f-24d18979fa82-maas-api-tls\") pod \"maas-api-e2e-ipp-b-a10ee3-cb74b475f-qmbpd\" (UID: \"265ebc75-4fd9-4f83-975f-24d18979fa82\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a10ee3-cb74b475f-qmbpd"
Jul 29 18:56:56.343937 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:56.343906    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/265ebc75-4fd9-4f83-975f-24d18979fa82-maas-api-tls\") pod \"maas-api-e2e-ipp-b-a10ee3-cb74b475f-qmbpd\" (UID: \"265ebc75-4fd9-4f83-975f-24d18979fa82\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a10ee3-cb74b475f-qmbpd"
Jul 29 18:56:56.531629 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:56.531577    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a10ee3-cb74b475f-qmbpd"
Jul 29 18:56:56.674529 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:56.674498    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-a10ee3-cb74b475f-qmbpd"]
Jul 29 18:56:56.676084 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:56:56.676060    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod265ebc75_4fd9_4f83_975f_24d18979fa82.slice/crio-312523fcaa5aee87628cf42cd15c91625d39bc96828ea769af5f24bf093e10dc WatchSource:0}: Error finding container 312523fcaa5aee87628cf42cd15c91625d39bc96828ea769af5f24bf093e10dc: Status 404 returned error can't find the container with id 312523fcaa5aee87628cf42cd15c91625d39bc96828ea769af5f24bf093e10dc
Jul 29 18:56:57.467572 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:57.467535    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a10ee3-cb74b475f-qmbpd" event={"ID":"265ebc75-4fd9-4f83-975f-24d18979fa82","Type":"ContainerStarted","Data":"78b803dbf1430b540bf3f7478ac8d9c099cd603146c471a5a27cd5845040e162"}
Jul 29 18:56:57.467776 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:57.467595    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a10ee3-cb74b475f-qmbpd" event={"ID":"265ebc75-4fd9-4f83-975f-24d18979fa82","Type":"ContainerStarted","Data":"312523fcaa5aee87628cf42cd15c91625d39bc96828ea769af5f24bf093e10dc"}
Jul 29 18:56:57.467776 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:57.467616    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a10ee3-cb74b475f-qmbpd"
Jul 29 18:56:57.485744 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:56:57.485677    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a10ee3-cb74b475f-qmbpd" podStartSLOduration=2.127593058 podStartE2EDuration="2.485661719s" totalImagesPullingTime="358.068661ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:56:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:56:57.483257207 +0000 UTC m=+3897.352480017" watchObservedRunningTime="2026-07-29 18:56:57.485661719 +0000 UTC m=+3897.354884525"
Jul 29 18:57:01.043303 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:57:01.043271    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gg2wj_2bf0bb71-4047-45a7-bd8e-5d6efabd26bd/console-operator/2.log"
Jul 29 18:57:01.061815 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:57:01.061791    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gg2wj_2bf0bb71-4047-45a7-bd8e-5d6efabd26bd/console-operator/2.log"
Jul 29 18:57:03.479809 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:57:03.479776    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a10ee3-cb74b475f-qmbpd"
Jul 29 18:57:27.376400 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:57:27.376363    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-p6kkw_0d65e55f-0460-4fd4-b46c-6d0bb6724295/payload-processing/0.log"
Jul 29 18:58:12.263119 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:12.263083    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-p6kkw_0d65e55f-0460-4fd4-b46c-6d0bb6724295/payload-processing/0.log"
Jul 29 18:58:16.044386 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:16.044350    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8c76b3-5dff5b8c9f-hm5w7"]
Jul 29 18:58:16.048292 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:16.048272    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8c76b3-5dff5b8c9f-hm5w7"
Jul 29 18:58:16.050897 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:16.050876    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-8c76b3-serving-cert\""
Jul 29 18:58:16.060079 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:16.060048    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8c76b3-5dff5b8c9f-hm5w7"]
Jul 29 18:58:16.123510 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:16.123471    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e28927b8-b869-482c-81a0-013b49e9cfd7-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-8c76b3-5dff5b8c9f-hm5w7\" (UID: \"e28927b8-b869-482c-81a0-013b49e9cfd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8c76b3-5dff5b8c9f-hm5w7"
Jul 29 18:58:16.123679 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:16.123519    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e28927b8-b869-482c-81a0-013b49e9cfd7-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-8c76b3-5dff5b8c9f-hm5w7\" (UID: \"e28927b8-b869-482c-81a0-013b49e9cfd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8c76b3-5dff5b8c9f-hm5w7"
Jul 29 18:58:16.123679 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:16.123573    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fptj5\" (UniqueName: \"kubernetes.io/projected/e28927b8-b869-482c-81a0-013b49e9cfd7-kube-api-access-fptj5\") pod \"maas-api-e2e-ipp-cleanup-8c76b3-5dff5b8c9f-hm5w7\" (UID: \"e28927b8-b869-482c-81a0-013b49e9cfd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8c76b3-5dff5b8c9f-hm5w7"
Jul 29 18:58:16.224964 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:16.224919    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e28927b8-b869-482c-81a0-013b49e9cfd7-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-8c76b3-5dff5b8c9f-hm5w7\" (UID: \"e28927b8-b869-482c-81a0-013b49e9cfd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8c76b3-5dff5b8c9f-hm5w7"
Jul 29 18:58:16.225174 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:16.225014    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e28927b8-b869-482c-81a0-013b49e9cfd7-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-8c76b3-5dff5b8c9f-hm5w7\" (UID: \"e28927b8-b869-482c-81a0-013b49e9cfd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8c76b3-5dff5b8c9f-hm5w7"
Jul 29 18:58:16.225174 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:16.225057    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fptj5\" (UniqueName: \"kubernetes.io/projected/e28927b8-b869-482c-81a0-013b49e9cfd7-kube-api-access-fptj5\") pod \"maas-api-e2e-ipp-cleanup-8c76b3-5dff5b8c9f-hm5w7\" (UID: \"e28927b8-b869-482c-81a0-013b49e9cfd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8c76b3-5dff5b8c9f-hm5w7"
Jul 29 18:58:16.225674 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:16.225646    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e28927b8-b869-482c-81a0-013b49e9cfd7-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-8c76b3-5dff5b8c9f-hm5w7\" (UID: \"e28927b8-b869-482c-81a0-013b49e9cfd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8c76b3-5dff5b8c9f-hm5w7"
Jul 29 18:58:16.227744 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:16.227702    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e28927b8-b869-482c-81a0-013b49e9cfd7-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-8c76b3-5dff5b8c9f-hm5w7\" (UID: \"e28927b8-b869-482c-81a0-013b49e9cfd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8c76b3-5dff5b8c9f-hm5w7"
Jul 29 18:58:16.237687 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:16.237635    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fptj5\" (UniqueName: \"kubernetes.io/projected/e28927b8-b869-482c-81a0-013b49e9cfd7-kube-api-access-fptj5\") pod \"maas-api-e2e-ipp-cleanup-8c76b3-5dff5b8c9f-hm5w7\" (UID: \"e28927b8-b869-482c-81a0-013b49e9cfd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8c76b3-5dff5b8c9f-hm5w7"
Jul 29 18:58:16.360652 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:16.360568    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8c76b3-5dff5b8c9f-hm5w7"
Jul 29 18:58:16.720127 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:16.720094    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8c76b3-5dff5b8c9f-hm5w7"]
Jul 29 18:58:16.722276 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:58:16.722238    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode28927b8_b869_482c_81a0_013b49e9cfd7.slice/crio-1754e2a270faa6514100306af7a8c448bee53e0081c8ab9f2ad5d3aff4a5caa0 WatchSource:0}: Error finding container 1754e2a270faa6514100306af7a8c448bee53e0081c8ab9f2ad5d3aff4a5caa0: Status 404 returned error can't find the container with id 1754e2a270faa6514100306af7a8c448bee53e0081c8ab9f2ad5d3aff4a5caa0
Jul 29 18:58:16.939009 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:16.938974    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8c76b3-5dff5b8c9f-hm5w7" event={"ID":"e28927b8-b869-482c-81a0-013b49e9cfd7","Type":"ContainerStarted","Data":"1754e2a270faa6514100306af7a8c448bee53e0081c8ab9f2ad5d3aff4a5caa0"}
Jul 29 18:58:17.946844 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:17.946809    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8c76b3-5dff5b8c9f-hm5w7" event={"ID":"e28927b8-b869-482c-81a0-013b49e9cfd7","Type":"ContainerStarted","Data":"e3340d86a9cf444cb25fce4b2bdd4021bddcf76ca987a88382fab3793ceb4ba6"}
Jul 29 18:58:17.947312 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:17.946918    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8c76b3-5dff5b8c9f-hm5w7"
Jul 29 18:58:17.967022 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:17.966979    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8c76b3-5dff5b8c9f-hm5w7" podStartSLOduration=1.544308339 podStartE2EDuration="1.966937922s" totalImagesPullingTime="422.629583ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:58:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:58:17.963331355 +0000 UTC m=+3977.832554164" watchObservedRunningTime="2026-07-29 18:58:17.966937922 +0000 UTC m=+3977.836160731"
Jul 29 18:58:23.958861 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:23.958829    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8c76b3-5dff5b8c9f-hm5w7"
Jul 29 18:58:41.628919 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:41.628887    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8c76b3-9mdpg"]
Jul 29 18:58:41.632830 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:41.632804    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8c76b3-9mdpg"
Jul 29 18:58:41.635557 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:41.635536    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qrbhl\""
Jul 29 18:58:41.642639 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:41.642612    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8c76b3-9mdpg"]
Jul 29 18:58:41.771462 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:41.771432    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf9f99d6-9ee3-4f14-8deb-113f0692d6fc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-8c76b3-9mdpg\" (UID: \"bf9f99d6-9ee3-4f14-8deb-113f0692d6fc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8c76b3-9mdpg"
Jul 29 18:58:41.872769 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:41.872736    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf9f99d6-9ee3-4f14-8deb-113f0692d6fc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-8c76b3-9mdpg\" (UID: \"bf9f99d6-9ee3-4f14-8deb-113f0692d6fc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8c76b3-9mdpg"
Jul 29 18:58:41.873318 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:41.873302    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf9f99d6-9ee3-4f14-8deb-113f0692d6fc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-8c76b3-9mdpg\" (UID: \"bf9f99d6-9ee3-4f14-8deb-113f0692d6fc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8c76b3-9mdpg"
Jul 29 18:58:41.946565 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:41.946482    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8c76b3-9mdpg"
Jul 29 18:58:42.096753 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:42.096699    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8c76b3-9mdpg"]
Jul 29 18:58:42.097224 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:58:42.097190    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf9f99d6_9ee3_4f14_8deb_113f0692d6fc.slice/crio-67336c14ba337aaba2843d663296c59949a3df9928bfa55682c929a6695e95b4 WatchSource:0}: Error finding container 67336c14ba337aaba2843d663296c59949a3df9928bfa55682c929a6695e95b4: Status 404 returned error can't find the container with id 67336c14ba337aaba2843d663296c59949a3df9928bfa55682c929a6695e95b4
Jul 29 18:58:43.108328 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:43.108289    2557 generic.go:356] "Generic (PLEG): container finished" podID="bf9f99d6-9ee3-4f14-8deb-113f0692d6fc" containerID="0adbb840d6be9e38c1692a8557c0be7aea6662168f297ce3d769e32491977049" exitCode=0
Jul 29 18:58:43.108776 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:43.108338    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8c76b3-9mdpg" event={"ID":"bf9f99d6-9ee3-4f14-8deb-113f0692d6fc","Type":"ContainerDied","Data":"0adbb840d6be9e38c1692a8557c0be7aea6662168f297ce3d769e32491977049"}
Jul 29 18:58:43.108776 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:43.108397    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8c76b3-9mdpg" event={"ID":"bf9f99d6-9ee3-4f14-8deb-113f0692d6fc","Type":"ContainerStarted","Data":"67336c14ba337aaba2843d663296c59949a3df9928bfa55682c929a6695e95b4"}
Jul 29 18:58:44.256494 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:44.256464    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8c76b3-9mdpg"
Jul 29 18:58:44.396241 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:44.396142    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf9f99d6-9ee3-4f14-8deb-113f0692d6fc-maas-api-service-ca\") pod \"bf9f99d6-9ee3-4f14-8deb-113f0692d6fc\" (UID: \"bf9f99d6-9ee3-4f14-8deb-113f0692d6fc\") "
Jul 29 18:58:44.396546 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:44.396521    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf9f99d6-9ee3-4f14-8deb-113f0692d6fc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bf9f99d6-9ee3-4f14-8deb-113f0692d6fc" (UID: "bf9f99d6-9ee3-4f14-8deb-113f0692d6fc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:58:44.497592 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:44.497564    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf9f99d6-9ee3-4f14-8deb-113f0692d6fc-maas-api-service-ca\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:58:45.123382 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:45.123352    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8c76b3-9mdpg"
Jul 29 18:58:45.123572 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:45.123346    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8c76b3-9mdpg" event={"ID":"bf9f99d6-9ee3-4f14-8deb-113f0692d6fc","Type":"ContainerDied","Data":"67336c14ba337aaba2843d663296c59949a3df9928bfa55682c929a6695e95b4"}
Jul 29 18:58:45.123572 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:45.123475    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="67336c14ba337aaba2843d663296c59949a3df9928bfa55682c929a6695e95b4"
Jul 29 18:58:46.592335 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:46.592304    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8c76b3-5dff5b8c9f-hm5w7"]
Jul 29 18:58:46.592818 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:46.592522    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8c76b3-5dff5b8c9f-hm5w7" podUID="e28927b8-b869-482c-81a0-013b49e9cfd7" containerName="maas-api" containerID="cri-o://e3340d86a9cf444cb25fce4b2bdd4021bddcf76ca987a88382fab3793ceb4ba6" gracePeriod=30
Jul 29 18:58:46.763913 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:46.763842    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8c76b3-9mdpg"]
Jul 29 18:58:46.768587 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:46.768560    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8c76b3-9mdpg"]
Jul 29 18:58:46.864110 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:46.864087    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8c76b3-5dff5b8c9f-hm5w7"
Jul 29 18:58:46.920577 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:46.920549    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fptj5\" (UniqueName: \"kubernetes.io/projected/e28927b8-b869-482c-81a0-013b49e9cfd7-kube-api-access-fptj5\") pod \"e28927b8-b869-482c-81a0-013b49e9cfd7\" (UID: \"e28927b8-b869-482c-81a0-013b49e9cfd7\") "
Jul 29 18:58:46.920761 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:46.920613    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e28927b8-b869-482c-81a0-013b49e9cfd7-maas-api-tls\") pod \"e28927b8-b869-482c-81a0-013b49e9cfd7\" (UID: \"e28927b8-b869-482c-81a0-013b49e9cfd7\") "
Jul 29 18:58:46.920761 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:46.920657    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e28927b8-b869-482c-81a0-013b49e9cfd7-openshift-service-ca\") pod \"e28927b8-b869-482c-81a0-013b49e9cfd7\" (UID: \"e28927b8-b869-482c-81a0-013b49e9cfd7\") "
Jul 29 18:58:46.921068 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:46.921044    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e28927b8-b869-482c-81a0-013b49e9cfd7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e28927b8-b869-482c-81a0-013b49e9cfd7" (UID: "e28927b8-b869-482c-81a0-013b49e9cfd7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:58:46.922912 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:46.922881    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e28927b8-b869-482c-81a0-013b49e9cfd7-kube-api-access-fptj5" (OuterVolumeSpecName: "kube-api-access-fptj5") pod "e28927b8-b869-482c-81a0-013b49e9cfd7" (UID: "e28927b8-b869-482c-81a0-013b49e9cfd7"). InnerVolumeSpecName "kube-api-access-fptj5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:58:46.922912 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:46.922879    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e28927b8-b869-482c-81a0-013b49e9cfd7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e28927b8-b869-482c-81a0-013b49e9cfd7" (UID: "e28927b8-b869-482c-81a0-013b49e9cfd7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:58:47.022338 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:47.022307    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fptj5\" (UniqueName: \"kubernetes.io/projected/e28927b8-b869-482c-81a0-013b49e9cfd7-kube-api-access-fptj5\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:58:47.022338 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:47.022332    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e28927b8-b869-482c-81a0-013b49e9cfd7-maas-api-tls\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:58:47.022338 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:47.022341    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e28927b8-b869-482c-81a0-013b49e9cfd7-openshift-service-ca\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:58:47.136902 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:47.136803    2557 generic.go:356] "Generic (PLEG): container finished" podID="e28927b8-b869-482c-81a0-013b49e9cfd7" containerID="e3340d86a9cf444cb25fce4b2bdd4021bddcf76ca987a88382fab3793ceb4ba6" exitCode=0
Jul 29 18:58:47.136902 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:47.136869    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8c76b3-5dff5b8c9f-hm5w7"
Jul 29 18:58:47.136902 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:47.136886    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8c76b3-5dff5b8c9f-hm5w7" event={"ID":"e28927b8-b869-482c-81a0-013b49e9cfd7","Type":"ContainerDied","Data":"e3340d86a9cf444cb25fce4b2bdd4021bddcf76ca987a88382fab3793ceb4ba6"}
Jul 29 18:58:47.137192 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:47.136969    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8c76b3-5dff5b8c9f-hm5w7" event={"ID":"e28927b8-b869-482c-81a0-013b49e9cfd7","Type":"ContainerDied","Data":"1754e2a270faa6514100306af7a8c448bee53e0081c8ab9f2ad5d3aff4a5caa0"}
Jul 29 18:58:47.137192 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:47.136995    2557 scope.go:117] "RemoveContainer" containerID="e3340d86a9cf444cb25fce4b2bdd4021bddcf76ca987a88382fab3793ceb4ba6"
Jul 29 18:58:47.150068 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:47.149921    2557 scope.go:117] "RemoveContainer" containerID="e3340d86a9cf444cb25fce4b2bdd4021bddcf76ca987a88382fab3793ceb4ba6"
Jul 29 18:58:47.150904 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:58:47.150878    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3340d86a9cf444cb25fce4b2bdd4021bddcf76ca987a88382fab3793ceb4ba6\": container with ID starting with e3340d86a9cf444cb25fce4b2bdd4021bddcf76ca987a88382fab3793ceb4ba6 not found: ID does not exist" containerID="e3340d86a9cf444cb25fce4b2bdd4021bddcf76ca987a88382fab3793ceb4ba6"
Jul 29 18:58:47.151021 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:47.150911    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3340d86a9cf444cb25fce4b2bdd4021bddcf76ca987a88382fab3793ceb4ba6"} err="failed to get container status \"e3340d86a9cf444cb25fce4b2bdd4021bddcf76ca987a88382fab3793ceb4ba6\": rpc error: code = NotFound desc = could not find container \"e3340d86a9cf444cb25fce4b2bdd4021bddcf76ca987a88382fab3793ceb4ba6\": container with ID starting with e3340d86a9cf444cb25fce4b2bdd4021bddcf76ca987a88382fab3793ceb4ba6 not found: ID does not exist"
Jul 29 18:58:47.161774 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:47.161748    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8c76b3-5dff5b8c9f-hm5w7"]
Jul 29 18:58:47.165817 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:47.165780    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8c76b3-5dff5b8c9f-hm5w7"]
Jul 29 18:58:48.653654 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:48.653624    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf9f99d6-9ee3-4f14-8deb-113f0692d6fc" path="/var/lib/kubelet/pods/bf9f99d6-9ee3-4f14-8deb-113f0692d6fc/volumes"
Jul 29 18:58:48.654101 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:48.654084    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e28927b8-b869-482c-81a0-013b49e9cfd7" path="/var/lib/kubelet/pods/e28927b8-b869-482c-81a0-013b49e9cfd7/volumes"
Jul 29 18:58:56.664850 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:56.664815    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3140cc-qt62n"]
Jul 29 18:58:56.665296 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:56.665210    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e28927b8-b869-482c-81a0-013b49e9cfd7" containerName="maas-api"
Jul 29 18:58:56.665296 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:56.665221    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="e28927b8-b869-482c-81a0-013b49e9cfd7" containerName="maas-api"
Jul 29 18:58:56.665296 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:56.665228    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf9f99d6-9ee3-4f14-8deb-113f0692d6fc" containerName="revoke-keys"
Jul 29 18:58:56.665296 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:56.665235    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf9f99d6-9ee3-4f14-8deb-113f0692d6fc" containerName="revoke-keys"
Jul 29 18:58:56.665436 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:56.665358    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="e28927b8-b869-482c-81a0-013b49e9cfd7" containerName="maas-api"
Jul 29 18:58:56.665436 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:56.665367    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf9f99d6-9ee3-4f14-8deb-113f0692d6fc" containerName="revoke-keys"
Jul 29 18:58:56.669820 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:56.669802    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3140cc-qt62n"
Jul 29 18:58:56.672329 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:56.672301    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qrbhl\""
Jul 29 18:58:56.675609 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:56.675581    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3140cc-qt62n"]
Jul 29 18:58:56.824004 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:56.823974    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ceb2c7be-5b2f-440d-824a-d6efdfe297fb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-3140cc-qt62n\" (UID: \"ceb2c7be-5b2f-440d-824a-d6efdfe297fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3140cc-qt62n"
Jul 29 18:58:56.925265 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:56.925177    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ceb2c7be-5b2f-440d-824a-d6efdfe297fb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-3140cc-qt62n\" (UID: \"ceb2c7be-5b2f-440d-824a-d6efdfe297fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3140cc-qt62n"
Jul 29 18:58:56.925786 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:56.925765    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ceb2c7be-5b2f-440d-824a-d6efdfe297fb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-3140cc-qt62n\" (UID: \"ceb2c7be-5b2f-440d-824a-d6efdfe297fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3140cc-qt62n"
Jul 29 18:58:56.981807 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:56.981781    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3140cc-qt62n"
Jul 29 18:58:57.121455 ip-10-0-142-156 kubenswrapper[2557]: W0729 18:58:57.121420    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podceb2c7be_5b2f_440d_824a_d6efdfe297fb.slice/crio-5a4f1533bf39079712bdb941db5a030d9fdda88be0c4b3dc3f3b9a54425047ee WatchSource:0}: Error finding container 5a4f1533bf39079712bdb941db5a030d9fdda88be0c4b3dc3f3b9a54425047ee: Status 404 returned error can't find the container with id 5a4f1533bf39079712bdb941db5a030d9fdda88be0c4b3dc3f3b9a54425047ee
Jul 29 18:58:57.122396 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:57.122368    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3140cc-qt62n"]
Jul 29 18:58:57.201592 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:57.201518    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3140cc-qt62n" event={"ID":"ceb2c7be-5b2f-440d-824a-d6efdfe297fb","Type":"ContainerStarted","Data":"5a4f1533bf39079712bdb941db5a030d9fdda88be0c4b3dc3f3b9a54425047ee"}
Jul 29 18:58:58.209034 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:58.208946    2557 generic.go:356] "Generic (PLEG): container finished" podID="ceb2c7be-5b2f-440d-824a-d6efdfe297fb" containerID="aac529dbb02924a3bb9506352bcf94e02ce0446c68f793db371173592a797a4b" exitCode=0
Jul 29 18:58:58.209034 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:58.209026    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3140cc-qt62n" event={"ID":"ceb2c7be-5b2f-440d-824a-d6efdfe297fb","Type":"ContainerDied","Data":"aac529dbb02924a3bb9506352bcf94e02ce0446c68f793db371173592a797a4b"}
Jul 29 18:58:59.355707 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:59.355679    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3140cc-qt62n"
Jul 29 18:58:59.551448 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:59.551419    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ceb2c7be-5b2f-440d-824a-d6efdfe297fb-maas-api-service-ca\") pod \"ceb2c7be-5b2f-440d-824a-d6efdfe297fb\" (UID: \"ceb2c7be-5b2f-440d-824a-d6efdfe297fb\") "
Jul 29 18:58:59.551772 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:59.551752    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ceb2c7be-5b2f-440d-824a-d6efdfe297fb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ceb2c7be-5b2f-440d-824a-d6efdfe297fb" (UID: "ceb2c7be-5b2f-440d-824a-d6efdfe297fb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:58:59.652851 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:58:59.652821    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ceb2c7be-5b2f-440d-824a-d6efdfe297fb-maas-api-service-ca\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:59:00.222169 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:59:00.222132    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3140cc-qt62n" event={"ID":"ceb2c7be-5b2f-440d-824a-d6efdfe297fb","Type":"ContainerDied","Data":"5a4f1533bf39079712bdb941db5a030d9fdda88be0c4b3dc3f3b9a54425047ee"}
Jul 29 18:59:00.222169 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:59:00.222162    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3140cc-qt62n"
Jul 29 18:59:00.222385 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:59:00.222191    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5a4f1533bf39079712bdb941db5a030d9fdda88be0c4b3dc3f3b9a54425047ee"
Jul 29 18:59:12.244142 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:59:12.244105    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-3140cc-6c4fc77759-6xgcn"]
Jul 29 18:59:12.245899 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:59:12.244344    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3140cc-6c4fc77759-6xgcn" podUID="0d1b8d9b-4131-4713-b2e0-1c42247852d8" containerName="maas-api" containerID="cri-o://4160da590409dcba1a0506fe5eefbb469bc747c91c30ee2d90c3c3db4b19a06d" gracePeriod=30
Jul 29 18:59:12.445462 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:59:12.445384    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3140cc-qt62n"]
Jul 29 18:59:12.448815 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:59:12.448781    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3140cc-qt62n"]
Jul 29 18:59:12.519392 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:59:12.519365    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3140cc-6c4fc77759-6xgcn"
Jul 29 18:59:12.556255 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:59:12.556224    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d1b8d9b-4131-4713-b2e0-1c42247852d8-openshift-service-ca\") pod \"0d1b8d9b-4131-4713-b2e0-1c42247852d8\" (UID: \"0d1b8d9b-4131-4713-b2e0-1c42247852d8\") "
Jul 29 18:59:12.556402 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:59:12.556276    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d1b8d9b-4131-4713-b2e0-1c42247852d8-maas-api-tls\") pod \"0d1b8d9b-4131-4713-b2e0-1c42247852d8\" (UID: \"0d1b8d9b-4131-4713-b2e0-1c42247852d8\") "
Jul 29 18:59:12.556485 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:59:12.556412    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dplhg\" (UniqueName: \"kubernetes.io/projected/0d1b8d9b-4131-4713-b2e0-1c42247852d8-kube-api-access-dplhg\") pod \"0d1b8d9b-4131-4713-b2e0-1c42247852d8\" (UID: \"0d1b8d9b-4131-4713-b2e0-1c42247852d8\") "
Jul 29 18:59:12.556544 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:59:12.556527    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d1b8d9b-4131-4713-b2e0-1c42247852d8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0d1b8d9b-4131-4713-b2e0-1c42247852d8" (UID: "0d1b8d9b-4131-4713-b2e0-1c42247852d8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:59:12.556759 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:59:12.556709    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d1b8d9b-4131-4713-b2e0-1c42247852d8-openshift-service-ca\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:59:12.558544 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:59:12.558516    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d1b8d9b-4131-4713-b2e0-1c42247852d8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0d1b8d9b-4131-4713-b2e0-1c42247852d8" (UID: "0d1b8d9b-4131-4713-b2e0-1c42247852d8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:59:12.558700 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:59:12.558676    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d1b8d9b-4131-4713-b2e0-1c42247852d8-kube-api-access-dplhg" (OuterVolumeSpecName: "kube-api-access-dplhg") pod "0d1b8d9b-4131-4713-b2e0-1c42247852d8" (UID: "0d1b8d9b-4131-4713-b2e0-1c42247852d8"). InnerVolumeSpecName "kube-api-access-dplhg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:59:12.653017 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:59:12.652975    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ceb2c7be-5b2f-440d-824a-d6efdfe297fb" path="/var/lib/kubelet/pods/ceb2c7be-5b2f-440d-824a-d6efdfe297fb/volumes"
Jul 29 18:59:12.657329 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:59:12.657298    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dplhg\" (UniqueName: \"kubernetes.io/projected/0d1b8d9b-4131-4713-b2e0-1c42247852d8-kube-api-access-dplhg\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:59:12.657329 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:59:12.657328    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d1b8d9b-4131-4713-b2e0-1c42247852d8-maas-api-tls\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 18:59:13.308288 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:59:13.308255    2557 generic.go:356] "Generic (PLEG): container finished" podID="0d1b8d9b-4131-4713-b2e0-1c42247852d8" containerID="4160da590409dcba1a0506fe5eefbb469bc747c91c30ee2d90c3c3db4b19a06d" exitCode=0
Jul 29 18:59:13.308677 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:59:13.308300    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3140cc-6c4fc77759-6xgcn"
Jul 29 18:59:13.308677 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:59:13.308336    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3140cc-6c4fc77759-6xgcn" event={"ID":"0d1b8d9b-4131-4713-b2e0-1c42247852d8","Type":"ContainerDied","Data":"4160da590409dcba1a0506fe5eefbb469bc747c91c30ee2d90c3c3db4b19a06d"}
Jul 29 18:59:13.308677 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:59:13.308387    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3140cc-6c4fc77759-6xgcn" event={"ID":"0d1b8d9b-4131-4713-b2e0-1c42247852d8","Type":"ContainerDied","Data":"567f1879af06b0512680e0765a8dc3b41def0622805e4b3b6bc4cad85cbde60a"}
Jul 29 18:59:13.308677 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:59:13.308410    2557 scope.go:117] "RemoveContainer" containerID="4160da590409dcba1a0506fe5eefbb469bc747c91c30ee2d90c3c3db4b19a06d"
Jul 29 18:59:13.317701 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:59:13.317685    2557 scope.go:117] "RemoveContainer" containerID="4160da590409dcba1a0506fe5eefbb469bc747c91c30ee2d90c3c3db4b19a06d"
Jul 29 18:59:13.317941 ip-10-0-142-156 kubenswrapper[2557]: E0729 18:59:13.317924    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4160da590409dcba1a0506fe5eefbb469bc747c91c30ee2d90c3c3db4b19a06d\": container with ID starting with 4160da590409dcba1a0506fe5eefbb469bc747c91c30ee2d90c3c3db4b19a06d not found: ID does not exist" containerID="4160da590409dcba1a0506fe5eefbb469bc747c91c30ee2d90c3c3db4b19a06d"
Jul 29 18:59:13.318003 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:59:13.317949    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4160da590409dcba1a0506fe5eefbb469bc747c91c30ee2d90c3c3db4b19a06d"} err="failed to get container status \"4160da590409dcba1a0506fe5eefbb469bc747c91c30ee2d90c3c3db4b19a06d\": rpc error: code = NotFound desc = could not find container \"4160da590409dcba1a0506fe5eefbb469bc747c91c30ee2d90c3c3db4b19a06d\": container with ID starting with 4160da590409dcba1a0506fe5eefbb469bc747c91c30ee2d90c3c3db4b19a06d not found: ID does not exist"
Jul 29 18:59:13.328910 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:59:13.328889    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-3140cc-6c4fc77759-6xgcn"]
Jul 29 18:59:13.332601 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:59:13.332580    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-3140cc-6c4fc77759-6xgcn"]
Jul 29 18:59:14.653558 ip-10-0-142-156 kubenswrapper[2557]: I0729 18:59:14.653526    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d1b8d9b-4131-4713-b2e0-1c42247852d8" path="/var/lib/kubelet/pods/0d1b8d9b-4131-4713-b2e0-1c42247852d8/volumes"
Jul 29 19:00:00.142179 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:00.142135    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6e9884-29755860-rr5ff"]
Jul 29 19:00:00.142827 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:00.142555    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d1b8d9b-4131-4713-b2e0-1c42247852d8" containerName="maas-api"
Jul 29 19:00:00.142827 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:00.142568    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d1b8d9b-4131-4713-b2e0-1c42247852d8" containerName="maas-api"
Jul 29 19:00:00.142827 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:00.142581    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ceb2c7be-5b2f-440d-824a-d6efdfe297fb" containerName="revoke-keys"
Jul 29 19:00:00.142827 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:00.142587    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="ceb2c7be-5b2f-440d-824a-d6efdfe297fb" containerName="revoke-keys"
Jul 29 19:00:00.142827 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:00.142668    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d1b8d9b-4131-4713-b2e0-1c42247852d8" containerName="maas-api"
Jul 29 19:00:00.142827 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:00.142676    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="ceb2c7be-5b2f-440d-824a-d6efdfe297fb" containerName="revoke-keys"
Jul 29 19:00:00.145662 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:00.145640    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755860-mnntr"]
Jul 29 19:00:00.145810 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:00.145769    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6e9884-29755860-rr5ff"
Jul 29 19:00:00.148639 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:00.148621    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qrbhl\""
Jul 29 19:00:00.148812 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:00.148794    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-750f0d-29755860-mb4xv"]
Jul 29 19:00:00.148890 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:00.148860    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755860-mnntr"
Jul 29 19:00:00.152565 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:00.152434    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-a10ee3-29755860-zvwtm"]
Jul 29 19:00:00.152698 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:00.152580    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-750f0d-29755860-mb4xv"
Jul 29 19:00:00.158164 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:00.158140    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6e9884-29755860-rr5ff"]
Jul 29 19:00:00.158357 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:00.158242    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-a10ee3-29755860-zvwtm"
Jul 29 19:00:00.160692 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:00.160634    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-750f0d-29755860-mb4xv"]
Jul 29 19:00:00.160839 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:00.160713    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6e9884-29755860-rr5ff"
Jul 29 19:00:00.166080 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:00.166057    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-a10ee3-29755860-zvwtm"]
Jul 29 19:00:00.172488 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:00.172463    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755860-mnntr"
Jul 29 19:00:00.178962 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:00.178935    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755860-mnntr"]
Jul 29 19:00:00.220443 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:00.220415    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-750f0d-29755860-mb4xv"
Jul 29 19:00:00.228326 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:00.228301    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-a10ee3-29755860-zvwtm"
Jul 29 19:00:00.381697 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:00.380139    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6e9884-29755860-rr5ff"]
Jul 29 19:00:00.384457 ip-10-0-142-156 kubenswrapper[2557]: W0729 19:00:00.384415    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0388b169_bb4a_41c8_b3b1_e396073f3192.slice/crio-9352fea003493bb91a4bbad103088f4da49da960af3b69119154212415670c88 WatchSource:0}: Error finding container 9352fea003493bb91a4bbad103088f4da49da960af3b69119154212415670c88: Status 404 returned error can't find the container with id 9352fea003493bb91a4bbad103088f4da49da960af3b69119154212415670c88
Jul 29 19:00:00.422196 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:00.422166    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755860-mnntr"]
Jul 29 19:00:00.433499 ip-10-0-142-156 kubenswrapper[2557]: W0729 19:00:00.433470    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4eccd738_5454_46bd_aa09_32e1ab3c8ad8.slice/crio-72b69a115757f9b638176466880e365cfb0839313d28fc998ecff54955e48b6a WatchSource:0}: Error finding container 72b69a115757f9b638176466880e365cfb0839313d28fc998ecff54955e48b6a: Status 404 returned error can't find the container with id 72b69a115757f9b638176466880e365cfb0839313d28fc998ecff54955e48b6a
Jul 29 19:00:00.593952 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:00.593920    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755860-mnntr" event={"ID":"4eccd738-5454-46bd-aa09-32e1ab3c8ad8","Type":"ContainerStarted","Data":"72b69a115757f9b638176466880e365cfb0839313d28fc998ecff54955e48b6a"}
Jul 29 19:00:00.594977 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:00.594954    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6e9884-29755860-rr5ff" event={"ID":"0388b169-bb4a-41c8-b3b1-e396073f3192","Type":"ContainerStarted","Data":"9352fea003493bb91a4bbad103088f4da49da960af3b69119154212415670c88"}
Jul 29 19:00:00.670706 ip-10-0-142-156 kubenswrapper[2557]: W0729 19:00:00.670674    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01212819_67ad_4072_8773_78eb551e3dd5.slice/crio-1ba07b552331d491b40e91ee2c19cab07811b4f02c53ccb465ee6573000069e1 WatchSource:0}: Error finding container 1ba07b552331d491b40e91ee2c19cab07811b4f02c53ccb465ee6573000069e1: Status 404 returned error can't find the container with id 1ba07b552331d491b40e91ee2c19cab07811b4f02c53ccb465ee6573000069e1
Jul 29 19:00:00.670941 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:00.670921    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-a10ee3-29755860-zvwtm"]
Jul 29 19:00:00.674426 ip-10-0-142-156 kubenswrapper[2557]: W0729 19:00:00.674403    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9453b129_4d68_4982_943e_5315ba154998.slice/crio-ce08b4b85ff358a86dfb21620cbbf41668a940be808532b421f3dddb738118ae WatchSource:0}: Error finding container ce08b4b85ff358a86dfb21620cbbf41668a940be808532b421f3dddb738118ae: Status 404 returned error can't find the container with id ce08b4b85ff358a86dfb21620cbbf41668a940be808532b421f3dddb738118ae
Jul 29 19:00:00.674630 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:00.674610    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-750f0d-29755860-mb4xv"]
Jul 29 19:00:01.603349 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:01.603310    2557 generic.go:356] "Generic (PLEG): container finished" podID="0388b169-bb4a-41c8-b3b1-e396073f3192" containerID="23e81c7a33c33f6dd7bc11c47e62542d65fe61c1ba79d3cc44aac559fdd12fa8" exitCode=0
Jul 29 19:00:01.603754 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:01.603407    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6e9884-29755860-rr5ff" event={"ID":"0388b169-bb4a-41c8-b3b1-e396073f3192","Type":"ContainerDied","Data":"23e81c7a33c33f6dd7bc11c47e62542d65fe61c1ba79d3cc44aac559fdd12fa8"}
Jul 29 19:00:01.605060 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:01.605031    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-a10ee3-29755860-zvwtm" event={"ID":"01212819-67ad-4072-8773-78eb551e3dd5","Type":"ContainerStarted","Data":"c8dec8465bddb1f72d641316bc4d2316fba12fb5fc042fe7a5d791cfd881b409"}
Jul 29 19:00:01.605179 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:01.605077    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-a10ee3-29755860-zvwtm" event={"ID":"01212819-67ad-4072-8773-78eb551e3dd5","Type":"ContainerStarted","Data":"1ba07b552331d491b40e91ee2c19cab07811b4f02c53ccb465ee6573000069e1"}
Jul 29 19:00:01.606352 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:01.606330    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-750f0d-29755860-mb4xv" event={"ID":"9453b129-4d68-4982-943e-5315ba154998","Type":"ContainerStarted","Data":"c7ee80b6c085129ef89333cb35f8c044f7b7d50a19a6be8127e47a82b5e91539"}
Jul 29 19:00:01.606526 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:01.606372    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-750f0d-29755860-mb4xv" event={"ID":"9453b129-4d68-4982-943e-5315ba154998","Type":"ContainerStarted","Data":"ce08b4b85ff358a86dfb21620cbbf41668a940be808532b421f3dddb738118ae"}
Jul 29 19:00:01.607817 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:01.607796    2557 generic.go:356] "Generic (PLEG): container finished" podID="4eccd738-5454-46bd-aa09-32e1ab3c8ad8" containerID="25d26257def35a80dabc2ad5bce06d80c30fa1d70a223b375f55a1709570236a" exitCode=0
Jul 29 19:00:01.607900 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:01.607837    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755860-mnntr" event={"ID":"4eccd738-5454-46bd-aa09-32e1ab3c8ad8","Type":"ContainerDied","Data":"25d26257def35a80dabc2ad5bce06d80c30fa1d70a223b375f55a1709570236a"}
Jul 29 19:00:01.633367 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:01.633282    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-a10ee3-29755860-zvwtm" podStartSLOduration=1.633268765 podStartE2EDuration="1.633268765s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:00:01.631826 +0000 UTC m=+4081.501048802" watchObservedRunningTime="2026-07-29 19:00:01.633268765 +0000 UTC m=+4081.502491572"
Jul 29 19:00:01.659655 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:01.659611    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-750f0d-29755860-mb4xv" podStartSLOduration=1.6595985500000001 podStartE2EDuration="1.65959855s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:00:01.658160811 +0000 UTC m=+4081.527383618" watchObservedRunningTime="2026-07-29 19:00:01.65959855 +0000 UTC m=+4081.528821357"
Jul 29 19:00:02.615111 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:02.615076    2557 generic.go:356] "Generic (PLEG): container finished" podID="9453b129-4d68-4982-943e-5315ba154998" containerID="c7ee80b6c085129ef89333cb35f8c044f7b7d50a19a6be8127e47a82b5e91539" exitCode=0
Jul 29 19:00:02.615534 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:02.615165    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-750f0d-29755860-mb4xv" event={"ID":"9453b129-4d68-4982-943e-5315ba154998","Type":"ContainerDied","Data":"c7ee80b6c085129ef89333cb35f8c044f7b7d50a19a6be8127e47a82b5e91539"}
Jul 29 19:00:02.616797 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:02.616776    2557 generic.go:356] "Generic (PLEG): container finished" podID="01212819-67ad-4072-8773-78eb551e3dd5" containerID="c8dec8465bddb1f72d641316bc4d2316fba12fb5fc042fe7a5d791cfd881b409" exitCode=0
Jul 29 19:00:02.616922 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:02.616834    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-a10ee3-29755860-zvwtm" event={"ID":"01212819-67ad-4072-8773-78eb551e3dd5","Type":"ContainerDied","Data":"c8dec8465bddb1f72d641316bc4d2316fba12fb5fc042fe7a5d791cfd881b409"}
Jul 29 19:00:02.767552 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:02.767513    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6e9884-29755860-rr5ff"
Jul 29 19:00:02.797080 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:02.797052    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755860-mnntr"
Jul 29 19:00:03.625503 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:03.625466    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755860-mnntr"
Jul 29 19:00:03.625503 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:03.625489    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755860-mnntr" event={"ID":"4eccd738-5454-46bd-aa09-32e1ab3c8ad8","Type":"ContainerDied","Data":"72b69a115757f9b638176466880e365cfb0839313d28fc998ecff54955e48b6a"}
Jul 29 19:00:03.626018 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:03.625538    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="72b69a115757f9b638176466880e365cfb0839313d28fc998ecff54955e48b6a"
Jul 29 19:00:03.626886 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:03.626866    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6e9884-29755860-rr5ff"
Jul 29 19:00:03.627000 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:03.626902    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6e9884-29755860-rr5ff" event={"ID":"0388b169-bb4a-41c8-b3b1-e396073f3192","Type":"ContainerDied","Data":"9352fea003493bb91a4bbad103088f4da49da960af3b69119154212415670c88"}
Jul 29 19:00:03.627000 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:03.626940    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9352fea003493bb91a4bbad103088f4da49da960af3b69119154212415670c88"
Jul 29 19:00:03.767592 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:03.767564    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-750f0d-29755860-mb4xv"
Jul 29 19:00:03.802061 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:03.802029    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-a10ee3-29755860-zvwtm"
Jul 29 19:00:03.858224 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:03.858174    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755815-584jp"]
Jul 29 19:00:03.858433 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:03.858243    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755815-584jp"]
Jul 29 19:00:04.635209 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:04.635178    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-a10ee3-29755860-zvwtm"
Jul 29 19:00:04.635702 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:04.635233    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-a10ee3-29755860-zvwtm" event={"ID":"01212819-67ad-4072-8773-78eb551e3dd5","Type":"ContainerDied","Data":"1ba07b552331d491b40e91ee2c19cab07811b4f02c53ccb465ee6573000069e1"}
Jul 29 19:00:04.635702 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:04.635307    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1ba07b552331d491b40e91ee2c19cab07811b4f02c53ccb465ee6573000069e1"
Jul 29 19:00:04.636612 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:04.636594    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-750f0d-29755860-mb4xv"
Jul 29 19:00:04.636712 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:04.636606    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-750f0d-29755860-mb4xv" event={"ID":"9453b129-4d68-4982-943e-5315ba154998","Type":"ContainerDied","Data":"ce08b4b85ff358a86dfb21620cbbf41668a940be808532b421f3dddb738118ae"}
Jul 29 19:00:04.636712 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:04.636644    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ce08b4b85ff358a86dfb21620cbbf41668a940be808532b421f3dddb738118ae"
Jul 29 19:00:04.652687 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:04.652660    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89eea13b-8c96-44cb-8269-0d60628bd39c" path="/var/lib/kubelet/pods/89eea13b-8c96-44cb-8269-0d60628bd39c/volumes"
Jul 29 19:00:23.041504 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:23.041464    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw"]
Jul 29 19:00:23.041948 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:23.041701    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw" podUID="9cf34b68-ba5b-49ad-bdc7-fb7bc133b863" containerName="istio-proxy" containerID="cri-o://1b861e342c68da278abcacd6a45d443629eea5ffadc8f509154e91f632500e76" gracePeriod=30
Jul 29 19:00:23.413089 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:23.413004    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a10ee3-ms8fg"]
Jul 29 19:00:23.413436 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:23.413423    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9453b129-4d68-4982-943e-5315ba154998" containerName="cleanup"
Jul 29 19:00:23.413485 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:23.413438    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="9453b129-4d68-4982-943e-5315ba154998" containerName="cleanup"
Jul 29 19:00:23.413485 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:23.413453    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4eccd738-5454-46bd-aa09-32e1ab3c8ad8" containerName="cleanup"
Jul 29 19:00:23.413485 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:23.413459    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="4eccd738-5454-46bd-aa09-32e1ab3c8ad8" containerName="cleanup"
Jul 29 19:00:23.413485 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:23.413466    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0388b169-bb4a-41c8-b3b1-e396073f3192" containerName="cleanup"
Jul 29 19:00:23.413485 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:23.413472    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="0388b169-bb4a-41c8-b3b1-e396073f3192" containerName="cleanup"
Jul 29 19:00:23.413485 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:23.413483    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01212819-67ad-4072-8773-78eb551e3dd5" containerName="cleanup"
Jul 29 19:00:23.413485 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:23.413489    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="01212819-67ad-4072-8773-78eb551e3dd5" containerName="cleanup"
Jul 29 19:00:23.413711 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:23.413555    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="9453b129-4d68-4982-943e-5315ba154998" containerName="cleanup"
Jul 29 19:00:23.413711 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:23.413564    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="0388b169-bb4a-41c8-b3b1-e396073f3192" containerName="cleanup"
Jul 29 19:00:23.413711 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:23.413571    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="4eccd738-5454-46bd-aa09-32e1ab3c8ad8" containerName="cleanup"
Jul 29 19:00:23.413711 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:23.413578    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="01212819-67ad-4072-8773-78eb551e3dd5" containerName="cleanup"
Jul 29 19:00:23.416792 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:23.416776    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a10ee3-ms8fg"
Jul 29 19:00:23.419501 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:23.419479    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qrbhl\""
Jul 29 19:00:23.427168 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:23.427135    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a10ee3-ms8fg"]
Jul 29 19:00:23.558826 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:23.558788    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/52eb5117-0323-48aa-999c-62386fd6e157-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-a10ee3-ms8fg\" (UID: \"52eb5117-0323-48aa-999c-62386fd6e157\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a10ee3-ms8fg"
Jul 29 19:00:23.660059 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:23.660017    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/52eb5117-0323-48aa-999c-62386fd6e157-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-a10ee3-ms8fg\" (UID: \"52eb5117-0323-48aa-999c-62386fd6e157\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a10ee3-ms8fg"
Jul 29 19:00:23.660656 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:23.660635    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/52eb5117-0323-48aa-999c-62386fd6e157-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-a10ee3-ms8fg\" (UID: \"52eb5117-0323-48aa-999c-62386fd6e157\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a10ee3-ms8fg"
Jul 29 19:00:23.738005 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:23.737923    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a10ee3-ms8fg"
Jul 29 19:00:23.878942 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:23.878910    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a10ee3-ms8fg"]
Jul 29 19:00:23.879271 ip-10-0-142-156 kubenswrapper[2557]: W0729 19:00:23.879243    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52eb5117_0323_48aa_999c_62386fd6e157.slice/crio-917c6ddcfa26ce8e7e4872234cf3ff69601557f6a2810bfa0884ba4c00c413ee WatchSource:0}: Error finding container 917c6ddcfa26ce8e7e4872234cf3ff69601557f6a2810bfa0884ba4c00c413ee: Status 404 returned error can't find the container with id 917c6ddcfa26ce8e7e4872234cf3ff69601557f6a2810bfa0884ba4c00c413ee
Jul 29 19:00:24.758404 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:24.758364    2557 generic.go:356] "Generic (PLEG): container finished" podID="52eb5117-0323-48aa-999c-62386fd6e157" containerID="44939198e28de6e1f7e0b1b3a395880f59e3c27ca436a68172a7c74f822dfab4" exitCode=0
Jul 29 19:00:24.758826 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:24.758455    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a10ee3-ms8fg" event={"ID":"52eb5117-0323-48aa-999c-62386fd6e157","Type":"ContainerDied","Data":"44939198e28de6e1f7e0b1b3a395880f59e3c27ca436a68172a7c74f822dfab4"}
Jul 29 19:00:24.758826 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:24.758516    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a10ee3-ms8fg" event={"ID":"52eb5117-0323-48aa-999c-62386fd6e157","Type":"ContainerStarted","Data":"917c6ddcfa26ce8e7e4872234cf3ff69601557f6a2810bfa0884ba4c00c413ee"}
Jul 29 19:00:25.906241 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:25.906215    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a10ee3-ms8fg"
Jul 29 19:00:25.982742 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:25.982696    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/52eb5117-0323-48aa-999c-62386fd6e157-maas-api-service-ca\") pod \"52eb5117-0323-48aa-999c-62386fd6e157\" (UID: \"52eb5117-0323-48aa-999c-62386fd6e157\") "
Jul 29 19:00:25.983017 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:25.982996    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52eb5117-0323-48aa-999c-62386fd6e157-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "52eb5117-0323-48aa-999c-62386fd6e157" (UID: "52eb5117-0323-48aa-999c-62386fd6e157"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:00:26.083510 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:26.083483    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/52eb5117-0323-48aa-999c-62386fd6e157-maas-api-service-ca\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 19:00:26.771736 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:26.771706    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a10ee3-ms8fg"
Jul 29 19:00:26.771922 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:26.771755    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a10ee3-ms8fg" event={"ID":"52eb5117-0323-48aa-999c-62386fd6e157","Type":"ContainerDied","Data":"917c6ddcfa26ce8e7e4872234cf3ff69601557f6a2810bfa0884ba4c00c413ee"}
Jul 29 19:00:26.771922 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:26.771807    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="917c6ddcfa26ce8e7e4872234cf3ff69601557f6a2810bfa0884ba4c00c413ee"
Jul 29 19:00:28.310009 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:28.309953    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw"
Jul 29 19:00:28.402229 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:28.402201    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cf34b68-ba5b-49ad-bdc7-fb7bc133b863-workload-socket\") pod \"9cf34b68-ba5b-49ad-bdc7-fb7bc133b863\" (UID: \"9cf34b68-ba5b-49ad-bdc7-fb7bc133b863\") "
Jul 29 19:00:28.402408 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:28.402246    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qvzr7\" (UniqueName: \"kubernetes.io/projected/9cf34b68-ba5b-49ad-bdc7-fb7bc133b863-kube-api-access-qvzr7\") pod \"9cf34b68-ba5b-49ad-bdc7-fb7bc133b863\" (UID: \"9cf34b68-ba5b-49ad-bdc7-fb7bc133b863\") "
Jul 29 19:00:28.402408 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:28.402275    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9cf34b68-ba5b-49ad-bdc7-fb7bc133b863-istio-data\") pod \"9cf34b68-ba5b-49ad-bdc7-fb7bc133b863\" (UID: \"9cf34b68-ba5b-49ad-bdc7-fb7bc133b863\") "
Jul 29 19:00:28.402408 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:28.402314    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9cf34b68-ba5b-49ad-bdc7-fb7bc133b863-istio-token\") pod \"9cf34b68-ba5b-49ad-bdc7-fb7bc133b863\" (UID: \"9cf34b68-ba5b-49ad-bdc7-fb7bc133b863\") "
Jul 29 19:00:28.402408 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:28.402341    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9cf34b68-ba5b-49ad-bdc7-fb7bc133b863-istiod-ca-cert\") pod \"9cf34b68-ba5b-49ad-bdc7-fb7bc133b863\" (UID: \"9cf34b68-ba5b-49ad-bdc7-fb7bc133b863\") "
Jul 29 19:00:28.402408 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:28.402406    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9cf34b68-ba5b-49ad-bdc7-fb7bc133b863-istio-podinfo\") pod \"9cf34b68-ba5b-49ad-bdc7-fb7bc133b863\" (UID: \"9cf34b68-ba5b-49ad-bdc7-fb7bc133b863\") "
Jul 29 19:00:28.402664 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:28.402450    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9cf34b68-ba5b-49ad-bdc7-fb7bc133b863-workload-certs\") pod \"9cf34b68-ba5b-49ad-bdc7-fb7bc133b863\" (UID: \"9cf34b68-ba5b-49ad-bdc7-fb7bc133b863\") "
Jul 29 19:00:28.402664 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:28.402505    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cf34b68-ba5b-49ad-bdc7-fb7bc133b863-credential-socket\") pod \"9cf34b68-ba5b-49ad-bdc7-fb7bc133b863\" (UID: \"9cf34b68-ba5b-49ad-bdc7-fb7bc133b863\") "
Jul 29 19:00:28.402664 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:28.402530    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9cf34b68-ba5b-49ad-bdc7-fb7bc133b863-istio-envoy\") pod \"9cf34b68-ba5b-49ad-bdc7-fb7bc133b863\" (UID: \"9cf34b68-ba5b-49ad-bdc7-fb7bc133b863\") "
Jul 29 19:00:28.402664 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:28.402544    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cf34b68-ba5b-49ad-bdc7-fb7bc133b863-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9cf34b68-ba5b-49ad-bdc7-fb7bc133b863" (UID: "9cf34b68-ba5b-49ad-bdc7-fb7bc133b863"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:00:28.402996 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:28.402754    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cf34b68-ba5b-49ad-bdc7-fb7bc133b863-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9cf34b68-ba5b-49ad-bdc7-fb7bc133b863" (UID: "9cf34b68-ba5b-49ad-bdc7-fb7bc133b863"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:00:28.402996 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:28.402773    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cf34b68-ba5b-49ad-bdc7-fb7bc133b863-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9cf34b68-ba5b-49ad-bdc7-fb7bc133b863" (UID: "9cf34b68-ba5b-49ad-bdc7-fb7bc133b863"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:00:28.402996 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:28.402816    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9cf34b68-ba5b-49ad-bdc7-fb7bc133b863-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9cf34b68-ba5b-49ad-bdc7-fb7bc133b863" (UID: "9cf34b68-ba5b-49ad-bdc7-fb7bc133b863"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:00:28.402996 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:28.402910    2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9cf34b68-ba5b-49ad-bdc7-fb7bc133b863-istiod-ca-cert\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 19:00:28.402996 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:28.402929    2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9cf34b68-ba5b-49ad-bdc7-fb7bc133b863-workload-certs\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 19:00:28.402996 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:28.402943    2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cf34b68-ba5b-49ad-bdc7-fb7bc133b863-credential-socket\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 19:00:28.402996 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:28.402957    2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cf34b68-ba5b-49ad-bdc7-fb7bc133b863-workload-socket\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 19:00:28.403515 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:28.403495    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cf34b68-ba5b-49ad-bdc7-fb7bc133b863-istio-data" (OuterVolumeSpecName: "istio-data") pod "9cf34b68-ba5b-49ad-bdc7-fb7bc133b863" (UID: "9cf34b68-ba5b-49ad-bdc7-fb7bc133b863"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:00:28.404803 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:28.404779    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cf34b68-ba5b-49ad-bdc7-fb7bc133b863-kube-api-access-qvzr7" (OuterVolumeSpecName: "kube-api-access-qvzr7") pod "9cf34b68-ba5b-49ad-bdc7-fb7bc133b863" (UID: "9cf34b68-ba5b-49ad-bdc7-fb7bc133b863"). InnerVolumeSpecName "kube-api-access-qvzr7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:00:28.404906 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:28.404804    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9cf34b68-ba5b-49ad-bdc7-fb7bc133b863-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9cf34b68-ba5b-49ad-bdc7-fb7bc133b863" (UID: "9cf34b68-ba5b-49ad-bdc7-fb7bc133b863"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 19:00:28.404906 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:28.404811    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cf34b68-ba5b-49ad-bdc7-fb7bc133b863-istio-token" (OuterVolumeSpecName: "istio-token") pod "9cf34b68-ba5b-49ad-bdc7-fb7bc133b863" (UID: "9cf34b68-ba5b-49ad-bdc7-fb7bc133b863"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:00:28.404906 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:28.404832    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cf34b68-ba5b-49ad-bdc7-fb7bc133b863-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9cf34b68-ba5b-49ad-bdc7-fb7bc133b863" (UID: "9cf34b68-ba5b-49ad-bdc7-fb7bc133b863"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:00:28.504177 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:28.504148    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qvzr7\" (UniqueName: \"kubernetes.io/projected/9cf34b68-ba5b-49ad-bdc7-fb7bc133b863-kube-api-access-qvzr7\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 19:00:28.504177 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:28.504174    2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9cf34b68-ba5b-49ad-bdc7-fb7bc133b863-istio-data\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 19:00:28.504177 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:28.504183    2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9cf34b68-ba5b-49ad-bdc7-fb7bc133b863-istio-token\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 19:00:28.504377 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:28.504191    2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9cf34b68-ba5b-49ad-bdc7-fb7bc133b863-istio-podinfo\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 19:00:28.504377 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:28.504198    2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9cf34b68-ba5b-49ad-bdc7-fb7bc133b863-istio-envoy\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 19:00:28.786808 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:28.786775    2557 generic.go:356] "Generic (PLEG): container finished" podID="9cf34b68-ba5b-49ad-bdc7-fb7bc133b863" containerID="1b861e342c68da278abcacd6a45d443629eea5ffadc8f509154e91f632500e76" exitCode=0
Jul 29 19:00:28.787001 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:28.786839    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw" event={"ID":"9cf34b68-ba5b-49ad-bdc7-fb7bc133b863","Type":"ContainerDied","Data":"1b861e342c68da278abcacd6a45d443629eea5ffadc8f509154e91f632500e76"}
Jul 29 19:00:28.787001 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:28.786850    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw"
Jul 29 19:00:28.787001 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:28.786882    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw" event={"ID":"9cf34b68-ba5b-49ad-bdc7-fb7bc133b863","Type":"ContainerDied","Data":"df7107124d5bcdadd7ca959b9c46d696ac2b27648bf59d3e9b1a01df2c53eea2"}
Jul 29 19:00:28.787001 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:28.786903    2557 scope.go:117] "RemoveContainer" containerID="1b861e342c68da278abcacd6a45d443629eea5ffadc8f509154e91f632500e76"
Jul 29 19:00:28.796670 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:28.796652    2557 scope.go:117] "RemoveContainer" containerID="cf0478470e1980752206530b4dd3ce6648ffacaf7266fd887f6de32ae872313b"
Jul 29 19:00:28.805497 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:28.805474    2557 scope.go:117] "RemoveContainer" containerID="1b861e342c68da278abcacd6a45d443629eea5ffadc8f509154e91f632500e76"
Jul 29 19:00:28.805813 ip-10-0-142-156 kubenswrapper[2557]: E0729 19:00:28.805788    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b861e342c68da278abcacd6a45d443629eea5ffadc8f509154e91f632500e76\": container with ID starting with 1b861e342c68da278abcacd6a45d443629eea5ffadc8f509154e91f632500e76 not found: ID does not exist" containerID="1b861e342c68da278abcacd6a45d443629eea5ffadc8f509154e91f632500e76"
Jul 29 19:00:28.805894 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:28.805826    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b861e342c68da278abcacd6a45d443629eea5ffadc8f509154e91f632500e76"} err="failed to get container status \"1b861e342c68da278abcacd6a45d443629eea5ffadc8f509154e91f632500e76\": rpc error: code = NotFound desc = could not find container \"1b861e342c68da278abcacd6a45d443629eea5ffadc8f509154e91f632500e76\": container with ID starting with 1b861e342c68da278abcacd6a45d443629eea5ffadc8f509154e91f632500e76 not found: ID does not exist"
Jul 29 19:00:28.805894 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:28.805849    2557 scope.go:117] "RemoveContainer" containerID="cf0478470e1980752206530b4dd3ce6648ffacaf7266fd887f6de32ae872313b"
Jul 29 19:00:28.806134 ip-10-0-142-156 kubenswrapper[2557]: E0729 19:00:28.806114    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf0478470e1980752206530b4dd3ce6648ffacaf7266fd887f6de32ae872313b\": container with ID starting with cf0478470e1980752206530b4dd3ce6648ffacaf7266fd887f6de32ae872313b not found: ID does not exist" containerID="cf0478470e1980752206530b4dd3ce6648ffacaf7266fd887f6de32ae872313b"
Jul 29 19:00:28.806209 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:28.806142    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf0478470e1980752206530b4dd3ce6648ffacaf7266fd887f6de32ae872313b"} err="failed to get container status \"cf0478470e1980752206530b4dd3ce6648ffacaf7266fd887f6de32ae872313b\": rpc error: code = NotFound desc = could not find container \"cf0478470e1980752206530b4dd3ce6648ffacaf7266fd887f6de32ae872313b\": container with ID starting with cf0478470e1980752206530b4dd3ce6648ffacaf7266fd887f6de32ae872313b not found: ID does not exist"
Jul 29 19:00:28.807453 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:28.807435    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw"]
Jul 29 19:00:28.810155 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:28.810136    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-3140cc-openshift-default-59c4f497db-7trxw"]
Jul 29 19:00:30.652467 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:30.652431    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9cf34b68-ba5b-49ad-bdc7-fb7bc133b863" path="/var/lib/kubelet/pods/9cf34b68-ba5b-49ad-bdc7-fb7bc133b863/volumes"
Jul 29 19:00:33.798566 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:33.798381    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-a10ee3-cb74b475f-qmbpd"]
Jul 29 19:00:33.799270 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:33.798799    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a10ee3-cb74b475f-qmbpd" podUID="265ebc75-4fd9-4f83-975f-24d18979fa82" containerName="maas-api" containerID="cri-o://78b803dbf1430b540bf3f7478ac8d9c099cd603146c471a5a27cd5845040e162" gracePeriod=30
Jul 29 19:00:33.825939 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:33.825910    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-a10ee3-29755860-zvwtm"]
Jul 29 19:00:33.828593 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:33.828570    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-a10ee3-29755860-zvwtm"]
Jul 29 19:00:33.996213 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:33.996179    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a10ee3-ms8fg"]
Jul 29 19:00:34.000573 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:34.000546    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a10ee3-ms8fg"]
Jul 29 19:00:34.061638 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:34.061525    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a10ee3-cb74b475f-qmbpd"
Jul 29 19:00:34.154804 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:34.154773    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n5n7l\" (UniqueName: \"kubernetes.io/projected/265ebc75-4fd9-4f83-975f-24d18979fa82-kube-api-access-n5n7l\") pod \"265ebc75-4fd9-4f83-975f-24d18979fa82\" (UID: \"265ebc75-4fd9-4f83-975f-24d18979fa82\") "
Jul 29 19:00:34.154961 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:34.154821    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/265ebc75-4fd9-4f83-975f-24d18979fa82-maas-api-tls\") pod \"265ebc75-4fd9-4f83-975f-24d18979fa82\" (UID: \"265ebc75-4fd9-4f83-975f-24d18979fa82\") "
Jul 29 19:00:34.154961 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:34.154906    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/265ebc75-4fd9-4f83-975f-24d18979fa82-openshift-service-ca\") pod \"265ebc75-4fd9-4f83-975f-24d18979fa82\" (UID: \"265ebc75-4fd9-4f83-975f-24d18979fa82\") "
Jul 29 19:00:34.155296 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:34.155268    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/265ebc75-4fd9-4f83-975f-24d18979fa82-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "265ebc75-4fd9-4f83-975f-24d18979fa82" (UID: "265ebc75-4fd9-4f83-975f-24d18979fa82"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:00:34.157010 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:34.156986    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/265ebc75-4fd9-4f83-975f-24d18979fa82-kube-api-access-n5n7l" (OuterVolumeSpecName: "kube-api-access-n5n7l") pod "265ebc75-4fd9-4f83-975f-24d18979fa82" (UID: "265ebc75-4fd9-4f83-975f-24d18979fa82"). InnerVolumeSpecName "kube-api-access-n5n7l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:00:34.157117 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:34.157012    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/265ebc75-4fd9-4f83-975f-24d18979fa82-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "265ebc75-4fd9-4f83-975f-24d18979fa82" (UID: "265ebc75-4fd9-4f83-975f-24d18979fa82"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:00:34.256188 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:34.256156    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/265ebc75-4fd9-4f83-975f-24d18979fa82-openshift-service-ca\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 19:00:34.256188 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:34.256184    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5n7l\" (UniqueName: \"kubernetes.io/projected/265ebc75-4fd9-4f83-975f-24d18979fa82-kube-api-access-n5n7l\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 19:00:34.256188 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:34.256195    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/265ebc75-4fd9-4f83-975f-24d18979fa82-maas-api-tls\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 19:00:34.653622 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:34.653588    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01212819-67ad-4072-8773-78eb551e3dd5" path="/var/lib/kubelet/pods/01212819-67ad-4072-8773-78eb551e3dd5/volumes"
Jul 29 19:00:34.653959 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:34.653944    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52eb5117-0323-48aa-999c-62386fd6e157" path="/var/lib/kubelet/pods/52eb5117-0323-48aa-999c-62386fd6e157/volumes"
Jul 29 19:00:34.828576 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:34.828542    2557 generic.go:356] "Generic (PLEG): container finished" podID="265ebc75-4fd9-4f83-975f-24d18979fa82" containerID="78b803dbf1430b540bf3f7478ac8d9c099cd603146c471a5a27cd5845040e162" exitCode=0
Jul 29 19:00:34.829090 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:34.828589    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a10ee3-cb74b475f-qmbpd" event={"ID":"265ebc75-4fd9-4f83-975f-24d18979fa82","Type":"ContainerDied","Data":"78b803dbf1430b540bf3f7478ac8d9c099cd603146c471a5a27cd5845040e162"}
Jul 29 19:00:34.829090 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:34.828612    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a10ee3-cb74b475f-qmbpd"
Jul 29 19:00:34.829090 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:34.828646    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a10ee3-cb74b475f-qmbpd" event={"ID":"265ebc75-4fd9-4f83-975f-24d18979fa82","Type":"ContainerDied","Data":"312523fcaa5aee87628cf42cd15c91625d39bc96828ea769af5f24bf093e10dc"}
Jul 29 19:00:34.829090 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:34.828669    2557 scope.go:117] "RemoveContainer" containerID="78b803dbf1430b540bf3f7478ac8d9c099cd603146c471a5a27cd5845040e162"
Jul 29 19:00:34.839420 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:34.839403    2557 scope.go:117] "RemoveContainer" containerID="78b803dbf1430b540bf3f7478ac8d9c099cd603146c471a5a27cd5845040e162"
Jul 29 19:00:34.839708 ip-10-0-142-156 kubenswrapper[2557]: E0729 19:00:34.839690    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78b803dbf1430b540bf3f7478ac8d9c099cd603146c471a5a27cd5845040e162\": container with ID starting with 78b803dbf1430b540bf3f7478ac8d9c099cd603146c471a5a27cd5845040e162 not found: ID does not exist" containerID="78b803dbf1430b540bf3f7478ac8d9c099cd603146c471a5a27cd5845040e162"
Jul 29 19:00:34.839773 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:34.839744    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78b803dbf1430b540bf3f7478ac8d9c099cd603146c471a5a27cd5845040e162"} err="failed to get container status \"78b803dbf1430b540bf3f7478ac8d9c099cd603146c471a5a27cd5845040e162\": rpc error: code = NotFound desc = could not find container \"78b803dbf1430b540bf3f7478ac8d9c099cd603146c471a5a27cd5845040e162\": container with ID starting with 78b803dbf1430b540bf3f7478ac8d9c099cd603146c471a5a27cd5845040e162 not found: ID does not exist"
Jul 29 19:00:34.850861 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:34.850838    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-a10ee3-cb74b475f-qmbpd"]
Jul 29 19:00:34.853456 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:34.853438    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-a10ee3-cb74b475f-qmbpd"]
Jul 29 19:00:36.652855 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:36.652817    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="265ebc75-4fd9-4f83-975f-24d18979fa82" path="/var/lib/kubelet/pods/265ebc75-4fd9-4f83-975f-24d18979fa82/volumes"
Jul 29 19:00:40.723612 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:40.723572    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6e9884-chzhw"]
Jul 29 19:00:40.724162 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:40.724142    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cf34b68-ba5b-49ad-bdc7-fb7bc133b863" containerName="istio-proxy"
Jul 29 19:00:40.724250 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:40.724165    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cf34b68-ba5b-49ad-bdc7-fb7bc133b863" containerName="istio-proxy"
Jul 29 19:00:40.724250 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:40.724177    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52eb5117-0323-48aa-999c-62386fd6e157" containerName="revoke-keys"
Jul 29 19:00:40.724250 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:40.724185    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="52eb5117-0323-48aa-999c-62386fd6e157" containerName="revoke-keys"
Jul 29 19:00:40.724250 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:40.724209    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="265ebc75-4fd9-4f83-975f-24d18979fa82" containerName="maas-api"
Jul 29 19:00:40.724250 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:40.724218    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="265ebc75-4fd9-4f83-975f-24d18979fa82" containerName="maas-api"
Jul 29 19:00:40.724250 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:40.724229    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cf34b68-ba5b-49ad-bdc7-fb7bc133b863" containerName="istio-proxy"
Jul 29 19:00:40.724250 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:40.724237    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cf34b68-ba5b-49ad-bdc7-fb7bc133b863" containerName="istio-proxy"
Jul 29 19:00:40.724604 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:40.724330    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cf34b68-ba5b-49ad-bdc7-fb7bc133b863" containerName="istio-proxy"
Jul 29 19:00:40.724604 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:40.724342    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="265ebc75-4fd9-4f83-975f-24d18979fa82" containerName="maas-api"
Jul 29 19:00:40.724604 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:40.724358    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="52eb5117-0323-48aa-999c-62386fd6e157" containerName="revoke-keys"
Jul 29 19:00:40.729207 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:40.729186    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6e9884-chzhw"
Jul 29 19:00:40.732024 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:40.731999    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qrbhl\""
Jul 29 19:00:40.733462 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:40.733439    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6e9884-chzhw"]
Jul 29 19:00:40.818114 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:40.818087    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbae6386-b8dd-4668-b573-1b460f835b7c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-6e9884-chzhw\" (UID: \"cbae6386-b8dd-4668-b573-1b460f835b7c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6e9884-chzhw"
Jul 29 19:00:40.919197 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:40.919167    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbae6386-b8dd-4668-b573-1b460f835b7c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-6e9884-chzhw\" (UID: \"cbae6386-b8dd-4668-b573-1b460f835b7c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6e9884-chzhw"
Jul 29 19:00:40.919832 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:40.919813    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbae6386-b8dd-4668-b573-1b460f835b7c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-6e9884-chzhw\" (UID: \"cbae6386-b8dd-4668-b573-1b460f835b7c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6e9884-chzhw"
Jul 29 19:00:41.044815 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:41.044782    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6e9884-chzhw"
Jul 29 19:00:41.187829 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:41.187755    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6e9884-chzhw"]
Jul 29 19:00:41.188487 ip-10-0-142-156 kubenswrapper[2557]: W0729 19:00:41.188453    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbae6386_b8dd_4668_b573_1b460f835b7c.slice/crio-02487569bfd94146c38df3b4c359ba2f838be965dfa15cd9e683176608baf089 WatchSource:0}: Error finding container 02487569bfd94146c38df3b4c359ba2f838be965dfa15cd9e683176608baf089: Status 404 returned error can't find the container with id 02487569bfd94146c38df3b4c359ba2f838be965dfa15cd9e683176608baf089
Jul 29 19:00:41.876602 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:41.876572    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6e9884-chzhw" event={"ID":"cbae6386-b8dd-4668-b573-1b460f835b7c","Type":"ContainerStarted","Data":"7f65a4a859bd86937bf1c768bd9a564da97435407e47d6a120008828a67eb30b"}
Jul 29 19:00:41.877077 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:41.876632    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6e9884-chzhw" event={"ID":"cbae6386-b8dd-4668-b573-1b460f835b7c","Type":"ContainerStarted","Data":"02487569bfd94146c38df3b4c359ba2f838be965dfa15cd9e683176608baf089"}
Jul 29 19:00:41.895269 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:41.895225    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6e9884-chzhw" podStartSLOduration=1.895211356 podStartE2EDuration="1.895211356s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:00:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:00:41.893356467 +0000 UTC m=+4121.762579276" watchObservedRunningTime="2026-07-29 19:00:41.895211356 +0000 UTC m=+4121.764434182"
Jul 29 19:00:42.884254 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:42.884215    2557 generic.go:356] "Generic (PLEG): container finished" podID="cbae6386-b8dd-4668-b573-1b460f835b7c" containerID="7f65a4a859bd86937bf1c768bd9a564da97435407e47d6a120008828a67eb30b" exitCode=0
Jul 29 19:00:42.884668 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:42.884309    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6e9884-chzhw" event={"ID":"cbae6386-b8dd-4668-b573-1b460f835b7c","Type":"ContainerDied","Data":"7f65a4a859bd86937bf1c768bd9a564da97435407e47d6a120008828a67eb30b"}
Jul 29 19:00:44.028573 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:44.028549    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6e9884-chzhw"
Jul 29 19:00:44.149303 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:44.149216    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbae6386-b8dd-4668-b573-1b460f835b7c-maas-api-service-ca\") pod \"cbae6386-b8dd-4668-b573-1b460f835b7c\" (UID: \"cbae6386-b8dd-4668-b573-1b460f835b7c\") "
Jul 29 19:00:44.149631 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:44.149601    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cbae6386-b8dd-4668-b573-1b460f835b7c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "cbae6386-b8dd-4668-b573-1b460f835b7c" (UID: "cbae6386-b8dd-4668-b573-1b460f835b7c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:00:44.250142 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:44.250110    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbae6386-b8dd-4668-b573-1b460f835b7c-maas-api-service-ca\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 19:00:44.898077 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:44.898043    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6e9884-chzhw" event={"ID":"cbae6386-b8dd-4668-b573-1b460f835b7c","Type":"ContainerDied","Data":"02487569bfd94146c38df3b4c359ba2f838be965dfa15cd9e683176608baf089"}
Jul 29 19:00:44.898077 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:44.898078    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6e9884-chzhw"
Jul 29 19:00:44.898277 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:44.898092    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="02487569bfd94146c38df3b4c359ba2f838be965dfa15cd9e683176608baf089"
Jul 29 19:00:51.201669 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:51.201632    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-6e9884-5bc8f8d7fc-qp5js"]
Jul 29 19:00:51.202125 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:51.201897    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6e9884-5bc8f8d7fc-qp5js" podUID="14f681f3-cbb6-423c-a25f-43b9bec512d3" containerName="maas-api" containerID="cri-o://94a30aaa74ed0c65f1cec53afa7c32784b757853ef0f68615fba61ed77ccca00" gracePeriod=30
Jul 29 19:00:51.214960 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:51.214901    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6e9884-29755860-rr5ff"]
Jul 29 19:00:51.217990 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:51.217969    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6e9884-29755845-z4gws"]
Jul 29 19:00:51.220669 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:51.220644    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6e9884-29755845-z4gws"]
Jul 29 19:00:51.222903 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:51.222885    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6e9884-29755860-rr5ff"]
Jul 29 19:00:51.382881 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:51.382855    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6e9884-chzhw"]
Jul 29 19:00:51.387009 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:51.386983    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6e9884-chzhw"]
Jul 29 19:00:51.464176 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:51.464122    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6e9884-5bc8f8d7fc-qp5js"
Jul 29 19:00:51.518940 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:51.518912    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/14f681f3-cbb6-423c-a25f-43b9bec512d3-openshift-service-ca\") pod \"14f681f3-cbb6-423c-a25f-43b9bec512d3\" (UID: \"14f681f3-cbb6-423c-a25f-43b9bec512d3\") "
Jul 29 19:00:51.519074 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:51.518984    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/14f681f3-cbb6-423c-a25f-43b9bec512d3-maas-api-tls\") pod \"14f681f3-cbb6-423c-a25f-43b9bec512d3\" (UID: \"14f681f3-cbb6-423c-a25f-43b9bec512d3\") "
Jul 29 19:00:51.519119 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:51.519082    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2q6gk\" (UniqueName: \"kubernetes.io/projected/14f681f3-cbb6-423c-a25f-43b9bec512d3-kube-api-access-2q6gk\") pod \"14f681f3-cbb6-423c-a25f-43b9bec512d3\" (UID: \"14f681f3-cbb6-423c-a25f-43b9bec512d3\") "
Jul 29 19:00:51.519298 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:51.519268    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14f681f3-cbb6-423c-a25f-43b9bec512d3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "14f681f3-cbb6-423c-a25f-43b9bec512d3" (UID: "14f681f3-cbb6-423c-a25f-43b9bec512d3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:00:51.519425 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:51.519410    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/14f681f3-cbb6-423c-a25f-43b9bec512d3-openshift-service-ca\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 19:00:51.521138 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:51.521117    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14f681f3-cbb6-423c-a25f-43b9bec512d3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "14f681f3-cbb6-423c-a25f-43b9bec512d3" (UID: "14f681f3-cbb6-423c-a25f-43b9bec512d3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:00:51.521202 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:51.521166    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14f681f3-cbb6-423c-a25f-43b9bec512d3-kube-api-access-2q6gk" (OuterVolumeSpecName: "kube-api-access-2q6gk") pod "14f681f3-cbb6-423c-a25f-43b9bec512d3" (UID: "14f681f3-cbb6-423c-a25f-43b9bec512d3"). InnerVolumeSpecName "kube-api-access-2q6gk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:00:51.620325 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:51.620295    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/14f681f3-cbb6-423c-a25f-43b9bec512d3-maas-api-tls\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 19:00:51.620325 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:51.620320    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2q6gk\" (UniqueName: \"kubernetes.io/projected/14f681f3-cbb6-423c-a25f-43b9bec512d3-kube-api-access-2q6gk\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 19:00:51.945661 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:51.945626    2557 generic.go:356] "Generic (PLEG): container finished" podID="14f681f3-cbb6-423c-a25f-43b9bec512d3" containerID="94a30aaa74ed0c65f1cec53afa7c32784b757853ef0f68615fba61ed77ccca00" exitCode=0
Jul 29 19:00:51.945885 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:51.945666    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6e9884-5bc8f8d7fc-qp5js" event={"ID":"14f681f3-cbb6-423c-a25f-43b9bec512d3","Type":"ContainerDied","Data":"94a30aaa74ed0c65f1cec53afa7c32784b757853ef0f68615fba61ed77ccca00"}
Jul 29 19:00:51.945885 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:51.945755    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6e9884-5bc8f8d7fc-qp5js" event={"ID":"14f681f3-cbb6-423c-a25f-43b9bec512d3","Type":"ContainerDied","Data":"e9581b6f0c46136f0c34e7c95dc1c0b7ba6bd0249a16f73e77605ea6f964fb61"}
Jul 29 19:00:51.945885 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:51.945680    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6e9884-5bc8f8d7fc-qp5js"
Jul 29 19:00:51.945885 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:51.945779    2557 scope.go:117] "RemoveContainer" containerID="94a30aaa74ed0c65f1cec53afa7c32784b757853ef0f68615fba61ed77ccca00"
Jul 29 19:00:51.956082 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:51.956045    2557 scope.go:117] "RemoveContainer" containerID="94a30aaa74ed0c65f1cec53afa7c32784b757853ef0f68615fba61ed77ccca00"
Jul 29 19:00:51.956547 ip-10-0-142-156 kubenswrapper[2557]: E0729 19:00:51.956526    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94a30aaa74ed0c65f1cec53afa7c32784b757853ef0f68615fba61ed77ccca00\": container with ID starting with 94a30aaa74ed0c65f1cec53afa7c32784b757853ef0f68615fba61ed77ccca00 not found: ID does not exist" containerID="94a30aaa74ed0c65f1cec53afa7c32784b757853ef0f68615fba61ed77ccca00"
Jul 29 19:00:51.956642 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:51.956553    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94a30aaa74ed0c65f1cec53afa7c32784b757853ef0f68615fba61ed77ccca00"} err="failed to get container status \"94a30aaa74ed0c65f1cec53afa7c32784b757853ef0f68615fba61ed77ccca00\": rpc error: code = NotFound desc = could not find container \"94a30aaa74ed0c65f1cec53afa7c32784b757853ef0f68615fba61ed77ccca00\": container with ID starting with 94a30aaa74ed0c65f1cec53afa7c32784b757853ef0f68615fba61ed77ccca00 not found: ID does not exist"
Jul 29 19:00:51.968363 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:51.968342    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-6e9884-5bc8f8d7fc-qp5js"]
Jul 29 19:00:51.974136 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:51.974119    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-6e9884-5bc8f8d7fc-qp5js"]
Jul 29 19:00:52.653436 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:52.653402    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0388b169-bb4a-41c8-b3b1-e396073f3192" path="/var/lib/kubelet/pods/0388b169-bb4a-41c8-b3b1-e396073f3192/volumes"
Jul 29 19:00:52.653820 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:52.653759    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14f681f3-cbb6-423c-a25f-43b9bec512d3" path="/var/lib/kubelet/pods/14f681f3-cbb6-423c-a25f-43b9bec512d3/volumes"
Jul 29 19:00:52.654005 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:52.653992    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="beeb4c2d-edaa-4153-bb1d-f1dfd95e5eb9" path="/var/lib/kubelet/pods/beeb4c2d-edaa-4153-bb1d-f1dfd95e5eb9/volumes"
Jul 29 19:00:52.654238 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:52.654227    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cbae6386-b8dd-4668-b573-1b460f835b7c" path="/var/lib/kubelet/pods/cbae6386-b8dd-4668-b573-1b460f835b7c/volumes"
Jul 29 19:00:58.079396 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:58.079355    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-750f0d-zxc7v"]
Jul 29 19:00:58.079907 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:58.079802    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14f681f3-cbb6-423c-a25f-43b9bec512d3" containerName="maas-api"
Jul 29 19:00:58.079907 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:58.079816    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="14f681f3-cbb6-423c-a25f-43b9bec512d3" containerName="maas-api"
Jul 29 19:00:58.079907 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:58.079824    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbae6386-b8dd-4668-b573-1b460f835b7c" containerName="revoke-keys"
Jul 29 19:00:58.079907 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:58.079830    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbae6386-b8dd-4668-b573-1b460f835b7c" containerName="revoke-keys"
Jul 29 19:00:58.079907 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:58.079892    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="14f681f3-cbb6-423c-a25f-43b9bec512d3" containerName="maas-api"
Jul 29 19:00:58.079907 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:58.079902    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="cbae6386-b8dd-4668-b573-1b460f835b7c" containerName="revoke-keys"
Jul 29 19:00:58.079907 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:58.079910    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cf34b68-ba5b-49ad-bdc7-fb7bc133b863" containerName="istio-proxy"
Jul 29 19:00:58.084567 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:58.084545    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-750f0d-zxc7v"
Jul 29 19:00:58.087822 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:58.087756    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qrbhl\""
Jul 29 19:00:58.091311 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:58.091283    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-750f0d-zxc7v"]
Jul 29 19:00:58.177018 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:58.176969    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b239cfc-2473-4e6e-a9a3-a5a37bf6df1a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-750f0d-zxc7v\" (UID: \"7b239cfc-2473-4e6e-a9a3-a5a37bf6df1a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-750f0d-zxc7v"
Jul 29 19:00:58.278590 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:58.278552    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b239cfc-2473-4e6e-a9a3-a5a37bf6df1a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-750f0d-zxc7v\" (UID: \"7b239cfc-2473-4e6e-a9a3-a5a37bf6df1a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-750f0d-zxc7v"
Jul 29 19:00:58.279206 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:58.279188    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b239cfc-2473-4e6e-a9a3-a5a37bf6df1a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-750f0d-zxc7v\" (UID: \"7b239cfc-2473-4e6e-a9a3-a5a37bf6df1a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-750f0d-zxc7v"
Jul 29 19:00:58.398383 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:58.398273    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-750f0d-zxc7v"
Jul 29 19:00:58.548028 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:58.548000    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-750f0d-zxc7v"]
Jul 29 19:00:58.549266 ip-10-0-142-156 kubenswrapper[2557]: W0729 19:00:58.549240    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b239cfc_2473_4e6e_a9a3_a5a37bf6df1a.slice/crio-933ab8cb215e3265615d8f55bd7f2493e6ff989b6b5747bb2f6652a4424a8750 WatchSource:0}: Error finding container 933ab8cb215e3265615d8f55bd7f2493e6ff989b6b5747bb2f6652a4424a8750: Status 404 returned error can't find the container with id 933ab8cb215e3265615d8f55bd7f2493e6ff989b6b5747bb2f6652a4424a8750
Jul 29 19:00:58.991752 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:58.991698    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-750f0d-zxc7v" event={"ID":"7b239cfc-2473-4e6e-a9a3-a5a37bf6df1a","Type":"ContainerStarted","Data":"cac73a35ed11bc9a386d600bf7cc43cab265420471a0a4b4e9c1640590a7eb9f"}
Jul 29 19:00:58.991933 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:58.991771    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-750f0d-zxc7v" event={"ID":"7b239cfc-2473-4e6e-a9a3-a5a37bf6df1a","Type":"ContainerStarted","Data":"933ab8cb215e3265615d8f55bd7f2493e6ff989b6b5747bb2f6652a4424a8750"}
Jul 29 19:00:59.017565 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:59.017465    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-750f0d-zxc7v" podStartSLOduration=1.017448883 podStartE2EDuration="1.017448883s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:00:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:00:59.014042113 +0000 UTC m=+4138.883264932" watchObservedRunningTime="2026-07-29 19:00:59.017448883 +0000 UTC m=+4138.886671707"
Jul 29 19:00:59.999340 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:59.999304    2557 generic.go:356] "Generic (PLEG): container finished" podID="7b239cfc-2473-4e6e-a9a3-a5a37bf6df1a" containerID="cac73a35ed11bc9a386d600bf7cc43cab265420471a0a4b4e9c1640590a7eb9f" exitCode=0
Jul 29 19:00:59.999777 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:00:59.999389    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-750f0d-zxc7v" event={"ID":"7b239cfc-2473-4e6e-a9a3-a5a37bf6df1a","Type":"ContainerDied","Data":"cac73a35ed11bc9a386d600bf7cc43cab265420471a0a4b4e9c1640590a7eb9f"}
Jul 29 19:01:01.147094 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:01.147063    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-750f0d-zxc7v"
Jul 29 19:01:01.308023 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:01.307985    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b239cfc-2473-4e6e-a9a3-a5a37bf6df1a-maas-api-service-ca\") pod \"7b239cfc-2473-4e6e-a9a3-a5a37bf6df1a\" (UID: \"7b239cfc-2473-4e6e-a9a3-a5a37bf6df1a\") "
Jul 29 19:01:01.308397 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:01.308371    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b239cfc-2473-4e6e-a9a3-a5a37bf6df1a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7b239cfc-2473-4e6e-a9a3-a5a37bf6df1a" (UID: "7b239cfc-2473-4e6e-a9a3-a5a37bf6df1a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:01:01.389019 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:01.388988    2557 scope.go:117] "RemoveContainer" containerID="a42f934467ed81f5958050806e8554d0367b8d12509166f856ed5f6ac9490979"
Jul 29 19:01:01.399166 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:01.399144    2557 scope.go:117] "RemoveContainer" containerID="558c151349384852c9dd1cd3cf7a966d7d944e1a1472a8ba0ed745758578a1b1"
Jul 29 19:01:01.409531 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:01.409495    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b239cfc-2473-4e6e-a9a3-a5a37bf6df1a-maas-api-service-ca\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 19:01:02.013211 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:02.013181    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-750f0d-zxc7v"
Jul 29 19:01:02.013419 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:02.013213    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-750f0d-zxc7v" event={"ID":"7b239cfc-2473-4e6e-a9a3-a5a37bf6df1a","Type":"ContainerDied","Data":"933ab8cb215e3265615d8f55bd7f2493e6ff989b6b5747bb2f6652a4424a8750"}
Jul 29 19:01:02.013419 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:02.013255    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="933ab8cb215e3265615d8f55bd7f2493e6ff989b6b5747bb2f6652a4424a8750"
Jul 29 19:01:08.470203 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:08.470168    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-750f0d-6f5bdf6cfb-sn7sj"]
Jul 29 19:01:08.470825 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:08.470421    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-750f0d-6f5bdf6cfb-sn7sj" podUID="63f41491-af00-4be7-ab6e-7b2d955d2f62" containerName="maas-api" containerID="cri-o://a5635c164365da0745bb9233664965ca940be701b3178e47b2d341af82c69d23" gracePeriod=30
Jul 29 19:01:08.503710 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:08.503680    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-750f0d-29755860-mb4xv"]
Jul 29 19:01:08.507428 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:08.507400    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-750f0d-29755860-mb4xv"]
Jul 29 19:01:08.655745 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:08.655440    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9453b129-4d68-4982-943e-5315ba154998" path="/var/lib/kubelet/pods/9453b129-4d68-4982-943e-5315ba154998/volumes"
Jul 29 19:01:08.659594 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:08.659571    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-750f0d-zxc7v"]
Jul 29 19:01:08.662103 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:08.662078    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-750f0d-zxc7v"]
Jul 29 19:01:08.733955 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:08.733896    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-750f0d-6f5bdf6cfb-sn7sj"
Jul 29 19:01:08.879653 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:08.879620    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63f41491-af00-4be7-ab6e-7b2d955d2f62-maas-api-tls\") pod \"63f41491-af00-4be7-ab6e-7b2d955d2f62\" (UID: \"63f41491-af00-4be7-ab6e-7b2d955d2f62\") "
Jul 29 19:01:08.879653 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:08.879655    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9xhv7\" (UniqueName: \"kubernetes.io/projected/63f41491-af00-4be7-ab6e-7b2d955d2f62-kube-api-access-9xhv7\") pod \"63f41491-af00-4be7-ab6e-7b2d955d2f62\" (UID: \"63f41491-af00-4be7-ab6e-7b2d955d2f62\") "
Jul 29 19:01:08.879946 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:08.879804    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63f41491-af00-4be7-ab6e-7b2d955d2f62-openshift-service-ca\") pod \"63f41491-af00-4be7-ab6e-7b2d955d2f62\" (UID: \"63f41491-af00-4be7-ab6e-7b2d955d2f62\") "
Jul 29 19:01:08.880117 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:08.880090    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63f41491-af00-4be7-ab6e-7b2d955d2f62-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "63f41491-af00-4be7-ab6e-7b2d955d2f62" (UID: "63f41491-af00-4be7-ab6e-7b2d955d2f62"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:01:08.881944 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:08.881924    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63f41491-af00-4be7-ab6e-7b2d955d2f62-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "63f41491-af00-4be7-ab6e-7b2d955d2f62" (UID: "63f41491-af00-4be7-ab6e-7b2d955d2f62"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:01:08.882016 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:08.881999    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63f41491-af00-4be7-ab6e-7b2d955d2f62-kube-api-access-9xhv7" (OuterVolumeSpecName: "kube-api-access-9xhv7") pod "63f41491-af00-4be7-ab6e-7b2d955d2f62" (UID: "63f41491-af00-4be7-ab6e-7b2d955d2f62"). InnerVolumeSpecName "kube-api-access-9xhv7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:01:08.980604 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:08.980571    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63f41491-af00-4be7-ab6e-7b2d955d2f62-openshift-service-ca\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 19:01:08.980604 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:08.980604    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63f41491-af00-4be7-ab6e-7b2d955d2f62-maas-api-tls\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 19:01:08.980837 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:08.980618    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9xhv7\" (UniqueName: \"kubernetes.io/projected/63f41491-af00-4be7-ab6e-7b2d955d2f62-kube-api-access-9xhv7\") on node \"ip-10-0-142-156.ec2.internal\" DevicePath \"\""
Jul 29 19:01:09.058524 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:09.058490    2557 generic.go:356] "Generic (PLEG): container finished" podID="63f41491-af00-4be7-ab6e-7b2d955d2f62" containerID="a5635c164365da0745bb9233664965ca940be701b3178e47b2d341af82c69d23" exitCode=0
Jul 29 19:01:09.058693 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:09.058540    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-750f0d-6f5bdf6cfb-sn7sj"
Jul 29 19:01:09.058693 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:09.058576    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-750f0d-6f5bdf6cfb-sn7sj" event={"ID":"63f41491-af00-4be7-ab6e-7b2d955d2f62","Type":"ContainerDied","Data":"a5635c164365da0745bb9233664965ca940be701b3178e47b2d341af82c69d23"}
Jul 29 19:01:09.058693 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:09.058630    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-750f0d-6f5bdf6cfb-sn7sj" event={"ID":"63f41491-af00-4be7-ab6e-7b2d955d2f62","Type":"ContainerDied","Data":"8be1932d873b018d4a8dda6e1084f10a9f4d9c2abaf6c4aa86e65814daace638"}
Jul 29 19:01:09.058693 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:09.058656    2557 scope.go:117] "RemoveContainer" containerID="a5635c164365da0745bb9233664965ca940be701b3178e47b2d341af82c69d23"
Jul 29 19:01:09.068577 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:09.068558    2557 scope.go:117] "RemoveContainer" containerID="a5635c164365da0745bb9233664965ca940be701b3178e47b2d341af82c69d23"
Jul 29 19:01:09.068859 ip-10-0-142-156 kubenswrapper[2557]: E0729 19:01:09.068838    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5635c164365da0745bb9233664965ca940be701b3178e47b2d341af82c69d23\": container with ID starting with a5635c164365da0745bb9233664965ca940be701b3178e47b2d341af82c69d23 not found: ID does not exist" containerID="a5635c164365da0745bb9233664965ca940be701b3178e47b2d341af82c69d23"
Jul 29 19:01:09.068949 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:09.068871    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5635c164365da0745bb9233664965ca940be701b3178e47b2d341af82c69d23"} err="failed to get container status \"a5635c164365da0745bb9233664965ca940be701b3178e47b2d341af82c69d23\": rpc error: code = NotFound desc = could not find container \"a5635c164365da0745bb9233664965ca940be701b3178e47b2d341af82c69d23\": container with ID starting with a5635c164365da0745bb9233664965ca940be701b3178e47b2d341af82c69d23 not found: ID does not exist"
Jul 29 19:01:09.081423 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:09.081398    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-750f0d-6f5bdf6cfb-sn7sj"]
Jul 29 19:01:09.086365 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:09.086341    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-750f0d-6f5bdf6cfb-sn7sj"]
Jul 29 19:01:10.653548 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:10.653506    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63f41491-af00-4be7-ab6e-7b2d955d2f62" path="/var/lib/kubelet/pods/63f41491-af00-4be7-ab6e-7b2d955d2f62/volumes"
Jul 29 19:01:10.654063 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:10.654032    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b239cfc-2473-4e6e-a9a3-a5a37bf6df1a" path="/var/lib/kubelet/pods/7b239cfc-2473-4e6e-a9a3-a5a37bf6df1a/volumes"
Jul 29 19:01:21.199132 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:21.199098    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5c67f77dd8-9kj8h_10654f46-4178-4840-81e6-52d53aacb1ec/manager/0.log"
Jul 29 19:01:21.311023 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:21.310997    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-mdsvb_dad1b1dc-9f1a-4fbd-bbbc-860e74a95cd2/server/0.log"
Jul 29 19:01:21.433423 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:21.433395    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-672fv_5645b830-d2f9-4164-8e78-a7be8c9af2b5/manager/0.log"
Jul 29 19:01:21.780779 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:21.780751    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d84c6c79c-qq57z_49eb35f6-74a3-42ba-b8de-d69123427363/copy-manifests/0.log"
Jul 29 19:01:21.787587 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:21.787557    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d84c6c79c-qq57z_49eb35f6-74a3-42ba-b8de-d69123427363/manager/0.log"
Jul 29 19:01:22.147881 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:22.147843    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-85d97bb9f9-r6vwc_a852fdcf-7a6d-41ee-ab42-00e93f24e51f/maas-api/0.log"
Jul 29 19:01:22.252915 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:22.252883    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29755830-fcch4_7e8a09f2-7bd4-4fcf-a517-7819546f9fd4/cleanup/0.log"
Jul 29 19:01:22.356806 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:22.356773    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29755845-9cbbh_345eaead-a9fb-4a6f-8bbe-5451b03b4925/cleanup/0.log"
Jul 29 19:01:22.465588 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:22.465498    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29755860-mnntr_4eccd738-5454-46bd-aa09-32e1ab3c8ad8/cleanup/0.log"
Jul 29 19:01:22.573526 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:22.573499    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-4cbzt_9353b89d-9f69-4bca-ba18-a0ee0bef1631/postgres/0.log"
Jul 29 19:01:23.339943 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:23.339918    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r8j6_ece83ae2-c4e0-4df9-95fd-616c5ab8990e/util/0.log"
Jul 29 19:01:23.346438 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:23.346399    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r8j6_ece83ae2-c4e0-4df9-95fd-616c5ab8990e/pull/0.log"
Jul 29 19:01:23.351959 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:23.351944    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r8j6_ece83ae2-c4e0-4df9-95fd-616c5ab8990e/extract/0.log"
Jul 29 19:01:23.464948 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:23.464921    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rsngv_da62e93b-16c5-4747-b35e-628e7636fcc8/pull/0.log"
Jul 29 19:01:23.470805 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:23.470782    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rsngv_da62e93b-16c5-4747-b35e-628e7636fcc8/extract/0.log"
Jul 29 19:01:23.476333 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:23.476314    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rsngv_da62e93b-16c5-4747-b35e-628e7636fcc8/util/0.log"
Jul 29 19:01:23.582353 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:23.582310    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gjbh_5f32a898-ddb9-4a13-8e36-83c7533a9755/util/0.log"
Jul 29 19:01:23.588970 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:23.588949    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gjbh_5f32a898-ddb9-4a13-8e36-83c7533a9755/pull/0.log"
Jul 29 19:01:23.596252 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:23.596178    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gjbh_5f32a898-ddb9-4a13-8e36-83c7533a9755/extract/0.log"
Jul 29 19:01:23.711301 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:23.711271    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8brs_26ba8c29-8db7-4faf-bbd9-187f92b54bab/util/0.log"
Jul 29 19:01:23.717218 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:23.717185    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8brs_26ba8c29-8db7-4faf-bbd9-187f92b54bab/pull/0.log"
Jul 29 19:01:23.723049 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:23.723031    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8brs_26ba8c29-8db7-4faf-bbd9-187f92b54bab/extract/0.log"
Jul 29 19:01:24.085870 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:24.085838    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-l8cgv_d1921098-bba3-4024-9220-fdefd73975ea/manager/0.log"
Jul 29 19:01:24.435211 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:24.435132    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-xvr6r_fa3433e1-e499-4bb0-803d-97906bc639f2/manager/0.log"
Jul 29 19:01:24.996301 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:24.996268    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5_920ab95a-edbf-425b-8f80-45937925ae16/istio-proxy/0.log"
Jul 29 19:01:25.116100 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:25.116074    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-j5k56_8b9ee721-eb64-4ab6-a806-c6562ce45797/discovery/0.log"
Jul 29 19:01:25.336730 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:25.336707    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-869f956dcc-fp29f_3877a269-9027-424f-8903-b06036bbbcb2/kube-auth-proxy/0.log"
Jul 29 19:01:25.457904 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:25.457871    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-vzxzx_5bb4f610-8260-474c-bae4-e5d9ed9a5280/istio-proxy/1.log"
Jul 29 19:01:25.576143 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:25.576111    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-tkpf5_4be56e67-7714-47b7-b0c5-254141f6b6c5/payload-pre-processing/0.log"
Jul 29 19:01:25.700360 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:25.700263    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-p6kkw_0d65e55f-0460-4fd4-b46c-6d0bb6724295/payload-processing/0.log"
Jul 29 19:01:25.812711 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:25.812681    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-676c59fd4f-tvcgb_0da2e7ec-9cb4-42dc-a537-a5b46687ee32/router/0.log"
Jul 29 19:01:26.266440 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:26.266392    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-2b6c6_335de777-811d-421e-b05d-729871464949/main/0.log"
Jul 29 19:01:32.126920 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:32.126883    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5k8jh/must-gather-6r99l"]
Jul 29 19:01:32.127337 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:32.127308    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b239cfc-2473-4e6e-a9a3-a5a37bf6df1a" containerName="revoke-keys"
Jul 29 19:01:32.127337 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:32.127320    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b239cfc-2473-4e6e-a9a3-a5a37bf6df1a" containerName="revoke-keys"
Jul 29 19:01:32.127337 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:32.127335    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63f41491-af00-4be7-ab6e-7b2d955d2f62" containerName="maas-api"
Jul 29 19:01:32.127450 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:32.127341    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="63f41491-af00-4be7-ab6e-7b2d955d2f62" containerName="maas-api"
Jul 29 19:01:32.127450 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:32.127404    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="63f41491-af00-4be7-ab6e-7b2d955d2f62" containerName="maas-api"
Jul 29 19:01:32.127450 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:32.127416    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b239cfc-2473-4e6e-a9a3-a5a37bf6df1a" containerName="revoke-keys"
Jul 29 19:01:32.130704 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:32.130683    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5k8jh/must-gather-6r99l"
Jul 29 19:01:32.133402 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:32.133381    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5k8jh\"/\"openshift-service-ca.crt\""
Jul 29 19:01:32.133529 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:32.133410    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5k8jh\"/\"kube-root-ca.crt\""
Jul 29 19:01:32.133529 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:32.133429    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-5k8jh\"/\"default-dockercfg-9zp9k\""
Jul 29 19:01:32.141616 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:32.141573    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5k8jh/must-gather-6r99l"]
Jul 29 19:01:32.185030 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:32.185002    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/6e7f9759-f2d9-4a01-9fc9-14159c4a294b-must-gather-output\") pod \"must-gather-6r99l\" (UID: \"6e7f9759-f2d9-4a01-9fc9-14159c4a294b\") " pod="openshift-must-gather-5k8jh/must-gather-6r99l"
Jul 29 19:01:32.185205 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:32.185050    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkbkr\" (UniqueName: \"kubernetes.io/projected/6e7f9759-f2d9-4a01-9fc9-14159c4a294b-kube-api-access-wkbkr\") pod \"must-gather-6r99l\" (UID: \"6e7f9759-f2d9-4a01-9fc9-14159c4a294b\") " pod="openshift-must-gather-5k8jh/must-gather-6r99l"
Jul 29 19:01:32.286145 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:32.286107    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/6e7f9759-f2d9-4a01-9fc9-14159c4a294b-must-gather-output\") pod \"must-gather-6r99l\" (UID: \"6e7f9759-f2d9-4a01-9fc9-14159c4a294b\") " pod="openshift-must-gather-5k8jh/must-gather-6r99l"
Jul 29 19:01:32.286316 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:32.286166    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkbkr\" (UniqueName: \"kubernetes.io/projected/6e7f9759-f2d9-4a01-9fc9-14159c4a294b-kube-api-access-wkbkr\") pod \"must-gather-6r99l\" (UID: \"6e7f9759-f2d9-4a01-9fc9-14159c4a294b\") " pod="openshift-must-gather-5k8jh/must-gather-6r99l"
Jul 29 19:01:32.286464 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:32.286442    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/6e7f9759-f2d9-4a01-9fc9-14159c4a294b-must-gather-output\") pod \"must-gather-6r99l\" (UID: \"6e7f9759-f2d9-4a01-9fc9-14159c4a294b\") " pod="openshift-must-gather-5k8jh/must-gather-6r99l"
Jul 29 19:01:32.295319 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:32.295292    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkbkr\" (UniqueName: \"kubernetes.io/projected/6e7f9759-f2d9-4a01-9fc9-14159c4a294b-kube-api-access-wkbkr\") pod \"must-gather-6r99l\" (UID: \"6e7f9759-f2d9-4a01-9fc9-14159c4a294b\") " pod="openshift-must-gather-5k8jh/must-gather-6r99l"
Jul 29 19:01:32.440992 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:32.440905    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5k8jh/must-gather-6r99l"
Jul 29 19:01:32.574798 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:32.574765    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5k8jh/must-gather-6r99l"]
Jul 29 19:01:32.576196 ip-10-0-142-156 kubenswrapper[2557]: W0729 19:01:32.576174    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6e7f9759_f2d9_4a01_9fc9_14159c4a294b.slice/crio-04d399041f7d97c63e31023743956e6c6b9526f4e947126434c61987c4874e24 WatchSource:0}: Error finding container 04d399041f7d97c63e31023743956e6c6b9526f4e947126434c61987c4874e24: Status 404 returned error can't find the container with id 04d399041f7d97c63e31023743956e6c6b9526f4e947126434c61987c4874e24
Jul 29 19:01:32.578108 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:32.578092    2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 19:01:33.210446 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:33.210408    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5k8jh/must-gather-6r99l" event={"ID":"6e7f9759-f2d9-4a01-9fc9-14159c4a294b","Type":"ContainerStarted","Data":"04d399041f7d97c63e31023743956e6c6b9526f4e947126434c61987c4874e24"}
Jul 29 19:01:34.221709 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:34.221664    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5k8jh/must-gather-6r99l" event={"ID":"6e7f9759-f2d9-4a01-9fc9-14159c4a294b","Type":"ContainerStarted","Data":"c322440c1cd886468dece606a22a793acec3f913e0420780d0d0ea5243685e15"}
Jul 29 19:01:34.222111 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:34.221749    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5k8jh/must-gather-6r99l" event={"ID":"6e7f9759-f2d9-4a01-9fc9-14159c4a294b","Type":"ContainerStarted","Data":"7600a96421e8f26e1583dad778eff7af5c60d8e8105ad71817f8dad7ef32b12f"}
Jul 29 19:01:34.240390 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:34.240338    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5k8jh/must-gather-6r99l" podStartSLOduration=1.1788318 podStartE2EDuration="2.240321133s" totalImagesPullingTime="1.061489333s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:01:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:01:34.236989648 +0000 UTC m=+4174.106212459" watchObservedRunningTime="2026-07-29 19:01:34.240321133 +0000 UTC m=+4174.109543940"
Jul 29 19:01:35.220495 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:35.220459    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-tdktb_905f9420-cd73-4e49-a85f-2163a04b8fcd/global-pull-secret-syncer/0.log"
Jul 29 19:01:35.356015 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:35.355981    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-mb6fv_1eecf48c-b25b-4449-a782-c3b702cc0368/konnectivity-agent/0.log"
Jul 29 19:01:35.492777 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:35.492659    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-156.ec2.internal_591bc9043de6ffb60a0ece0a3a196263/haproxy/0.log"
Jul 29 19:01:40.539830 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:40.539775    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r8j6_ece83ae2-c4e0-4df9-95fd-616c5ab8990e/extract/0.log"
Jul 29 19:01:40.561189 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:40.561155    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r8j6_ece83ae2-c4e0-4df9-95fd-616c5ab8990e/util/0.log"
Jul 29 19:01:40.586960 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:40.586922    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r8j6_ece83ae2-c4e0-4df9-95fd-616c5ab8990e/pull/0.log"
Jul 29 19:01:40.614283 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:40.614256    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rsngv_da62e93b-16c5-4747-b35e-628e7636fcc8/extract/0.log"
Jul 29 19:01:40.662692 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:40.662442    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rsngv_da62e93b-16c5-4747-b35e-628e7636fcc8/util/0.log"
Jul 29 19:01:40.685298 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:40.685269    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rsngv_da62e93b-16c5-4747-b35e-628e7636fcc8/pull/0.log"
Jul 29 19:01:40.715215 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:40.715181    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gjbh_5f32a898-ddb9-4a13-8e36-83c7533a9755/extract/0.log"
Jul 29 19:01:40.735963 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:40.735938    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gjbh_5f32a898-ddb9-4a13-8e36-83c7533a9755/util/0.log"
Jul 29 19:01:40.760428 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:40.760393    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gjbh_5f32a898-ddb9-4a13-8e36-83c7533a9755/pull/0.log"
Jul 29 19:01:40.790693 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:40.790612    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8brs_26ba8c29-8db7-4faf-bbd9-187f92b54bab/extract/0.log"
Jul 29 19:01:40.814994 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:40.814967    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8brs_26ba8c29-8db7-4faf-bbd9-187f92b54bab/util/0.log"
Jul 29 19:01:40.840658 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:40.840631    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8brs_26ba8c29-8db7-4faf-bbd9-187f92b54bab/pull/0.log"
Jul 29 19:01:41.359296 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:41.359272    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-l8cgv_d1921098-bba3-4024-9220-fdefd73975ea/manager/0.log"
Jul 29 19:01:41.573122 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:41.573047    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-xvr6r_fa3433e1-e499-4bb0-803d-97906bc639f2/manager/0.log"
Jul 29 19:01:43.532917 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:43.532872    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-rgj9m_427a5e33-ee40-4ef0-a223-32a6ba9e52cb/cluster-monitoring-operator/0.log"
Jul 29 19:01:43.696838 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:43.696775    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8x4dk_5e92be4a-f9b0-4f36-8a4d-09ae165c21e4/node-exporter/0.log"
Jul 29 19:01:43.716991 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:43.716962    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8x4dk_5e92be4a-f9b0-4f36-8a4d-09ae165c21e4/kube-rbac-proxy/0.log"
Jul 29 19:01:43.742700 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:43.742672    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8x4dk_5e92be4a-f9b0-4f36-8a4d-09ae165c21e4/init-textfile/0.log"
Jul 29 19:01:44.174338 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:44.174296    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-cshwb_527c7cfd-bdea-432e-886a-ccdefd744e0a/prometheus-operator-admission-webhook/0.log"
Jul 29 19:01:45.497030 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:45.496995    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-grlrl_3aad4b60-ee91-42d3-bbf1-865219602575/networking-console-plugin/0.log"
Jul 29 19:01:45.606059 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:45.606015    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5k8jh/perf-node-gather-daemonset-8lk2j"]
Jul 29 19:01:45.613761 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:45.613706    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-8lk2j"
Jul 29 19:01:45.619547 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:45.619508    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5k8jh/perf-node-gather-daemonset-8lk2j"]
Jul 29 19:01:45.669525 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:45.669479    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/08e99fa4-f349-401c-9079-541a437a66d3-lib-modules\") pod \"perf-node-gather-daemonset-8lk2j\" (UID: \"08e99fa4-f349-401c-9079-541a437a66d3\") " pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-8lk2j"
Jul 29 19:01:45.669738 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:45.669567    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/08e99fa4-f349-401c-9079-541a437a66d3-proc\") pod \"perf-node-gather-daemonset-8lk2j\" (UID: \"08e99fa4-f349-401c-9079-541a437a66d3\") " pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-8lk2j"
Jul 29 19:01:45.669738 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:45.669595    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/08e99fa4-f349-401c-9079-541a437a66d3-sys\") pod \"perf-node-gather-daemonset-8lk2j\" (UID: \"08e99fa4-f349-401c-9079-541a437a66d3\") " pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-8lk2j"
Jul 29 19:01:45.669738 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:45.669651    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzhp7\" (UniqueName: \"kubernetes.io/projected/08e99fa4-f349-401c-9079-541a437a66d3-kube-api-access-rzhp7\") pod \"perf-node-gather-daemonset-8lk2j\" (UID: \"08e99fa4-f349-401c-9079-541a437a66d3\") " pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-8lk2j"
Jul 29 19:01:45.669738 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:45.669696    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/08e99fa4-f349-401c-9079-541a437a66d3-podres\") pod \"perf-node-gather-daemonset-8lk2j\" (UID: \"08e99fa4-f349-401c-9079-541a437a66d3\") " pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-8lk2j"
Jul 29 19:01:45.770925 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:45.770840    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzhp7\" (UniqueName: \"kubernetes.io/projected/08e99fa4-f349-401c-9079-541a437a66d3-kube-api-access-rzhp7\") pod \"perf-node-gather-daemonset-8lk2j\" (UID: \"08e99fa4-f349-401c-9079-541a437a66d3\") " pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-8lk2j"
Jul 29 19:01:45.771203 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:45.771184    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/08e99fa4-f349-401c-9079-541a437a66d3-podres\") pod \"perf-node-gather-daemonset-8lk2j\" (UID: \"08e99fa4-f349-401c-9079-541a437a66d3\") " pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-8lk2j"
Jul 29 19:01:45.771590 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:45.771567    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/08e99fa4-f349-401c-9079-541a437a66d3-lib-modules\") pod \"perf-node-gather-daemonset-8lk2j\" (UID: \"08e99fa4-f349-401c-9079-541a437a66d3\") " pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-8lk2j"
Jul 29 19:01:45.771920 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:45.771902    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/08e99fa4-f349-401c-9079-541a437a66d3-proc\") pod \"perf-node-gather-daemonset-8lk2j\" (UID: \"08e99fa4-f349-401c-9079-541a437a66d3\") " pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-8lk2j"
Jul 29 19:01:45.772151 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:45.772134    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/08e99fa4-f349-401c-9079-541a437a66d3-sys\") pod \"perf-node-gather-daemonset-8lk2j\" (UID: \"08e99fa4-f349-401c-9079-541a437a66d3\") " pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-8lk2j"
Jul 29 19:01:45.772383 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:45.771426    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/08e99fa4-f349-401c-9079-541a437a66d3-podres\") pod \"perf-node-gather-daemonset-8lk2j\" (UID: \"08e99fa4-f349-401c-9079-541a437a66d3\") " pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-8lk2j"
Jul 29 19:01:45.772503 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:45.772089    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/08e99fa4-f349-401c-9079-541a437a66d3-proc\") pod \"perf-node-gather-daemonset-8lk2j\" (UID: \"08e99fa4-f349-401c-9079-541a437a66d3\") " pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-8lk2j"
Jul 29 19:01:45.772604 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:45.772288    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/08e99fa4-f349-401c-9079-541a437a66d3-sys\") pod \"perf-node-gather-daemonset-8lk2j\" (UID: \"08e99fa4-f349-401c-9079-541a437a66d3\") " pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-8lk2j"
Jul 29 19:01:45.772703 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:45.771818    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/08e99fa4-f349-401c-9079-541a437a66d3-lib-modules\") pod \"perf-node-gather-daemonset-8lk2j\" (UID: \"08e99fa4-f349-401c-9079-541a437a66d3\") " pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-8lk2j"
Jul 29 19:01:45.778855 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:45.778802    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzhp7\" (UniqueName: \"kubernetes.io/projected/08e99fa4-f349-401c-9079-541a437a66d3-kube-api-access-rzhp7\") pod \"perf-node-gather-daemonset-8lk2j\" (UID: \"08e99fa4-f349-401c-9079-541a437a66d3\") " pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-8lk2j"
Jul 29 19:01:45.930700 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:45.930664    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-8lk2j"
Jul 29 19:01:46.119229 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:46.119196    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5k8jh/perf-node-gather-daemonset-8lk2j"]
Jul 29 19:01:46.119635 ip-10-0-142-156 kubenswrapper[2557]: W0729 19:01:46.119606    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod08e99fa4_f349_401c_9079_541a437a66d3.slice/crio-8ffb471ca4db872d393ed664066ca294d595b03778d49b491b104d3e8f63999b WatchSource:0}: Error finding container 8ffb471ca4db872d393ed664066ca294d595b03778d49b491b104d3e8f63999b: Status 404 returned error can't find the container with id 8ffb471ca4db872d393ed664066ca294d595b03778d49b491b104d3e8f63999b
Jul 29 19:01:46.131071 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:46.130957    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gg2wj_2bf0bb71-4047-45a7-bd8e-5d6efabd26bd/console-operator/2.log"
Jul 29 19:01:46.144241 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:46.144174    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gg2wj_2bf0bb71-4047-45a7-bd8e-5d6efabd26bd/console-operator/3.log"
Jul 29 19:01:46.326824 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:46.326705    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-8lk2j" event={"ID":"08e99fa4-f349-401c-9079-541a437a66d3","Type":"ContainerStarted","Data":"e16a91eb3ed83183c691d14ff551fdfcc0ed37dc012b882816e3587b9e2bf102"}
Jul 29 19:01:46.326824 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:46.326803    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-8lk2j" event={"ID":"08e99fa4-f349-401c-9079-541a437a66d3","Type":"ContainerStarted","Data":"8ffb471ca4db872d393ed664066ca294d595b03778d49b491b104d3e8f63999b"}
Jul 29 19:01:46.327270 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:46.327242    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-8lk2j"
Jul 29 19:01:46.354292 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:46.354210    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-8lk2j" podStartSLOduration=1.354188444 podStartE2EDuration="1.354188444s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 19:01:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:01:46.34440222 +0000 UTC m=+4186.213625027" watchObservedRunningTime="2026-07-29 19:01:46.354188444 +0000 UTC m=+4186.223411252"
Jul 29 19:01:46.611610 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:46.611512    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-54dbc47d44-n8w42_5d8bb6be-286e-4058-a065-e8c037015e0c/console/0.log"
Jul 29 19:01:47.242244 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:47.242209    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-549c6d54cf-m2x44_6554ff13-9d69-4062-be7c-a8d3ff442945/volume-data-source-validator/0.log"
Jul 29 19:01:48.342535 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:48.342508    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bwjgj_dedf5ed1-ad33-4658-b79c-ae80134b18c7/dns/0.log"
Jul 29 19:01:48.362959 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:48.362925    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bwjgj_dedf5ed1-ad33-4658-b79c-ae80134b18c7/kube-rbac-proxy/0.log"
Jul 29 19:01:48.430745 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:48.430700    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5wxd2_85585baa-2c99-4feb-91ec-4a34a98df21b/dns-node-resolver/0.log"
Jul 29 19:01:49.044926 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:49.044901    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vmhq9_8ef211d7-e236-4476-b5df-48aadf6e6195/node-ca/0.log"
Jul 29 19:01:49.966634 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:49.966597    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5_920ab95a-edbf-425b-8f80-45937925ae16/istio-proxy/0.log"
Jul 29 19:01:50.273651 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:50.273564    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-j5k56_8b9ee721-eb64-4ab6-a806-c6562ce45797/discovery/0.log"
Jul 29 19:01:50.325163 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:50.325105    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-869f956dcc-fp29f_3877a269-9027-424f-8903-b06036bbbcb2/kube-auth-proxy/0.log"
Jul 29 19:01:50.355570 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:50.355536    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-vzxzx_5bb4f610-8260-474c-bae4-e5d9ed9a5280/istio-proxy/0.log"
Jul 29 19:01:50.412790 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:50.412757    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-vzxzx_5bb4f610-8260-474c-bae4-e5d9ed9a5280/istio-proxy/1.log"
Jul 29 19:01:50.466260 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:50.466229    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-tkpf5_4be56e67-7714-47b7-b0c5-254141f6b6c5/payload-pre-processing/0.log"
Jul 29 19:01:50.508350 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:50.508311    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-p6kkw_0d65e55f-0460-4fd4-b46c-6d0bb6724295/payload-processing/0.log"
Jul 29 19:01:50.536689 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:50.536626    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-676c59fd4f-tvcgb_0da2e7ec-9cb4-42dc-a537-a5b46687ee32/router/0.log"
Jul 29 19:01:51.322064 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:51.322030    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6kfpl_f9eb2a9b-3f29-455c-9cc2-0d3924a4bef2/serve-healthcheck-canary/0.log"
Jul 29 19:01:51.862427 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:51.862356    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-4j8zv_a9042be4-9aa4-4bc5-b9b8-364a7f782353/insights-operator/0.log"
Jul 29 19:01:51.864977 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:51.864948    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-4j8zv_a9042be4-9aa4-4bc5-b9b8-364a7f782353/insights-operator/1.log"
Jul 29 19:01:52.022125 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:52.022095    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wtc5s_ee11bff0-fc98-47a5-9f2b-8a1f2675c98c/kube-rbac-proxy/0.log"
Jul 29 19:01:52.042162 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:52.042132    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wtc5s_ee11bff0-fc98-47a5-9f2b-8a1f2675c98c/exporter/0.log"
Jul 29 19:01:52.063243 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:52.063215    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wtc5s_ee11bff0-fc98-47a5-9f2b-8a1f2675c98c/extractor/0.log"
Jul 29 19:01:52.351547 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:52.351508    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-8lk2j"
Jul 29 19:01:55.161164 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:55.161122    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5c67f77dd8-9kj8h_10654f46-4178-4840-81e6-52d53aacb1ec/manager/0.log"
Jul 29 19:01:55.183277 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:55.183249    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-mdsvb_dad1b1dc-9f1a-4fbd-bbbc-860e74a95cd2/server/0.log"
Jul 29 19:01:55.295892 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:55.295857    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-672fv_5645b830-d2f9-4164-8e78-a7be8c9af2b5/manager/0.log"
Jul 29 19:01:55.475418 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:55.475332    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d84c6c79c-qq57z_49eb35f6-74a3-42ba-b8de-d69123427363/manager/0.log"
Jul 29 19:01:55.505970 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:55.505934    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d84c6c79c-qq57z_49eb35f6-74a3-42ba-b8de-d69123427363/copy-manifests/0.log"
Jul 29 19:01:56.672660 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:56.672629    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5_920ab95a-edbf-425b-8f80-45937925ae16/istio-proxy/0.log"
Jul 29 19:01:56.952476 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:56.952433    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-j5k56_8b9ee721-eb64-4ab6-a806-c6562ce45797/discovery/0.log"
Jul 29 19:01:56.984128 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:56.984089    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-869f956dcc-fp29f_3877a269-9027-424f-8903-b06036bbbcb2/kube-auth-proxy/0.log"
Jul 29 19:01:57.002101 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:57.002068    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-vzxzx_5bb4f610-8260-474c-bae4-e5d9ed9a5280/istio-proxy/0.log"
Jul 29 19:01:57.061812 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:57.061772    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-vzxzx_5bb4f610-8260-474c-bae4-e5d9ed9a5280/istio-proxy/1.log"
Jul 29 19:01:57.093849 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:57.093818    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-tkpf5_4be56e67-7714-47b7-b0c5-254141f6b6c5/payload-pre-processing/0.log"
Jul 29 19:01:57.124014 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:57.123965    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-p6kkw_0d65e55f-0460-4fd4-b46c-6d0bb6724295/payload-processing/0.log"
Jul 29 19:01:57.139946 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:57.139920    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-676c59fd4f-tvcgb_0da2e7ec-9cb4-42dc-a537-a5b46687ee32/router/0.log"
Jul 29 19:01:58.456342 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:58.456314    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-xnpd5_920ab95a-edbf-425b-8f80-45937925ae16/istio-proxy/0.log"
Jul 29 19:01:58.727780 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:58.727643    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-j5k56_8b9ee721-eb64-4ab6-a806-c6562ce45797/discovery/0.log"
Jul 29 19:01:58.760990 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:58.760961    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-869f956dcc-fp29f_3877a269-9027-424f-8903-b06036bbbcb2/kube-auth-proxy/0.log"
Jul 29 19:01:58.781935 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:58.781896    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-vzxzx_5bb4f610-8260-474c-bae4-e5d9ed9a5280/istio-proxy/0.log"
Jul 29 19:01:58.840813 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:58.840715    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-vzxzx_5bb4f610-8260-474c-bae4-e5d9ed9a5280/istio-proxy/1.log"
Jul 29 19:01:58.874008 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:58.873973    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-tkpf5_4be56e67-7714-47b7-b0c5-254141f6b6c5/payload-pre-processing/0.log"
Jul 29 19:01:58.905283 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:58.905255    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-p6kkw_0d65e55f-0460-4fd4-b46c-6d0bb6724295/payload-processing/0.log"
Jul 29 19:01:58.925063 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:01:58.925033    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-676c59fd4f-tvcgb_0da2e7ec-9cb4-42dc-a537-a5b46687ee32/router/0.log"
Jul 29 19:02:01.127347 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:02:01.126296    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gg2wj_2bf0bb71-4047-45a7-bd8e-5d6efabd26bd/console-operator/2.log"
Jul 29 19:02:01.139306 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:02:01.139278    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gg2wj_2bf0bb71-4047-45a7-bd8e-5d6efabd26bd/console-operator/2.log"
Jul 29 19:02:02.163351 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:02:02.163314    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-hhwbp_a91229a9-7326-45f0-9bed-57c0f1868780/kube-storage-version-migrator-operator/1.log"
Jul 29 19:02:02.165530 ip-10-0-142-156 kubenswrapper[2557]: I0729 19:02:02.165503    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-hhwbp_a91229a9-7326-45f0-9bed-57c0f1868780/kube-storage-version-migrator-operator/0.log"