Jul 28 17:32:20.383968 ip-10-0-136-126 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 28 17:32:20.383977 ip-10-0-136-126 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 28 17:32:20.383984 ip-10-0-136-126 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 28 17:32:20.384189 ip-10-0-136-126 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 28 17:32:30.419837 ip-10-0-136-126 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 28 17:32:30.419851 ip-10-0-136-126 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 411dc07e8c4f48f8bb2389d2200b1e9d --
Jul 28 17:34:21.214728 ip-10-0-136-126 systemd[1]: Starting Kubernetes Kubelet...
Jul 28 17:34:21.627662 ip-10-0-136-126 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 17:34:21.627662 ip-10-0-136-126 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 28 17:34:21.627662 ip-10-0-136-126 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 17:34:21.627662 ip-10-0-136-126 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 28 17:34:21.627662 ip-10-0-136-126 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 17:34:21.629392 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.629324    2563 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 28 17:34:21.634699 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634677    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 17:34:21.634699 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634695    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 17:34:21.634699 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634700    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 17:34:21.634699 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634704    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 17:34:21.634924 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634708    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 17:34:21.634924 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634712    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 17:34:21.634924 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634716    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 17:34:21.634924 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634719    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 17:34:21.634924 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634723    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 17:34:21.634924 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634726    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 17:34:21.634924 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634731    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 17:34:21.634924 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634737    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 17:34:21.634924 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634741    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 17:34:21.634924 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634745    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 17:34:21.634924 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634749    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 17:34:21.634924 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634752    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 17:34:21.634924 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634756    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 17:34:21.634924 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634759    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 17:34:21.634924 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634763    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 17:34:21.634924 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634766    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 17:34:21.634924 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634770    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 17:34:21.634924 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634773    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 17:34:21.634924 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634777    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 17:34:21.635691 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634780    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 17:34:21.635691 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634784    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 17:34:21.635691 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634787    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 17:34:21.635691 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634793    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 17:34:21.635691 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634798    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 17:34:21.635691 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634802    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 17:34:21.635691 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634806    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 17:34:21.635691 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634810    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 17:34:21.635691 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634813    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 17:34:21.635691 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634826    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 17:34:21.635691 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634831    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 17:34:21.635691 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634834    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 17:34:21.635691 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634838    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 17:34:21.635691 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634842    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 17:34:21.635691 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634847    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 17:34:21.635691 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634851    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 17:34:21.635691 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634857    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 17:34:21.635691 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634860    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 17:34:21.635691 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634864    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 17:34:21.635691 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634868    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 17:34:21.636481 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634872    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 17:34:21.636481 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634876    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 17:34:21.636481 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634880    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 17:34:21.636481 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634883    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 17:34:21.636481 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634887    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 17:34:21.636481 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634893    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 17:34:21.636481 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634899    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 17:34:21.636481 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634903    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 17:34:21.636481 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634907    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 17:34:21.636481 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634911    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 17:34:21.636481 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634915    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 17:34:21.636481 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634919    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 17:34:21.636481 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634922    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 17:34:21.636481 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634926    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 17:34:21.636481 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634930    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 17:34:21.636481 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634934    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 17:34:21.636481 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634938    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 17:34:21.636481 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634942    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 17:34:21.636481 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634946    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 17:34:21.637200 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634950    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 17:34:21.637200 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634954    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 17:34:21.637200 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634958    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 17:34:21.637200 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634962    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 17:34:21.637200 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634966    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 17:34:21.637200 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634970    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 17:34:21.637200 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634974    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 17:34:21.637200 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634978    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 17:34:21.637200 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634982    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 17:34:21.637200 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634986    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 17:34:21.637200 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634991    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 17:34:21.637200 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634994    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 17:34:21.637200 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.634999    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 17:34:21.637200 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.635002    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 17:34:21.637200 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.635006    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 17:34:21.637200 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.635010    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 17:34:21.637200 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.635014    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 17:34:21.637200 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.635017    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 17:34:21.637200 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.635021    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 17:34:21.637985 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.635025    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 17:34:21.637985 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.635031    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 17:34:21.637985 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.635036    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 17:34:21.637985 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.635040    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 17:34:21.637985 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.635044    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 17:34:21.637985 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.635048    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 17:34:21.637985 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.635052    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 17:34:21.637985 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.635056    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 17:34:21.637985 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.635060    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 17:34:21.637985 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.635064    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 17:34:21.637985 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.635068    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 17:34:21.637985 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.635072    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 17:34:21.637985 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.635076    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 17:34:21.637985 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.635079    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 17:34:21.637985 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.635083    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 17:34:21.637985 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.635087    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 17:34:21.637985 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.635090    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 17:34:21.637985 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.635094    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 17:34:21.637985 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636212    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 17:34:21.637985 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636221    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 17:34:21.638809 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636226    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 17:34:21.638809 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636230    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 17:34:21.638809 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636234    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 17:34:21.638809 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636238    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 17:34:21.638809 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636243    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 17:34:21.638809 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636250    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 17:34:21.638809 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636255    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 17:34:21.638809 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636260    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 17:34:21.638809 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636264    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 17:34:21.638809 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636269    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 17:34:21.638809 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636273    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 17:34:21.638809 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636278    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 17:34:21.638809 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636282    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 17:34:21.638809 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636286    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 17:34:21.638809 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636290    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 17:34:21.638809 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636295    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 17:34:21.638809 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636299    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 17:34:21.638809 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636305    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 17:34:21.638809 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636310    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 17:34:21.639387 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636314    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 17:34:21.639387 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636318    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 17:34:21.639387 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636323    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 17:34:21.639387 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636327    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 17:34:21.639387 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636331    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 17:34:21.639387 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636335    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 17:34:21.639387 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636339    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 17:34:21.639387 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636344    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 17:34:21.639387 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636357    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 17:34:21.639387 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636362    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 17:34:21.639387 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636366    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 17:34:21.639387 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636370    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 17:34:21.639387 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636374    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 17:34:21.639387 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636378    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 17:34:21.639387 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636382    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 17:34:21.639387 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636385    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 17:34:21.639387 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636390    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 17:34:21.639387 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636393    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 17:34:21.639387 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636397    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 17:34:21.639923 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636401    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 17:34:21.639923 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636405    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 17:34:21.639923 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636409    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 17:34:21.639923 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636414    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 17:34:21.639923 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636417    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 17:34:21.639923 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636421    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 17:34:21.639923 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636442    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 17:34:21.639923 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636446    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 17:34:21.639923 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636450    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 17:34:21.639923 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636454    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 17:34:21.639923 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636459    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 17:34:21.639923 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636463    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 17:34:21.639923 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636467    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 17:34:21.639923 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636471    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 17:34:21.639923 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636475    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 17:34:21.639923 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636478    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 17:34:21.639923 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636483    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 17:34:21.639923 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636486    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 17:34:21.639923 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636491    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 17:34:21.639923 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636495    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 17:34:21.640382 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636498    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 17:34:21.640382 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636504    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 17:34:21.640382 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636508    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 17:34:21.640382 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636512    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 17:34:21.640382 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636516    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 17:34:21.640382 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636520    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 17:34:21.640382 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636525    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 17:34:21.640382 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636529    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 17:34:21.640382 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636533    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 17:34:21.640382 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636537    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 17:34:21.640382 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636541    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 17:34:21.640382 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636544    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 17:34:21.640382 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636548    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 17:34:21.640382 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636552    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 17:34:21.640382 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636556    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 17:34:21.640382 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636560    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 17:34:21.640382 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636564    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 17:34:21.640382 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636568    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 17:34:21.640382 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636572    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 17:34:21.640382 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636576    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 17:34:21.641095 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636580    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 17:34:21.641095 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636583    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 17:34:21.641095 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636587    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 17:34:21.641095 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636593    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 17:34:21.641095 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636597    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 17:34:21.641095 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636601    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 17:34:21.641095 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636605    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 17:34:21.641095 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636609    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 17:34:21.641095 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636613    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 17:34:21.641095 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636617    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 17:34:21.641095 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636621    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 17:34:21.641095 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636625    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 17:34:21.641095 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636629    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 17:34:21.641095 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636632    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 17:34:21.641095 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636636    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 17:34:21.641095 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636643    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 17:34:21.641095 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636648    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 17:34:21.641095 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636652    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 17:34:21.641095 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.636657    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 17:34:21.641566 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637212    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 17:34:21.641566 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637220    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 17:34:21.641566 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637225    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 17:34:21.641566 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637230    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 17:34:21.641566 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637235    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 17:34:21.641566 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637239    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 17:34:21.641566 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637244    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 17:34:21.641566 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637247    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 17:34:21.641566 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637251    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 17:34:21.641566 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637255    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 17:34:21.641566 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637259    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 17:34:21.641566 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637262    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 17:34:21.641566 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637266    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 17:34:21.641566 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637270    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 17:34:21.641566 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637274    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 17:34:21.641566 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637278    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 17:34:21.641566 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637282    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 17:34:21.641566 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637286    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 17:34:21.641566 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637290    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 17:34:21.642057 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637294    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 17:34:21.642057 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637297    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 17:34:21.642057 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637301    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 17:34:21.642057 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637305    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 17:34:21.642057 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637309    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 17:34:21.642057 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637313    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 17:34:21.642057 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637317    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 17:34:21.642057 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637321    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 17:34:21.642057 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637325    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 17:34:21.642057 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637329    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 17:34:21.642057 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637333    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 17:34:21.642057 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637337    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 17:34:21.642057 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637341    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 17:34:21.642057 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637346    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 17:34:21.642057 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637359    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 17:34:21.642057 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637363    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 17:34:21.642057 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637367    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 17:34:21.642057 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637370    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 17:34:21.642057 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637374    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 17:34:21.642057 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637378    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 17:34:21.642582 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637382    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 17:34:21.642582 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637386    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 17:34:21.642582 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637390    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 17:34:21.642582 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637393    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 17:34:21.642582 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637398    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 17:34:21.642582 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637402    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 17:34:21.642582 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637408    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 17:34:21.642582 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637413    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 17:34:21.642582 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637418    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 17:34:21.642582 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637422    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 17:34:21.642582 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637446    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 17:34:21.642582 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637453    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 17:34:21.642582 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637457    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 17:34:21.642582 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637462    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 17:34:21.642582 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637467    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 17:34:21.642582 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637471    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 17:34:21.642582 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637475    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 17:34:21.642582 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637479    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 17:34:21.642582 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637483    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 17:34:21.643032 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637487    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 17:34:21.643032 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637490    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 17:34:21.643032 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637494    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 17:34:21.643032 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637498    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 17:34:21.643032 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637502    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 17:34:21.643032 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637507    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 17:34:21.643032 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637511    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 17:34:21.643032 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637515    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 17:34:21.643032 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637519    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 17:34:21.643032 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637522    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 17:34:21.643032 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637526    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 17:34:21.643032 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637530    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 17:34:21.643032 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637534    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 17:34:21.643032 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637538    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 17:34:21.643032 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637542    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 17:34:21.643032 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637546    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 17:34:21.643032 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637549    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 17:34:21.643032 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637556    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 17:34:21.643032 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637559    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 17:34:21.643032 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637566    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 17:34:21.643523 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637570    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 17:34:21.643523 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637574    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 17:34:21.643523 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637579    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 17:34:21.643523 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637583    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 17:34:21.643523 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637587    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 17:34:21.643523 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637591    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 17:34:21.643523 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637595    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 17:34:21.643523 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637601    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 17:34:21.643523 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637606    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 17:34:21.643523 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637611    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 17:34:21.643523 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637616    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 17:34:21.643523 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637620    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 17:34:21.643523 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637624    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 17:34:21.643523 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637628    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 17:34:21.643523 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637631    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 17:34:21.643523 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637635    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 17:34:21.643523 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637639    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 17:34:21.643523 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637643    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 17:34:21.643523 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637646    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 17:34:21.643523 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637650    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 17:34:21.643973 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.637654    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 17:34:21.643973 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.637740    2563 flags.go:64] FLAG: --address="0.0.0.0"
Jul 28 17:34:21.643973 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.637750    2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 28 17:34:21.643973 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.637760    2563 flags.go:64] FLAG: --anonymous-auth="true"
Jul 28 17:34:21.643973 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.637766    2563 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 28 17:34:21.643973 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.637772    2563 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 28 17:34:21.643973 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.637776    2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 28 17:34:21.643973 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.637782    2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 28 17:34:21.643973 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.637788    2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 28 17:34:21.643973 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.637792    2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 28 17:34:21.643973 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.637797    2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 28 17:34:21.643973 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.637804    2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 28 17:34:21.643973 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.637809    2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 28 17:34:21.643973 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.637814    2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 28 17:34:21.643973 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.637819    2563 flags.go:64] FLAG: --cgroup-root=""
Jul 28 17:34:21.643973 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.637823    2563 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 28 17:34:21.643973 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.637827    2563 flags.go:64] FLAG: --client-ca-file=""
Jul 28 17:34:21.643973 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.637832    2563 flags.go:64] FLAG: --cloud-provider="external"
Jul 28 17:34:21.643973 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.637837    2563 flags.go:64] FLAG: --cluster-dns="[]"
Jul 28 17:34:21.643973 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.637843    2563 flags.go:64] FLAG: --cluster-domain=""
Jul 28 17:34:21.643973 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.637847    2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 28 17:34:21.643973 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.637851    2563 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 28 17:34:21.643973 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.637856    2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 28 17:34:21.643973 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.637861    2563 flags.go:64] FLAG: --container-log-max-files="5"
Jul 28 17:34:21.644583 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.637874    2563 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 28 17:34:21.644583 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.637878    2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 28 17:34:21.644583 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.637883    2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 28 17:34:21.644583 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.637888    2563 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 28 17:34:21.644583 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.637893    2563 flags.go:64] FLAG: --contention-profiling="false"
Jul 28 17:34:21.644583 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.637897    2563 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 28 17:34:21.644583 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.637901    2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 28 17:34:21.644583 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.637906    2563 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 28 17:34:21.644583 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.637910    2563 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 28 17:34:21.644583 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.637916    2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 28 17:34:21.644583 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.637920    2563 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 28 17:34:21.644583 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.637925    2563 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 28 17:34:21.644583 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.637929    2563 flags.go:64] FLAG: --enable-load-reader="false"
Jul 28 17:34:21.644583 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.637933    2563 flags.go:64] FLAG: --enable-server="true"
Jul 28 17:34:21.644583 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.637938    2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 28 17:34:21.644583 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.637945    2563 flags.go:64] FLAG: --event-burst="100"
Jul 28 17:34:21.644583 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.637949    2563 flags.go:64] FLAG: --event-qps="50"
Jul 28 17:34:21.644583 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.637953    2563 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 28 17:34:21.644583 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.637958    2563 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 28 17:34:21.644583 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.637964    2563 flags.go:64] FLAG: --eviction-hard=""
Jul 28 17:34:21.644583 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.637970    2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 28 17:34:21.644583 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.637974    2563 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 28 17:34:21.644583 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.637980    2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 28 17:34:21.644583 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.637984    2563 flags.go:64] FLAG: --eviction-soft=""
Jul 28 17:34:21.644583 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.637988    2563 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 28 17:34:21.645143 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.637992    2563 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 28 17:34:21.645143 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.637997    2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 28 17:34:21.645143 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638001    2563 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 28 17:34:21.645143 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638005    2563 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 28 17:34:21.645143 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638009    2563 flags.go:64] FLAG: --fail-swap-on="true"
Jul 28 17:34:21.645143 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638013    2563 flags.go:64] FLAG: --feature-gates=""
Jul 28 17:34:21.645143 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638019    2563 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 28 17:34:21.645143 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638024    2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 28 17:34:21.645143 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638028    2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 28 17:34:21.645143 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638033    2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 28 17:34:21.645143 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638037    2563 flags.go:64] FLAG: --healthz-port="10248"
Jul 28 17:34:21.645143 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638042    2563 flags.go:64] FLAG: --help="false"
Jul 28 17:34:21.645143 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638046    2563 flags.go:64] FLAG: --hostname-override="ip-10-0-136-126.ec2.internal"
Jul 28 17:34:21.645143 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638050    2563 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 28 17:34:21.645143 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638055    2563 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 28 17:34:21.645143 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638059    2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 28 17:34:21.645143 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638064    2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 28 17:34:21.645143 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638069    2563 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 28 17:34:21.645143 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638074    2563 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 28 17:34:21.645143 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638078    2563 flags.go:64] FLAG: --image-service-endpoint=""
Jul 28 17:34:21.645143 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638082    2563 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 28 17:34:21.645143 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638086    2563 flags.go:64] FLAG: --kube-api-burst="100"
Jul 28 17:34:21.645143 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638090    2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 28 17:34:21.645768 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638095    2563 flags.go:64] FLAG: --kube-api-qps="50"
Jul 28 17:34:21.645768 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638099    2563 flags.go:64] FLAG: --kube-reserved=""
Jul 28 17:34:21.645768 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638104    2563 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 28 17:34:21.645768 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638108    2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 28 17:34:21.645768 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638113    2563 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 28 17:34:21.645768 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638117    2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 28 17:34:21.645768 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638121    2563 flags.go:64] FLAG: --lock-file=""
Jul 28 17:34:21.645768 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638125    2563 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 28 17:34:21.645768 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638129    2563 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 28 17:34:21.645768 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638134    2563 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 28 17:34:21.645768 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638142    2563 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 28 17:34:21.645768 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638147    2563 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 28 17:34:21.645768 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638151    2563 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 28 17:34:21.645768 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638155    2563 flags.go:64] FLAG: --logging-format="text"
Jul 28 17:34:21.645768 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638159    2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 28 17:34:21.645768 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638164    2563 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 28 17:34:21.645768 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638168    2563 flags.go:64] FLAG: --manifest-url=""
Jul 28 17:34:21.645768 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638172    2563 flags.go:64] FLAG: --manifest-url-header=""
Jul 28 17:34:21.645768 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638178    2563 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 28 17:34:21.645768 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638183    2563 flags.go:64] FLAG: --max-open-files="1000000"
Jul 28 17:34:21.645768 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638188    2563 flags.go:64] FLAG: --max-pods="110"
Jul 28 17:34:21.645768 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638192    2563 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 28 17:34:21.645768 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638197    2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 28 17:34:21.645768 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638201    2563 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 28 17:34:21.645768 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638205    2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 28 17:34:21.646318 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638210    2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 28 17:34:21.646318 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638214    2563 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 28 17:34:21.646318 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638218    2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 28 17:34:21.646318 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638228    2563 flags.go:64] FLAG: --node-status-max-images="50"
Jul 28 17:34:21.646318 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638232    2563 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 28 17:34:21.646318 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638237    2563 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 28 17:34:21.646318 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638241    2563 flags.go:64] FLAG: --pod-cidr=""
Jul 28 17:34:21.646318 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638245    2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 28 17:34:21.646318 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638252    2563 flags.go:64] FLAG: --pod-manifest-path=""
Jul 28 17:34:21.646318 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638256    2563 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 28 17:34:21.646318 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638261    2563 flags.go:64] FLAG: --pods-per-core="0"
Jul 28 17:34:21.646318 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638265    2563 flags.go:64] FLAG: --port="10250"
Jul 28 17:34:21.646318 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638270    2563 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 28 17:34:21.646318 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638274    2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0cd5de67d40bddd89"
Jul 28 17:34:21.646318 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638279    2563 flags.go:64] FLAG: --qos-reserved=""
Jul 28 17:34:21.646318 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638283    2563 flags.go:64] FLAG: --read-only-port="10255"
Jul 28 17:34:21.646318 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638288    2563 flags.go:64] FLAG: --register-node="true"
Jul 28 17:34:21.646318 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638292    2563 flags.go:64] FLAG: --register-with-taints=""
Jul 28 17:34:21.646318 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638298    2563 flags.go:64] FLAG: --registry-burst="10"
Jul 28 17:34:21.646318 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638303    2563 flags.go:64] FLAG: --registry-qps="5"
Jul 28 17:34:21.646318 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638308    2563 flags.go:64] FLAG: --reserved-cpus=""
Jul 28 17:34:21.646318 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638312    2563 flags.go:64] FLAG: --reserved-memory=""
Jul 28 17:34:21.646318 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638317    2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 28 17:34:21.646318 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638322    2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 28 17:34:21.646318 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638326    2563 flags.go:64] FLAG: --rotate-certificates="false"
Jul 28 17:34:21.646942 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638330    2563 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 28 17:34:21.646942 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638335    2563 flags.go:64] FLAG: --runonce="false"
Jul 28 17:34:21.646942 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638339    2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 28 17:34:21.646942 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638344    2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 28 17:34:21.646942 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638356    2563 flags.go:64] FLAG: --seccomp-default="false"
Jul 28 17:34:21.646942 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638360    2563 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 28 17:34:21.646942 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638365    2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 28 17:34:21.646942 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638369    2563 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 28 17:34:21.646942 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638374    2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 28 17:34:21.646942 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638378    2563 flags.go:64] FLAG: --storage-driver-password="root"
Jul 28 17:34:21.646942 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638383    2563 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 28 17:34:21.646942 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638387    2563 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 28 17:34:21.646942 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638391    2563 flags.go:64] FLAG: --storage-driver-user="root"
Jul 28 17:34:21.646942 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638395    2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 28 17:34:21.646942 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638400    2563 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 28 17:34:21.646942 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638404    2563 flags.go:64] FLAG: --system-cgroups=""
Jul 28 17:34:21.646942 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638408    2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 28 17:34:21.646942 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638416    2563 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 28 17:34:21.646942 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638421    2563 flags.go:64] FLAG: --tls-cert-file=""
Jul 28 17:34:21.646942 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638443    2563 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 28 17:34:21.646942 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638449    2563 flags.go:64] FLAG: --tls-min-version=""
Jul 28 17:34:21.646942 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638454    2563 flags.go:64] FLAG: --tls-private-key-file=""
Jul 28 17:34:21.646942 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638458    2563 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 28 17:34:21.646942 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638462    2563 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 28 17:34:21.646942 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638466    2563 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 28 17:34:21.647531 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638470    2563 flags.go:64] FLAG: --v="2"
Jul 28 17:34:21.647531 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638476    2563 flags.go:64] FLAG: --version="false"
Jul 28 17:34:21.647531 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638483    2563 flags.go:64] FLAG: --vmodule=""
Jul 28 17:34:21.647531 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638488    2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 28 17:34:21.647531 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.638493    2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 28 17:34:21.647531 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638653    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 17:34:21.647531 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638659    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 17:34:21.647531 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638663    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 17:34:21.647531 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638668    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 17:34:21.647531 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638672    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 17:34:21.647531 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638676    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 17:34:21.647531 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638680    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 17:34:21.647531 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638684    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 17:34:21.647531 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638687    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 17:34:21.647531 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638691    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 17:34:21.647531 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638695    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 17:34:21.647531 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638699    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 17:34:21.647531 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638703    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 17:34:21.647531 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638707    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 17:34:21.647531 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638711    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 17:34:21.647531 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638715    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 17:34:21.647991 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638718    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 17:34:21.647991 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638722    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 17:34:21.647991 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638726    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 17:34:21.647991 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638729    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 17:34:21.647991 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638733    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 17:34:21.647991 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638737    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 17:34:21.647991 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638741    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 17:34:21.647991 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638745    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 17:34:21.647991 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638749    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 17:34:21.647991 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638752    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 17:34:21.647991 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638756    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 17:34:21.647991 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638760    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 17:34:21.647991 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638764    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 17:34:21.647991 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638768    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 17:34:21.647991 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638773    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 17:34:21.647991 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638777    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 17:34:21.647991 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638781    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 17:34:21.647991 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638785    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 17:34:21.647991 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638789    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 17:34:21.647991 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638792    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 17:34:21.648459 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638798    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 17:34:21.648459 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638804    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 17:34:21.648459 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638807    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 17:34:21.648459 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638811    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 17:34:21.648459 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638814    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 17:34:21.648459 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638818    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 17:34:21.648459 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638821    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 17:34:21.648459 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638825    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 17:34:21.648459 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638828    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 17:34:21.648459 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638832    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 17:34:21.648459 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638838    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 17:34:21.648459 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638842    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 17:34:21.648459 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638846    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 17:34:21.648459 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638850    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 17:34:21.648459 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638853    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 17:34:21.648459 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638859    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 17:34:21.648459 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638862    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 17:34:21.648459 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638866    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 17:34:21.648459 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638870    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 17:34:21.648894 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638874    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 17:34:21.648894 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638878    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 17:34:21.648894 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638882    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 17:34:21.648894 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638886    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 17:34:21.648894 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638890    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 17:34:21.648894 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638894    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 17:34:21.648894 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638897    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 17:34:21.648894 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638901    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 17:34:21.648894 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638912    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 17:34:21.648894 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638917    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 17:34:21.648894 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638922    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 17:34:21.648894 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638926    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 17:34:21.648894 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638930    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 17:34:21.648894 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638935    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 17:34:21.648894 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638939    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 17:34:21.648894 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638943    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 17:34:21.648894 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638947    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 17:34:21.648894 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638951    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 17:34:21.648894 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638955    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 17:34:21.649316 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638958    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 17:34:21.649316 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638962    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 17:34:21.649316 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638966    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 17:34:21.649316 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638970    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 17:34:21.649316 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638975    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 17:34:21.649316 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638979    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 17:34:21.649316 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638983    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 17:34:21.649316 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638986    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 17:34:21.649316 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638990    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 17:34:21.649316 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638995    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 17:34:21.649316 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.638999    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 17:34:21.649316 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.639003    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 17:34:21.649316 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.639008    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 17:34:21.649316 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.639011    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 17:34:21.649316 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.639015    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 17:34:21.649316 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.639019    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 17:34:21.649316 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.639022    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 17:34:21.649316 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.639028    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 17:34:21.649316 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.639034    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 17:34:21.649316 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.639040    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 17:34:21.649789 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.639044    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 17:34:21.649789 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.639047    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 17:34:21.649789 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.639052    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 17:34:21.649789 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.639055    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 17:34:21.649789 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.639059    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 17:34:21.649789 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.639638    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 17:34:21.649789 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.645894    2563 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 28 17:34:21.649789 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.645908    2563 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 28 17:34:21.649789 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.646988    2563 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 28 17:34:21.649789 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.646994    2563 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 28 17:34:21.649789 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647042    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 17:34:21.649789 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647047    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 17:34:21.649789 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647050    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 17:34:21.649789 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647053    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 17:34:21.650110 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647055    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 17:34:21.650110 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647058    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 17:34:21.650110 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647061    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 17:34:21.650110 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647063    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 17:34:21.650110 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647066    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 17:34:21.650110 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647068    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 17:34:21.650110 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647071    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 17:34:21.650110 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647073    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 17:34:21.650110 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647075    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 17:34:21.650110 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647078    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 17:34:21.650110 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647080    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 17:34:21.650110 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647082    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 17:34:21.650110 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647085    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 17:34:21.650110 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647087    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 17:34:21.650110 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647089    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 17:34:21.650110 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647091    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 17:34:21.650110 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647094    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 17:34:21.650110 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647096    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 17:34:21.650110 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647099    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 17:34:21.650110 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647101    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 17:34:21.650622 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647103    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 17:34:21.650622 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647106    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 17:34:21.650622 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647108    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 17:34:21.650622 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647110    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 17:34:21.650622 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647113    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 17:34:21.650622 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647116    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 17:34:21.650622 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647119    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 17:34:21.650622 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647123    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 17:34:21.650622 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647126    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 17:34:21.650622 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647128    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 17:34:21.650622 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647131    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 17:34:21.650622 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647133    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 17:34:21.650622 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647136    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 17:34:21.650622 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647138    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 17:34:21.650622 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647141    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 17:34:21.650622 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647143    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 17:34:21.650622 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647146    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 17:34:21.650622 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647148    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 17:34:21.650622 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647151    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 17:34:21.651044 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647153    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 17:34:21.651044 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647155    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 17:34:21.651044 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647158    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 17:34:21.651044 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647160    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 17:34:21.651044 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647162    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 17:34:21.651044 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647164    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 17:34:21.651044 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647167    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 17:34:21.651044 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647169    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 17:34:21.651044 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647172    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 17:34:21.651044 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647174    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 17:34:21.651044 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647176    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 17:34:21.651044 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647179    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 17:34:21.651044 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647181    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 17:34:21.651044 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647183    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 17:34:21.651044 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647185    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 17:34:21.651044 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647188    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 17:34:21.651044 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647190    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 17:34:21.651044 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647192    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 17:34:21.651044 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647195    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 17:34:21.651044 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647198    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 17:34:21.651518 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647200    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 17:34:21.651518 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647204    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 17:34:21.651518 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647207    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 17:34:21.651518 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647210    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 17:34:21.651518 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647212    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 17:34:21.651518 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647215    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 17:34:21.651518 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647217    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 17:34:21.651518 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647219    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 17:34:21.651518 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647222    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 17:34:21.651518 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647224    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 17:34:21.651518 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647226    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 17:34:21.651518 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647229    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 17:34:21.651518 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647231    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 17:34:21.651518 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647233    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 17:34:21.651518 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647236    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 17:34:21.651518 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647238    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 17:34:21.651518 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647240    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 17:34:21.651518 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647242    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 17:34:21.651518 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647245    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 17:34:21.651946 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647247    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 17:34:21.651946 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647249    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 17:34:21.651946 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647252    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 17:34:21.651946 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647254    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 17:34:21.651946 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647258    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 17:34:21.651946 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647261    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 17:34:21.651946 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647263    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 17:34:21.651946 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647266    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 17:34:21.651946 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647268    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 17:34:21.651946 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647271    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 17:34:21.651946 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647273    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 17:34:21.651946 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647275    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 17:34:21.651946 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647278    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 17:34:21.651946 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647280    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 17:34:21.651946 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647283    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 17:34:21.651946 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647285    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 17:34:21.651946 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647287    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 17:34:21.652320 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.647292    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 17:34:21.652320 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647389    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 17:34:21.652320 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647392    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 17:34:21.652320 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647400    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 17:34:21.652320 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647402    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 17:34:21.652320 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647405    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 17:34:21.652320 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647407    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 17:34:21.652320 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647410    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 17:34:21.652320 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647413    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 17:34:21.652320 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647415    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 17:34:21.652320 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647418    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 17:34:21.652320 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647420    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 17:34:21.652320 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647422    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 17:34:21.652320 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647425    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 17:34:21.652320 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647427    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 17:34:21.652684 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647443    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 17:34:21.652684 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647445    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 17:34:21.652684 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647448    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 17:34:21.652684 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647450    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 17:34:21.652684 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647452    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 17:34:21.652684 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647455    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 17:34:21.652684 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647457    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 17:34:21.652684 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647459    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 17:34:21.652684 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647462    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 17:34:21.652684 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647464    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 17:34:21.652684 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647467    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 17:34:21.652684 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647469    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 17:34:21.652684 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647471    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 17:34:21.652684 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647474    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 17:34:21.652684 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647477    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 17:34:21.652684 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647479    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 17:34:21.652684 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647481    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 17:34:21.652684 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647483    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 17:34:21.652684 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647486    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 17:34:21.652684 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647488    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 17:34:21.653118 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647490    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 17:34:21.653118 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647493    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 17:34:21.653118 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647495    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 17:34:21.653118 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647497    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 17:34:21.653118 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647499    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 17:34:21.653118 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647502    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 17:34:21.653118 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647504    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 17:34:21.653118 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647507    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 17:34:21.653118 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647509    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 17:34:21.653118 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647511    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 17:34:21.653118 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647513    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 17:34:21.653118 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647516    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 17:34:21.653118 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647518    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 17:34:21.653118 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647520    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 17:34:21.653118 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647522    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 17:34:21.653118 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647525    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 17:34:21.653118 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647527    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 17:34:21.653118 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647529    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 17:34:21.653118 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647531    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 17:34:21.653118 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647534    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 17:34:21.653598 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647536    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 17:34:21.653598 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647538    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 17:34:21.653598 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647541    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 17:34:21.653598 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647543    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 17:34:21.653598 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647546    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 17:34:21.653598 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647548    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 17:34:21.653598 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647551    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 17:34:21.653598 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647554    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 17:34:21.653598 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647556    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 17:34:21.653598 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647559    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 17:34:21.653598 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647561    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 17:34:21.653598 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647563    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 17:34:21.653598 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647566    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 17:34:21.653598 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647569    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 17:34:21.653598 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647572    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 17:34:21.653598 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647576    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 17:34:21.653598 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647578    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 17:34:21.653598 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647581    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 17:34:21.653598 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647584    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 17:34:21.654010 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647586    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 17:34:21.654010 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647589    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 17:34:21.654010 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647591    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 17:34:21.654010 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647593    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 17:34:21.654010 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647595    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 17:34:21.654010 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647598    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 17:34:21.654010 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647601    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 17:34:21.654010 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647604    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 17:34:21.654010 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647607    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 17:34:21.654010 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647609    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 17:34:21.654010 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647612    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 17:34:21.654010 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647614    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 17:34:21.654010 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647616    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 17:34:21.654010 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647619    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 17:34:21.654010 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647621    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 17:34:21.654010 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647624    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 17:34:21.654010 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647626    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 17:34:21.654010 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647628    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 17:34:21.654010 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647631    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 17:34:21.654010 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647633    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 17:34:21.654475 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647636    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 17:34:21.654475 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647638    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 17:34:21.654475 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647641    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 17:34:21.654475 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647643    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 17:34:21.654475 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647645    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 17:34:21.654475 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:21.647647    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 17:34:21.654475 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.647652    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 17:34:21.654475 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.647755    2563 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 28 17:34:21.654475 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.649507    2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 28 17:34:21.654475 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.650852    2563 server.go:1019] "Starting client certificate rotation"
Jul 28 17:34:21.654475 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.650942    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 17:34:21.654475 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.650965    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 17:34:21.654749 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.654564    2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 17:34:21.659393 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.659376    2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 17:34:21.672808 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.672787    2563 log.go:25] "Validated CRI v1 runtime API"
Jul 28 17:34:21.674588 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.674575    2563 log.go:25] "Validated CRI v1 image API"
Jul 28 17:34:21.675328 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.675316    2563 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 28 17:34:21.677529 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.677505    2563 fs.go:135] Filesystem UUIDs: map[6f0edc9e-496a-4c19-b201-86a59699c4d9:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 9fa26bfe-0135-4ee3-9e8e-44cac6874ec6:/dev/nvme0n1p4]
Jul 28 17:34:21.677598 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.677528    2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 28 17:34:21.681621 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.681604    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 17:34:21.683008 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.682910    2563 manager.go:216] Machine: {Timestamp:2026-07-28 17:34:21.681624448 +0000 UTC m=+0.342780791 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099825 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d72cb7dedd08a141fb5267ac33b73 SystemUUID:ec2d72cb-7ded-d08a-141f-b5267ac33b73 BootID:411dc07e-8c4f-48f8-bb23-89d2200b1e9d Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:86:b6:58:09 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:86:b6:58:09 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:26:d2:df:24:b3:74 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 28 17:34:21.683008 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.683001    2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 28 17:34:21.683146 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.683065    2563 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 28 17:34:21.686623 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.686601    2563 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 28 17:34:21.686760 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.686626    2563 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-126.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 28 17:34:21.687366 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.687354    2563 topology_manager.go:138] "Creating topology manager with none policy"
Jul 28 17:34:21.687420 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.687370    2563 container_manager_linux.go:309] "Creating device plugin manager"
Jul 28 17:34:21.687420 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.687379    2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 17:34:21.688059 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.688048    2563 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 17:34:21.688813 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.688803    2563 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 28 17:34:21.690458 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.690446    2563 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 17:34:21.690572 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.690562    2563 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 28 17:34:21.690627 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.690596    2563 kubelet.go:486] "Attempting to sync node with API server"
Jul 28 17:34:21.690627 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.690609    2563 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 28 17:34:21.690627 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.690622    2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 28 17:34:21.690738 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.690634    2563 kubelet.go:398] "Adding apiserver pod source"
Jul 28 17:34:21.690738 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.690645    2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 28 17:34:21.690941 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.690929    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 17:34:21.690999 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.690948    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 17:34:21.692875 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.692850    2563 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 28 17:34:21.695972 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.695949    2563 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 28 17:34:21.696059 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.695992    2563 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 28 17:34:21.698916 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.698898    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 28 17:34:21.698996 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.698920    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 28 17:34:21.698996 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.698927    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 28 17:34:21.698996 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.698935    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 28 17:34:21.698996 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.698947    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 28 17:34:21.698996 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.698954    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 28 17:34:21.698996 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.698962    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 28 17:34:21.698996 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.698966    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 28 17:34:21.698996 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.698973    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 28 17:34:21.698996 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.698978    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 28 17:34:21.698996 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.698988    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 28 17:34:21.698996 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.698993    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 28 17:34:21.699924 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.699914    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 28 17:34:21.699969 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.699925    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 28 17:34:21.700477 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:21.700457    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-126.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 28 17:34:21.700538 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:21.700489    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 28 17:34:21.702589 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.702577    2563 server.go:1268] "Started kubelet"
Jul 28 17:34:21.702684 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.702662    2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 28 17:34:21.702735 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.702652    2563 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 28 17:34:21.702735 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.702712    2563 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 28 17:34:21.703221 ip-10-0-136-126 systemd[1]: Started Kubernetes Kubelet.
Jul 28 17:34:21.703821 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.703705    2563 server.go:310] "Adding debug handlers to kubelet server"
Jul 28 17:34:21.704335 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.704311    2563 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 28 17:34:21.707777 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:21.707729    2563 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 28 17:34:21.707866 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.707854    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-126.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 28 17:34:21.708767 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:21.707858    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-126.ec2.internal.18c6849141790471  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-126.ec2.internal,UID:ip-10-0-136-126.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-126.ec2.internal,},FirstTimestamp:2026-07-28 17:34:21.702554737 +0000 UTC m=+0.363711080,LastTimestamp:2026-07-28 17:34:21.702554737 +0000 UTC m=+0.363711080,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-126.ec2.internal,}"
Jul 28 17:34:21.710538 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.710525    2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 28 17:34:21.710621 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.710576    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 28 17:34:21.710621 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:21.710604    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-126.ec2.internal\" not found"
Jul 28 17:34:21.711069 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.711052    2563 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 28 17:34:21.711138 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.711078    2563 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 28 17:34:21.711138 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.711092    2563 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 28 17:34:21.711236 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.711178    2563 reconstruct.go:97] "Volume reconstruction finished"
Jul 28 17:34:21.711236 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.711187    2563 reconciler.go:29] "Reconciler: start to sync state"
Jul 28 17:34:21.711329 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.711312    2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 28 17:34:21.711329 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.711328    2563 factory.go:55] Registering systemd factory
Jul 28 17:34:21.711504 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.711337    2563 factory.go:223] Registration of the systemd container factory successfully
Jul 28 17:34:21.711642 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.711627    2563 factory.go:153] Registering CRI-O factory
Jul 28 17:34:21.711716 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.711648    2563 factory.go:223] Registration of the crio container factory successfully
Jul 28 17:34:21.711716 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.711667    2563 factory.go:103] Registering Raw factory
Jul 28 17:34:21.711716 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.711676    2563 manager.go:1196] Started watching for new ooms in manager
Jul 28 17:34:21.712006 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.711996    2563 manager.go:349] Starting recovery of all containers
Jul 28 17:34:21.714641 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:21.714617    2563 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-126.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 28 17:34:21.714701 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:21.714666    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 28 17:34:21.723053 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.723030    2563 manager.go:354] Recovery completed
Jul 28 17:34:21.724268 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:21.724244    2563 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": readdirent /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Jul 28 17:34:21.727011 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.727000    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 17:34:21.728626 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.728607    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-126.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 17:34:21.728711 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.728641    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-126.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 17:34:21.728711 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.728655    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-126.ec2.internal" event="NodeHasSufficientPID"
Jul 28 17:34:21.729112 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.729095    2563 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 28 17:34:21.729112 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.729106    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 17:34:21.729224 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.729110    2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 28 17:34:21.729224 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.729145    2563 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 17:34:21.729793 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.729780    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-126.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 17:34:21.729855 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.729802    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-126.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 17:34:21.729855 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.729812    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-126.ec2.internal" event="NodeHasSufficientPID"
Jul 28 17:34:21.730073 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:21.730015    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-126.ec2.internal.18c684914306d62b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-126.ec2.internal,UID:ip-10-0-136-126.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-126.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-126.ec2.internal,},FirstTimestamp:2026-07-28 17:34:21.728626219 +0000 UTC m=+0.389782565,LastTimestamp:2026-07-28 17:34:21.728626219 +0000 UTC m=+0.389782565,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-126.ec2.internal,}"
Jul 28 17:34:21.731159 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.731148    2563 policy_none.go:49] "None policy: Start"
Jul 28 17:34:21.731205 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.731162    2563 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 28 17:34:21.731205 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.731171    2563 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 28 17:34:21.732065 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.732055    2563 policy_none.go:47] "Start"
Jul 28 17:34:21.739483 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:21.739410    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-126.ec2.internal.18c6849143072a26  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-126.ec2.internal,UID:ip-10-0-136-126.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-136-126.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-136-126.ec2.internal,},FirstTimestamp:2026-07-28 17:34:21.728647718 +0000 UTC m=+0.389804067,LastTimestamp:2026-07-28 17:34:21.728647718 +0000 UTC m=+0.389804067,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-126.ec2.internal,}"
Jul 28 17:34:21.740308 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.740295    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rgp6q"
Jul 28 17:34:21.748056 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:21.748008    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-126.ec2.internal.18c6849143075b4e  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-126.ec2.internal,UID:ip-10-0-136-126.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-136-126.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-136-126.ec2.internal,},FirstTimestamp:2026-07-28 17:34:21.728660302 +0000 UTC m=+0.389816654,LastTimestamp:2026-07-28 17:34:21.728660302 +0000 UTC m=+0.389816654,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-126.ec2.internal,}"
Jul 28 17:34:21.749535 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.749520    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rgp6q"
Jul 28 17:34:21.755858 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:21.755694    2563 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-136-126.ec2.internal.18c684914306d62b\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-126.ec2.internal.18c684914306d62b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-126.ec2.internal,UID:ip-10-0-136-126.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-126.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-126.ec2.internal,},FirstTimestamp:2026-07-28 17:34:21.728626219 +0000 UTC m=+0.389782565,LastTimestamp:2026-07-28 17:34:21.72979193 +0000 UTC m=+0.390948273,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-126.ec2.internal,}"
Jul 28 17:34:21.766481 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.766470    2563 manager.go:337] "Starting Device Plugin manager"
Jul 28 17:34:21.766533 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:21.766494    2563 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 28 17:34:21.766533 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.766502    2563 server.go:87] "Starting device plugin registration server"
Jul 28 17:34:21.766688 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.766680    2563 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 28 17:34:21.766747 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.766690    2563 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 28 17:34:21.766800 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.766788    2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 28 17:34:21.766879 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.766866    2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 28 17:34:21.766879 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.766879    2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 28 17:34:21.767305 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:21.767286    2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 28 17:34:21.767370 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:21.767338    2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-126.ec2.internal\" not found"
Jul 28 17:34:21.845730 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.845702    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 28 17:34:21.847042 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.846758    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 28 17:34:21.847042 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.846799    2563 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 28 17:34:21.847042 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.846820    2563 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 28 17:34:21.847042 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:21.846860    2563 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 28 17:34:21.848951 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.848935    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 17:34:21.867716 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.867699    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 17:34:21.872728 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.872711    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-126.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 17:34:21.872806 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.872740    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-126.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 17:34:21.872806 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.872750    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-126.ec2.internal" event="NodeHasSufficientPID"
Jul 28 17:34:21.872806 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.872771    2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-126.ec2.internal"
Jul 28 17:34:21.880887 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.880874    2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-126.ec2.internal"
Jul 28 17:34:21.880943 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:21.880897    2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-126.ec2.internal\": node \"ip-10-0-136-126.ec2.internal\" not found"
Jul 28 17:34:21.896535 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:21.896513    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-126.ec2.internal\" not found"
Jul 28 17:34:21.947882 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.947859    2563 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-126.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-126.ec2.internal"]
Jul 28 17:34:21.947977 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.947927    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 17:34:21.948776 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.948763    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-126.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 17:34:21.948838 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.948786    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-126.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 17:34:21.948838 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.948795    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-126.ec2.internal" event="NodeHasSufficientPID"
Jul 28 17:34:21.950217 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.950202    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 17:34:21.950331 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.950318    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-126.ec2.internal"
Jul 28 17:34:21.950390 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.950357    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 17:34:21.950836 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.950823    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-126.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 17:34:21.950899 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.950837    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-126.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 17:34:21.950899 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.950849    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-126.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 17:34:21.950899 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.950858    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-126.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 17:34:21.950899 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.950864    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-126.ec2.internal" event="NodeHasSufficientPID"
Jul 28 17:34:21.950899 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.950870    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-126.ec2.internal" event="NodeHasSufficientPID"
Jul 28 17:34:21.953271 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.953254    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-126.ec2.internal"
Jul 28 17:34:21.953351 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.953280    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 17:34:21.954118 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.954103    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-126.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 17:34:21.954199 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.954129    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-126.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 17:34:21.954199 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:21.954140    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-126.ec2.internal" event="NodeHasSufficientPID"
Jul 28 17:34:21.988962 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:21.988945    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-126.ec2.internal\" not found" node="ip-10-0-136-126.ec2.internal"
Jul 28 17:34:21.992900 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:21.992882    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-126.ec2.internal\" not found" node="ip-10-0-136-126.ec2.internal"
Jul 28 17:34:21.997169 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:21.997148    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-126.ec2.internal\" not found"
Jul 28 17:34:22.098115 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:22.098084    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-126.ec2.internal\" not found"
Jul 28 17:34:22.112313 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:22.112290    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/17457ce2fa8ddf46113d3222bb973d5a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-126.ec2.internal\" (UID: \"17457ce2fa8ddf46113d3222bb973d5a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-126.ec2.internal"
Jul 28 17:34:22.112400 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:22.112333    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/17457ce2fa8ddf46113d3222bb973d5a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-126.ec2.internal\" (UID: \"17457ce2fa8ddf46113d3222bb973d5a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-126.ec2.internal"
Jul 28 17:34:22.112400 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:22.112357    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bf90ceb74d2562be9fba76df0fbf255c-config\") pod \"kube-apiserver-proxy-ip-10-0-136-126.ec2.internal\" (UID: \"bf90ceb74d2562be9fba76df0fbf255c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-126.ec2.internal"
Jul 28 17:34:22.198606 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:22.198579    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-126.ec2.internal\" not found"
Jul 28 17:34:22.212756 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:22.212739    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/17457ce2fa8ddf46113d3222bb973d5a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-126.ec2.internal\" (UID: \"17457ce2fa8ddf46113d3222bb973d5a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-126.ec2.internal"
Jul 28 17:34:22.212810 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:22.212761    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bf90ceb74d2562be9fba76df0fbf255c-config\") pod \"kube-apiserver-proxy-ip-10-0-136-126.ec2.internal\" (UID: \"bf90ceb74d2562be9fba76df0fbf255c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-126.ec2.internal"
Jul 28 17:34:22.212810 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:22.212779    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/17457ce2fa8ddf46113d3222bb973d5a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-126.ec2.internal\" (UID: \"17457ce2fa8ddf46113d3222bb973d5a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-126.ec2.internal"
Jul 28 17:34:22.212891 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:22.212813    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/17457ce2fa8ddf46113d3222bb973d5a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-126.ec2.internal\" (UID: \"17457ce2fa8ddf46113d3222bb973d5a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-126.ec2.internal"
Jul 28 17:34:22.212891 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:22.212843    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bf90ceb74d2562be9fba76df0fbf255c-config\") pod \"kube-apiserver-proxy-ip-10-0-136-126.ec2.internal\" (UID: \"bf90ceb74d2562be9fba76df0fbf255c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-126.ec2.internal"
Jul 28 17:34:22.212891 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:22.212844    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/17457ce2fa8ddf46113d3222bb973d5a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-126.ec2.internal\" (UID: \"17457ce2fa8ddf46113d3222bb973d5a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-126.ec2.internal"
Jul 28 17:34:22.292888 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:22.292848    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-126.ec2.internal"
Jul 28 17:34:22.297243 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:22.296739    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-126.ec2.internal"
Jul 28 17:34:22.298826 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:22.298808    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-126.ec2.internal\" not found"
Jul 28 17:34:22.399308 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:22.399221    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-126.ec2.internal\" not found"
Jul 28 17:34:22.499666 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:22.499628    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-126.ec2.internal\" not found"
Jul 28 17:34:22.600063 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:22.600035    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-126.ec2.internal\" not found"
Jul 28 17:34:22.650493 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:22.650411    2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 28 17:34:22.650885 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:22.650563    2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 28 17:34:22.700813 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:22.700781    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-126.ec2.internal\" not found"
Jul 28 17:34:22.755974 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:22.755919    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-27 17:29:21 +0000 UTC" deadline="2028-02-18 03:11:43.634255332 +0000 UTC"
Jul 28 17:34:22.755974 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:22.755971    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13665h37m20.878287717s"
Jul 28 17:34:22.759064 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:22.759041    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 17:34:22.760658 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:22.760642    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-126.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 17:34:22.760763 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:22.760673    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-126.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 17:34:22.760763 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:22.760684    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-126.ec2.internal" event="NodeHasSufficientPID"
Jul 28 17:34:22.763764 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:22.763744    2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 17:34:22.801066 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:22.801038    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-126.ec2.internal\" not found"
Jul 28 17:34:22.855880 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:22.855860    2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 17:34:22.863162 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:22.863131    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17457ce2fa8ddf46113d3222bb973d5a.slice/crio-c3cf6ff2ef0e07ad3a5391148514f22a10e903d630ff6d4eb799251f6d70b59f WatchSource:0}: Error finding container c3cf6ff2ef0e07ad3a5391148514f22a10e903d630ff6d4eb799251f6d70b59f: Status 404 returned error can't find the container with id c3cf6ff2ef0e07ad3a5391148514f22a10e903d630ff6d4eb799251f6d70b59f
Jul 28 17:34:22.863325 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:22.863306    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf90ceb74d2562be9fba76df0fbf255c.slice/crio-35140f5d4ba8c63db08a260c4833b14b7695ed8655b76edd9ff47193651842fc WatchSource:0}: Error finding container 35140f5d4ba8c63db08a260c4833b14b7695ed8655b76edd9ff47193651842fc: Status 404 returned error can't find the container with id 35140f5d4ba8c63db08a260c4833b14b7695ed8655b76edd9ff47193651842fc
Jul 28 17:34:22.870539 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:22.870523    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 17:34:22.901497 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:22.901416    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-126.ec2.internal\" not found"
Jul 28 17:34:23.001791 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:23.001765    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-126.ec2.internal\" not found"
Jul 28 17:34:23.034337 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.034317    2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 17:34:23.111153 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.111130    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-126.ec2.internal"
Jul 28 17:34:23.120228 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.120211    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 17:34:23.121211 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.121199    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-126.ec2.internal"
Jul 28 17:34:23.136309 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.136291    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 17:34:23.691745 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.691711    2563 apiserver.go:52] "Watching apiserver"
Jul 28 17:34:23.701228 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.701201    2563 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 28 17:34:23.702231 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.702205    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-2qvs8","kube-system/kube-apiserver-proxy-ip-10-0-136-126.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t7h6","openshift-dns/node-resolver-j4wkp","openshift-multus/multus-additional-cni-plugins-x2h5c","openshift-cluster-node-tuning-operator/tuned-25z5w","openshift-image-registry/node-ca-z8fhq","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-126.ec2.internal","openshift-multus/multus-8mhf9","openshift-multus/network-metrics-daemon-4tthg","openshift-network-diagnostics/network-check-target-74vbl","openshift-network-operator/iptables-alerter-nwwzc","openshift-ovn-kubernetes/ovnkube-node-krssz","kube-system/global-pull-secret-syncer-z75m5"]
Jul 28 17:34:23.704056 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.704037    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-z8fhq"
Jul 28 17:34:23.705404 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.705378    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.706696 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.706680    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t7h6"
Jul 28 17:34:23.707053 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.707037    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 28 17:34:23.707222 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.707207    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-jdtlk\""
Jul 28 17:34:23.707391 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.707040    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 28 17:34:23.707675 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.707658    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 28 17:34:23.707808 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.707765    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 28 17:34:23.707917 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.707900    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 28 17:34:23.707995 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.707962    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-fwsnf\""
Jul 28 17:34:23.708071 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.708056    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 28 17:34:23.709969 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.708884    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-j4wkp"
Jul 28 17:34:23.709969 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.709482    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 28 17:34:23.709969 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.709757    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 28 17:34:23.709969 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.709916    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 28 17:34:23.710625 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.710599    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 28 17:34:23.710815 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.710796    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-x2h5c"
Jul 28 17:34:23.711598 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.711582    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 28 17:34:23.712138 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.712119    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 28 17:34:23.713552 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.713535    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-88zs2\""
Jul 28 17:34:23.714387 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.714371    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 28 17:34:23.714529 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.714516    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 28 17:34:23.714815 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.714798    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-25z5w"
Jul 28 17:34:23.715499 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.715273    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 28 17:34:23.717308 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.716367    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-2qcbp\""
Jul 28 17:34:23.717308 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.716564    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 28 17:34:23.717308 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.716756    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 28 17:34:23.717308 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.716810    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 28 17:34:23.717308 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.716756    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 28 17:34:23.717308 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.717031    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 28 17:34:23.717308 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.717057    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-sn5x6\""
Jul 28 17:34:23.717698 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.717479    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-nzlx8\""
Jul 28 17:34:23.717749 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.717734    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8mhf9"
Jul 28 17:34:23.718638 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.718580    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/502adcd9-be65-4041-8d71-97c51f0af4a4-run-systemd\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.718638 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.718595    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 28 17:34:23.718638 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.718614    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-x2h5c\" (UID: \"3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b\") " pod="openshift-multus/multus-additional-cni-plugins-x2h5c"
Jul 28 17:34:23.718930 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.718641    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 28 17:34:23.718930 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.718648    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddn4j\" (UniqueName: \"kubernetes.io/projected/502adcd9-be65-4041-8d71-97c51f0af4a4-kube-api-access-ddn4j\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.718930 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.718684    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/502adcd9-be65-4041-8d71-97c51f0af4a4-run-openvswitch\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.718930 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.718705    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/502adcd9-be65-4041-8d71-97c51f0af4a4-ovn-node-metrics-cert\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.718930 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.718732    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvt4w\" (UniqueName: \"kubernetes.io/projected/654d1f40-f526-4325-b40c-fb544079bff4-kube-api-access-bvt4w\") pod \"node-resolver-j4wkp\" (UID: \"654d1f40-f526-4325-b40c-fb544079bff4\") " pod="openshift-dns/node-resolver-j4wkp"
Jul 28 17:34:23.718930 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.718753    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-x2h5c\" (UID: \"3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b\") " pod="openshift-multus/multus-additional-cni-plugins-x2h5c"
Jul 28 17:34:23.718930 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.718772    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6f0b63c9-e5d8-4bb4-8244-eefc95d9ff67-registration-dir\") pod \"aws-ebs-csi-driver-node-8t7h6\" (UID: \"6f0b63c9-e5d8-4bb4-8244-eefc95d9ff67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t7h6"
Jul 28 17:34:23.718930 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.718795    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rhmj\" (UniqueName: \"kubernetes.io/projected/3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b-kube-api-access-4rhmj\") pod \"multus-additional-cni-plugins-x2h5c\" (UID: \"3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b\") " pod="openshift-multus/multus-additional-cni-plugins-x2h5c"
Jul 28 17:34:23.718930 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.718864    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/502adcd9-be65-4041-8d71-97c51f0af4a4-systemd-units\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.718930 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.718890    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/502adcd9-be65-4041-8d71-97c51f0af4a4-var-lib-openvswitch\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.718930 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.718905    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/502adcd9-be65-4041-8d71-97c51f0af4a4-etc-openvswitch\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.718930 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.718920    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/502adcd9-be65-4041-8d71-97c51f0af4a4-host-run-ovn-kubernetes\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.719523 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.718939    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b-system-cni-dir\") pod \"multus-additional-cni-plugins-x2h5c\" (UID: \"3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b\") " pod="openshift-multus/multus-additional-cni-plugins-x2h5c"
Jul 28 17:34:23.719523 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.718963    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d6004102-aca2-4f56-8ddb-94374d3bc900-serviceca\") pod \"node-ca-z8fhq\" (UID: \"d6004102-aca2-4f56-8ddb-94374d3bc900\") " pod="openshift-image-registry/node-ca-z8fhq"
Jul 28 17:34:23.719523 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.718988    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/654d1f40-f526-4325-b40c-fb544079bff4-tmp-dir\") pod \"node-resolver-j4wkp\" (UID: \"654d1f40-f526-4325-b40c-fb544079bff4\") " pod="openshift-dns/node-resolver-j4wkp"
Jul 28 17:34:23.719523 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.719011    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/118595be-d78d-40e2-9d5b-937167697690-var-lib-kubelet\") pod \"tuned-25z5w\" (UID: \"118595be-d78d-40e2-9d5b-937167697690\") " pod="openshift-cluster-node-tuning-operator/tuned-25z5w"
Jul 28 17:34:23.719523 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.719033    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b-os-release\") pod \"multus-additional-cni-plugins-x2h5c\" (UID: \"3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b\") " pod="openshift-multus/multus-additional-cni-plugins-x2h5c"
Jul 28 17:34:23.719523 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.719055    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/118595be-d78d-40e2-9d5b-937167697690-run\") pod \"tuned-25z5w\" (UID: \"118595be-d78d-40e2-9d5b-937167697690\") " pod="openshift-cluster-node-tuning-operator/tuned-25z5w"
Jul 28 17:34:23.719523 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.719085    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6f0b63c9-e5d8-4bb4-8244-eefc95d9ff67-socket-dir\") pod \"aws-ebs-csi-driver-node-8t7h6\" (UID: \"6f0b63c9-e5d8-4bb4-8244-eefc95d9ff67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t7h6"
Jul 28 17:34:23.719523 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.719129    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2qvs8"
Jul 28 17:34:23.719523 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.719123    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/502adcd9-be65-4041-8d71-97c51f0af4a4-host-kubelet\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.719523 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.719301    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/502adcd9-be65-4041-8d71-97c51f0af4a4-host-cni-bin\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.719523 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.719329    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/502adcd9-be65-4041-8d71-97c51f0af4a4-env-overrides\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.719523 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.719366    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-x2h5c\" (UID: \"3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b\") " pod="openshift-multus/multus-additional-cni-plugins-x2h5c"
Jul 28 17:34:23.719523 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.719387    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d6004102-aca2-4f56-8ddb-94374d3bc900-host\") pod \"node-ca-z8fhq\" (UID: \"d6004102-aca2-4f56-8ddb-94374d3bc900\") " pod="openshift-image-registry/node-ca-z8fhq"
Jul 28 17:34:23.719523 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.719420    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/654d1f40-f526-4325-b40c-fb544079bff4-hosts-file\") pod \"node-resolver-j4wkp\" (UID: \"654d1f40-f526-4325-b40c-fb544079bff4\") " pod="openshift-dns/node-resolver-j4wkp"
Jul 28 17:34:23.719523 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.719473    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/118595be-d78d-40e2-9d5b-937167697690-etc-sysctl-conf\") pod \"tuned-25z5w\" (UID: \"118595be-d78d-40e2-9d5b-937167697690\") " pod="openshift-cluster-node-tuning-operator/tuned-25z5w"
Jul 28 17:34:23.719523 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.719495    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/118595be-d78d-40e2-9d5b-937167697690-etc-systemd\") pod \"tuned-25z5w\" (UID: \"118595be-d78d-40e2-9d5b-937167697690\") " pod="openshift-cluster-node-tuning-operator/tuned-25z5w"
Jul 28 17:34:23.720165 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.719543    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/118595be-d78d-40e2-9d5b-937167697690-tmp\") pod \"tuned-25z5w\" (UID: \"118595be-d78d-40e2-9d5b-937167697690\") " pod="openshift-cluster-node-tuning-operator/tuned-25z5w"
Jul 28 17:34:23.720165 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.719577    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/502adcd9-be65-4041-8d71-97c51f0af4a4-run-ovn\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.720165 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.719601    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/502adcd9-be65-4041-8d71-97c51f0af4a4-ovnkube-config\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.720165 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.719624    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f49bc\" (UniqueName: \"kubernetes.io/projected/6f0b63c9-e5d8-4bb4-8244-eefc95d9ff67-kube-api-access-f49bc\") pod \"aws-ebs-csi-driver-node-8t7h6\" (UID: \"6f0b63c9-e5d8-4bb4-8244-eefc95d9ff67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t7h6"
Jul 28 17:34:23.720165 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.719645    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b-cnibin\") pod \"multus-additional-cni-plugins-x2h5c\" (UID: \"3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b\") " pod="openshift-multus/multus-additional-cni-plugins-x2h5c"
Jul 28 17:34:23.720165 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.719673    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b-cni-binary-copy\") pod \"multus-additional-cni-plugins-x2h5c\" (UID: \"3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b\") " pod="openshift-multus/multus-additional-cni-plugins-x2h5c"
Jul 28 17:34:23.720165 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.719678    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 28 17:34:23.720165 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.719693    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/502adcd9-be65-4041-8d71-97c51f0af4a4-host-run-netns\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.720165 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.719745    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/118595be-d78d-40e2-9d5b-937167697690-etc-sysctl-d\") pod \"tuned-25z5w\" (UID: \"118595be-d78d-40e2-9d5b-937167697690\") " pod="openshift-cluster-node-tuning-operator/tuned-25z5w"
Jul 28 17:34:23.720165 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.719764    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/118595be-d78d-40e2-9d5b-937167697690-host\") pod \"tuned-25z5w\" (UID: \"118595be-d78d-40e2-9d5b-937167697690\") " pod="openshift-cluster-node-tuning-operator/tuned-25z5w"
Jul 28 17:34:23.720165 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.719802    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/502adcd9-be65-4041-8d71-97c51f0af4a4-host-slash\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.720165 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.719823    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/502adcd9-be65-4041-8d71-97c51f0af4a4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.720165 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.719844    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/118595be-d78d-40e2-9d5b-937167697690-etc-kubernetes\") pod \"tuned-25z5w\" (UID: \"118595be-d78d-40e2-9d5b-937167697690\") " pod="openshift-cluster-node-tuning-operator/tuned-25z5w"
Jul 28 17:34:23.720165 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.719864    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrqm9\" (UniqueName: \"kubernetes.io/projected/118595be-d78d-40e2-9d5b-937167697690-kube-api-access-xrqm9\") pod \"tuned-25z5w\" (UID: \"118595be-d78d-40e2-9d5b-937167697690\") " pod="openshift-cluster-node-tuning-operator/tuned-25z5w"
Jul 28 17:34:23.720165 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.719896    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6f0b63c9-e5d8-4bb4-8244-eefc95d9ff67-etc-selinux\") pod \"aws-ebs-csi-driver-node-8t7h6\" (UID: \"6f0b63c9-e5d8-4bb4-8244-eefc95d9ff67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t7h6"
Jul 28 17:34:23.720165 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.719927    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6f0b63c9-e5d8-4bb4-8244-eefc95d9ff67-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8t7h6\" (UID: \"6f0b63c9-e5d8-4bb4-8244-eefc95d9ff67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t7h6"
Jul 28 17:34:23.720165 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.719950    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/118595be-d78d-40e2-9d5b-937167697690-etc-modprobe-d\") pod \"tuned-25z5w\" (UID: \"118595be-d78d-40e2-9d5b-937167697690\") " pod="openshift-cluster-node-tuning-operator/tuned-25z5w"
Jul 28 17:34:23.720883 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.719979    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-4gtmw\""
Jul 28 17:34:23.720883 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.719985    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/118595be-d78d-40e2-9d5b-937167697690-etc-sysconfig\") pod \"tuned-25z5w\" (UID: \"118595be-d78d-40e2-9d5b-937167697690\") " pod="openshift-cluster-node-tuning-operator/tuned-25z5w"
Jul 28 17:34:23.720883 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.720026    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/118595be-d78d-40e2-9d5b-937167697690-sys\") pod \"tuned-25z5w\" (UID: \"118595be-d78d-40e2-9d5b-937167697690\") " pod="openshift-cluster-node-tuning-operator/tuned-25z5w"
Jul 28 17:34:23.720883 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.720049    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6f0b63c9-e5d8-4bb4-8244-eefc95d9ff67-sys-fs\") pod \"aws-ebs-csi-driver-node-8t7h6\" (UID: \"6f0b63c9-e5d8-4bb4-8244-eefc95d9ff67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t7h6"
Jul 28 17:34:23.720883 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.720069    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/118595be-d78d-40e2-9d5b-937167697690-lib-modules\") pod \"tuned-25z5w\" (UID: \"118595be-d78d-40e2-9d5b-937167697690\") " pod="openshift-cluster-node-tuning-operator/tuned-25z5w"
Jul 28 17:34:23.720883 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.720097    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/118595be-d78d-40e2-9d5b-937167697690-etc-tuned\") pod \"tuned-25z5w\" (UID: \"118595be-d78d-40e2-9d5b-937167697690\") " pod="openshift-cluster-node-tuning-operator/tuned-25z5w"
Jul 28 17:34:23.720883 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.720148    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zbbv\" (UniqueName: \"kubernetes.io/projected/d6004102-aca2-4f56-8ddb-94374d3bc900-kube-api-access-8zbbv\") pod \"node-ca-z8fhq\" (UID: \"d6004102-aca2-4f56-8ddb-94374d3bc900\") " pod="openshift-image-registry/node-ca-z8fhq"
Jul 28 17:34:23.720883 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.720190    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6f0b63c9-e5d8-4bb4-8244-eefc95d9ff67-device-dir\") pod \"aws-ebs-csi-driver-node-8t7h6\" (UID: \"6f0b63c9-e5d8-4bb4-8244-eefc95d9ff67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t7h6"
Jul 28 17:34:23.720883 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.720224    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/502adcd9-be65-4041-8d71-97c51f0af4a4-node-log\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.720883 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.720244    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/502adcd9-be65-4041-8d71-97c51f0af4a4-log-socket\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.720883 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.720269    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/502adcd9-be65-4041-8d71-97c51f0af4a4-host-cni-netd\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.720883 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.720299    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z75m5"
Jul 28 17:34:23.720883 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.720289    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/502adcd9-be65-4041-8d71-97c51f0af4a4-ovnkube-script-lib\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.720883 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:23.720367    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z75m5" podUID="8a8edb3a-34c9-4a66-81aa-49474fdfdf37"
Jul 28 17:34:23.721505 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.721155    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 28 17:34:23.721862 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.721825    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-xvgnl\""
Jul 28 17:34:23.722013 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.721998    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 28 17:34:23.722116 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.722092    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tthg"
Jul 28 17:34:23.722202 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:23.722175    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4tthg" podUID="d3c4a388-fdcc-4744-b6e2-d52dc701f235"
Jul 28 17:34:23.723543 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.723500    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-74vbl"
Jul 28 17:34:23.723722 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:23.723703    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-74vbl" podUID="41d54bc6-a997-48d3-9752-f78204bbdf43"
Jul 28 17:34:23.724142 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.724123    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 17:34:23.724818 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.724632    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nwwzc"
Jul 28 17:34:23.726956 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.726904    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 28 17:34:23.727267 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.727247    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 28 17:34:23.727410 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.727395    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-5hpl6\""
Jul 28 17:34:23.727592 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.727574    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 28 17:34:23.746749 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.746727    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-2hlxw"
Jul 28 17:34:23.754543 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.754520    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-2hlxw"
Jul 28 17:34:23.785686 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.785626    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 17:34:23.811890 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.811866    2563 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 28 17:34:23.820806 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.820783    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvt4w\" (UniqueName: \"kubernetes.io/projected/654d1f40-f526-4325-b40c-fb544079bff4-kube-api-access-bvt4w\") pod \"node-resolver-j4wkp\" (UID: \"654d1f40-f526-4325-b40c-fb544079bff4\") " pod="openshift-dns/node-resolver-j4wkp"
Jul 28 17:34:23.820917 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.820818    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-x2h5c\" (UID: \"3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b\") " pod="openshift-multus/multus-additional-cni-plugins-x2h5c"
Jul 28 17:34:23.820917 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.820844    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ea8d2cd4-eb75-479a-a7fa-730d68fb1cee-system-cni-dir\") pod \"multus-8mhf9\" (UID: \"ea8d2cd4-eb75-479a-a7fa-730d68fb1cee\") " pod="openshift-multus/multus-8mhf9"
Jul 28 17:34:23.820917 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.820865    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ea8d2cd4-eb75-479a-a7fa-730d68fb1cee-host-var-lib-kubelet\") pod \"multus-8mhf9\" (UID: \"ea8d2cd4-eb75-479a-a7fa-730d68fb1cee\") " pod="openshift-multus/multus-8mhf9"
Jul 28 17:34:23.820917 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.820884    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ea8d2cd4-eb75-479a-a7fa-730d68fb1cee-hostroot\") pod \"multus-8mhf9\" (UID: \"ea8d2cd4-eb75-479a-a7fa-730d68fb1cee\") " pod="openshift-multus/multus-8mhf9"
Jul 28 17:34:23.820917 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.820904    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6f0b63c9-e5d8-4bb4-8244-eefc95d9ff67-registration-dir\") pod \"aws-ebs-csi-driver-node-8t7h6\" (UID: \"6f0b63c9-e5d8-4bb4-8244-eefc95d9ff67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t7h6"
Jul 28 17:34:23.821150 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.820927    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rhmj\" (UniqueName: \"kubernetes.io/projected/3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b-kube-api-access-4rhmj\") pod \"multus-additional-cni-plugins-x2h5c\" (UID: \"3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b\") " pod="openshift-multus/multus-additional-cni-plugins-x2h5c"
Jul 28 17:34:23.821150 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.820947    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8a8edb3a-34c9-4a66-81aa-49474fdfdf37-dbus\") pod \"global-pull-secret-syncer-z75m5\" (UID: \"8a8edb3a-34c9-4a66-81aa-49474fdfdf37\") " pod="kube-system/global-pull-secret-syncer-z75m5"
Jul 28 17:34:23.821150 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.820974    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ea8d2cd4-eb75-479a-a7fa-730d68fb1cee-os-release\") pod \"multus-8mhf9\" (UID: \"ea8d2cd4-eb75-479a-a7fa-730d68fb1cee\") " pod="openshift-multus/multus-8mhf9"
Jul 28 17:34:23.821150 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.820993    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ea8d2cd4-eb75-479a-a7fa-730d68fb1cee-host-run-k8s-cni-cncf-io\") pod \"multus-8mhf9\" (UID: \"ea8d2cd4-eb75-479a-a7fa-730d68fb1cee\") " pod="openshift-multus/multus-8mhf9"
Jul 28 17:34:23.821150 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.821016    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/502adcd9-be65-4041-8d71-97c51f0af4a4-systemd-units\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.821150 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.821039    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/502adcd9-be65-4041-8d71-97c51f0af4a4-var-lib-openvswitch\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.821150 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.821061    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/502adcd9-be65-4041-8d71-97c51f0af4a4-etc-openvswitch\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.821150 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.821082    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/502adcd9-be65-4041-8d71-97c51f0af4a4-host-run-ovn-kubernetes\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.821150 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.821105    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b-system-cni-dir\") pod \"multus-additional-cni-plugins-x2h5c\" (UID: \"3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b\") " pod="openshift-multus/multus-additional-cni-plugins-x2h5c"
Jul 28 17:34:23.821150 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.821126    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d6004102-aca2-4f56-8ddb-94374d3bc900-serviceca\") pod \"node-ca-z8fhq\" (UID: \"d6004102-aca2-4f56-8ddb-94374d3bc900\") " pod="openshift-image-registry/node-ca-z8fhq"
Jul 28 17:34:23.821150 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.821148    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8a8edb3a-34c9-4a66-81aa-49474fdfdf37-original-pull-secret\") pod \"global-pull-secret-syncer-z75m5\" (UID: \"8a8edb3a-34c9-4a66-81aa-49474fdfdf37\") " pod="kube-system/global-pull-secret-syncer-z75m5"
Jul 28 17:34:23.821642 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.821169    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcgdp\" (UniqueName: \"kubernetes.io/projected/ea8d2cd4-eb75-479a-a7fa-730d68fb1cee-kube-api-access-vcgdp\") pod \"multus-8mhf9\" (UID: \"ea8d2cd4-eb75-479a-a7fa-730d68fb1cee\") " pod="openshift-multus/multus-8mhf9"
Jul 28 17:34:23.821642 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.821205    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/654d1f40-f526-4325-b40c-fb544079bff4-tmp-dir\") pod \"node-resolver-j4wkp\" (UID: \"654d1f40-f526-4325-b40c-fb544079bff4\") " pod="openshift-dns/node-resolver-j4wkp"
Jul 28 17:34:23.821642 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.821228    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/118595be-d78d-40e2-9d5b-937167697690-var-lib-kubelet\") pod \"tuned-25z5w\" (UID: \"118595be-d78d-40e2-9d5b-937167697690\") " pod="openshift-cluster-node-tuning-operator/tuned-25z5w"
Jul 28 17:34:23.821642 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.821251    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7e2a258e-828b-4bbd-96b8-56cb5a7dd36f-konnectivity-ca\") pod \"konnectivity-agent-2qvs8\" (UID: \"7e2a258e-828b-4bbd-96b8-56cb5a7dd36f\") " pod="kube-system/konnectivity-agent-2qvs8"
Jul 28 17:34:23.821642 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.821274    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b-os-release\") pod \"multus-additional-cni-plugins-x2h5c\" (UID: \"3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b\") " pod="openshift-multus/multus-additional-cni-plugins-x2h5c"
Jul 28 17:34:23.821642 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.821295    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/118595be-d78d-40e2-9d5b-937167697690-run\") pod \"tuned-25z5w\" (UID: \"118595be-d78d-40e2-9d5b-937167697690\") " pod="openshift-cluster-node-tuning-operator/tuned-25z5w"
Jul 28 17:34:23.821642 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.821316    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ea8d2cd4-eb75-479a-a7fa-730d68fb1cee-cnibin\") pod \"multus-8mhf9\" (UID: \"ea8d2cd4-eb75-479a-a7fa-730d68fb1cee\") " pod="openshift-multus/multus-8mhf9"
Jul 28 17:34:23.821642 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.821338    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6f0b63c9-e5d8-4bb4-8244-eefc95d9ff67-socket-dir\") pod \"aws-ebs-csi-driver-node-8t7h6\" (UID: \"6f0b63c9-e5d8-4bb4-8244-eefc95d9ff67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t7h6"
Jul 28 17:34:23.821642 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.821360    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/502adcd9-be65-4041-8d71-97c51f0af4a4-host-kubelet\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.821642 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.821377    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-x2h5c\" (UID: \"3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b\") " pod="openshift-multus/multus-additional-cni-plugins-x2h5c"
Jul 28 17:34:23.821642 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.821392    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/502adcd9-be65-4041-8d71-97c51f0af4a4-host-cni-bin\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.821642 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.821416    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/502adcd9-be65-4041-8d71-97c51f0af4a4-env-overrides\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.821642 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.821464    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-x2h5c\" (UID: \"3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b\") " pod="openshift-multus/multus-additional-cni-plugins-x2h5c"
Jul 28 17:34:23.821642 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.821484    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d6004102-aca2-4f56-8ddb-94374d3bc900-host\") pod \"node-ca-z8fhq\" (UID: \"d6004102-aca2-4f56-8ddb-94374d3bc900\") " pod="openshift-image-registry/node-ca-z8fhq"
Jul 28 17:34:23.821642 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.821467    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b-system-cni-dir\") pod \"multus-additional-cni-plugins-x2h5c\" (UID: \"3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b\") " pod="openshift-multus/multus-additional-cni-plugins-x2h5c"
Jul 28 17:34:23.821642 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.821515    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ea8d2cd4-eb75-479a-a7fa-730d68fb1cee-multus-cni-dir\") pod \"multus-8mhf9\" (UID: \"ea8d2cd4-eb75-479a-a7fa-730d68fb1cee\") " pod="openshift-multus/multus-8mhf9"
Jul 28 17:34:23.821642 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.821533    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ea8d2cd4-eb75-479a-a7fa-730d68fb1cee-multus-conf-dir\") pod \"multus-8mhf9\" (UID: \"ea8d2cd4-eb75-479a-a7fa-730d68fb1cee\") " pod="openshift-multus/multus-8mhf9"
Jul 28 17:34:23.822356 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.821557    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/654d1f40-f526-4325-b40c-fb544079bff4-hosts-file\") pod \"node-resolver-j4wkp\" (UID: \"654d1f40-f526-4325-b40c-fb544079bff4\") " pod="openshift-dns/node-resolver-j4wkp"
Jul 28 17:34:23.822356 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.821576    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/118595be-d78d-40e2-9d5b-937167697690-etc-sysctl-conf\") pod \"tuned-25z5w\" (UID: \"118595be-d78d-40e2-9d5b-937167697690\") " pod="openshift-cluster-node-tuning-operator/tuned-25z5w"
Jul 28 17:34:23.822356 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.821595    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/118595be-d78d-40e2-9d5b-937167697690-etc-systemd\") pod \"tuned-25z5w\" (UID: \"118595be-d78d-40e2-9d5b-937167697690\") " pod="openshift-cluster-node-tuning-operator/tuned-25z5w"
Jul 28 17:34:23.822356 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.821616    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/118595be-d78d-40e2-9d5b-937167697690-tmp\") pod \"tuned-25z5w\" (UID: \"118595be-d78d-40e2-9d5b-937167697690\") " pod="openshift-cluster-node-tuning-operator/tuned-25z5w"
Jul 28 17:34:23.822356 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.821636    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/502adcd9-be65-4041-8d71-97c51f0af4a4-run-ovn\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.822356 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.821656    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/502adcd9-be65-4041-8d71-97c51f0af4a4-ovnkube-config\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.822356 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.821671    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f49bc\" (UniqueName: \"kubernetes.io/projected/6f0b63c9-e5d8-4bb4-8244-eefc95d9ff67-kube-api-access-f49bc\") pod \"aws-ebs-csi-driver-node-8t7h6\" (UID: \"6f0b63c9-e5d8-4bb4-8244-eefc95d9ff67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t7h6"
Jul 28 17:34:23.822356 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.821687    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b-cnibin\") pod \"multus-additional-cni-plugins-x2h5c\" (UID: \"3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b\") " pod="openshift-multus/multus-additional-cni-plugins-x2h5c"
Jul 28 17:34:23.822356 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.821701    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b-cni-binary-copy\") pod \"multus-additional-cni-plugins-x2h5c\" (UID: \"3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b\") " pod="openshift-multus/multus-additional-cni-plugins-x2h5c"
Jul 28 17:34:23.822356 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.821715    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ea8d2cd4-eb75-479a-a7fa-730d68fb1cee-host-run-multus-certs\") pod \"multus-8mhf9\" (UID: \"ea8d2cd4-eb75-479a-a7fa-730d68fb1cee\") " pod="openshift-multus/multus-8mhf9"
Jul 28 17:34:23.822356 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.821731    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wv9jw\" (UniqueName: \"kubernetes.io/projected/a35e7d0f-47ba-4b8a-8987-47f0163cc156-kube-api-access-wv9jw\") pod \"iptables-alerter-nwwzc\" (UID: \"a35e7d0f-47ba-4b8a-8987-47f0163cc156\") " pod="openshift-network-operator/iptables-alerter-nwwzc"
Jul 28 17:34:23.822356 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.821754    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/502adcd9-be65-4041-8d71-97c51f0af4a4-host-run-netns\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.822356 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.821775    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/118595be-d78d-40e2-9d5b-937167697690-etc-sysctl-d\") pod \"tuned-25z5w\" (UID: \"118595be-d78d-40e2-9d5b-937167697690\") " pod="openshift-cluster-node-tuning-operator/tuned-25z5w"
Jul 28 17:34:23.822356 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.821793    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/654d1f40-f526-4325-b40c-fb544079bff4-tmp-dir\") pod \"node-resolver-j4wkp\" (UID: \"654d1f40-f526-4325-b40c-fb544079bff4\") " pod="openshift-dns/node-resolver-j4wkp"
Jul 28 17:34:23.822356 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.821796    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/118595be-d78d-40e2-9d5b-937167697690-host\") pod \"tuned-25z5w\" (UID: \"118595be-d78d-40e2-9d5b-937167697690\") " pod="openshift-cluster-node-tuning-operator/tuned-25z5w"
Jul 28 17:34:23.822356 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.821849    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/118595be-d78d-40e2-9d5b-937167697690-host\") pod \"tuned-25z5w\" (UID: \"118595be-d78d-40e2-9d5b-937167697690\") " pod="openshift-cluster-node-tuning-operator/tuned-25z5w"
Jul 28 17:34:23.822356 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.821848    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ea8d2cd4-eb75-479a-a7fa-730d68fb1cee-host-var-lib-cni-multus\") pod \"multus-8mhf9\" (UID: \"ea8d2cd4-eb75-479a-a7fa-730d68fb1cee\") " pod="openshift-multus/multus-8mhf9"
Jul 28 17:34:23.822974 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.821874    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d6004102-aca2-4f56-8ddb-94374d3bc900-serviceca\") pod \"node-ca-z8fhq\" (UID: \"d6004102-aca2-4f56-8ddb-94374d3bc900\") " pod="openshift-image-registry/node-ca-z8fhq"
Jul 28 17:34:23.822974 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.821878    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/502adcd9-be65-4041-8d71-97c51f0af4a4-host-slash\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.822974 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.821977    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/502adcd9-be65-4041-8d71-97c51f0af4a4-run-ovn\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.822974 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.821986    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d6004102-aca2-4f56-8ddb-94374d3bc900-host\") pod \"node-ca-z8fhq\" (UID: \"d6004102-aca2-4f56-8ddb-94374d3bc900\") " pod="openshift-image-registry/node-ca-z8fhq"
Jul 28 17:34:23.822974 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.822051    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/118595be-d78d-40e2-9d5b-937167697690-etc-sysctl-conf\") pod \"tuned-25z5w\" (UID: \"118595be-d78d-40e2-9d5b-937167697690\") " pod="openshift-cluster-node-tuning-operator/tuned-25z5w"
Jul 28 17:34:23.822974 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.822090    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b-cnibin\") pod \"multus-additional-cni-plugins-x2h5c\" (UID: \"3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b\") " pod="openshift-multus/multus-additional-cni-plugins-x2h5c"
Jul 28 17:34:23.822974 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.822114    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-x2h5c\" (UID: \"3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b\") " pod="openshift-multus/multus-additional-cni-plugins-x2h5c"
Jul 28 17:34:23.822974 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.822124    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/502adcd9-be65-4041-8d71-97c51f0af4a4-etc-openvswitch\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.822974 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.822156    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/502adcd9-be65-4041-8d71-97c51f0af4a4-var-lib-openvswitch\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.822974 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.822187    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/502adcd9-be65-4041-8d71-97c51f0af4a4-host-slash\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.822974 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.822205    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b-os-release\") pod \"multus-additional-cni-plugins-x2h5c\" (UID: \"3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b\") " pod="openshift-multus/multus-additional-cni-plugins-x2h5c"
Jul 28 17:34:23.822974 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.822224    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/502adcd9-be65-4041-8d71-97c51f0af4a4-host-cni-bin\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.822974 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.822265    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6f0b63c9-e5d8-4bb4-8244-eefc95d9ff67-registration-dir\") pod \"aws-ebs-csi-driver-node-8t7h6\" (UID: \"6f0b63c9-e5d8-4bb4-8244-eefc95d9ff67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t7h6"
Jul 28 17:34:23.822974 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.822296    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/502adcd9-be65-4041-8d71-97c51f0af4a4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.822974 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.822320    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/118595be-d78d-40e2-9d5b-937167697690-etc-kubernetes\") pod \"tuned-25z5w\" (UID: \"118595be-d78d-40e2-9d5b-937167697690\") " pod="openshift-cluster-node-tuning-operator/tuned-25z5w"
Jul 28 17:34:23.822974 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.822343    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrqm9\" (UniqueName: \"kubernetes.io/projected/118595be-d78d-40e2-9d5b-937167697690-kube-api-access-xrqm9\") pod \"tuned-25z5w\" (UID: \"118595be-d78d-40e2-9d5b-937167697690\") " pod="openshift-cluster-node-tuning-operator/tuned-25z5w"
Jul 28 17:34:23.822974 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.822377    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ea8d2cd4-eb75-479a-a7fa-730d68fb1cee-etc-kubernetes\") pod \"multus-8mhf9\" (UID: \"ea8d2cd4-eb75-479a-a7fa-730d68fb1cee\") " pod="openshift-multus/multus-8mhf9"
Jul 28 17:34:23.823531 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.822390    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/502adcd9-be65-4041-8d71-97c51f0af4a4-ovnkube-config\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.823531 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.822401    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7smn5\" (UniqueName: \"kubernetes.io/projected/d3c4a388-fdcc-4744-b6e2-d52dc701f235-kube-api-access-7smn5\") pod \"network-metrics-daemon-4tthg\" (UID: \"d3c4a388-fdcc-4744-b6e2-d52dc701f235\") " pod="openshift-multus/network-metrics-daemon-4tthg"
Jul 28 17:34:23.823531 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.822444    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6f0b63c9-e5d8-4bb4-8244-eefc95d9ff67-etc-selinux\") pod \"aws-ebs-csi-driver-node-8t7h6\" (UID: \"6f0b63c9-e5d8-4bb4-8244-eefc95d9ff67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t7h6"
Jul 28 17:34:23.823531 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.822472    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6f0b63c9-e5d8-4bb4-8244-eefc95d9ff67-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8t7h6\" (UID: \"6f0b63c9-e5d8-4bb4-8244-eefc95d9ff67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t7h6"
Jul 28 17:34:23.823531 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.822501    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/118595be-d78d-40e2-9d5b-937167697690-etc-modprobe-d\") pod \"tuned-25z5w\" (UID: \"118595be-d78d-40e2-9d5b-937167697690\") " pod="openshift-cluster-node-tuning-operator/tuned-25z5w"
Jul 28 17:34:23.823531 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.822527    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/118595be-d78d-40e2-9d5b-937167697690-etc-sysconfig\") pod \"tuned-25z5w\" (UID: \"118595be-d78d-40e2-9d5b-937167697690\") " pod="openshift-cluster-node-tuning-operator/tuned-25z5w"
Jul 28 17:34:23.823531 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.822536    2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 28 17:34:23.823531 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.822588    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/118595be-d78d-40e2-9d5b-937167697690-sys\") pod \"tuned-25z5w\" (UID: \"118595be-d78d-40e2-9d5b-937167697690\") " pod="openshift-cluster-node-tuning-operator/tuned-25z5w"
Jul 28 17:34:23.823531 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.822551    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/118595be-d78d-40e2-9d5b-937167697690-sys\") pod \"tuned-25z5w\" (UID: \"118595be-d78d-40e2-9d5b-937167697690\") " pod="openshift-cluster-node-tuning-operator/tuned-25z5w"
Jul 28 17:34:23.823531 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.822692    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/118595be-d78d-40e2-9d5b-937167697690-etc-sysctl-d\") pod \"tuned-25z5w\" (UID: \"118595be-d78d-40e2-9d5b-937167697690\") " pod="openshift-cluster-node-tuning-operator/tuned-25z5w"
Jul 28 17:34:23.823531 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.822716    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a35e7d0f-47ba-4b8a-8987-47f0163cc156-iptables-alerter-script\") pod \"iptables-alerter-nwwzc\" (UID: \"a35e7d0f-47ba-4b8a-8987-47f0163cc156\") " pod="openshift-network-operator/iptables-alerter-nwwzc"
Jul 28 17:34:23.823531 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.822728    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/118595be-d78d-40e2-9d5b-937167697690-etc-systemd\") pod \"tuned-25z5w\" (UID: \"118595be-d78d-40e2-9d5b-937167697690\") " pod="openshift-cluster-node-tuning-operator/tuned-25z5w"
Jul 28 17:34:23.823531 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.822157    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/118595be-d78d-40e2-9d5b-937167697690-var-lib-kubelet\") pod \"tuned-25z5w\" (UID: \"118595be-d78d-40e2-9d5b-937167697690\") " pod="openshift-cluster-node-tuning-operator/tuned-25z5w"
Jul 28 17:34:23.823531 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.822779    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6f0b63c9-e5d8-4bb4-8244-eefc95d9ff67-sys-fs\") pod \"aws-ebs-csi-driver-node-8t7h6\" (UID: \"6f0b63c9-e5d8-4bb4-8244-eefc95d9ff67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t7h6"
Jul 28 17:34:23.823531 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.822786    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/502adcd9-be65-4041-8d71-97c51f0af4a4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.823531 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.822797    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/118595be-d78d-40e2-9d5b-937167697690-lib-modules\") pod \"tuned-25z5w\" (UID: \"118595be-d78d-40e2-9d5b-937167697690\") " pod="openshift-cluster-node-tuning-operator/tuned-25z5w"
Jul 28 17:34:23.823531 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.822817    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/118595be-d78d-40e2-9d5b-937167697690-etc-tuned\") pod \"tuned-25z5w\" (UID: \"118595be-d78d-40e2-9d5b-937167697690\") " pod="openshift-cluster-node-tuning-operator/tuned-25z5w"
Jul 28 17:34:23.824267 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.822836    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zbbv\" (UniqueName: \"kubernetes.io/projected/d6004102-aca2-4f56-8ddb-94374d3bc900-kube-api-access-8zbbv\") pod \"node-ca-z8fhq\" (UID: \"d6004102-aca2-4f56-8ddb-94374d3bc900\") " pod="openshift-image-registry/node-ca-z8fhq"
Jul 28 17:34:23.824267 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.822864    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6f0b63c9-e5d8-4bb4-8244-eefc95d9ff67-device-dir\") pod \"aws-ebs-csi-driver-node-8t7h6\" (UID: \"6f0b63c9-e5d8-4bb4-8244-eefc95d9ff67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t7h6"
Jul 28 17:34:23.824267 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.822891    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6f0b63c9-e5d8-4bb4-8244-eefc95d9ff67-socket-dir\") pod \"aws-ebs-csi-driver-node-8t7h6\" (UID: \"6f0b63c9-e5d8-4bb4-8244-eefc95d9ff67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t7h6"
Jul 28 17:34:23.824267 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.822891    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8a8edb3a-34c9-4a66-81aa-49474fdfdf37-kubelet-config\") pod \"global-pull-secret-syncer-z75m5\" (UID: \"8a8edb3a-34c9-4a66-81aa-49474fdfdf37\") " pod="kube-system/global-pull-secret-syncer-z75m5"
Jul 28 17:34:23.824267 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.822797    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/118595be-d78d-40e2-9d5b-937167697690-etc-kubernetes\") pod \"tuned-25z5w\" (UID: \"118595be-d78d-40e2-9d5b-937167697690\") " pod="openshift-cluster-node-tuning-operator/tuned-25z5w"
Jul 28 17:34:23.824267 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.822939    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/502adcd9-be65-4041-8d71-97c51f0af4a4-systemd-units\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.824267 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.823000    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/502adcd9-be65-4041-8d71-97c51f0af4a4-host-run-ovn-kubernetes\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.824267 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.823038    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/502adcd9-be65-4041-8d71-97c51f0af4a4-host-run-netns\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.824267 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.823043    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/654d1f40-f526-4325-b40c-fb544079bff4-hosts-file\") pod \"node-resolver-j4wkp\" (UID: \"654d1f40-f526-4325-b40c-fb544079bff4\") " pod="openshift-dns/node-resolver-j4wkp"
Jul 28 17:34:23.824267 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.823088    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6f0b63c9-e5d8-4bb4-8244-eefc95d9ff67-sys-fs\") pod \"aws-ebs-csi-driver-node-8t7h6\" (UID: \"6f0b63c9-e5d8-4bb4-8244-eefc95d9ff67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t7h6"
Jul 28 17:34:23.824267 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.823098    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/118595be-d78d-40e2-9d5b-937167697690-etc-modprobe-d\") pod \"tuned-25z5w\" (UID: \"118595be-d78d-40e2-9d5b-937167697690\") " pod="openshift-cluster-node-tuning-operator/tuned-25z5w"
Jul 28 17:34:23.824267 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.823118    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3c4a388-fdcc-4744-b6e2-d52dc701f235-metrics-certs\") pod \"network-metrics-daemon-4tthg\" (UID: \"d3c4a388-fdcc-4744-b6e2-d52dc701f235\") " pod="openshift-multus/network-metrics-daemon-4tthg"
Jul 28 17:34:23.824267 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.823144    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gg7b\" (UniqueName: \"kubernetes.io/projected/41d54bc6-a997-48d3-9752-f78204bbdf43-kube-api-access-4gg7b\") pod \"network-check-target-74vbl\" (UID: \"41d54bc6-a997-48d3-9752-f78204bbdf43\") " pod="openshift-network-diagnostics/network-check-target-74vbl"
Jul 28 17:34:23.824267 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.823169    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/502adcd9-be65-4041-8d71-97c51f0af4a4-node-log\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.824267 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.823192    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/502adcd9-be65-4041-8d71-97c51f0af4a4-log-socket\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.824267 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.823227    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/502adcd9-be65-4041-8d71-97c51f0af4a4-host-cni-netd\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.824267 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.823250    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/502adcd9-be65-4041-8d71-97c51f0af4a4-ovnkube-script-lib\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.824838 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.823274    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ea8d2cd4-eb75-479a-a7fa-730d68fb1cee-multus-socket-dir-parent\") pod \"multus-8mhf9\" (UID: \"ea8d2cd4-eb75-479a-a7fa-730d68fb1cee\") " pod="openshift-multus/multus-8mhf9"
Jul 28 17:34:23.824838 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.823298    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ea8d2cd4-eb75-479a-a7fa-730d68fb1cee-host-run-netns\") pod \"multus-8mhf9\" (UID: \"ea8d2cd4-eb75-479a-a7fa-730d68fb1cee\") " pod="openshift-multus/multus-8mhf9"
Jul 28 17:34:23.824838 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.823314    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b-cni-binary-copy\") pod \"multus-additional-cni-plugins-x2h5c\" (UID: \"3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b\") " pod="openshift-multus/multus-additional-cni-plugins-x2h5c"
Jul 28 17:34:23.824838 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.823319    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ea8d2cd4-eb75-479a-a7fa-730d68fb1cee-host-var-lib-cni-bin\") pod \"multus-8mhf9\" (UID: \"ea8d2cd4-eb75-479a-a7fa-730d68fb1cee\") " pod="openshift-multus/multus-8mhf9"
Jul 28 17:34:23.824838 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.823343    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/502adcd9-be65-4041-8d71-97c51f0af4a4-run-systemd\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.824838 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.823377    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/502adcd9-be65-4041-8d71-97c51f0af4a4-host-kubelet\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.824838 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.823379    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-x2h5c\" (UID: \"3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b\") " pod="openshift-multus/multus-additional-cni-plugins-x2h5c"
Jul 28 17:34:23.824838 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.823464    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ea8d2cd4-eb75-479a-a7fa-730d68fb1cee-multus-daemon-config\") pod \"multus-8mhf9\" (UID: \"ea8d2cd4-eb75-479a-a7fa-730d68fb1cee\") " pod="openshift-multus/multus-8mhf9"
Jul 28 17:34:23.824838 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.823477    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/118595be-d78d-40e2-9d5b-937167697690-lib-modules\") pod \"tuned-25z5w\" (UID: \"118595be-d78d-40e2-9d5b-937167697690\") " pod="openshift-cluster-node-tuning-operator/tuned-25z5w"
Jul 28 17:34:23.824838 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.823491    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddn4j\" (UniqueName: \"kubernetes.io/projected/502adcd9-be65-4041-8d71-97c51f0af4a4-kube-api-access-ddn4j\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.824838 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.823515    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/502adcd9-be65-4041-8d71-97c51f0af4a4-log-socket\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.824838 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.823521    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ea8d2cd4-eb75-479a-a7fa-730d68fb1cee-cni-binary-copy\") pod \"multus-8mhf9\" (UID: \"ea8d2cd4-eb75-479a-a7fa-730d68fb1cee\") " pod="openshift-multus/multus-8mhf9"
Jul 28 17:34:23.824838 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.823545    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7e2a258e-828b-4bbd-96b8-56cb5a7dd36f-agent-certs\") pod \"konnectivity-agent-2qvs8\" (UID: \"7e2a258e-828b-4bbd-96b8-56cb5a7dd36f\") " pod="kube-system/konnectivity-agent-2qvs8"
Jul 28 17:34:23.824838 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.823567    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a35e7d0f-47ba-4b8a-8987-47f0163cc156-host-slash\") pod \"iptables-alerter-nwwzc\" (UID: \"a35e7d0f-47ba-4b8a-8987-47f0163cc156\") " pod="openshift-network-operator/iptables-alerter-nwwzc"
Jul 28 17:34:23.824838 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.823592    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/502adcd9-be65-4041-8d71-97c51f0af4a4-run-openvswitch\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.824838 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.823615    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/502adcd9-be65-4041-8d71-97c51f0af4a4-ovn-node-metrics-cert\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.824838 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.823884    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6f0b63c9-e5d8-4bb4-8244-eefc95d9ff67-etc-selinux\") pod \"aws-ebs-csi-driver-node-8t7h6\" (UID: \"6f0b63c9-e5d8-4bb4-8244-eefc95d9ff67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t7h6"
Jul 28 17:34:23.825290 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.823932    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6f0b63c9-e5d8-4bb4-8244-eefc95d9ff67-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8t7h6\" (UID: \"6f0b63c9-e5d8-4bb4-8244-eefc95d9ff67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t7h6"
Jul 28 17:34:23.825290 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.823993    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/118595be-d78d-40e2-9d5b-937167697690-etc-sysconfig\") pod \"tuned-25z5w\" (UID: \"118595be-d78d-40e2-9d5b-937167697690\") " pod="openshift-cluster-node-tuning-operator/tuned-25z5w"
Jul 28 17:34:23.825290 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.824031    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6f0b63c9-e5d8-4bb4-8244-eefc95d9ff67-device-dir\") pod \"aws-ebs-csi-driver-node-8t7h6\" (UID: \"6f0b63c9-e5d8-4bb4-8244-eefc95d9ff67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t7h6"
Jul 28 17:34:23.825290 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.824031    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/502adcd9-be65-4041-8d71-97c51f0af4a4-env-overrides\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.825290 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.824066    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/502adcd9-be65-4041-8d71-97c51f0af4a4-run-systemd\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.825290 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.824178    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/502adcd9-be65-4041-8d71-97c51f0af4a4-host-cni-netd\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.825290 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.824189    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/118595be-d78d-40e2-9d5b-937167697690-run\") pod \"tuned-25z5w\" (UID: \"118595be-d78d-40e2-9d5b-937167697690\") " pod="openshift-cluster-node-tuning-operator/tuned-25z5w"
Jul 28 17:34:23.825290 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.824239    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/502adcd9-be65-4041-8d71-97c51f0af4a4-node-log\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.825290 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.824282    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/502adcd9-be65-4041-8d71-97c51f0af4a4-run-openvswitch\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.825290 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.824510    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/502adcd9-be65-4041-8d71-97c51f0af4a4-ovnkube-script-lib\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.825290 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.824667    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-x2h5c\" (UID: \"3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b\") " pod="openshift-multus/multus-additional-cni-plugins-x2h5c"
Jul 28 17:34:23.825788 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.825772    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/118595be-d78d-40e2-9d5b-937167697690-tmp\") pod \"tuned-25z5w\" (UID: \"118595be-d78d-40e2-9d5b-937167697690\") " pod="openshift-cluster-node-tuning-operator/tuned-25z5w"
Jul 28 17:34:23.825864 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.825841    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/118595be-d78d-40e2-9d5b-937167697690-etc-tuned\") pod \"tuned-25z5w\" (UID: \"118595be-d78d-40e2-9d5b-937167697690\") " pod="openshift-cluster-node-tuning-operator/tuned-25z5w"
Jul 28 17:34:23.826468 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.826451    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/502adcd9-be65-4041-8d71-97c51f0af4a4-ovn-node-metrics-cert\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.829451 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.829396    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvt4w\" (UniqueName: \"kubernetes.io/projected/654d1f40-f526-4325-b40c-fb544079bff4-kube-api-access-bvt4w\") pod \"node-resolver-j4wkp\" (UID: \"654d1f40-f526-4325-b40c-fb544079bff4\") " pod="openshift-dns/node-resolver-j4wkp"
Jul 28 17:34:23.830557 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.830528    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f49bc\" (UniqueName: \"kubernetes.io/projected/6f0b63c9-e5d8-4bb4-8244-eefc95d9ff67-kube-api-access-f49bc\") pod \"aws-ebs-csi-driver-node-8t7h6\" (UID: \"6f0b63c9-e5d8-4bb4-8244-eefc95d9ff67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t7h6"
Jul 28 17:34:23.831484 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.831376    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrqm9\" (UniqueName: \"kubernetes.io/projected/118595be-d78d-40e2-9d5b-937167697690-kube-api-access-xrqm9\") pod \"tuned-25z5w\" (UID: \"118595be-d78d-40e2-9d5b-937167697690\") " pod="openshift-cluster-node-tuning-operator/tuned-25z5w"
Jul 28 17:34:23.831972 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.831930    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rhmj\" (UniqueName: \"kubernetes.io/projected/3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b-kube-api-access-4rhmj\") pod \"multus-additional-cni-plugins-x2h5c\" (UID: \"3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b\") " pod="openshift-multus/multus-additional-cni-plugins-x2h5c"
Jul 28 17:34:23.832302 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.832282    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zbbv\" (UniqueName: \"kubernetes.io/projected/d6004102-aca2-4f56-8ddb-94374d3bc900-kube-api-access-8zbbv\") pod \"node-ca-z8fhq\" (UID: \"d6004102-aca2-4f56-8ddb-94374d3bc900\") " pod="openshift-image-registry/node-ca-z8fhq"
Jul 28 17:34:23.832609 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.832592    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddn4j\" (UniqueName: \"kubernetes.io/projected/502adcd9-be65-4041-8d71-97c51f0af4a4-kube-api-access-ddn4j\") pod \"ovnkube-node-krssz\" (UID: \"502adcd9-be65-4041-8d71-97c51f0af4a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:23.851069 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.851037    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-126.ec2.internal" event={"ID":"bf90ceb74d2562be9fba76df0fbf255c","Type":"ContainerStarted","Data":"35140f5d4ba8c63db08a260c4833b14b7695ed8655b76edd9ff47193651842fc"}
Jul 28 17:34:23.852120 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.852097    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-126.ec2.internal" event={"ID":"17457ce2fa8ddf46113d3222bb973d5a","Type":"ContainerStarted","Data":"c3cf6ff2ef0e07ad3a5391148514f22a10e903d630ff6d4eb799251f6d70b59f"}
Jul 28 17:34:23.924882 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.924853    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a35e7d0f-47ba-4b8a-8987-47f0163cc156-host-slash\") pod \"iptables-alerter-nwwzc\" (UID: \"a35e7d0f-47ba-4b8a-8987-47f0163cc156\") " pod="openshift-network-operator/iptables-alerter-nwwzc"
Jul 28 17:34:23.924882 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.924883    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ea8d2cd4-eb75-479a-a7fa-730d68fb1cee-system-cni-dir\") pod \"multus-8mhf9\" (UID: \"ea8d2cd4-eb75-479a-a7fa-730d68fb1cee\") " pod="openshift-multus/multus-8mhf9"
Jul 28 17:34:23.925089 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.924901    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ea8d2cd4-eb75-479a-a7fa-730d68fb1cee-host-var-lib-kubelet\") pod \"multus-8mhf9\" (UID: \"ea8d2cd4-eb75-479a-a7fa-730d68fb1cee\") " pod="openshift-multus/multus-8mhf9"
Jul 28 17:34:23.925089 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.924920    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ea8d2cd4-eb75-479a-a7fa-730d68fb1cee-hostroot\") pod \"multus-8mhf9\" (UID: \"ea8d2cd4-eb75-479a-a7fa-730d68fb1cee\") " pod="openshift-multus/multus-8mhf9"
Jul 28 17:34:23.925089 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.924943    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8a8edb3a-34c9-4a66-81aa-49474fdfdf37-dbus\") pod \"global-pull-secret-syncer-z75m5\" (UID: \"8a8edb3a-34c9-4a66-81aa-49474fdfdf37\") " pod="kube-system/global-pull-secret-syncer-z75m5"
Jul 28 17:34:23.925089 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.924965    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ea8d2cd4-eb75-479a-a7fa-730d68fb1cee-os-release\") pod \"multus-8mhf9\" (UID: \"ea8d2cd4-eb75-479a-a7fa-730d68fb1cee\") " pod="openshift-multus/multus-8mhf9"
Jul 28 17:34:23.925089 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.924965    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a35e7d0f-47ba-4b8a-8987-47f0163cc156-host-slash\") pod \"iptables-alerter-nwwzc\" (UID: \"a35e7d0f-47ba-4b8a-8987-47f0163cc156\") " pod="openshift-network-operator/iptables-alerter-nwwzc"
Jul 28 17:34:23.925089 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.924984    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ea8d2cd4-eb75-479a-a7fa-730d68fb1cee-system-cni-dir\") pod \"multus-8mhf9\" (UID: \"ea8d2cd4-eb75-479a-a7fa-730d68fb1cee\") " pod="openshift-multus/multus-8mhf9"
Jul 28 17:34:23.925089 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.924986    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ea8d2cd4-eb75-479a-a7fa-730d68fb1cee-host-run-k8s-cni-cncf-io\") pod \"multus-8mhf9\" (UID: \"ea8d2cd4-eb75-479a-a7fa-730d68fb1cee\") " pod="openshift-multus/multus-8mhf9"
Jul 28 17:34:23.925089 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.924985    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ea8d2cd4-eb75-479a-a7fa-730d68fb1cee-host-var-lib-kubelet\") pod \"multus-8mhf9\" (UID: \"ea8d2cd4-eb75-479a-a7fa-730d68fb1cee\") " pod="openshift-multus/multus-8mhf9"
Jul 28 17:34:23.925089 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.925020    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ea8d2cd4-eb75-479a-a7fa-730d68fb1cee-hostroot\") pod \"multus-8mhf9\" (UID: \"ea8d2cd4-eb75-479a-a7fa-730d68fb1cee\") " pod="openshift-multus/multus-8mhf9"
Jul 28 17:34:23.925089 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.925020    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ea8d2cd4-eb75-479a-a7fa-730d68fb1cee-host-run-k8s-cni-cncf-io\") pod \"multus-8mhf9\" (UID: \"ea8d2cd4-eb75-479a-a7fa-730d68fb1cee\") " pod="openshift-multus/multus-8mhf9"
Jul 28 17:34:23.925089 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.925040    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8a8edb3a-34c9-4a66-81aa-49474fdfdf37-original-pull-secret\") pod \"global-pull-secret-syncer-z75m5\" (UID: \"8a8edb3a-34c9-4a66-81aa-49474fdfdf37\") " pod="kube-system/global-pull-secret-syncer-z75m5"
Jul 28 17:34:23.925089 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.925052    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ea8d2cd4-eb75-479a-a7fa-730d68fb1cee-os-release\") pod \"multus-8mhf9\" (UID: \"ea8d2cd4-eb75-479a-a7fa-730d68fb1cee\") " pod="openshift-multus/multus-8mhf9"
Jul 28 17:34:23.925089 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.925058    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcgdp\" (UniqueName: \"kubernetes.io/projected/ea8d2cd4-eb75-479a-a7fa-730d68fb1cee-kube-api-access-vcgdp\") pod \"multus-8mhf9\" (UID: \"ea8d2cd4-eb75-479a-a7fa-730d68fb1cee\") " pod="openshift-multus/multus-8mhf9"
Jul 28 17:34:23.925666 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.925099    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7e2a258e-828b-4bbd-96b8-56cb5a7dd36f-konnectivity-ca\") pod \"konnectivity-agent-2qvs8\" (UID: \"7e2a258e-828b-4bbd-96b8-56cb5a7dd36f\") " pod="kube-system/konnectivity-agent-2qvs8"
Jul 28 17:34:23.925666 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.925123    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ea8d2cd4-eb75-479a-a7fa-730d68fb1cee-cnibin\") pod \"multus-8mhf9\" (UID: \"ea8d2cd4-eb75-479a-a7fa-730d68fb1cee\") " pod="openshift-multus/multus-8mhf9"
Jul 28 17:34:23.925666 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:23.925138    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 17:34:23.925666 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.925141    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8a8edb3a-34c9-4a66-81aa-49474fdfdf37-dbus\") pod \"global-pull-secret-syncer-z75m5\" (UID: \"8a8edb3a-34c9-4a66-81aa-49474fdfdf37\") " pod="kube-system/global-pull-secret-syncer-z75m5"
Jul 28 17:34:23.925666 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.925149    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ea8d2cd4-eb75-479a-a7fa-730d68fb1cee-multus-cni-dir\") pod \"multus-8mhf9\" (UID: \"ea8d2cd4-eb75-479a-a7fa-730d68fb1cee\") " pod="openshift-multus/multus-8mhf9"
Jul 28 17:34:23.925666 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.925192    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ea8d2cd4-eb75-479a-a7fa-730d68fb1cee-cnibin\") pod \"multus-8mhf9\" (UID: \"ea8d2cd4-eb75-479a-a7fa-730d68fb1cee\") " pod="openshift-multus/multus-8mhf9"
Jul 28 17:34:23.925666 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:23.925198    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a8edb3a-34c9-4a66-81aa-49474fdfdf37-original-pull-secret podName:8a8edb3a-34c9-4a66-81aa-49474fdfdf37 nodeName:}" failed. No retries permitted until 2026-07-28 17:34:24.425178793 +0000 UTC m=+3.086335143 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8a8edb3a-34c9-4a66-81aa-49474fdfdf37-original-pull-secret") pod "global-pull-secret-syncer-z75m5" (UID: "8a8edb3a-34c9-4a66-81aa-49474fdfdf37") : object "kube-system"/"original-pull-secret" not registered
Jul 28 17:34:23.925666 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.925200    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ea8d2cd4-eb75-479a-a7fa-730d68fb1cee-multus-cni-dir\") pod \"multus-8mhf9\" (UID: \"ea8d2cd4-eb75-479a-a7fa-730d68fb1cee\") " pod="openshift-multus/multus-8mhf9"
Jul 28 17:34:23.925666 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.925215    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ea8d2cd4-eb75-479a-a7fa-730d68fb1cee-multus-conf-dir\") pod \"multus-8mhf9\" (UID: \"ea8d2cd4-eb75-479a-a7fa-730d68fb1cee\") " pod="openshift-multus/multus-8mhf9"
Jul 28 17:34:23.925666 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.925245    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ea8d2cd4-eb75-479a-a7fa-730d68fb1cee-host-run-multus-certs\") pod \"multus-8mhf9\" (UID: \"ea8d2cd4-eb75-479a-a7fa-730d68fb1cee\") " pod="openshift-multus/multus-8mhf9"
Jul 28 17:34:23.925666 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.925270    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ea8d2cd4-eb75-479a-a7fa-730d68fb1cee-multus-conf-dir\") pod \"multus-8mhf9\" (UID: \"ea8d2cd4-eb75-479a-a7fa-730d68fb1cee\") " pod="openshift-multus/multus-8mhf9"
Jul 28 17:34:23.925666 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.925272    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wv9jw\" (UniqueName: \"kubernetes.io/projected/a35e7d0f-47ba-4b8a-8987-47f0163cc156-kube-api-access-wv9jw\") pod \"iptables-alerter-nwwzc\" (UID: \"a35e7d0f-47ba-4b8a-8987-47f0163cc156\") " pod="openshift-network-operator/iptables-alerter-nwwzc"
Jul 28 17:34:23.925666 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.925310    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ea8d2cd4-eb75-479a-a7fa-730d68fb1cee-host-run-multus-certs\") pod \"multus-8mhf9\" (UID: \"ea8d2cd4-eb75-479a-a7fa-730d68fb1cee\") " pod="openshift-multus/multus-8mhf9"
Jul 28 17:34:23.925666 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.925325    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ea8d2cd4-eb75-479a-a7fa-730d68fb1cee-host-var-lib-cni-multus\") pod \"multus-8mhf9\" (UID: \"ea8d2cd4-eb75-479a-a7fa-730d68fb1cee\") " pod="openshift-multus/multus-8mhf9"
Jul 28 17:34:23.925666 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.925347    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ea8d2cd4-eb75-479a-a7fa-730d68fb1cee-etc-kubernetes\") pod \"multus-8mhf9\" (UID: \"ea8d2cd4-eb75-479a-a7fa-730d68fb1cee\") " pod="openshift-multus/multus-8mhf9"
Jul 28 17:34:23.925666 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.925360    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7smn5\" (UniqueName: \"kubernetes.io/projected/d3c4a388-fdcc-4744-b6e2-d52dc701f235-kube-api-access-7smn5\") pod \"network-metrics-daemon-4tthg\" (UID: \"d3c4a388-fdcc-4744-b6e2-d52dc701f235\") " pod="openshift-multus/network-metrics-daemon-4tthg"
Jul 28 17:34:23.925666 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.925408    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ea8d2cd4-eb75-479a-a7fa-730d68fb1cee-host-var-lib-cni-multus\") pod \"multus-8mhf9\" (UID: \"ea8d2cd4-eb75-479a-a7fa-730d68fb1cee\") " pod="openshift-multus/multus-8mhf9"
Jul 28 17:34:23.925666 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.925420    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ea8d2cd4-eb75-479a-a7fa-730d68fb1cee-etc-kubernetes\") pod \"multus-8mhf9\" (UID: \"ea8d2cd4-eb75-479a-a7fa-730d68fb1cee\") " pod="openshift-multus/multus-8mhf9"
Jul 28 17:34:23.926469 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.925458    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a35e7d0f-47ba-4b8a-8987-47f0163cc156-iptables-alerter-script\") pod \"iptables-alerter-nwwzc\" (UID: \"a35e7d0f-47ba-4b8a-8987-47f0163cc156\") " pod="openshift-network-operator/iptables-alerter-nwwzc"
Jul 28 17:34:23.926469 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.925491    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8a8edb3a-34c9-4a66-81aa-49474fdfdf37-kubelet-config\") pod \"global-pull-secret-syncer-z75m5\" (UID: \"8a8edb3a-34c9-4a66-81aa-49474fdfdf37\") " pod="kube-system/global-pull-secret-syncer-z75m5"
Jul 28 17:34:23.926469 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.925519    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3c4a388-fdcc-4744-b6e2-d52dc701f235-metrics-certs\") pod \"network-metrics-daemon-4tthg\" (UID: \"d3c4a388-fdcc-4744-b6e2-d52dc701f235\") " pod="openshift-multus/network-metrics-daemon-4tthg"
Jul 28 17:34:23.926469 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.925543    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gg7b\" (UniqueName: \"kubernetes.io/projected/41d54bc6-a997-48d3-9752-f78204bbdf43-kube-api-access-4gg7b\") pod \"network-check-target-74vbl\" (UID: \"41d54bc6-a997-48d3-9752-f78204bbdf43\") " pod="openshift-network-diagnostics/network-check-target-74vbl"
Jul 28 17:34:23.926469 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.925553    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8a8edb3a-34c9-4a66-81aa-49474fdfdf37-kubelet-config\") pod \"global-pull-secret-syncer-z75m5\" (UID: \"8a8edb3a-34c9-4a66-81aa-49474fdfdf37\") " pod="kube-system/global-pull-secret-syncer-z75m5"
Jul 28 17:34:23.926469 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:23.925637    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:34:23.926469 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.925656    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ea8d2cd4-eb75-479a-a7fa-730d68fb1cee-multus-socket-dir-parent\") pod \"multus-8mhf9\" (UID: \"ea8d2cd4-eb75-479a-a7fa-730d68fb1cee\") " pod="openshift-multus/multus-8mhf9"
Jul 28 17:34:23.926469 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:23.925682    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3c4a388-fdcc-4744-b6e2-d52dc701f235-metrics-certs podName:d3c4a388-fdcc-4744-b6e2-d52dc701f235 nodeName:}" failed. No retries permitted until 2026-07-28 17:34:24.425669349 +0000 UTC m=+3.086825680 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d3c4a388-fdcc-4744-b6e2-d52dc701f235-metrics-certs") pod "network-metrics-daemon-4tthg" (UID: "d3c4a388-fdcc-4744-b6e2-d52dc701f235") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:34:23.926469 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.925701    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ea8d2cd4-eb75-479a-a7fa-730d68fb1cee-host-run-netns\") pod \"multus-8mhf9\" (UID: \"ea8d2cd4-eb75-479a-a7fa-730d68fb1cee\") " pod="openshift-multus/multus-8mhf9"
Jul 28 17:34:23.926469 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.925706    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ea8d2cd4-eb75-479a-a7fa-730d68fb1cee-multus-socket-dir-parent\") pod \"multus-8mhf9\" (UID: \"ea8d2cd4-eb75-479a-a7fa-730d68fb1cee\") " pod="openshift-multus/multus-8mhf9"
Jul 28 17:34:23.926469 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.925718    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ea8d2cd4-eb75-479a-a7fa-730d68fb1cee-host-var-lib-cni-bin\") pod \"multus-8mhf9\" (UID: \"ea8d2cd4-eb75-479a-a7fa-730d68fb1cee\") " pod="openshift-multus/multus-8mhf9"
Jul 28 17:34:23.926469 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.925738    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ea8d2cd4-eb75-479a-a7fa-730d68fb1cee-host-run-netns\") pod \"multus-8mhf9\" (UID: \"ea8d2cd4-eb75-479a-a7fa-730d68fb1cee\") " pod="openshift-multus/multus-8mhf9"
Jul 28 17:34:23.926469 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.925738    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ea8d2cd4-eb75-479a-a7fa-730d68fb1cee-multus-daemon-config\") pod \"multus-8mhf9\" (UID: \"ea8d2cd4-eb75-479a-a7fa-730d68fb1cee\") " pod="openshift-multus/multus-8mhf9"
Jul 28 17:34:23.926469 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.925771    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ea8d2cd4-eb75-479a-a7fa-730d68fb1cee-cni-binary-copy\") pod \"multus-8mhf9\" (UID: \"ea8d2cd4-eb75-479a-a7fa-730d68fb1cee\") " pod="openshift-multus/multus-8mhf9"
Jul 28 17:34:23.926469 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.925814    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ea8d2cd4-eb75-479a-a7fa-730d68fb1cee-host-var-lib-cni-bin\") pod \"multus-8mhf9\" (UID: \"ea8d2cd4-eb75-479a-a7fa-730d68fb1cee\") " pod="openshift-multus/multus-8mhf9"
Jul 28 17:34:23.926469 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.925849    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7e2a258e-828b-4bbd-96b8-56cb5a7dd36f-agent-certs\") pod \"konnectivity-agent-2qvs8\" (UID: \"7e2a258e-828b-4bbd-96b8-56cb5a7dd36f\") " pod="kube-system/konnectivity-agent-2qvs8"
Jul 28 17:34:23.926469 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.925911    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a35e7d0f-47ba-4b8a-8987-47f0163cc156-iptables-alerter-script\") pod \"iptables-alerter-nwwzc\" (UID: \"a35e7d0f-47ba-4b8a-8987-47f0163cc156\") " pod="openshift-network-operator/iptables-alerter-nwwzc"
Jul 28 17:34:23.927201 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.926223    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7e2a258e-828b-4bbd-96b8-56cb5a7dd36f-konnectivity-ca\") pod \"konnectivity-agent-2qvs8\" (UID: \"7e2a258e-828b-4bbd-96b8-56cb5a7dd36f\") " pod="kube-system/konnectivity-agent-2qvs8"
Jul 28 17:34:23.927201 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.926238    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ea8d2cd4-eb75-479a-a7fa-730d68fb1cee-cni-binary-copy\") pod \"multus-8mhf9\" (UID: \"ea8d2cd4-eb75-479a-a7fa-730d68fb1cee\") " pod="openshift-multus/multus-8mhf9"
Jul 28 17:34:23.927201 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.926280    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ea8d2cd4-eb75-479a-a7fa-730d68fb1cee-multus-daemon-config\") pod \"multus-8mhf9\" (UID: \"ea8d2cd4-eb75-479a-a7fa-730d68fb1cee\") " pod="openshift-multus/multus-8mhf9"
Jul 28 17:34:23.928121 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.928102    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7e2a258e-828b-4bbd-96b8-56cb5a7dd36f-agent-certs\") pod \"konnectivity-agent-2qvs8\" (UID: \"7e2a258e-828b-4bbd-96b8-56cb5a7dd36f\") " pod="kube-system/konnectivity-agent-2qvs8"
Jul 28 17:34:23.933631 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:23.933615    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 17:34:23.933735 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:23.933654    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 17:34:23.933735 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:23.933667    2563 projected.go:196] Error preparing data for projected volume kube-api-access-4gg7b for pod openshift-network-diagnostics/network-check-target-74vbl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:34:23.933735 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:23.933723    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/41d54bc6-a997-48d3-9752-f78204bbdf43-kube-api-access-4gg7b podName:41d54bc6-a997-48d3-9752-f78204bbdf43 nodeName:}" failed. No retries permitted until 2026-07-28 17:34:24.433708216 +0000 UTC m=+3.094864558 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-4gg7b" (UniqueName: "kubernetes.io/projected/41d54bc6-a997-48d3-9752-f78204bbdf43-kube-api-access-4gg7b") pod "network-check-target-74vbl" (UID: "41d54bc6-a997-48d3-9752-f78204bbdf43") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:34:23.935628 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.935595    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7smn5\" (UniqueName: \"kubernetes.io/projected/d3c4a388-fdcc-4744-b6e2-d52dc701f235-kube-api-access-7smn5\") pod \"network-metrics-daemon-4tthg\" (UID: \"d3c4a388-fdcc-4744-b6e2-d52dc701f235\") " pod="openshift-multus/network-metrics-daemon-4tthg"
Jul 28 17:34:23.935772 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.935756    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcgdp\" (UniqueName: \"kubernetes.io/projected/ea8d2cd4-eb75-479a-a7fa-730d68fb1cee-kube-api-access-vcgdp\") pod \"multus-8mhf9\" (UID: \"ea8d2cd4-eb75-479a-a7fa-730d68fb1cee\") " pod="openshift-multus/multus-8mhf9"
Jul 28 17:34:23.935872 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:23.935858    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wv9jw\" (UniqueName: \"kubernetes.io/projected/a35e7d0f-47ba-4b8a-8987-47f0163cc156-kube-api-access-wv9jw\") pod \"iptables-alerter-nwwzc\" (UID: \"a35e7d0f-47ba-4b8a-8987-47f0163cc156\") " pod="openshift-network-operator/iptables-alerter-nwwzc"
Jul 28 17:34:24.017301 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:24.017233    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-z8fhq"
Jul 28 17:34:24.025353 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:24.025325    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:24.033228 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:24.033214    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t7h6"
Jul 28 17:34:24.038757 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:24.038741    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-j4wkp"
Jul 28 17:34:24.045133 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:24.045117    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-x2h5c"
Jul 28 17:34:24.055918 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:24.055902    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-25z5w"
Jul 28 17:34:24.061759 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:24.061742    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8mhf9"
Jul 28 17:34:24.067749 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:24.067734    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2qvs8"
Jul 28 17:34:24.074139 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:24.074123    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nwwzc"
Jul 28 17:34:24.429008 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:24.428979    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3c4a388-fdcc-4744-b6e2-d52dc701f235-metrics-certs\") pod \"network-metrics-daemon-4tthg\" (UID: \"d3c4a388-fdcc-4744-b6e2-d52dc701f235\") " pod="openshift-multus/network-metrics-daemon-4tthg"
Jul 28 17:34:24.429171 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:24.429034    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8a8edb3a-34c9-4a66-81aa-49474fdfdf37-original-pull-secret\") pod \"global-pull-secret-syncer-z75m5\" (UID: \"8a8edb3a-34c9-4a66-81aa-49474fdfdf37\") " pod="kube-system/global-pull-secret-syncer-z75m5"
Jul 28 17:34:24.429171 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:24.429120    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:34:24.429171 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:24.429141    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 17:34:24.429309 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:24.429182    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3c4a388-fdcc-4744-b6e2-d52dc701f235-metrics-certs podName:d3c4a388-fdcc-4744-b6e2-d52dc701f235 nodeName:}" failed. No retries permitted until 2026-07-28 17:34:25.429168958 +0000 UTC m=+4.090325289 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d3c4a388-fdcc-4744-b6e2-d52dc701f235-metrics-certs") pod "network-metrics-daemon-4tthg" (UID: "d3c4a388-fdcc-4744-b6e2-d52dc701f235") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:34:24.429309 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:24.429215    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a8edb3a-34c9-4a66-81aa-49474fdfdf37-original-pull-secret podName:8a8edb3a-34c9-4a66-81aa-49474fdfdf37 nodeName:}" failed. No retries permitted until 2026-07-28 17:34:25.429197735 +0000 UTC m=+4.090354084 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8a8edb3a-34c9-4a66-81aa-49474fdfdf37-original-pull-secret") pod "global-pull-secret-syncer-z75m5" (UID: "8a8edb3a-34c9-4a66-81aa-49474fdfdf37") : object "kube-system"/"original-pull-secret" not registered
Jul 28 17:34:24.529960 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:24.529930    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gg7b\" (UniqueName: \"kubernetes.io/projected/41d54bc6-a997-48d3-9752-f78204bbdf43-kube-api-access-4gg7b\") pod \"network-check-target-74vbl\" (UID: \"41d54bc6-a997-48d3-9752-f78204bbdf43\") " pod="openshift-network-diagnostics/network-check-target-74vbl"
Jul 28 17:34:24.530094 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:24.530083    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 17:34:24.530141 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:24.530101    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 17:34:24.530141 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:24.530114    2563 projected.go:196] Error preparing data for projected volume kube-api-access-4gg7b for pod openshift-network-diagnostics/network-check-target-74vbl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:34:24.530212 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:24.530171    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/41d54bc6-a997-48d3-9752-f78204bbdf43-kube-api-access-4gg7b podName:41d54bc6-a997-48d3-9752-f78204bbdf43 nodeName:}" failed. No retries permitted until 2026-07-28 17:34:25.530153091 +0000 UTC m=+4.191309436 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-4gg7b" (UniqueName: "kubernetes.io/projected/41d54bc6-a997-48d3-9752-f78204bbdf43-kube-api-access-4gg7b") pod "network-check-target-74vbl" (UID: "41d54bc6-a997-48d3-9752-f78204bbdf43") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:34:24.621149 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:24.621127    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6004102_aca2_4f56_8ddb_94374d3bc900.slice/crio-010dc31330c741c1296511f6ff0c55a5f72393fef7d6fb99b8d0a51938510b3b WatchSource:0}: Error finding container 010dc31330c741c1296511f6ff0c55a5f72393fef7d6fb99b8d0a51938510b3b: Status 404 returned error can't find the container with id 010dc31330c741c1296511f6ff0c55a5f72393fef7d6fb99b8d0a51938510b3b
Jul 28 17:34:24.640507 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:24.640487    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod654d1f40_f526_4325_b40c_fb544079bff4.slice/crio-77e9024c869e2d1215aa78439a57270bf017cdc9f1790e1899b4031476978580 WatchSource:0}: Error finding container 77e9024c869e2d1215aa78439a57270bf017cdc9f1790e1899b4031476978580: Status 404 returned error can't find the container with id 77e9024c869e2d1215aa78439a57270bf017cdc9f1790e1899b4031476978580
Jul 28 17:34:24.640973 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:24.640953    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f0b63c9_e5d8_4bb4_8244_eefc95d9ff67.slice/crio-89b8c720fd2c24c6aa1bd4c3e3d0e6bb6299dbd651f9caa28085a498afc15d51 WatchSource:0}: Error finding container 89b8c720fd2c24c6aa1bd4c3e3d0e6bb6299dbd651f9caa28085a498afc15d51: Status 404 returned error can't find the container with id 89b8c720fd2c24c6aa1bd4c3e3d0e6bb6299dbd651f9caa28085a498afc15d51
Jul 28 17:34:24.641811 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:24.641794    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea8d2cd4_eb75_479a_a7fa_730d68fb1cee.slice/crio-91e691268b6ddcd437859a60ef64e2caa922728861a425cddd8cf4d6cc21206b WatchSource:0}: Error finding container 91e691268b6ddcd437859a60ef64e2caa922728861a425cddd8cf4d6cc21206b: Status 404 returned error can't find the container with id 91e691268b6ddcd437859a60ef64e2caa922728861a425cddd8cf4d6cc21206b
Jul 28 17:34:24.642906 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:24.642842    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda35e7d0f_47ba_4b8a_8987_47f0163cc156.slice/crio-126d1f87cef98afaea7d51e5ee0969eec103214fcf290e45574421252338a470 WatchSource:0}: Error finding container 126d1f87cef98afaea7d51e5ee0969eec103214fcf290e45574421252338a470: Status 404 returned error can't find the container with id 126d1f87cef98afaea7d51e5ee0969eec103214fcf290e45574421252338a470
Jul 28 17:34:24.643843 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:24.643682    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3aef368b_c04d_4b4e_94a2_0ac3e76f2f0b.slice/crio-f26a8fe1164cd04bba5be1b4b88302813880dca47aa7da99d6292129afc308ab WatchSource:0}: Error finding container f26a8fe1164cd04bba5be1b4b88302813880dca47aa7da99d6292129afc308ab: Status 404 returned error can't find the container with id f26a8fe1164cd04bba5be1b4b88302813880dca47aa7da99d6292129afc308ab
Jul 28 17:34:24.644748 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:24.644463    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e2a258e_828b_4bbd_96b8_56cb5a7dd36f.slice/crio-9a7e50cd77cb4b6695671b02c91573518f614d78f587c4dd54c2bf8cc066a046 WatchSource:0}: Error finding container 9a7e50cd77cb4b6695671b02c91573518f614d78f587c4dd54c2bf8cc066a046: Status 404 returned error can't find the container with id 9a7e50cd77cb4b6695671b02c91573518f614d78f587c4dd54c2bf8cc066a046
Jul 28 17:34:24.645140 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:24.645070    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod118595be_d78d_40e2_9d5b_937167697690.slice/crio-7f429032ad73b994bdbb077e5ef3082e14df74a1fd62246de0bb4bc09cac4b30 WatchSource:0}: Error finding container 7f429032ad73b994bdbb077e5ef3082e14df74a1fd62246de0bb4bc09cac4b30: Status 404 returned error can't find the container with id 7f429032ad73b994bdbb077e5ef3082e14df74a1fd62246de0bb4bc09cac4b30
Jul 28 17:34:24.645927 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:24.645884    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod502adcd9_be65_4041_8d71_97c51f0af4a4.slice/crio-a191d866429e4d6455e04344bd3088b1f2ac9ce1d2c62598e95fd076a881851d WatchSource:0}: Error finding container a191d866429e4d6455e04344bd3088b1f2ac9ce1d2c62598e95fd076a881851d: Status 404 returned error can't find the container with id a191d866429e4d6455e04344bd3088b1f2ac9ce1d2c62598e95fd076a881851d
Jul 28 17:34:24.756906 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:24.756847    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 17:29:23 +0000 UTC" deadline="2028-03-30 22:33:36.93068977 +0000 UTC"
Jul 28 17:34:24.756906 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:24.756903    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14668h59m12.173789959s"
Jul 28 17:34:24.847944 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:24.847819    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-74vbl"
Jul 28 17:34:24.848026 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:24.848010    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-74vbl" podUID="41d54bc6-a997-48d3-9752-f78204bbdf43"
Jul 28 17:34:24.854884 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:24.854864    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2qvs8" event={"ID":"7e2a258e-828b-4bbd-96b8-56cb5a7dd36f","Type":"ContainerStarted","Data":"9a7e50cd77cb4b6695671b02c91573518f614d78f587c4dd54c2bf8cc066a046"}
Jul 28 17:34:24.855778 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:24.855754    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nwwzc" event={"ID":"a35e7d0f-47ba-4b8a-8987-47f0163cc156","Type":"ContainerStarted","Data":"126d1f87cef98afaea7d51e5ee0969eec103214fcf290e45574421252338a470"}
Jul 28 17:34:24.856608 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:24.856588    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8mhf9" event={"ID":"ea8d2cd4-eb75-479a-a7fa-730d68fb1cee","Type":"ContainerStarted","Data":"91e691268b6ddcd437859a60ef64e2caa922728861a425cddd8cf4d6cc21206b"}
Jul 28 17:34:24.857421 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:24.857403    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-j4wkp" event={"ID":"654d1f40-f526-4325-b40c-fb544079bff4","Type":"ContainerStarted","Data":"77e9024c869e2d1215aa78439a57270bf017cdc9f1790e1899b4031476978580"}
Jul 28 17:34:24.858950 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:24.858931    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-126.ec2.internal" event={"ID":"bf90ceb74d2562be9fba76df0fbf255c","Type":"ContainerStarted","Data":"21e896504164ef203f98f2279805b75eac17be620ef9539c3dcd5848b801211b"}
Jul 28 17:34:24.860083 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:24.860065    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-126.ec2.internal" event={"ID":"17457ce2fa8ddf46113d3222bb973d5a","Type":"ContainerStarted","Data":"37d70f5d108b5d512cd6ce6b408cbbcd474c898e99a93aa1b30e1c11d27b18af"}
Jul 28 17:34:24.860983 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:24.860958    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-krssz" event={"ID":"502adcd9-be65-4041-8d71-97c51f0af4a4","Type":"ContainerStarted","Data":"a191d866429e4d6455e04344bd3088b1f2ac9ce1d2c62598e95fd076a881851d"}
Jul 28 17:34:24.861862 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:24.861846    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-25z5w" event={"ID":"118595be-d78d-40e2-9d5b-937167697690","Type":"ContainerStarted","Data":"7f429032ad73b994bdbb077e5ef3082e14df74a1fd62246de0bb4bc09cac4b30"}
Jul 28 17:34:24.862763 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:24.862746    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x2h5c" event={"ID":"3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b","Type":"ContainerStarted","Data":"f26a8fe1164cd04bba5be1b4b88302813880dca47aa7da99d6292129afc308ab"}
Jul 28 17:34:24.863723 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:24.863706    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t7h6" event={"ID":"6f0b63c9-e5d8-4bb4-8244-eefc95d9ff67","Type":"ContainerStarted","Data":"89b8c720fd2c24c6aa1bd4c3e3d0e6bb6299dbd651f9caa28085a498afc15d51"}
Jul 28 17:34:24.864658 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:24.864643    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-z8fhq" event={"ID":"d6004102-aca2-4f56-8ddb-94374d3bc900","Type":"ContainerStarted","Data":"010dc31330c741c1296511f6ff0c55a5f72393fef7d6fb99b8d0a51938510b3b"}
Jul 28 17:34:24.886932 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:24.886899    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-126.ec2.internal" podStartSLOduration=2.886889391 podStartE2EDuration="2.886889391s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:34:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:34:24.885174995 +0000 UTC m=+3.546331359" watchObservedRunningTime="2026-07-28 17:34:24.886889391 +0000 UTC m=+3.548045742"
Jul 28 17:34:25.435940 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:25.435904    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3c4a388-fdcc-4744-b6e2-d52dc701f235-metrics-certs\") pod \"network-metrics-daemon-4tthg\" (UID: \"d3c4a388-fdcc-4744-b6e2-d52dc701f235\") " pod="openshift-multus/network-metrics-daemon-4tthg"
Jul 28 17:34:25.436104 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:25.435988    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8a8edb3a-34c9-4a66-81aa-49474fdfdf37-original-pull-secret\") pod \"global-pull-secret-syncer-z75m5\" (UID: \"8a8edb3a-34c9-4a66-81aa-49474fdfdf37\") " pod="kube-system/global-pull-secret-syncer-z75m5"
Jul 28 17:34:25.436172 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:25.436118    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 17:34:25.436217 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:25.436174    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a8edb3a-34c9-4a66-81aa-49474fdfdf37-original-pull-secret podName:8a8edb3a-34c9-4a66-81aa-49474fdfdf37 nodeName:}" failed. No retries permitted until 2026-07-28 17:34:27.436157872 +0000 UTC m=+6.097314209 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8a8edb3a-34c9-4a66-81aa-49474fdfdf37-original-pull-secret") pod "global-pull-secret-syncer-z75m5" (UID: "8a8edb3a-34c9-4a66-81aa-49474fdfdf37") : object "kube-system"/"original-pull-secret" not registered
Jul 28 17:34:25.436674 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:25.436559    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:34:25.436674 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:25.436619    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3c4a388-fdcc-4744-b6e2-d52dc701f235-metrics-certs podName:d3c4a388-fdcc-4744-b6e2-d52dc701f235 nodeName:}" failed. No retries permitted until 2026-07-28 17:34:27.436602956 +0000 UTC m=+6.097759293 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d3c4a388-fdcc-4744-b6e2-d52dc701f235-metrics-certs") pod "network-metrics-daemon-4tthg" (UID: "d3c4a388-fdcc-4744-b6e2-d52dc701f235") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:34:25.536764 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:25.536730    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gg7b\" (UniqueName: \"kubernetes.io/projected/41d54bc6-a997-48d3-9752-f78204bbdf43-kube-api-access-4gg7b\") pod \"network-check-target-74vbl\" (UID: \"41d54bc6-a997-48d3-9752-f78204bbdf43\") " pod="openshift-network-diagnostics/network-check-target-74vbl"
Jul 28 17:34:25.536925 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:25.536899    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 17:34:25.536925 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:25.536918    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 17:34:25.537016 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:25.536929    2563 projected.go:196] Error preparing data for projected volume kube-api-access-4gg7b for pod openshift-network-diagnostics/network-check-target-74vbl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:34:25.537016 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:25.536984    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/41d54bc6-a997-48d3-9752-f78204bbdf43-kube-api-access-4gg7b podName:41d54bc6-a997-48d3-9752-f78204bbdf43 nodeName:}" failed. No retries permitted until 2026-07-28 17:34:27.536965875 +0000 UTC m=+6.198122206 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-4gg7b" (UniqueName: "kubernetes.io/projected/41d54bc6-a997-48d3-9752-f78204bbdf43-kube-api-access-4gg7b") pod "network-check-target-74vbl" (UID: "41d54bc6-a997-48d3-9752-f78204bbdf43") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:34:25.757856 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:25.757672    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 17:29:23 +0000 UTC" deadline="2028-02-02 02:00:33.451990872 +0000 UTC"
Jul 28 17:34:25.757856 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:25.757704    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13280h26m7.694290644s"
Jul 28 17:34:25.847674 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:25.847640    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tthg"
Jul 28 17:34:25.847810 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:25.847770    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4tthg" podUID="d3c4a388-fdcc-4744-b6e2-d52dc701f235"
Jul 28 17:34:25.848318 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:25.848302    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z75m5"
Jul 28 17:34:25.848451 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:25.848396    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z75m5" podUID="8a8edb3a-34c9-4a66-81aa-49474fdfdf37"
Jul 28 17:34:25.882519 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:25.882466    2563 generic.go:356] "Generic (PLEG): container finished" podID="17457ce2fa8ddf46113d3222bb973d5a" containerID="37d70f5d108b5d512cd6ce6b408cbbcd474c898e99a93aa1b30e1c11d27b18af" exitCode=0
Jul 28 17:34:25.882666 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:25.882636    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-126.ec2.internal" event={"ID":"17457ce2fa8ddf46113d3222bb973d5a","Type":"ContainerDied","Data":"37d70f5d108b5d512cd6ce6b408cbbcd474c898e99a93aa1b30e1c11d27b18af"}
Jul 28 17:34:26.848162 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:26.847658    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-74vbl"
Jul 28 17:34:26.848162 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:26.847785    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-74vbl" podUID="41d54bc6-a997-48d3-9752-f78204bbdf43"
Jul 28 17:34:26.906226 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:26.905518    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-126.ec2.internal" event={"ID":"17457ce2fa8ddf46113d3222bb973d5a","Type":"ContainerStarted","Data":"f763489eb2d3580d308b5260d34e678fe86cbe227aaa20ee027eae675ad9a996"}
Jul 28 17:34:26.922609 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:26.922232    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-126.ec2.internal" podStartSLOduration=4.922210809 podStartE2EDuration="4.922210809s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:34:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:34:26.921965817 +0000 UTC m=+5.583122177" watchObservedRunningTime="2026-07-28 17:34:26.922210809 +0000 UTC m=+5.583367165"
Jul 28 17:34:27.451601 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:27.451563    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3c4a388-fdcc-4744-b6e2-d52dc701f235-metrics-certs\") pod \"network-metrics-daemon-4tthg\" (UID: \"d3c4a388-fdcc-4744-b6e2-d52dc701f235\") " pod="openshift-multus/network-metrics-daemon-4tthg"
Jul 28 17:34:27.451763 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:27.451641    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8a8edb3a-34c9-4a66-81aa-49474fdfdf37-original-pull-secret\") pod \"global-pull-secret-syncer-z75m5\" (UID: \"8a8edb3a-34c9-4a66-81aa-49474fdfdf37\") " pod="kube-system/global-pull-secret-syncer-z75m5"
Jul 28 17:34:27.451763 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:27.451750    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 17:34:27.451888 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:27.451806    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a8edb3a-34c9-4a66-81aa-49474fdfdf37-original-pull-secret podName:8a8edb3a-34c9-4a66-81aa-49474fdfdf37 nodeName:}" failed. No retries permitted until 2026-07-28 17:34:31.451788231 +0000 UTC m=+10.112944566 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8a8edb3a-34c9-4a66-81aa-49474fdfdf37-original-pull-secret") pod "global-pull-secret-syncer-z75m5" (UID: "8a8edb3a-34c9-4a66-81aa-49474fdfdf37") : object "kube-system"/"original-pull-secret" not registered
Jul 28 17:34:27.451888 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:27.451887    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:34:27.451984 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:27.451921    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3c4a388-fdcc-4744-b6e2-d52dc701f235-metrics-certs podName:d3c4a388-fdcc-4744-b6e2-d52dc701f235 nodeName:}" failed. No retries permitted until 2026-07-28 17:34:31.451908908 +0000 UTC m=+10.113065258 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d3c4a388-fdcc-4744-b6e2-d52dc701f235-metrics-certs") pod "network-metrics-daemon-4tthg" (UID: "d3c4a388-fdcc-4744-b6e2-d52dc701f235") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:34:27.552038 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:27.552006    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gg7b\" (UniqueName: \"kubernetes.io/projected/41d54bc6-a997-48d3-9752-f78204bbdf43-kube-api-access-4gg7b\") pod \"network-check-target-74vbl\" (UID: \"41d54bc6-a997-48d3-9752-f78204bbdf43\") " pod="openshift-network-diagnostics/network-check-target-74vbl"
Jul 28 17:34:27.552206 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:27.552189    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 17:34:27.552270 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:27.552207    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 17:34:27.552270 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:27.552220    2563 projected.go:196] Error preparing data for projected volume kube-api-access-4gg7b for pod openshift-network-diagnostics/network-check-target-74vbl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:34:27.552362 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:27.552277    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/41d54bc6-a997-48d3-9752-f78204bbdf43-kube-api-access-4gg7b podName:41d54bc6-a997-48d3-9752-f78204bbdf43 nodeName:}" failed. No retries permitted until 2026-07-28 17:34:31.552259228 +0000 UTC m=+10.213415587 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-4gg7b" (UniqueName: "kubernetes.io/projected/41d54bc6-a997-48d3-9752-f78204bbdf43-kube-api-access-4gg7b") pod "network-check-target-74vbl" (UID: "41d54bc6-a997-48d3-9752-f78204bbdf43") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:34:27.848106 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:27.847965    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z75m5"
Jul 28 17:34:27.848106 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:27.847991    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tthg"
Jul 28 17:34:27.848594 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:27.848105    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z75m5" podUID="8a8edb3a-34c9-4a66-81aa-49474fdfdf37"
Jul 28 17:34:27.848594 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:27.848259    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4tthg" podUID="d3c4a388-fdcc-4744-b6e2-d52dc701f235"
Jul 28 17:34:28.847896 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:28.847863    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-74vbl"
Jul 28 17:34:28.848071 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:28.848001    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-74vbl" podUID="41d54bc6-a997-48d3-9752-f78204bbdf43"
Jul 28 17:34:29.847931 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:29.847888    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tthg"
Jul 28 17:34:29.848402 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:29.848020    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4tthg" podUID="d3c4a388-fdcc-4744-b6e2-d52dc701f235"
Jul 28 17:34:29.848402 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:29.848297    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z75m5"
Jul 28 17:34:29.848567 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:29.848412    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z75m5" podUID="8a8edb3a-34c9-4a66-81aa-49474fdfdf37"
Jul 28 17:34:30.847413 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:30.847369    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-74vbl"
Jul 28 17:34:30.847596 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:30.847531    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-74vbl" podUID="41d54bc6-a997-48d3-9752-f78204bbdf43"
Jul 28 17:34:31.489502 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:31.489145    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3c4a388-fdcc-4744-b6e2-d52dc701f235-metrics-certs\") pod \"network-metrics-daemon-4tthg\" (UID: \"d3c4a388-fdcc-4744-b6e2-d52dc701f235\") " pod="openshift-multus/network-metrics-daemon-4tthg"
Jul 28 17:34:31.489502 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:31.489234    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8a8edb3a-34c9-4a66-81aa-49474fdfdf37-original-pull-secret\") pod \"global-pull-secret-syncer-z75m5\" (UID: \"8a8edb3a-34c9-4a66-81aa-49474fdfdf37\") " pod="kube-system/global-pull-secret-syncer-z75m5"
Jul 28 17:34:31.489502 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:31.489377    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 17:34:31.489502 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:31.489379    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:34:31.489502 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:31.489457    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a8edb3a-34c9-4a66-81aa-49474fdfdf37-original-pull-secret podName:8a8edb3a-34c9-4a66-81aa-49474fdfdf37 nodeName:}" failed. No retries permitted until 2026-07-28 17:34:39.489420544 +0000 UTC m=+18.150576877 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8a8edb3a-34c9-4a66-81aa-49474fdfdf37-original-pull-secret") pod "global-pull-secret-syncer-z75m5" (UID: "8a8edb3a-34c9-4a66-81aa-49474fdfdf37") : object "kube-system"/"original-pull-secret" not registered
Jul 28 17:34:31.489502 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:31.489479    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3c4a388-fdcc-4744-b6e2-d52dc701f235-metrics-certs podName:d3c4a388-fdcc-4744-b6e2-d52dc701f235 nodeName:}" failed. No retries permitted until 2026-07-28 17:34:39.489468767 +0000 UTC m=+18.150625107 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d3c4a388-fdcc-4744-b6e2-d52dc701f235-metrics-certs") pod "network-metrics-daemon-4tthg" (UID: "d3c4a388-fdcc-4744-b6e2-d52dc701f235") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:34:31.589841 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:31.589532    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gg7b\" (UniqueName: \"kubernetes.io/projected/41d54bc6-a997-48d3-9752-f78204bbdf43-kube-api-access-4gg7b\") pod \"network-check-target-74vbl\" (UID: \"41d54bc6-a997-48d3-9752-f78204bbdf43\") " pod="openshift-network-diagnostics/network-check-target-74vbl"
Jul 28 17:34:31.589841 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:31.589752    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 17:34:31.589841 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:31.589773    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 17:34:31.589841 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:31.589787    2563 projected.go:196] Error preparing data for projected volume kube-api-access-4gg7b for pod openshift-network-diagnostics/network-check-target-74vbl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:34:31.589841 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:31.589845    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/41d54bc6-a997-48d3-9752-f78204bbdf43-kube-api-access-4gg7b podName:41d54bc6-a997-48d3-9752-f78204bbdf43 nodeName:}" failed. No retries permitted until 2026-07-28 17:34:39.589827822 +0000 UTC m=+18.250984170 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-4gg7b" (UniqueName: "kubernetes.io/projected/41d54bc6-a997-48d3-9752-f78204bbdf43-kube-api-access-4gg7b") pod "network-check-target-74vbl" (UID: "41d54bc6-a997-48d3-9752-f78204bbdf43") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:34:31.847122 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:31.847092    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z75m5"
Jul 28 17:34:31.848733 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:31.848198    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z75m5" podUID="8a8edb3a-34c9-4a66-81aa-49474fdfdf37"
Jul 28 17:34:31.848733 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:31.848575    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tthg"
Jul 28 17:34:31.848733 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:31.848676    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4tthg" podUID="d3c4a388-fdcc-4744-b6e2-d52dc701f235"
Jul 28 17:34:32.848219 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:32.847728    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-74vbl"
Jul 28 17:34:32.848219 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:32.847871    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-74vbl" podUID="41d54bc6-a997-48d3-9752-f78204bbdf43"
Jul 28 17:34:33.847907 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:33.847869    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z75m5"
Jul 28 17:34:33.848086 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:33.848003    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z75m5" podUID="8a8edb3a-34c9-4a66-81aa-49474fdfdf37"
Jul 28 17:34:33.848086 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:33.848070    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tthg"
Jul 28 17:34:33.848241 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:33.848215    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4tthg" podUID="d3c4a388-fdcc-4744-b6e2-d52dc701f235"
Jul 28 17:34:34.847547 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:34.847521    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-74vbl"
Jul 28 17:34:34.847699 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:34.847613    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-74vbl" podUID="41d54bc6-a997-48d3-9752-f78204bbdf43"
Jul 28 17:34:35.847696 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:35.847660    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tthg"
Jul 28 17:34:35.848158 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:35.847662    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z75m5"
Jul 28 17:34:35.848158 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:35.847791    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4tthg" podUID="d3c4a388-fdcc-4744-b6e2-d52dc701f235"
Jul 28 17:34:35.848158 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:35.847874    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z75m5" podUID="8a8edb3a-34c9-4a66-81aa-49474fdfdf37"
Jul 28 17:34:36.847609 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:36.847571    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-74vbl"
Jul 28 17:34:36.847766 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:36.847699    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-74vbl" podUID="41d54bc6-a997-48d3-9752-f78204bbdf43"
Jul 28 17:34:37.847181 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:37.847087    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z75m5"
Jul 28 17:34:37.847181 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:37.847122    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tthg"
Jul 28 17:34:37.847414 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:37.847201    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z75m5" podUID="8a8edb3a-34c9-4a66-81aa-49474fdfdf37"
Jul 28 17:34:37.847414 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:37.847326    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4tthg" podUID="d3c4a388-fdcc-4744-b6e2-d52dc701f235"
Jul 28 17:34:38.847981 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:38.847949    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-74vbl"
Jul 28 17:34:38.848560 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:38.848070    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-74vbl" podUID="41d54bc6-a997-48d3-9752-f78204bbdf43"
Jul 28 17:34:39.547582 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:39.547497    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3c4a388-fdcc-4744-b6e2-d52dc701f235-metrics-certs\") pod \"network-metrics-daemon-4tthg\" (UID: \"d3c4a388-fdcc-4744-b6e2-d52dc701f235\") " pod="openshift-multus/network-metrics-daemon-4tthg"
Jul 28 17:34:39.547582 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:39.547567    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8a8edb3a-34c9-4a66-81aa-49474fdfdf37-original-pull-secret\") pod \"global-pull-secret-syncer-z75m5\" (UID: \"8a8edb3a-34c9-4a66-81aa-49474fdfdf37\") " pod="kube-system/global-pull-secret-syncer-z75m5"
Jul 28 17:34:39.547834 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:39.547662    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:34:39.547834 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:39.547733    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3c4a388-fdcc-4744-b6e2-d52dc701f235-metrics-certs podName:d3c4a388-fdcc-4744-b6e2-d52dc701f235 nodeName:}" failed. No retries permitted until 2026-07-28 17:34:55.547710437 +0000 UTC m=+34.208866799 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d3c4a388-fdcc-4744-b6e2-d52dc701f235-metrics-certs") pod "network-metrics-daemon-4tthg" (UID: "d3c4a388-fdcc-4744-b6e2-d52dc701f235") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:34:39.547834 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:39.547669    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 17:34:39.547834 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:39.547806    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a8edb3a-34c9-4a66-81aa-49474fdfdf37-original-pull-secret podName:8a8edb3a-34c9-4a66-81aa-49474fdfdf37 nodeName:}" failed. No retries permitted until 2026-07-28 17:34:55.547790905 +0000 UTC m=+34.208947239 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8a8edb3a-34c9-4a66-81aa-49474fdfdf37-original-pull-secret") pod "global-pull-secret-syncer-z75m5" (UID: "8a8edb3a-34c9-4a66-81aa-49474fdfdf37") : object "kube-system"/"original-pull-secret" not registered
Jul 28 17:34:39.648123 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:39.648081    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gg7b\" (UniqueName: \"kubernetes.io/projected/41d54bc6-a997-48d3-9752-f78204bbdf43-kube-api-access-4gg7b\") pod \"network-check-target-74vbl\" (UID: \"41d54bc6-a997-48d3-9752-f78204bbdf43\") " pod="openshift-network-diagnostics/network-check-target-74vbl"
Jul 28 17:34:39.648317 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:39.648273    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 17:34:39.648317 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:39.648299    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 17:34:39.648317 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:39.648313    2563 projected.go:196] Error preparing data for projected volume kube-api-access-4gg7b for pod openshift-network-diagnostics/network-check-target-74vbl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:34:39.648512 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:39.648380    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/41d54bc6-a997-48d3-9752-f78204bbdf43-kube-api-access-4gg7b podName:41d54bc6-a997-48d3-9752-f78204bbdf43 nodeName:}" failed. No retries permitted until 2026-07-28 17:34:55.648359797 +0000 UTC m=+34.309516132 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-4gg7b" (UniqueName: "kubernetes.io/projected/41d54bc6-a997-48d3-9752-f78204bbdf43-kube-api-access-4gg7b") pod "network-check-target-74vbl" (UID: "41d54bc6-a997-48d3-9752-f78204bbdf43") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:34:39.847583 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:39.847528    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z75m5"
Jul 28 17:34:39.847776 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:39.847540    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tthg"
Jul 28 17:34:39.847776 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:39.847671    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z75m5" podUID="8a8edb3a-34c9-4a66-81aa-49474fdfdf37"
Jul 28 17:34:39.847776 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:39.847717    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4tthg" podUID="d3c4a388-fdcc-4744-b6e2-d52dc701f235"
Jul 28 17:34:40.847545 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:40.847510    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-74vbl"
Jul 28 17:34:40.848001 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:40.847631    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-74vbl" podUID="41d54bc6-a997-48d3-9752-f78204bbdf43"
Jul 28 17:34:41.853519 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:41.853487    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z75m5"
Jul 28 17:34:41.853956 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:41.853495    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tthg"
Jul 28 17:34:41.853956 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:41.853585    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z75m5" podUID="8a8edb3a-34c9-4a66-81aa-49474fdfdf37"
Jul 28 17:34:41.853956 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:41.853715    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4tthg" podUID="d3c4a388-fdcc-4744-b6e2-d52dc701f235"
Jul 28 17:34:42.847960 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:42.847929    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-74vbl"
Jul 28 17:34:42.848116 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:42.848030    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-74vbl" podUID="41d54bc6-a997-48d3-9752-f78204bbdf43"
Jul 28 17:34:43.850374 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:43.850213    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tthg"
Jul 28 17:34:43.850738 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:43.850256    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z75m5"
Jul 28 17:34:43.850738 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:43.850482    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4tthg" podUID="d3c4a388-fdcc-4744-b6e2-d52dc701f235"
Jul 28 17:34:43.850738 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:43.850520    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z75m5" podUID="8a8edb3a-34c9-4a66-81aa-49474fdfdf37"
Jul 28 17:34:43.938751 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:43.938709    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-krssz" event={"ID":"502adcd9-be65-4041-8d71-97c51f0af4a4","Type":"ContainerStarted","Data":"5d17df40ba219c6610dc2e978e378521be9d0e3d1879e4597c4564a3938573f3"}
Jul 28 17:34:43.938844 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:43.938811    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-krssz" event={"ID":"502adcd9-be65-4041-8d71-97c51f0af4a4","Type":"ContainerStarted","Data":"e4944f2d3e4900cdc41f5feea59a365485d384ab1317960eb053d0c79c0b992b"}
Jul 28 17:34:43.940199 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:43.940172    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-25z5w" event={"ID":"118595be-d78d-40e2-9d5b-937167697690","Type":"ContainerStarted","Data":"d40ecd9faa4a158cc115718da70f3562f1cc99100fc8cb1922e059146d34db5a"}
Jul 28 17:34:43.941533 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:43.941510    2563 generic.go:356] "Generic (PLEG): container finished" podID="3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b" containerID="181807c879a52e2f929c0ec0afb9723ec1af697d0572a36100ee41de43c626c8" exitCode=0
Jul 28 17:34:43.941614 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:43.941592    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x2h5c" event={"ID":"3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b","Type":"ContainerDied","Data":"181807c879a52e2f929c0ec0afb9723ec1af697d0572a36100ee41de43c626c8"}
Jul 28 17:34:43.943111 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:43.943030    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t7h6" event={"ID":"6f0b63c9-e5d8-4bb4-8244-eefc95d9ff67","Type":"ContainerStarted","Data":"1bae432f63267f180b7d011dcf133ff7a083cc7773705fbbf5cf6f87d14eadbd"}
Jul 28 17:34:43.945213 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:43.945183    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-z8fhq" event={"ID":"d6004102-aca2-4f56-8ddb-94374d3bc900","Type":"ContainerStarted","Data":"2d0473711433a962bf527b1d55b6ec94e38d4970190909b579c99e1d74cbf850"}
Jul 28 17:34:43.948866 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:43.948844    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2qvs8" event={"ID":"7e2a258e-828b-4bbd-96b8-56cb5a7dd36f","Type":"ContainerStarted","Data":"0ff5542fe4195f42732794599bd0245133ec01a2be9bcd32850ae19691486605"}
Jul 28 17:34:43.950067 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:43.950049    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8mhf9" event={"ID":"ea8d2cd4-eb75-479a-a7fa-730d68fb1cee","Type":"ContainerStarted","Data":"2f88e449d5e51944dbc7f832933e4197c5c6183593d8b7f5336e67d9e1442bcc"}
Jul 28 17:34:43.951104 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:43.951088    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-j4wkp" event={"ID":"654d1f40-f526-4325-b40c-fb544079bff4","Type":"ContainerStarted","Data":"8d4171a418805080fa5cc3d949f96e495d87cdd8496e2ddcd46adc51cd736714"}
Jul 28 17:34:43.958718 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:43.958683    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-25z5w" podStartSLOduration=4.351046257 podStartE2EDuration="22.958672392s" totalImagesPullingTime="18.607626135s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:34:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:34:43.958380083 +0000 UTC m=+22.619536441" watchObservedRunningTime="2026-07-28 17:34:43.958672392 +0000 UTC m=+22.619828747"
Jul 28 17:34:43.977955 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:43.977923    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-z8fhq" podStartSLOduration=4.355909311 podStartE2EDuration="22.977912429s" totalImagesPullingTime="18.622003118s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:34:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:34:43.977586215 +0000 UTC m=+22.638742571" watchObservedRunningTime="2026-07-28 17:34:43.977912429 +0000 UTC m=+22.639068785"
Jul 28 17:34:44.017348 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:44.017210    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-2qvs8" podStartSLOduration=8.46955712 podStartE2EDuration="23.017200102s" totalImagesPullingTime="14.547642982s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:34:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:34:44.016861935 +0000 UTC m=+22.678018290" watchObservedRunningTime="2026-07-28 17:34:44.017200102 +0000 UTC m=+22.678356456"
Jul 28 17:34:44.033297 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:44.033257    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-j4wkp" podStartSLOduration=4.422573015 podStartE2EDuration="23.033242746s" totalImagesPullingTime="18.610669731s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:34:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:34:44.032366961 +0000 UTC m=+22.693523315" watchObservedRunningTime="2026-07-28 17:34:44.033242746 +0000 UTC m=+22.694399102"
Jul 28 17:34:44.053529 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:44.053489    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-8mhf9" podStartSLOduration=4.411372642 podStartE2EDuration="23.0534717s" totalImagesPullingTime="18.642099058s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:34:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:34:44.053113814 +0000 UTC m=+22.714270169" watchObservedRunningTime="2026-07-28 17:34:44.0534717 +0000 UTC m=+22.714628056"
Jul 28 17:34:44.821187 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:44.821160    2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 28 17:34:44.847428 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:44.847403    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-74vbl"
Jul 28 17:34:44.847566 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:44.847541    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-74vbl" podUID="41d54bc6-a997-48d3-9752-f78204bbdf43"
Jul 28 17:34:44.956541 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:44.956502    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-krssz" event={"ID":"502adcd9-be65-4041-8d71-97c51f0af4a4","Type":"ContainerStarted","Data":"1b7a07a7aa8897ce9fa1a56bf5a3c39c3492c8b4a6d6988471266fb4ca12cb6d"}
Jul 28 17:34:44.957315 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:44.956592    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-krssz" event={"ID":"502adcd9-be65-4041-8d71-97c51f0af4a4","Type":"ContainerStarted","Data":"875b71f23c130bcc69b71ee985dc0464b58c5da8a81fb13cbed52c5e73bab9ba"}
Jul 28 17:34:44.957315 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:44.956641    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-krssz" event={"ID":"502adcd9-be65-4041-8d71-97c51f0af4a4","Type":"ContainerStarted","Data":"1032240a88a216a3ea78643f1d3fdb56ebb6f636c35d687bfdd4004437fb352b"}
Jul 28 17:34:44.957315 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:44.956690    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-krssz" event={"ID":"502adcd9-be65-4041-8d71-97c51f0af4a4","Type":"ContainerStarted","Data":"50238266cb5788c58e9014d92feedb391c06d9f27aebeb589be88a8ad1761ea2"}
Jul 28 17:34:44.958448 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:44.958397    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t7h6" event={"ID":"6f0b63c9-e5d8-4bb4-8244-eefc95d9ff67","Type":"ContainerStarted","Data":"93f58dae35df678924562fe26e708dda1c4841d6ddd305baa37f9788e6e71951"}
Jul 28 17:34:44.960044 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:44.959992    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nwwzc" event={"ID":"a35e7d0f-47ba-4b8a-8987-47f0163cc156","Type":"ContainerStarted","Data":"0d7241c963bfb80d19ed763a0633762f12ef819333618a4bad72a559b094c623"}
Jul 28 17:34:44.976977 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:44.976937    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-nwwzc" podStartSLOduration=5.368464909 podStartE2EDuration="23.976923323s" totalImagesPullingTime="18.608458414s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:34:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:34:44.976830242 +0000 UTC m=+23.637986597" watchObservedRunningTime="2026-07-28 17:34:44.976923323 +0000 UTC m=+23.638079658"
Jul 28 17:34:45.779348 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:45.779244    2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-28T17:34:44.821180225Z","UUID":"b6037546-c00a-47a2-8d66-48413ab4235f","Handler":null,"Name":"","Endpoint":""}
Jul 28 17:34:45.782551 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:45.782524    2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 28 17:34:45.782680 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:45.782569    2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 28 17:34:45.848145 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:45.848115    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tthg"
Jul 28 17:34:45.848316 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:45.848240    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4tthg" podUID="d3c4a388-fdcc-4744-b6e2-d52dc701f235"
Jul 28 17:34:45.848382 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:45.848304    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z75m5"
Jul 28 17:34:45.848474 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:45.848452    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z75m5" podUID="8a8edb3a-34c9-4a66-81aa-49474fdfdf37"
Jul 28 17:34:46.328420 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:46.328382    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-2qvs8"
Jul 28 17:34:46.329154 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:46.329134    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-2qvs8"
Jul 28 17:34:46.847377 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:46.847299    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-74vbl"
Jul 28 17:34:46.847545 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:46.847416    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-74vbl" podUID="41d54bc6-a997-48d3-9752-f78204bbdf43"
Jul 28 17:34:46.968524 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:46.968467    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-krssz" event={"ID":"502adcd9-be65-4041-8d71-97c51f0af4a4","Type":"ContainerStarted","Data":"bcdbf79803192c8bfb5c27d7ed2fa5b698fe98d5788857613e8bfcd29b33b83a"}
Jul 28 17:34:46.970841 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:46.970809    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t7h6" event={"ID":"6f0b63c9-e5d8-4bb4-8244-eefc95d9ff67","Type":"ContainerStarted","Data":"ec8f0928aa65ca58d47ef6060b1a12067ed3d8cdeaacab263c7b5c6f33e26dfc"}
Jul 28 17:34:46.971222 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:46.971115    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-2qvs8"
Jul 28 17:34:46.971487 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:46.971469    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-2qvs8"
Jul 28 17:34:47.005295 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:47.005237    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t7h6" podStartSLOduration=5.1484107 podStartE2EDuration="26.005220443s" totalImagesPullingTime="20.856809743s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:34:21 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:34:46.990773136 +0000 UTC m=+25.651929485" watchObservedRunningTime="2026-07-28 17:34:47.005220443 +0000 UTC m=+25.666376797"
Jul 28 17:34:47.851402 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:47.851377    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z75m5"
Jul 28 17:34:47.851808 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:47.851518    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z75m5" podUID="8a8edb3a-34c9-4a66-81aa-49474fdfdf37"
Jul 28 17:34:47.851808 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:47.851560    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tthg"
Jul 28 17:34:47.851808 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:47.851686    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4tthg" podUID="d3c4a388-fdcc-4744-b6e2-d52dc701f235"
Jul 28 17:34:48.847266 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:48.847135    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-74vbl"
Jul 28 17:34:48.847402 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:48.847378    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-74vbl" podUID="41d54bc6-a997-48d3-9752-f78204bbdf43"
Jul 28 17:34:48.979003 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:48.978929    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-krssz" event={"ID":"502adcd9-be65-4041-8d71-97c51f0af4a4","Type":"ContainerStarted","Data":"28f03bccb01e1575b7c91a4946f189674cfce7723a88ca5c695a066248b02dd7"}
Jul 28 17:34:48.979793 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:48.979130    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:48.979793 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:48.979189    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:48.980833 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:48.980811    2563 generic.go:356] "Generic (PLEG): container finished" podID="3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b" containerID="98ba48ad84861a1bad78bca1ab42d269c27ef1cc8c20468b0e82ef768d4c5bfa" exitCode=0
Jul 28 17:34:48.980959 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:48.980879    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x2h5c" event={"ID":"3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b","Type":"ContainerDied","Data":"98ba48ad84861a1bad78bca1ab42d269c27ef1cc8c20468b0e82ef768d4c5bfa"}
Jul 28 17:34:48.994067 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:48.994042    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:49.010665 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:49.010630    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-krssz" podStartSLOduration=8.949580461 podStartE2EDuration="28.010619436s" totalImagesPullingTime="19.061038975s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:34:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:34:49.009342931 +0000 UTC m=+27.670499287" watchObservedRunningTime="2026-07-28 17:34:49.010619436 +0000 UTC m=+27.671775790"
Jul 28 17:34:49.847921 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:49.847684    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z75m5"
Jul 28 17:34:49.848047 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:49.847743    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tthg"
Jul 28 17:34:49.848094 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:49.848038    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z75m5" podUID="8a8edb3a-34c9-4a66-81aa-49474fdfdf37"
Jul 28 17:34:49.848094 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:49.848084    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4tthg" podUID="d3c4a388-fdcc-4744-b6e2-d52dc701f235"
Jul 28 17:34:49.983270 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:49.983150    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:49.998354 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:49.998316    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:34:50.847957 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:50.847932    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-74vbl"
Jul 28 17:34:50.848131 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:50.848046    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-74vbl" podUID="41d54bc6-a997-48d3-9752-f78204bbdf43"
Jul 28 17:34:50.986657 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:50.986628    2563 generic.go:356] "Generic (PLEG): container finished" podID="3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b" containerID="00b4e5130480f0aa2b8b7246c0284c3b1f29334cf17b76f7b68a231194e7f07d" exitCode=0
Jul 28 17:34:50.987039 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:50.986706    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x2h5c" event={"ID":"3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b","Type":"ContainerDied","Data":"00b4e5130480f0aa2b8b7246c0284c3b1f29334cf17b76f7b68a231194e7f07d"}
Jul 28 17:34:51.071264 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:51.071237    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-74vbl"]
Jul 28 17:34:51.071412 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:51.071323    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-74vbl"
Jul 28 17:34:51.071496 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:51.071406    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-74vbl" podUID="41d54bc6-a997-48d3-9752-f78204bbdf43"
Jul 28 17:34:51.074076 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:51.074053    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-z75m5"]
Jul 28 17:34:51.074194 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:51.074151    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z75m5"
Jul 28 17:34:51.074265 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:51.074246    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z75m5" podUID="8a8edb3a-34c9-4a66-81aa-49474fdfdf37"
Jul 28 17:34:51.074671 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:51.074650    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4tthg"]
Jul 28 17:34:51.074791 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:51.074755    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tthg"
Jul 28 17:34:51.074873 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:51.074854    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4tthg" podUID="d3c4a388-fdcc-4744-b6e2-d52dc701f235"
Jul 28 17:34:52.847297 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:52.847261    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-74vbl"
Jul 28 17:34:52.847684 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:52.847305    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z75m5"
Jul 28 17:34:52.847684 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:52.847359    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tthg"
Jul 28 17:34:52.847684 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:52.847362    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-74vbl" podUID="41d54bc6-a997-48d3-9752-f78204bbdf43"
Jul 28 17:34:52.847684 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:52.847463    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z75m5" podUID="8a8edb3a-34c9-4a66-81aa-49474fdfdf37"
Jul 28 17:34:52.847684 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:52.847565    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4tthg" podUID="d3c4a388-fdcc-4744-b6e2-d52dc701f235"
Jul 28 17:34:52.993842 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:52.993808    2563 generic.go:356] "Generic (PLEG): container finished" podID="3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b" containerID="3a09cfad142c3922df94ce972f5e2ae13f93cc750a270bd98797acfed4fe4f7d" exitCode=0
Jul 28 17:34:52.994053 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:52.993861    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x2h5c" event={"ID":"3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b","Type":"ContainerDied","Data":"3a09cfad142c3922df94ce972f5e2ae13f93cc750a270bd98797acfed4fe4f7d"}
Jul 28 17:34:54.848014 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:54.847774    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-74vbl"
Jul 28 17:34:54.848427 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:54.847807    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z75m5"
Jul 28 17:34:54.848427 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:54.848121    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-74vbl" podUID="41d54bc6-a997-48d3-9752-f78204bbdf43"
Jul 28 17:34:54.848427 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:54.847820    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tthg"
Jul 28 17:34:54.848427 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:54.848190    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z75m5" podUID="8a8edb3a-34c9-4a66-81aa-49474fdfdf37"
Jul 28 17:34:54.848427 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:54.848288    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4tthg" podUID="d3c4a388-fdcc-4744-b6e2-d52dc701f235"
Jul 28 17:34:55.408069 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:55.408038    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:34:55.492828 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:55.492784    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:34:55.565967 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:55.565928    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8a8edb3a-34c9-4a66-81aa-49474fdfdf37-original-pull-secret\") pod \"global-pull-secret-syncer-z75m5\" (UID: \"8a8edb3a-34c9-4a66-81aa-49474fdfdf37\") " pod="kube-system/global-pull-secret-syncer-z75m5"
Jul 28 17:34:55.566131 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:55.565989    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3c4a388-fdcc-4744-b6e2-d52dc701f235-metrics-certs\") pod \"network-metrics-daemon-4tthg\" (UID: \"d3c4a388-fdcc-4744-b6e2-d52dc701f235\") " pod="openshift-multus/network-metrics-daemon-4tthg"
Jul 28 17:34:55.566131 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:55.566048    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 17:34:55.566131 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:55.566083    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:34:55.566131 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:55.566112    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a8edb3a-34c9-4a66-81aa-49474fdfdf37-original-pull-secret podName:8a8edb3a-34c9-4a66-81aa-49474fdfdf37 nodeName:}" failed. No retries permitted until 2026-07-28 17:35:27.566097029 +0000 UTC m=+66.227253367 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8a8edb3a-34c9-4a66-81aa-49474fdfdf37-original-pull-secret") pod "global-pull-secret-syncer-z75m5" (UID: "8a8edb3a-34c9-4a66-81aa-49474fdfdf37") : object "kube-system"/"original-pull-secret" not registered
Jul 28 17:34:55.566131 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:55.566131    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3c4a388-fdcc-4744-b6e2-d52dc701f235-metrics-certs podName:d3c4a388-fdcc-4744-b6e2-d52dc701f235 nodeName:}" failed. No retries permitted until 2026-07-28 17:35:27.566120901 +0000 UTC m=+66.227277234 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d3c4a388-fdcc-4744-b6e2-d52dc701f235-metrics-certs") pod "network-metrics-daemon-4tthg" (UID: "d3c4a388-fdcc-4744-b6e2-d52dc701f235") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:34:55.667282 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:55.667211    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gg7b\" (UniqueName: \"kubernetes.io/projected/41d54bc6-a997-48d3-9752-f78204bbdf43-kube-api-access-4gg7b\") pod \"network-check-target-74vbl\" (UID: \"41d54bc6-a997-48d3-9752-f78204bbdf43\") " pod="openshift-network-diagnostics/network-check-target-74vbl"
Jul 28 17:34:55.667428 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:55.667332    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 17:34:55.667428 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:55.667348    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 17:34:55.667428 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:55.667359    2563 projected.go:196] Error preparing data for projected volume kube-api-access-4gg7b for pod openshift-network-diagnostics/network-check-target-74vbl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:34:55.667428 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:55.667418    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/41d54bc6-a997-48d3-9752-f78204bbdf43-kube-api-access-4gg7b podName:41d54bc6-a997-48d3-9752-f78204bbdf43 nodeName:}" failed. No retries permitted until 2026-07-28 17:35:27.667401121 +0000 UTC m=+66.328557470 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-4gg7b" (UniqueName: "kubernetes.io/projected/41d54bc6-a997-48d3-9752-f78204bbdf43-kube-api-access-4gg7b") pod "network-check-target-74vbl" (UID: "41d54bc6-a997-48d3-9752-f78204bbdf43") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:34:56.847593 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:56.847572    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-74vbl"
Jul 28 17:34:56.848035 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:56.847604    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z75m5"
Jul 28 17:34:56.848035 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:56.847608    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tthg"
Jul 28 17:34:56.848035 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:56.847674    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-74vbl" podUID="41d54bc6-a997-48d3-9752-f78204bbdf43"
Jul 28 17:34:56.848035 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:56.847748    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z75m5" podUID="8a8edb3a-34c9-4a66-81aa-49474fdfdf37"
Jul 28 17:34:56.848035 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:56.847845    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4tthg" podUID="d3c4a388-fdcc-4744-b6e2-d52dc701f235"
Jul 28 17:34:57.005556 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.005526    2563 generic.go:356] "Generic (PLEG): container finished" podID="3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b" containerID="d40b664dc8954ac466c8ad9246d55f259b9dc15c9c8ed9d49d50de0384c9f382" exitCode=0
Jul 28 17:34:57.005664 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.005595    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x2h5c" event={"ID":"3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b","Type":"ContainerDied","Data":"d40b664dc8954ac466c8ad9246d55f259b9dc15c9c8ed9d49d50de0384c9f382"}
Jul 28 17:34:57.125649 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.125625    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-126.ec2.internal" event="NodeReady"
Jul 28 17:34:57.125787 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.125738    2563 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 28 17:34:57.181107 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.181074    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-lpwn9"]
Jul 28 17:34:57.182803 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.182788    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-lpwn9"
Jul 28 17:34:57.186229 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.186211    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-7mzg2\""
Jul 28 17:34:57.186327 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.186216    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 28 17:34:57.186512 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.186500    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 28 17:34:57.186590 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.186571    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 28 17:34:57.187783 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.187227    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-psd2g"]
Jul 28 17:34:57.189038 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.189022    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-psd2g"
Jul 28 17:34:57.203816 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.203794    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 28 17:34:57.203905 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.203819    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 28 17:34:57.203905 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.203845    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lxmvx\""
Jul 28 17:34:57.205239 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.205220    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-lpwn9"]
Jul 28 17:34:57.208586 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.208570    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-psd2g"]
Jul 28 17:34:57.280200 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.280160    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/897c5c1b-7fb1-40eb-8b4a-46e737d8e991-metrics-tls\") pod \"dns-default-psd2g\" (UID: \"897c5c1b-7fb1-40eb-8b4a-46e737d8e991\") " pod="openshift-dns/dns-default-psd2g"
Jul 28 17:34:57.280200 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.280202    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/093b4d76-2af2-4586-8421-68b7047ff74e-cert\") pod \"ingress-canary-lpwn9\" (UID: \"093b4d76-2af2-4586-8421-68b7047ff74e\") " pod="openshift-ingress-canary/ingress-canary-lpwn9"
Jul 28 17:34:57.282943 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.280617    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/897c5c1b-7fb1-40eb-8b4a-46e737d8e991-tmp-dir\") pod \"dns-default-psd2g\" (UID: \"897c5c1b-7fb1-40eb-8b4a-46e737d8e991\") " pod="openshift-dns/dns-default-psd2g"
Jul 28 17:34:57.282943 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.280730    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/897c5c1b-7fb1-40eb-8b4a-46e737d8e991-config-volume\") pod \"dns-default-psd2g\" (UID: \"897c5c1b-7fb1-40eb-8b4a-46e737d8e991\") " pod="openshift-dns/dns-default-psd2g"
Jul 28 17:34:57.282943 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.280830    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxztx\" (UniqueName: \"kubernetes.io/projected/093b4d76-2af2-4586-8421-68b7047ff74e-kube-api-access-rxztx\") pod \"ingress-canary-lpwn9\" (UID: \"093b4d76-2af2-4586-8421-68b7047ff74e\") " pod="openshift-ingress-canary/ingress-canary-lpwn9"
Jul 28 17:34:57.282943 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.280865    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vw25w\" (UniqueName: \"kubernetes.io/projected/897c5c1b-7fb1-40eb-8b4a-46e737d8e991-kube-api-access-vw25w\") pod \"dns-default-psd2g\" (UID: \"897c5c1b-7fb1-40eb-8b4a-46e737d8e991\") " pod="openshift-dns/dns-default-psd2g"
Jul 28 17:34:57.368908 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.368877    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:34:57.381864 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.381844    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/897c5c1b-7fb1-40eb-8b4a-46e737d8e991-metrics-tls\") pod \"dns-default-psd2g\" (UID: \"897c5c1b-7fb1-40eb-8b4a-46e737d8e991\") " pod="openshift-dns/dns-default-psd2g"
Jul 28 17:34:57.381943 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.381876    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/093b4d76-2af2-4586-8421-68b7047ff74e-cert\") pod \"ingress-canary-lpwn9\" (UID: \"093b4d76-2af2-4586-8421-68b7047ff74e\") " pod="openshift-ingress-canary/ingress-canary-lpwn9"
Jul 28 17:34:57.381943 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.381909    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/897c5c1b-7fb1-40eb-8b4a-46e737d8e991-tmp-dir\") pod \"dns-default-psd2g\" (UID: \"897c5c1b-7fb1-40eb-8b4a-46e737d8e991\") " pod="openshift-dns/dns-default-psd2g"
Jul 28 17:34:57.381943 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.381930    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/897c5c1b-7fb1-40eb-8b4a-46e737d8e991-config-volume\") pod \"dns-default-psd2g\" (UID: \"897c5c1b-7fb1-40eb-8b4a-46e737d8e991\") " pod="openshift-dns/dns-default-psd2g"
Jul 28 17:34:57.382043 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.381973    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxztx\" (UniqueName: \"kubernetes.io/projected/093b4d76-2af2-4586-8421-68b7047ff74e-kube-api-access-rxztx\") pod \"ingress-canary-lpwn9\" (UID: \"093b4d76-2af2-4586-8421-68b7047ff74e\") " pod="openshift-ingress-canary/ingress-canary-lpwn9"
Jul 28 17:34:57.382043 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.381988    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vw25w\" (UniqueName: \"kubernetes.io/projected/897c5c1b-7fb1-40eb-8b4a-46e737d8e991-kube-api-access-vw25w\") pod \"dns-default-psd2g\" (UID: \"897c5c1b-7fb1-40eb-8b4a-46e737d8e991\") " pod="openshift-dns/dns-default-psd2g"
Jul 28 17:34:57.382043 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:57.381994    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 17:34:57.382156 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:57.382051    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/897c5c1b-7fb1-40eb-8b4a-46e737d8e991-metrics-tls podName:897c5c1b-7fb1-40eb-8b4a-46e737d8e991 nodeName:}" failed. No retries permitted until 2026-07-28 17:34:57.882030671 +0000 UTC m=+36.543187005 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/897c5c1b-7fb1-40eb-8b4a-46e737d8e991-metrics-tls") pod "dns-default-psd2g" (UID: "897c5c1b-7fb1-40eb-8b4a-46e737d8e991") : secret "dns-default-metrics-tls" not found
Jul 28 17:34:57.382339 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.382308    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/897c5c1b-7fb1-40eb-8b4a-46e737d8e991-tmp-dir\") pod \"dns-default-psd2g\" (UID: \"897c5c1b-7fb1-40eb-8b4a-46e737d8e991\") " pod="openshift-dns/dns-default-psd2g"
Jul 28 17:34:57.382526 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.382509    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/897c5c1b-7fb1-40eb-8b4a-46e737d8e991-config-volume\") pod \"dns-default-psd2g\" (UID: \"897c5c1b-7fb1-40eb-8b4a-46e737d8e991\") " pod="openshift-dns/dns-default-psd2g"
Jul 28 17:34:57.386200 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.386178    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/093b4d76-2af2-4586-8421-68b7047ff74e-cert\") pod \"ingress-canary-lpwn9\" (UID: \"093b4d76-2af2-4586-8421-68b7047ff74e\") " pod="openshift-ingress-canary/ingress-canary-lpwn9"
Jul 28 17:34:57.392149 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.392128    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vw25w\" (UniqueName: \"kubernetes.io/projected/897c5c1b-7fb1-40eb-8b4a-46e737d8e991-kube-api-access-vw25w\") pod \"dns-default-psd2g\" (UID: \"897c5c1b-7fb1-40eb-8b4a-46e737d8e991\") " pod="openshift-dns/dns-default-psd2g"
Jul 28 17:34:57.392697 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.392681    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxztx\" (UniqueName: \"kubernetes.io/projected/093b4d76-2af2-4586-8421-68b7047ff74e-kube-api-access-rxztx\") pod \"ingress-canary-lpwn9\" (UID: \"093b4d76-2af2-4586-8421-68b7047ff74e\") " pod="openshift-ingress-canary/ingress-canary-lpwn9"
Jul 28 17:34:57.469796 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.469764    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:34:57.491412 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.491394    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-lpwn9"
Jul 28 17:34:57.637607 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.637538    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-lpwn9"]
Jul 28 17:34:57.641693 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:57.641667    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod093b4d76_2af2_4586_8421_68b7047ff74e.slice/crio-81932e1b77b116d842fbd248488eff2a41d1f3920cbf6eb1cfeaa17b0f4234bd WatchSource:0}: Error finding container 81932e1b77b116d842fbd248488eff2a41d1f3920cbf6eb1cfeaa17b0f4234bd: Status 404 returned error can't find the container with id 81932e1b77b116d842fbd248488eff2a41d1f3920cbf6eb1cfeaa17b0f4234bd
Jul 28 17:34:57.763470 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.763410    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-zc5p9"]
Jul 28 17:34:57.766306 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.766177    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-zc5p9"
Jul 28 17:34:57.767314 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.767080    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-jqft4"]
Jul 28 17:34:57.768749 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.768731    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 28 17:34:57.769128 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.769110    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-bmz8v\""
Jul 28 17:34:57.769186 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.769152    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 28 17:34:57.769186 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.769119    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7c67d86669-hftv7"]
Jul 28 17:34:57.769266 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.769187    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-jqft4"
Jul 28 17:34:57.769372 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.769357    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 28 17:34:57.769448 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.769412    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 28 17:34:57.770954 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.770939    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7c67d86669-hftv7"
Jul 28 17:34:57.774360 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.774121    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 28 17:34:57.774360 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.774151    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 28 17:34:57.774360 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.774157    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 28 17:34:57.774360 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.774157    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-bbm7j\""
Jul 28 17:34:57.774360 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.774222    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-7nq9j\""
Jul 28 17:34:57.774360 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.774230    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 28 17:34:57.774360 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.774247    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 28 17:34:57.774360 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.774164    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 28 17:34:57.774360 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.774364    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 28 17:34:57.774768 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.774417    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 28 17:34:57.774768 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.774510    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 28 17:34:57.774768 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.774627    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 28 17:34:57.779590 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.779573    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-zc5p9"]
Jul 28 17:34:57.784208 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.784188    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-jqft4"]
Jul 28 17:34:57.796487 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.796428    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7c67d86669-hftv7"]
Jul 28 17:34:57.883994 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.883964    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-25vvm"]
Jul 28 17:34:57.885901 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.885886    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-25vvm"
Jul 28 17:34:57.888326 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.888245    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-pnbkh\""
Jul 28 17:34:57.888473 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.888456    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 28 17:34:57.888554 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.888533    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 28 17:34:57.888554 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.888547    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 28 17:34:57.888651 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.888635    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 28 17:34:57.899230 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.899206    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sw5pj\" (UniqueName: \"kubernetes.io/projected/7393a17e-a626-450c-bcb9-04ea1774e002-kube-api-access-sw5pj\") pod \"cluster-monitoring-operator-548b489cb8-jqft4\" (UID: \"7393a17e-a626-450c-bcb9-04ea1774e002\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-jqft4"
Jul 28 17:34:57.899348 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.899244    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/0641104e-1805-4a32-97fd-e9ee7c550152-stats-auth\") pod \"router-default-7c67d86669-hftv7\" (UID: \"0641104e-1805-4a32-97fd-e9ee7c550152\") " pod="openshift-ingress/router-default-7c67d86669-hftv7"
Jul 28 17:34:57.899348 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.899268    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jhp9\" (UniqueName: \"kubernetes.io/projected/0641104e-1805-4a32-97fd-e9ee7c550152-kube-api-access-8jhp9\") pod \"router-default-7c67d86669-hftv7\" (UID: \"0641104e-1805-4a32-97fd-e9ee7c550152\") " pod="openshift-ingress/router-default-7c67d86669-hftv7"
Jul 28 17:34:57.899348 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.899304    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/897c5c1b-7fb1-40eb-8b4a-46e737d8e991-metrics-tls\") pod \"dns-default-psd2g\" (UID: \"897c5c1b-7fb1-40eb-8b4a-46e737d8e991\") " pod="openshift-dns/dns-default-psd2g"
Jul 28 17:34:57.899348 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.899329    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7393a17e-a626-450c-bcb9-04ea1774e002-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-jqft4\" (UID: \"7393a17e-a626-450c-bcb9-04ea1774e002\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-jqft4"
Jul 28 17:34:57.899576 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.899387    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0641104e-1805-4a32-97fd-e9ee7c550152-service-ca-bundle\") pod \"router-default-7c67d86669-hftv7\" (UID: \"0641104e-1805-4a32-97fd-e9ee7c550152\") " pod="openshift-ingress/router-default-7c67d86669-hftv7"
Jul 28 17:34:57.899576 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.899414    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0641104e-1805-4a32-97fd-e9ee7c550152-metrics-certs\") pod \"router-default-7c67d86669-hftv7\" (UID: \"0641104e-1805-4a32-97fd-e9ee7c550152\") " pod="openshift-ingress/router-default-7c67d86669-hftv7"
Jul 28 17:34:57.899576 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.899453    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/dfad001a-8bdb-474e-9c4a-cfed601c7748-config\") pod \"service-ca-operator-5d6c75f84f-zc5p9\" (UID: \"dfad001a-8bdb-474e-9c4a-cfed601c7748\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-zc5p9"
Jul 28 17:34:57.899576 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:57.899528    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 17:34:57.899576 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.899554    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/dfad001a-8bdb-474e-9c4a-cfed601c7748-serving-cert\") pod \"service-ca-operator-5d6c75f84f-zc5p9\" (UID: \"dfad001a-8bdb-474e-9c4a-cfed601c7748\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-zc5p9"
Jul 28 17:34:57.899795 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:57.899582    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/897c5c1b-7fb1-40eb-8b4a-46e737d8e991-metrics-tls podName:897c5c1b-7fb1-40eb-8b4a-46e737d8e991 nodeName:}" failed. No retries permitted until 2026-07-28 17:34:58.899564956 +0000 UTC m=+37.560721304 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/897c5c1b-7fb1-40eb-8b4a-46e737d8e991-metrics-tls") pod "dns-default-psd2g" (UID: "897c5c1b-7fb1-40eb-8b4a-46e737d8e991") : secret "dns-default-metrics-tls" not found
Jul 28 17:34:57.899795 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.899643    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/7393a17e-a626-450c-bcb9-04ea1774e002-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-jqft4\" (UID: \"7393a17e-a626-450c-bcb9-04ea1774e002\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-jqft4"
Jul 28 17:34:57.899795 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.899691    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/0641104e-1805-4a32-97fd-e9ee7c550152-default-certificate\") pod \"router-default-7c67d86669-hftv7\" (UID: \"0641104e-1805-4a32-97fd-e9ee7c550152\") " pod="openshift-ingress/router-default-7c67d86669-hftv7"
Jul 28 17:34:57.899795 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.899721    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltgj7\" (UniqueName: \"kubernetes.io/projected/dfad001a-8bdb-474e-9c4a-cfed601c7748-kube-api-access-ltgj7\") pod \"service-ca-operator-5d6c75f84f-zc5p9\" (UID: \"dfad001a-8bdb-474e-9c4a-cfed601c7748\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-zc5p9"
Jul 28 17:34:57.900199 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.900180    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-25vvm"]
Jul 28 17:34:57.951911 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.951883    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:34:57.969286 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.969264    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-ch2rb"]
Jul 28 17:34:57.971208 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.971194    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-75cf897b8-llxcl"]
Jul 28 17:34:57.971325 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.971307    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-ch2rb"
Jul 28 17:34:57.972923 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.972906    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75cf897b8-llxcl"
Jul 28 17:34:57.973736 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.973719    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-4fvb8\""
Jul 28 17:34:57.976336 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.974256    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 28 17:34:57.976336 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.974356    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 28 17:34:57.976336 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.974358    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 28 17:34:57.976336 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.975195    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 28 17:34:57.976336 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.975414    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 28 17:34:57.976336 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.975465    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 28 17:34:57.976336 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.975623    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-n7jzp\""
Jul 28 17:34:57.981652 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.981632    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-ch2rb"]
Jul 28 17:34:57.982602 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.982583    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 28 17:34:57.985206 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:57.985187    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-75cf897b8-llxcl"]
Jul 28 17:34:58.000123 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.000106    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ea3e1303-f4dc-4511-abbc-9a7c0e789164-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-25vvm\" (UID: \"ea3e1303-f4dc-4511-abbc-9a7c0e789164\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-25vvm"
Jul 28 17:34:58.000198 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.000133    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sw5pj\" (UniqueName: \"kubernetes.io/projected/7393a17e-a626-450c-bcb9-04ea1774e002-kube-api-access-sw5pj\") pod \"cluster-monitoring-operator-548b489cb8-jqft4\" (UID: \"7393a17e-a626-450c-bcb9-04ea1774e002\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-jqft4"
Jul 28 17:34:58.000198 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.000151    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wppnv\" (UniqueName: \"kubernetes.io/projected/ea3e1303-f4dc-4511-abbc-9a7c0e789164-kube-api-access-wppnv\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-25vvm\" (UID: \"ea3e1303-f4dc-4511-abbc-9a7c0e789164\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-25vvm"
Jul 28 17:34:58.000198 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.000170    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/0641104e-1805-4a32-97fd-e9ee7c550152-stats-auth\") pod \"router-default-7c67d86669-hftv7\" (UID: \"0641104e-1805-4a32-97fd-e9ee7c550152\") " pod="openshift-ingress/router-default-7c67d86669-hftv7"
Jul 28 17:34:58.000316 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.000212    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jhp9\" (UniqueName: \"kubernetes.io/projected/0641104e-1805-4a32-97fd-e9ee7c550152-kube-api-access-8jhp9\") pod \"router-default-7c67d86669-hftv7\" (UID: \"0641104e-1805-4a32-97fd-e9ee7c550152\") " pod="openshift-ingress/router-default-7c67d86669-hftv7"
Jul 28 17:34:58.000316 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.000254    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7393a17e-a626-450c-bcb9-04ea1774e002-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-jqft4\" (UID: \"7393a17e-a626-450c-bcb9-04ea1774e002\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-jqft4"
Jul 28 17:34:58.000410 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.000312    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0641104e-1805-4a32-97fd-e9ee7c550152-service-ca-bundle\") pod \"router-default-7c67d86669-hftv7\" (UID: \"0641104e-1805-4a32-97fd-e9ee7c550152\") " pod="openshift-ingress/router-default-7c67d86669-hftv7"
Jul 28 17:34:58.000410 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.000342    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0641104e-1805-4a32-97fd-e9ee7c550152-metrics-certs\") pod \"router-default-7c67d86669-hftv7\" (UID: \"0641104e-1805-4a32-97fd-e9ee7c550152\") " pod="openshift-ingress/router-default-7c67d86669-hftv7"
Jul 28 17:34:58.000410 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.000365    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/dfad001a-8bdb-474e-9c4a-cfed601c7748-config\") pod \"service-ca-operator-5d6c75f84f-zc5p9\" (UID: \"dfad001a-8bdb-474e-9c4a-cfed601c7748\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-zc5p9"
Jul 28 17:34:58.000410 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.000400    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/dfad001a-8bdb-474e-9c4a-cfed601c7748-serving-cert\") pod \"service-ca-operator-5d6c75f84f-zc5p9\" (UID: \"dfad001a-8bdb-474e-9c4a-cfed601c7748\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-zc5p9"
Jul 28 17:34:58.000622 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:58.000421    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 17:34:58.000622 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.000426    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/7393a17e-a626-450c-bcb9-04ea1774e002-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-jqft4\" (UID: \"7393a17e-a626-450c-bcb9-04ea1774e002\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-jqft4"
Jul 28 17:34:58.000622 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:58.000480    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0641104e-1805-4a32-97fd-e9ee7c550152-service-ca-bundle podName:0641104e-1805-4a32-97fd-e9ee7c550152 nodeName:}" failed. No retries permitted until 2026-07-28 17:34:58.500460383 +0000 UTC m=+37.161616720 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0641104e-1805-4a32-97fd-e9ee7c550152-service-ca-bundle") pod "router-default-7c67d86669-hftv7" (UID: "0641104e-1805-4a32-97fd-e9ee7c550152") : configmap references non-existent config key: service-ca.crt
Jul 28 17:34:58.000622 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:58.000516    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7393a17e-a626-450c-bcb9-04ea1774e002-cluster-monitoring-operator-tls podName:7393a17e-a626-450c-bcb9-04ea1774e002 nodeName:}" failed. No retries permitted until 2026-07-28 17:34:58.500499454 +0000 UTC m=+37.161655806 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7393a17e-a626-450c-bcb9-04ea1774e002-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-jqft4" (UID: "7393a17e-a626-450c-bcb9-04ea1774e002") : secret "cluster-monitoring-operator-tls" not found
Jul 28 17:34:58.000622 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.000560    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/0641104e-1805-4a32-97fd-e9ee7c550152-default-certificate\") pod \"router-default-7c67d86669-hftv7\" (UID: \"0641104e-1805-4a32-97fd-e9ee7c550152\") " pod="openshift-ingress/router-default-7c67d86669-hftv7"
Jul 28 17:34:58.000622 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.000588    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltgj7\" (UniqueName: \"kubernetes.io/projected/dfad001a-8bdb-474e-9c4a-cfed601c7748-kube-api-access-ltgj7\") pod \"service-ca-operator-5d6c75f84f-zc5p9\" (UID: \"dfad001a-8bdb-474e-9c4a-cfed601c7748\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-zc5p9"
Jul 28 17:34:58.000622 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.000618    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ea3e1303-f4dc-4511-abbc-9a7c0e789164-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-25vvm\" (UID: \"ea3e1303-f4dc-4511-abbc-9a7c0e789164\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-25vvm"
Jul 28 17:34:58.000938 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:58.000429    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 17:34:58.000938 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:58.000718    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0641104e-1805-4a32-97fd-e9ee7c550152-metrics-certs podName:0641104e-1805-4a32-97fd-e9ee7c550152 nodeName:}" failed. No retries permitted until 2026-07-28 17:34:58.500704972 +0000 UTC m=+37.161861318 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0641104e-1805-4a32-97fd-e9ee7c550152-metrics-certs") pod "router-default-7c67d86669-hftv7" (UID: "0641104e-1805-4a32-97fd-e9ee7c550152") : secret "router-metrics-certs-default" not found
Jul 28 17:34:58.001086 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.001063    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/dfad001a-8bdb-474e-9c4a-cfed601c7748-config\") pod \"service-ca-operator-5d6c75f84f-zc5p9\" (UID: \"dfad001a-8bdb-474e-9c4a-cfed601c7748\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-zc5p9"
Jul 28 17:34:58.001199 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.001179    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/7393a17e-a626-450c-bcb9-04ea1774e002-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-jqft4\" (UID: \"7393a17e-a626-450c-bcb9-04ea1774e002\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-jqft4"
Jul 28 17:34:58.002923 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.002903    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/dfad001a-8bdb-474e-9c4a-cfed601c7748-serving-cert\") pod \"service-ca-operator-5d6c75f84f-zc5p9\" (UID: \"dfad001a-8bdb-474e-9c4a-cfed601c7748\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-zc5p9"
Jul 28 17:34:58.003032 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.003014    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/0641104e-1805-4a32-97fd-e9ee7c550152-stats-auth\") pod \"router-default-7c67d86669-hftv7\" (UID: \"0641104e-1805-4a32-97fd-e9ee7c550152\") " pod="openshift-ingress/router-default-7c67d86669-hftv7"
Jul 28 17:34:58.003199 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.003180    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/0641104e-1805-4a32-97fd-e9ee7c550152-default-certificate\") pod \"router-default-7c67d86669-hftv7\" (UID: \"0641104e-1805-4a32-97fd-e9ee7c550152\") " pod="openshift-ingress/router-default-7c67d86669-hftv7"
Jul 28 17:34:58.012131 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.012073    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sw5pj\" (UniqueName: \"kubernetes.io/projected/7393a17e-a626-450c-bcb9-04ea1774e002-kube-api-access-sw5pj\") pod \"cluster-monitoring-operator-548b489cb8-jqft4\" (UID: \"7393a17e-a626-450c-bcb9-04ea1774e002\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-jqft4"
Jul 28 17:34:58.012218 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.012171    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jhp9\" (UniqueName: \"kubernetes.io/projected/0641104e-1805-4a32-97fd-e9ee7c550152-kube-api-access-8jhp9\") pod \"router-default-7c67d86669-hftv7\" (UID: \"0641104e-1805-4a32-97fd-e9ee7c550152\") " pod="openshift-ingress/router-default-7c67d86669-hftv7"
Jul 28 17:34:58.012770 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.012746    2563 generic.go:356] "Generic (PLEG): container finished" podID="3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b" containerID="8d8b904433f26af9fdcd23e1e2c741f4f862065ace7a249ff77040426b3e443f" exitCode=0
Jul 28 17:34:58.012868 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.012838    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x2h5c" event={"ID":"3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b","Type":"ContainerDied","Data":"8d8b904433f26af9fdcd23e1e2c741f4f862065ace7a249ff77040426b3e443f"}
Jul 28 17:34:58.013003 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.012986    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltgj7\" (UniqueName: \"kubernetes.io/projected/dfad001a-8bdb-474e-9c4a-cfed601c7748-kube-api-access-ltgj7\") pod \"service-ca-operator-5d6c75f84f-zc5p9\" (UID: \"dfad001a-8bdb-474e-9c4a-cfed601c7748\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-zc5p9"
Jul 28 17:34:58.014112 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.014081    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-lpwn9" event={"ID":"093b4d76-2af2-4586-8421-68b7047ff74e","Type":"ContainerStarted","Data":"81932e1b77b116d842fbd248488eff2a41d1f3920cbf6eb1cfeaa17b0f4234bd"}
Jul 28 17:34:58.033604 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.033583    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:34:58.077183 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.077158    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-zc5p9"
Jul 28 17:34:58.101695 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.101666    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ea3e1303-f4dc-4511-abbc-9a7c0e789164-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-25vvm\" (UID: \"ea3e1303-f4dc-4511-abbc-9a7c0e789164\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-25vvm"
Jul 28 17:34:58.101815 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.101711    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wppnv\" (UniqueName: \"kubernetes.io/projected/ea3e1303-f4dc-4511-abbc-9a7c0e789164-kube-api-access-wppnv\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-25vvm\" (UID: \"ea3e1303-f4dc-4511-abbc-9a7c0e789164\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-25vvm"
Jul 28 17:34:58.101815 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.101805    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/23a671ef-805b-4481-8950-689b84637ec5-image-registry-private-configuration\") pod \"image-registry-75cf897b8-llxcl\" (UID: \"23a671ef-805b-4481-8950-689b84637ec5\") " pod="openshift-image-registry/image-registry-75cf897b8-llxcl"
Jul 28 17:34:58.101920 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.101835    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4qb6\" (UniqueName: \"kubernetes.io/projected/23a671ef-805b-4481-8950-689b84637ec5-kube-api-access-v4qb6\") pod \"image-registry-75cf897b8-llxcl\" (UID: \"23a671ef-805b-4481-8950-689b84637ec5\") " pod="openshift-image-registry/image-registry-75cf897b8-llxcl"
Jul 28 17:34:58.101920 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.101903    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2r64k\" (UniqueName: \"kubernetes.io/projected/b3a94675-bd9d-4bdf-b73d-074135ab92e3-kube-api-access-2r64k\") pod \"cluster-samples-operator-6748489656-ch2rb\" (UID: \"b3a94675-bd9d-4bdf-b73d-074135ab92e3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-ch2rb"
Jul 28 17:34:58.102025 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.101928    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/23a671ef-805b-4481-8950-689b84637ec5-trusted-ca\") pod \"image-registry-75cf897b8-llxcl\" (UID: \"23a671ef-805b-4481-8950-689b84637ec5\") " pod="openshift-image-registry/image-registry-75cf897b8-llxcl"
Jul 28 17:34:58.102025 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.101956    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/23a671ef-805b-4481-8950-689b84637ec5-registry-certificates\") pod \"image-registry-75cf897b8-llxcl\" (UID: \"23a671ef-805b-4481-8950-689b84637ec5\") " pod="openshift-image-registry/image-registry-75cf897b8-llxcl"
Jul 28 17:34:58.102121 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.102046    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/23a671ef-805b-4481-8950-689b84637ec5-bound-sa-token\") pod \"image-registry-75cf897b8-llxcl\" (UID: \"23a671ef-805b-4481-8950-689b84637ec5\") " pod="openshift-image-registry/image-registry-75cf897b8-llxcl"
Jul 28 17:34:58.102121 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.102099    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/23a671ef-805b-4481-8950-689b84637ec5-ca-trust-extracted\") pod \"image-registry-75cf897b8-llxcl\" (UID: \"23a671ef-805b-4481-8950-689b84637ec5\") " pod="openshift-image-registry/image-registry-75cf897b8-llxcl"
Jul 28 17:34:58.102219 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.102145    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/23a671ef-805b-4481-8950-689b84637ec5-registry-tls\") pod \"image-registry-75cf897b8-llxcl\" (UID: \"23a671ef-805b-4481-8950-689b84637ec5\") " pod="openshift-image-registry/image-registry-75cf897b8-llxcl"
Jul 28 17:34:58.102219 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.102174    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3a94675-bd9d-4bdf-b73d-074135ab92e3-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-ch2rb\" (UID: \"b3a94675-bd9d-4bdf-b73d-074135ab92e3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-ch2rb"
Jul 28 17:34:58.102316 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.102236    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ea3e1303-f4dc-4511-abbc-9a7c0e789164-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-25vvm\" (UID: \"ea3e1303-f4dc-4511-abbc-9a7c0e789164\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-25vvm"
Jul 28 17:34:58.102316 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.102263    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ea3e1303-f4dc-4511-abbc-9a7c0e789164-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-25vvm\" (UID: \"ea3e1303-f4dc-4511-abbc-9a7c0e789164\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-25vvm"
Jul 28 17:34:58.102724 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.102685    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/23a671ef-805b-4481-8950-689b84637ec5-installation-pull-secrets\") pod \"image-registry-75cf897b8-llxcl\" (UID: \"23a671ef-805b-4481-8950-689b84637ec5\") " pod="openshift-image-registry/image-registry-75cf897b8-llxcl"
Jul 28 17:34:58.105925 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.105879    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ea3e1303-f4dc-4511-abbc-9a7c0e789164-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-25vvm\" (UID: \"ea3e1303-f4dc-4511-abbc-9a7c0e789164\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-25vvm"
Jul 28 17:34:58.117006 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.116978    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wppnv\" (UniqueName: \"kubernetes.io/projected/ea3e1303-f4dc-4511-abbc-9a7c0e789164-kube-api-access-wppnv\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-25vvm\" (UID: \"ea3e1303-f4dc-4511-abbc-9a7c0e789164\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-25vvm"
Jul 28 17:34:58.195231 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.195201    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-25vvm"
Jul 28 17:34:58.203351 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.203324    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/23a671ef-805b-4481-8950-689b84637ec5-image-registry-private-configuration\") pod \"image-registry-75cf897b8-llxcl\" (UID: \"23a671ef-805b-4481-8950-689b84637ec5\") " pod="openshift-image-registry/image-registry-75cf897b8-llxcl"
Jul 28 17:34:58.203505 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.203365    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4qb6\" (UniqueName: \"kubernetes.io/projected/23a671ef-805b-4481-8950-689b84637ec5-kube-api-access-v4qb6\") pod \"image-registry-75cf897b8-llxcl\" (UID: \"23a671ef-805b-4481-8950-689b84637ec5\") " pod="openshift-image-registry/image-registry-75cf897b8-llxcl"
Jul 28 17:34:58.203505 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.203413    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2r64k\" (UniqueName: \"kubernetes.io/projected/b3a94675-bd9d-4bdf-b73d-074135ab92e3-kube-api-access-2r64k\") pod \"cluster-samples-operator-6748489656-ch2rb\" (UID: \"b3a94675-bd9d-4bdf-b73d-074135ab92e3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-ch2rb"
Jul 28 17:34:58.203505 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.203456    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/23a671ef-805b-4481-8950-689b84637ec5-trusted-ca\") pod \"image-registry-75cf897b8-llxcl\" (UID: \"23a671ef-805b-4481-8950-689b84637ec5\") " pod="openshift-image-registry/image-registry-75cf897b8-llxcl"
Jul 28 17:34:58.203505 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.203481    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/23a671ef-805b-4481-8950-689b84637ec5-registry-certificates\") pod \"image-registry-75cf897b8-llxcl\" (UID: \"23a671ef-805b-4481-8950-689b84637ec5\") " pod="openshift-image-registry/image-registry-75cf897b8-llxcl"
Jul 28 17:34:58.203720 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.203514    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/23a671ef-805b-4481-8950-689b84637ec5-bound-sa-token\") pod \"image-registry-75cf897b8-llxcl\" (UID: \"23a671ef-805b-4481-8950-689b84637ec5\") " pod="openshift-image-registry/image-registry-75cf897b8-llxcl"
Jul 28 17:34:58.203720 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.203559    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/23a671ef-805b-4481-8950-689b84637ec5-ca-trust-extracted\") pod \"image-registry-75cf897b8-llxcl\" (UID: \"23a671ef-805b-4481-8950-689b84637ec5\") " pod="openshift-image-registry/image-registry-75cf897b8-llxcl"
Jul 28 17:34:58.203720 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.203602    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/23a671ef-805b-4481-8950-689b84637ec5-registry-tls\") pod \"image-registry-75cf897b8-llxcl\" (UID: \"23a671ef-805b-4481-8950-689b84637ec5\") " pod="openshift-image-registry/image-registry-75cf897b8-llxcl"
Jul 28 17:34:58.203720 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.203630    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3a94675-bd9d-4bdf-b73d-074135ab92e3-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-ch2rb\" (UID: \"b3a94675-bd9d-4bdf-b73d-074135ab92e3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-ch2rb"
Jul 28 17:34:58.203720 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.203666    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/23a671ef-805b-4481-8950-689b84637ec5-installation-pull-secrets\") pod \"image-registry-75cf897b8-llxcl\" (UID: \"23a671ef-805b-4481-8950-689b84637ec5\") " pod="openshift-image-registry/image-registry-75cf897b8-llxcl"
Jul 28 17:34:58.204288 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.203993    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/23a671ef-805b-4481-8950-689b84637ec5-ca-trust-extracted\") pod \"image-registry-75cf897b8-llxcl\" (UID: \"23a671ef-805b-4481-8950-689b84637ec5\") " pod="openshift-image-registry/image-registry-75cf897b8-llxcl"
Jul 28 17:34:58.204288 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:58.204168    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 17:34:58.204288 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:58.204191    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75cf897b8-llxcl: secret "image-registry-tls" not found
Jul 28 17:34:58.204288 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:58.204238    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/23a671ef-805b-4481-8950-689b84637ec5-registry-tls podName:23a671ef-805b-4481-8950-689b84637ec5 nodeName:}" failed. No retries permitted until 2026-07-28 17:34:58.704221439 +0000 UTC m=+37.365377786 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/23a671ef-805b-4481-8950-689b84637ec5-registry-tls") pod "image-registry-75cf897b8-llxcl" (UID: "23a671ef-805b-4481-8950-689b84637ec5") : secret "image-registry-tls" not found
Jul 28 17:34:58.204565 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.204305    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/23a671ef-805b-4481-8950-689b84637ec5-registry-certificates\") pod \"image-registry-75cf897b8-llxcl\" (UID: \"23a671ef-805b-4481-8950-689b84637ec5\") " pod="openshift-image-registry/image-registry-75cf897b8-llxcl"
Jul 28 17:34:58.204565 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:58.204317    2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 17:34:58.204565 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:58.204362    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3a94675-bd9d-4bdf-b73d-074135ab92e3-samples-operator-tls podName:b3a94675-bd9d-4bdf-b73d-074135ab92e3 nodeName:}" failed. No retries permitted until 2026-07-28 17:34:58.704347687 +0000 UTC m=+37.365504037 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b3a94675-bd9d-4bdf-b73d-074135ab92e3-samples-operator-tls") pod "cluster-samples-operator-6748489656-ch2rb" (UID: "b3a94675-bd9d-4bdf-b73d-074135ab92e3") : secret "samples-operator-tls" not found
Jul 28 17:34:58.204565 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.204420    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/23a671ef-805b-4481-8950-689b84637ec5-trusted-ca\") pod \"image-registry-75cf897b8-llxcl\" (UID: \"23a671ef-805b-4481-8950-689b84637ec5\") " pod="openshift-image-registry/image-registry-75cf897b8-llxcl"
Jul 28 17:34:58.206942 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.206905    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/23a671ef-805b-4481-8950-689b84637ec5-image-registry-private-configuration\") pod \"image-registry-75cf897b8-llxcl\" (UID: \"23a671ef-805b-4481-8950-689b84637ec5\") " pod="openshift-image-registry/image-registry-75cf897b8-llxcl"
Jul 28 17:34:58.206942 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.206923    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/23a671ef-805b-4481-8950-689b84637ec5-installation-pull-secrets\") pod \"image-registry-75cf897b8-llxcl\" (UID: \"23a671ef-805b-4481-8950-689b84637ec5\") " pod="openshift-image-registry/image-registry-75cf897b8-llxcl"
Jul 28 17:34:58.213515 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.213132    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-zc5p9"]
Jul 28 17:34:58.215931 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.215908    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2r64k\" (UniqueName: \"kubernetes.io/projected/b3a94675-bd9d-4bdf-b73d-074135ab92e3-kube-api-access-2r64k\") pod \"cluster-samples-operator-6748489656-ch2rb\" (UID: \"b3a94675-bd9d-4bdf-b73d-074135ab92e3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-ch2rb"
Jul 28 17:34:58.216097 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.216079    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/23a671ef-805b-4481-8950-689b84637ec5-bound-sa-token\") pod \"image-registry-75cf897b8-llxcl\" (UID: \"23a671ef-805b-4481-8950-689b84637ec5\") " pod="openshift-image-registry/image-registry-75cf897b8-llxcl"
Jul 28 17:34:58.216296 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.216275    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4qb6\" (UniqueName: \"kubernetes.io/projected/23a671ef-805b-4481-8950-689b84637ec5-kube-api-access-v4qb6\") pod \"image-registry-75cf897b8-llxcl\" (UID: \"23a671ef-805b-4481-8950-689b84637ec5\") " pod="openshift-image-registry/image-registry-75cf897b8-llxcl"
Jul 28 17:34:58.217559 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:58.217531    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfad001a_8bdb_474e_9c4a_cfed601c7748.slice/crio-b1c6192bfe7a3fd19bb2fb8da4a7f19cf5e0259ad0ed9a00d5548ec7e6ddb817 WatchSource:0}: Error finding container b1c6192bfe7a3fd19bb2fb8da4a7f19cf5e0259ad0ed9a00d5548ec7e6ddb817: Status 404 returned error can't find the container with id b1c6192bfe7a3fd19bb2fb8da4a7f19cf5e0259ad0ed9a00d5548ec7e6ddb817
Jul 28 17:34:58.335219 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.335100    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-25vvm"]
Jul 28 17:34:58.338127 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:34:58.338097    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea3e1303_f4dc_4511_abbc_9a7c0e789164.slice/crio-339f2630c1b1a2d8a04a1056fe0454dc469b9e75a1f5c5267a3148eedb2086cc WatchSource:0}: Error finding container 339f2630c1b1a2d8a04a1056fe0454dc469b9e75a1f5c5267a3148eedb2086cc: Status 404 returned error can't find the container with id 339f2630c1b1a2d8a04a1056fe0454dc469b9e75a1f5c5267a3148eedb2086cc
Jul 28 17:34:58.505834 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.505757    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0641104e-1805-4a32-97fd-e9ee7c550152-service-ca-bundle\") pod \"router-default-7c67d86669-hftv7\" (UID: \"0641104e-1805-4a32-97fd-e9ee7c550152\") " pod="openshift-ingress/router-default-7c67d86669-hftv7"
Jul 28 17:34:58.505834 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.505793    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0641104e-1805-4a32-97fd-e9ee7c550152-metrics-certs\") pod \"router-default-7c67d86669-hftv7\" (UID: \"0641104e-1805-4a32-97fd-e9ee7c550152\") " pod="openshift-ingress/router-default-7c67d86669-hftv7"
Jul 28 17:34:58.506032 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:58.505896    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 17:34:58.506032 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:58.505930    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0641104e-1805-4a32-97fd-e9ee7c550152-service-ca-bundle podName:0641104e-1805-4a32-97fd-e9ee7c550152 nodeName:}" failed. No retries permitted until 2026-07-28 17:34:59.505909095 +0000 UTC m=+38.167065432 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0641104e-1805-4a32-97fd-e9ee7c550152-service-ca-bundle") pod "router-default-7c67d86669-hftv7" (UID: "0641104e-1805-4a32-97fd-e9ee7c550152") : configmap references non-existent config key: service-ca.crt
Jul 28 17:34:58.506032 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:58.505954    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0641104e-1805-4a32-97fd-e9ee7c550152-metrics-certs podName:0641104e-1805-4a32-97fd-e9ee7c550152 nodeName:}" failed. No retries permitted until 2026-07-28 17:34:59.505946306 +0000 UTC m=+38.167102641 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0641104e-1805-4a32-97fd-e9ee7c550152-metrics-certs") pod "router-default-7c67d86669-hftv7" (UID: "0641104e-1805-4a32-97fd-e9ee7c550152") : secret "router-metrics-certs-default" not found
Jul 28 17:34:58.506032 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.505988    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7393a17e-a626-450c-bcb9-04ea1774e002-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-jqft4\" (UID: \"7393a17e-a626-450c-bcb9-04ea1774e002\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-jqft4"
Jul 28 17:34:58.506194 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:58.506111    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 17:34:58.506194 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:58.506147    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7393a17e-a626-450c-bcb9-04ea1774e002-cluster-monitoring-operator-tls podName:7393a17e-a626-450c-bcb9-04ea1774e002 nodeName:}" failed. No retries permitted until 2026-07-28 17:34:59.506136151 +0000 UTC m=+38.167292504 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7393a17e-a626-450c-bcb9-04ea1774e002-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-jqft4" (UID: "7393a17e-a626-450c-bcb9-04ea1774e002") : secret "cluster-monitoring-operator-tls" not found
Jul 28 17:34:58.708087 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.708051    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/23a671ef-805b-4481-8950-689b84637ec5-registry-tls\") pod \"image-registry-75cf897b8-llxcl\" (UID: \"23a671ef-805b-4481-8950-689b84637ec5\") " pod="openshift-image-registry/image-registry-75cf897b8-llxcl"
Jul 28 17:34:58.708235 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.708096    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3a94675-bd9d-4bdf-b73d-074135ab92e3-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-ch2rb\" (UID: \"b3a94675-bd9d-4bdf-b73d-074135ab92e3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-ch2rb"
Jul 28 17:34:58.708235 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:58.708210    2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 17:34:58.708235 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:58.708214    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 17:34:58.708235 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:58.708230    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75cf897b8-llxcl: secret "image-registry-tls" not found
Jul 28 17:34:58.708370 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:58.708274    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/23a671ef-805b-4481-8950-689b84637ec5-registry-tls podName:23a671ef-805b-4481-8950-689b84637ec5 nodeName:}" failed. No retries permitted until 2026-07-28 17:34:59.70825666 +0000 UTC m=+38.369413009 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/23a671ef-805b-4481-8950-689b84637ec5-registry-tls") pod "image-registry-75cf897b8-llxcl" (UID: "23a671ef-805b-4481-8950-689b84637ec5") : secret "image-registry-tls" not found
Jul 28 17:34:58.708370 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:58.708290    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3a94675-bd9d-4bdf-b73d-074135ab92e3-samples-operator-tls podName:b3a94675-bd9d-4bdf-b73d-074135ab92e3 nodeName:}" failed. No retries permitted until 2026-07-28 17:34:59.708281895 +0000 UTC m=+38.369438229 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b3a94675-bd9d-4bdf-b73d-074135ab92e3-samples-operator-tls") pod "cluster-samples-operator-6748489656-ch2rb" (UID: "b3a94675-bd9d-4bdf-b73d-074135ab92e3") : secret "samples-operator-tls" not found
Jul 28 17:34:58.847946 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.847896    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-74vbl"
Jul 28 17:34:58.848491 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.847896    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z75m5"
Jul 28 17:34:58.848491 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.847896    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tthg"
Jul 28 17:34:58.850895 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.850870    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 17:34:58.851034 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.850998    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-z5ccl\""
Jul 28 17:34:58.851034 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.851018    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 17:34:58.851136 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.851074    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 17:34:58.851365 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.851340    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-c22cz\""
Jul 28 17:34:58.851476 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.851377    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 17:34:58.909948 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.909922    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/897c5c1b-7fb1-40eb-8b4a-46e737d8e991-metrics-tls\") pod \"dns-default-psd2g\" (UID: \"897c5c1b-7fb1-40eb-8b4a-46e737d8e991\") " pod="openshift-dns/dns-default-psd2g"
Jul 28 17:34:58.910356 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:58.910111    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 17:34:58.910356 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:58.910191    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/897c5c1b-7fb1-40eb-8b4a-46e737d8e991-metrics-tls podName:897c5c1b-7fb1-40eb-8b4a-46e737d8e991 nodeName:}" failed. No retries permitted until 2026-07-28 17:35:00.910164442 +0000 UTC m=+39.571320807 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/897c5c1b-7fb1-40eb-8b4a-46e737d8e991-metrics-tls") pod "dns-default-psd2g" (UID: "897c5c1b-7fb1-40eb-8b4a-46e737d8e991") : secret "dns-default-metrics-tls" not found
Jul 28 17:34:58.931929 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:58.931860    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:34:59.021241 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:59.021207    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x2h5c" event={"ID":"3aef368b-c04d-4b4e-94a2-0ac3e76f2f0b","Type":"ContainerStarted","Data":"8f3413df45dd8789ccf2f7e50ccdace6cbc6dce1f7273f2890e6763e45f90bca"}
Jul 28 17:34:59.027215 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:59.023467    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-25vvm" event={"ID":"ea3e1303-f4dc-4511-abbc-9a7c0e789164","Type":"ContainerStarted","Data":"339f2630c1b1a2d8a04a1056fe0454dc469b9e75a1f5c5267a3148eedb2086cc"}
Jul 28 17:34:59.027215 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:59.025144    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-zc5p9" event={"ID":"dfad001a-8bdb-474e-9c4a-cfed601c7748","Type":"ContainerStarted","Data":"b1c6192bfe7a3fd19bb2fb8da4a7f19cf5e0259ad0ed9a00d5548ec7e6ddb817"}
Jul 28 17:34:59.047246 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:59.047194    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-x2h5c" podStartSLOduration=9.235907145 podStartE2EDuration="38.047176188s" totalImagesPullingTime="28.811269043s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:34:21 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:34:59.045556159 +0000 UTC m=+37.706712514" watchObservedRunningTime="2026-07-28 17:34:59.047176188 +0000 UTC m=+37.708332544"
Jul 28 17:34:59.054252 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:59.054222    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:34:59.252888 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:59.252611    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:34:59.352273 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:59.352185    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:34:59.515669 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:59.515585    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0641104e-1805-4a32-97fd-e9ee7c550152-service-ca-bundle\") pod \"router-default-7c67d86669-hftv7\" (UID: \"0641104e-1805-4a32-97fd-e9ee7c550152\") " pod="openshift-ingress/router-default-7c67d86669-hftv7"
Jul 28 17:34:59.515669 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:59.515635    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0641104e-1805-4a32-97fd-e9ee7c550152-metrics-certs\") pod \"router-default-7c67d86669-hftv7\" (UID: \"0641104e-1805-4a32-97fd-e9ee7c550152\") " pod="openshift-ingress/router-default-7c67d86669-hftv7"
Jul 28 17:34:59.515884 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:59.515712    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7393a17e-a626-450c-bcb9-04ea1774e002-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-jqft4\" (UID: \"7393a17e-a626-450c-bcb9-04ea1774e002\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-jqft4"
Jul 28 17:34:59.515884 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:59.515781    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0641104e-1805-4a32-97fd-e9ee7c550152-service-ca-bundle podName:0641104e-1805-4a32-97fd-e9ee7c550152 nodeName:}" failed. No retries permitted until 2026-07-28 17:35:01.515757478 +0000 UTC m=+40.176913816 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0641104e-1805-4a32-97fd-e9ee7c550152-service-ca-bundle") pod "router-default-7c67d86669-hftv7" (UID: "0641104e-1805-4a32-97fd-e9ee7c550152") : configmap references non-existent config key: service-ca.crt
Jul 28 17:34:59.515884 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:59.515830    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 17:34:59.515884 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:59.515841    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 17:34:59.516073 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:59.515888    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7393a17e-a626-450c-bcb9-04ea1774e002-cluster-monitoring-operator-tls podName:7393a17e-a626-450c-bcb9-04ea1774e002 nodeName:}" failed. No retries permitted until 2026-07-28 17:35:01.515872576 +0000 UTC m=+40.177028911 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7393a17e-a626-450c-bcb9-04ea1774e002-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-jqft4" (UID: "7393a17e-a626-450c-bcb9-04ea1774e002") : secret "cluster-monitoring-operator-tls" not found
Jul 28 17:34:59.516073 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:59.515916    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0641104e-1805-4a32-97fd-e9ee7c550152-metrics-certs podName:0641104e-1805-4a32-97fd-e9ee7c550152 nodeName:}" failed. No retries permitted until 2026-07-28 17:35:01.51590613 +0000 UTC m=+40.177062468 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0641104e-1805-4a32-97fd-e9ee7c550152-metrics-certs") pod "router-default-7c67d86669-hftv7" (UID: "0641104e-1805-4a32-97fd-e9ee7c550152") : secret "router-metrics-certs-default" not found
Jul 28 17:34:59.718092 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:59.718053    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/23a671ef-805b-4481-8950-689b84637ec5-registry-tls\") pod \"image-registry-75cf897b8-llxcl\" (UID: \"23a671ef-805b-4481-8950-689b84637ec5\") " pod="openshift-image-registry/image-registry-75cf897b8-llxcl"
Jul 28 17:34:59.718275 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:59.718102    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3a94675-bd9d-4bdf-b73d-074135ab92e3-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-ch2rb\" (UID: \"b3a94675-bd9d-4bdf-b73d-074135ab92e3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-ch2rb"
Jul 28 17:34:59.718275 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:59.718238    2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 17:34:59.718275 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:59.718244    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 17:34:59.718275 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:59.718266    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75cf897b8-llxcl: secret "image-registry-tls" not found
Jul 28 17:34:59.718466 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:59.718386    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3a94675-bd9d-4bdf-b73d-074135ab92e3-samples-operator-tls podName:b3a94675-bd9d-4bdf-b73d-074135ab92e3 nodeName:}" failed. No retries permitted until 2026-07-28 17:35:01.718289149 +0000 UTC m=+40.379445487 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b3a94675-bd9d-4bdf-b73d-074135ab92e3-samples-operator-tls") pod "cluster-samples-operator-6748489656-ch2rb" (UID: "b3a94675-bd9d-4bdf-b73d-074135ab92e3") : secret "samples-operator-tls" not found
Jul 28 17:34:59.718510 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:34:59.718489    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/23a671ef-805b-4481-8950-689b84637ec5-registry-tls podName:23a671ef-805b-4481-8950-689b84637ec5 nodeName:}" failed. No retries permitted until 2026-07-28 17:35:01.718467368 +0000 UTC m=+40.379623712 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/23a671ef-805b-4481-8950-689b84637ec5-registry-tls") pod "image-registry-75cf897b8-llxcl" (UID: "23a671ef-805b-4481-8950-689b84637ec5") : secret "image-registry-tls" not found
Jul 28 17:34:59.884693 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:59.884661    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:34:59.979482 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:34:59.979429    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:00.055274 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:00.055239    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:00.276106 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:00.276017    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:00.377609 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:00.377556    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:00.801421 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:00.801389    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:00.930629 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:00.930584    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/897c5c1b-7fb1-40eb-8b4a-46e737d8e991-metrics-tls\") pod \"dns-default-psd2g\" (UID: \"897c5c1b-7fb1-40eb-8b4a-46e737d8e991\") " pod="openshift-dns/dns-default-psd2g"
Jul 28 17:35:00.930859 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:35:00.930834    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 17:35:00.930986 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:35:00.930915    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/897c5c1b-7fb1-40eb-8b4a-46e737d8e991-metrics-tls podName:897c5c1b-7fb1-40eb-8b4a-46e737d8e991 nodeName:}" failed. No retries permitted until 2026-07-28 17:35:04.930896063 +0000 UTC m=+43.592052396 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/897c5c1b-7fb1-40eb-8b4a-46e737d8e991-metrics-tls") pod "dns-default-psd2g" (UID: "897c5c1b-7fb1-40eb-8b4a-46e737d8e991") : secret "dns-default-metrics-tls" not found
Jul 28 17:35:00.937305 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:00.937283    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:01.037785 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:01.037755    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:01.137948 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:01.137913    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:01.536699 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:01.536671    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0641104e-1805-4a32-97fd-e9ee7c550152-service-ca-bundle\") pod \"router-default-7c67d86669-hftv7\" (UID: \"0641104e-1805-4a32-97fd-e9ee7c550152\") " pod="openshift-ingress/router-default-7c67d86669-hftv7"
Jul 28 17:35:01.536699 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:01.536705    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0641104e-1805-4a32-97fd-e9ee7c550152-metrics-certs\") pod \"router-default-7c67d86669-hftv7\" (UID: \"0641104e-1805-4a32-97fd-e9ee7c550152\") " pod="openshift-ingress/router-default-7c67d86669-hftv7"
Jul 28 17:35:01.536853 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:01.536780    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7393a17e-a626-450c-bcb9-04ea1774e002-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-jqft4\" (UID: \"7393a17e-a626-450c-bcb9-04ea1774e002\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-jqft4"
Jul 28 17:35:01.536917 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:35:01.536874    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0641104e-1805-4a32-97fd-e9ee7c550152-service-ca-bundle podName:0641104e-1805-4a32-97fd-e9ee7c550152 nodeName:}" failed. No retries permitted until 2026-07-28 17:35:05.53684879 +0000 UTC m=+44.198005131 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0641104e-1805-4a32-97fd-e9ee7c550152-service-ca-bundle") pod "router-default-7c67d86669-hftv7" (UID: "0641104e-1805-4a32-97fd-e9ee7c550152") : configmap references non-existent config key: service-ca.crt
Jul 28 17:35:01.536917 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:35:01.536875    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 17:35:01.536917 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:35:01.536892    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 17:35:01.537050 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:35:01.536925    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7393a17e-a626-450c-bcb9-04ea1774e002-cluster-monitoring-operator-tls podName:7393a17e-a626-450c-bcb9-04ea1774e002 nodeName:}" failed. No retries permitted until 2026-07-28 17:35:05.536915558 +0000 UTC m=+44.198071892 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7393a17e-a626-450c-bcb9-04ea1774e002-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-jqft4" (UID: "7393a17e-a626-450c-bcb9-04ea1774e002") : secret "cluster-monitoring-operator-tls" not found
Jul 28 17:35:01.537050 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:35:01.536945    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0641104e-1805-4a32-97fd-e9ee7c550152-metrics-certs podName:0641104e-1805-4a32-97fd-e9ee7c550152 nodeName:}" failed. No retries permitted until 2026-07-28 17:35:05.536934358 +0000 UTC m=+44.198090697 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0641104e-1805-4a32-97fd-e9ee7c550152-metrics-certs") pod "router-default-7c67d86669-hftv7" (UID: "0641104e-1805-4a32-97fd-e9ee7c550152") : secret "router-metrics-certs-default" not found
Jul 28 17:35:01.545398 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:01.545376    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:01.645833 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:01.645806    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:01.738245 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:01.738202    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/23a671ef-805b-4481-8950-689b84637ec5-registry-tls\") pod \"image-registry-75cf897b8-llxcl\" (UID: \"23a671ef-805b-4481-8950-689b84637ec5\") " pod="openshift-image-registry/image-registry-75cf897b8-llxcl"
Jul 28 17:35:01.738465 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:01.738258    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3a94675-bd9d-4bdf-b73d-074135ab92e3-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-ch2rb\" (UID: \"b3a94675-bd9d-4bdf-b73d-074135ab92e3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-ch2rb"
Jul 28 17:35:01.738465 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:35:01.738344    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 17:35:01.738465 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:35:01.738371    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75cf897b8-llxcl: secret "image-registry-tls" not found
Jul 28 17:35:01.738465 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:35:01.738383    2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 17:35:01.738465 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:35:01.738455    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/23a671ef-805b-4481-8950-689b84637ec5-registry-tls podName:23a671ef-805b-4481-8950-689b84637ec5 nodeName:}" failed. No retries permitted until 2026-07-28 17:35:05.738420642 +0000 UTC m=+44.399576987 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/23a671ef-805b-4481-8950-689b84637ec5-registry-tls") pod "image-registry-75cf897b8-llxcl" (UID: "23a671ef-805b-4481-8950-689b84637ec5") : secret "image-registry-tls" not found
Jul 28 17:35:01.738760 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:35:01.738482    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3a94675-bd9d-4bdf-b73d-074135ab92e3-samples-operator-tls podName:b3a94675-bd9d-4bdf-b73d-074135ab92e3 nodeName:}" failed. No retries permitted until 2026-07-28 17:35:05.738463798 +0000 UTC m=+44.399620131 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b3a94675-bd9d-4bdf-b73d-074135ab92e3-samples-operator-tls") pod "cluster-samples-operator-6748489656-ch2rb" (UID: "b3a94675-bd9d-4bdf-b73d-074135ab92e3") : secret "samples-operator-tls" not found
Jul 28 17:35:02.034607 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:02.034524    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-lpwn9" event={"ID":"093b4d76-2af2-4586-8421-68b7047ff74e","Type":"ContainerStarted","Data":"064807f8b8d2fa0d89d27058e3cf60a69b39878245de04f15b2e0057a9d6d2dc"}
Jul 28 17:35:02.036031 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:02.036000    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-zc5p9" event={"ID":"dfad001a-8bdb-474e-9c4a-cfed601c7748","Type":"ContainerStarted","Data":"af915131b3badf50622caca677b736516d40ea6df5b14c802f5fa3f331ae9cb6"}
Jul 28 17:35:02.037444 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:02.037407    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-25vvm" event={"ID":"ea3e1303-f4dc-4511-abbc-9a7c0e789164","Type":"ContainerStarted","Data":"443ade54813b098490369137f585a28f1f7c2bdcdf1f90332338040fba1ad27e"}
Jul 28 17:35:02.053782 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:02.053726    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-lpwn9" podStartSLOduration=1.128357001 podStartE2EDuration="5.053708602s" totalImagesPullingTime="3.925351601s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:34:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:35:02.053528566 +0000 UTC m=+40.714684922" watchObservedRunningTime="2026-07-28 17:35:02.053708602 +0000 UTC m=+40.714864957"
Jul 28 17:35:02.070597 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:02.070544    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-zc5p9" podStartSLOduration=1.719636513 podStartE2EDuration="5.070526746s" totalImagesPullingTime="3.350890233s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:34:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:35:02.069745336 +0000 UTC m=+40.730901692" watchObservedRunningTime="2026-07-28 17:35:02.070526746 +0000 UTC m=+40.731683103"
Jul 28 17:35:02.085711 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:02.085663    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-25vvm" podStartSLOduration=1.851983639 podStartE2EDuration="5.085647897s" totalImagesPullingTime="3.233664258s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:34:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:35:02.084984897 +0000 UTC m=+40.746141253" watchObservedRunningTime="2026-07-28 17:35:02.085647897 +0000 UTC m=+40.746804251"
Jul 28 17:35:02.634511 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:02.634478    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:02.732598 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:02.732563    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:03.057668 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:03.057593    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:03.157878 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:03.157847    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:03.332829 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:03.332800    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:03.440928 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:03.440899    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:04.388269 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:04.388235    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:04.448771 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:04.448748    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:04.965167 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:04.965139    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/897c5c1b-7fb1-40eb-8b4a-46e737d8e991-metrics-tls\") pod \"dns-default-psd2g\" (UID: \"897c5c1b-7fb1-40eb-8b4a-46e737d8e991\") " pod="openshift-dns/dns-default-psd2g"
Jul 28 17:35:04.965329 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:35:04.965289    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 17:35:04.965376 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:35:04.965350    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/897c5c1b-7fb1-40eb-8b4a-46e737d8e991-metrics-tls podName:897c5c1b-7fb1-40eb-8b4a-46e737d8e991 nodeName:}" failed. No retries permitted until 2026-07-28 17:35:12.965333705 +0000 UTC m=+51.626490042 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/897c5c1b-7fb1-40eb-8b4a-46e737d8e991-metrics-tls") pod "dns-default-psd2g" (UID: "897c5c1b-7fb1-40eb-8b4a-46e737d8e991") : secret "dns-default-metrics-tls" not found
Jul 28 17:35:04.973885 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:04.973863    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:05.215087 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:05.215057    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:05.241257 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:05.241228    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-mj678"]
Jul 28 17:35:05.275450 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:05.275413    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mj678"]
Jul 28 17:35:05.275569 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:05.275539    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mj678"
Jul 28 17:35:05.278224 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:05.278203    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 28 17:35:05.278224 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:05.278221    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 28 17:35:05.278379 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:05.278281    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 28 17:35:05.279067 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:05.279045    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-xj8mn\""
Jul 28 17:35:05.279239 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:05.279213    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 28 17:35:05.291749 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:05.291729    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:05.368530 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:05.368506    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dda2a096-8121-4e4d-a074-1565b2bf0872-data-volume\") pod \"insights-runtime-extractor-mj678\" (UID: \"dda2a096-8121-4e4d-a074-1565b2bf0872\") " pod="openshift-insights/insights-runtime-extractor-mj678"
Jul 28 17:35:05.368633 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:05.368538    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dda2a096-8121-4e4d-a074-1565b2bf0872-crio-socket\") pod \"insights-runtime-extractor-mj678\" (UID: \"dda2a096-8121-4e4d-a074-1565b2bf0872\") " pod="openshift-insights/insights-runtime-extractor-mj678"
Jul 28 17:35:05.368633 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:05.368558    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dda2a096-8121-4e4d-a074-1565b2bf0872-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mj678\" (UID: \"dda2a096-8121-4e4d-a074-1565b2bf0872\") " pod="openshift-insights/insights-runtime-extractor-mj678"
Jul 28 17:35:05.368705 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:05.368690    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dda2a096-8121-4e4d-a074-1565b2bf0872-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mj678\" (UID: \"dda2a096-8121-4e4d-a074-1565b2bf0872\") " pod="openshift-insights/insights-runtime-extractor-mj678"
Jul 28 17:35:05.368752 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:05.368720    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66nf4\" (UniqueName: \"kubernetes.io/projected/dda2a096-8121-4e4d-a074-1565b2bf0872-kube-api-access-66nf4\") pod \"insights-runtime-extractor-mj678\" (UID: \"dda2a096-8121-4e4d-a074-1565b2bf0872\") " pod="openshift-insights/insights-runtime-extractor-mj678"
Jul 28 17:35:05.469520 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:05.469470    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dda2a096-8121-4e4d-a074-1565b2bf0872-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mj678\" (UID: \"dda2a096-8121-4e4d-a074-1565b2bf0872\") " pod="openshift-insights/insights-runtime-extractor-mj678"
Jul 28 17:35:05.469520 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:05.469498    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66nf4\" (UniqueName: \"kubernetes.io/projected/dda2a096-8121-4e4d-a074-1565b2bf0872-kube-api-access-66nf4\") pod \"insights-runtime-extractor-mj678\" (UID: \"dda2a096-8121-4e4d-a074-1565b2bf0872\") " pod="openshift-insights/insights-runtime-extractor-mj678"
Jul 28 17:35:05.469843 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:05.469572    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dda2a096-8121-4e4d-a074-1565b2bf0872-data-volume\") pod \"insights-runtime-extractor-mj678\" (UID: \"dda2a096-8121-4e4d-a074-1565b2bf0872\") " pod="openshift-insights/insights-runtime-extractor-mj678"
Jul 28 17:35:05.469843 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:05.469600    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dda2a096-8121-4e4d-a074-1565b2bf0872-crio-socket\") pod \"insights-runtime-extractor-mj678\" (UID: \"dda2a096-8121-4e4d-a074-1565b2bf0872\") " pod="openshift-insights/insights-runtime-extractor-mj678"
Jul 28 17:35:05.469843 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:35:05.469608    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 17:35:05.469843 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:05.469625    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dda2a096-8121-4e4d-a074-1565b2bf0872-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mj678\" (UID: \"dda2a096-8121-4e4d-a074-1565b2bf0872\") " pod="openshift-insights/insights-runtime-extractor-mj678"
Jul 28 17:35:05.469843 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:35:05.469674    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dda2a096-8121-4e4d-a074-1565b2bf0872-insights-runtime-extractor-tls podName:dda2a096-8121-4e4d-a074-1565b2bf0872 nodeName:}" failed. No retries permitted until 2026-07-28 17:35:05.96965797 +0000 UTC m=+44.630814303 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dda2a096-8121-4e4d-a074-1565b2bf0872-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mj678" (UID: "dda2a096-8121-4e4d-a074-1565b2bf0872") : secret "insights-runtime-extractor-tls" not found
Jul 28 17:35:05.469843 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:35:05.469681    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 17:35:05.469843 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:35:05.469736    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dda2a096-8121-4e4d-a074-1565b2bf0872-kube-rbac-proxy-cm podName:dda2a096-8121-4e4d-a074-1565b2bf0872 nodeName:}" failed. No retries permitted until 2026-07-28 17:35:05.969721255 +0000 UTC m=+44.630877596 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dda2a096-8121-4e4d-a074-1565b2bf0872-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mj678" (UID: "dda2a096-8121-4e4d-a074-1565b2bf0872") : configmap "kube-rbac-proxy" not found
Jul 28 17:35:05.469843 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:05.469818    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dda2a096-8121-4e4d-a074-1565b2bf0872-crio-socket\") pod \"insights-runtime-extractor-mj678\" (UID: \"dda2a096-8121-4e4d-a074-1565b2bf0872\") " pod="openshift-insights/insights-runtime-extractor-mj678"
Jul 28 17:35:05.470085 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:05.469903    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dda2a096-8121-4e4d-a074-1565b2bf0872-data-volume\") pod \"insights-runtime-extractor-mj678\" (UID: \"dda2a096-8121-4e4d-a074-1565b2bf0872\") " pod="openshift-insights/insights-runtime-extractor-mj678"
Jul 28 17:35:05.491073 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:05.491050    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66nf4\" (UniqueName: \"kubernetes.io/projected/dda2a096-8121-4e4d-a074-1565b2bf0872-kube-api-access-66nf4\") pod \"insights-runtime-extractor-mj678\" (UID: \"dda2a096-8121-4e4d-a074-1565b2bf0872\") " pod="openshift-insights/insights-runtime-extractor-mj678"
Jul 28 17:35:05.570124 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:05.570097    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0641104e-1805-4a32-97fd-e9ee7c550152-service-ca-bundle\") pod \"router-default-7c67d86669-hftv7\" (UID: \"0641104e-1805-4a32-97fd-e9ee7c550152\") " pod="openshift-ingress/router-default-7c67d86669-hftv7"
Jul 28 17:35:05.570225 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:05.570133    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0641104e-1805-4a32-97fd-e9ee7c550152-metrics-certs\") pod \"router-default-7c67d86669-hftv7\" (UID: \"0641104e-1805-4a32-97fd-e9ee7c550152\") " pod="openshift-ingress/router-default-7c67d86669-hftv7"
Jul 28 17:35:05.570261 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:05.570237    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7393a17e-a626-450c-bcb9-04ea1774e002-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-jqft4\" (UID: \"7393a17e-a626-450c-bcb9-04ea1774e002\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-jqft4"
Jul 28 17:35:05.570291 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:35:05.570269    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0641104e-1805-4a32-97fd-e9ee7c550152-service-ca-bundle podName:0641104e-1805-4a32-97fd-e9ee7c550152 nodeName:}" failed. No retries permitted until 2026-07-28 17:35:13.570251879 +0000 UTC m=+52.231408234 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0641104e-1805-4a32-97fd-e9ee7c550152-service-ca-bundle") pod "router-default-7c67d86669-hftv7" (UID: "0641104e-1805-4a32-97fd-e9ee7c550152") : configmap references non-existent config key: service-ca.crt
Jul 28 17:35:05.570328 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:35:05.570319    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 17:35:05.570361 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:35:05.570322    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 17:35:05.570361 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:35:05.570357    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7393a17e-a626-450c-bcb9-04ea1774e002-cluster-monitoring-operator-tls podName:7393a17e-a626-450c-bcb9-04ea1774e002 nodeName:}" failed. No retries permitted until 2026-07-28 17:35:13.57034622 +0000 UTC m=+52.231502560 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7393a17e-a626-450c-bcb9-04ea1774e002-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-jqft4" (UID: "7393a17e-a626-450c-bcb9-04ea1774e002") : secret "cluster-monitoring-operator-tls" not found
Jul 28 17:35:05.570427 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:35:05.570388    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0641104e-1805-4a32-97fd-e9ee7c550152-metrics-certs podName:0641104e-1805-4a32-97fd-e9ee7c550152 nodeName:}" failed. No retries permitted until 2026-07-28 17:35:13.570372176 +0000 UTC m=+52.231528523 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0641104e-1805-4a32-97fd-e9ee7c550152-metrics-certs") pod "router-default-7c67d86669-hftv7" (UID: "0641104e-1805-4a32-97fd-e9ee7c550152") : secret "router-metrics-certs-default" not found
Jul 28 17:35:05.771673 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:05.771594    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/23a671ef-805b-4481-8950-689b84637ec5-registry-tls\") pod \"image-registry-75cf897b8-llxcl\" (UID: \"23a671ef-805b-4481-8950-689b84637ec5\") " pod="openshift-image-registry/image-registry-75cf897b8-llxcl"
Jul 28 17:35:05.771673 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:05.771630    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3a94675-bd9d-4bdf-b73d-074135ab92e3-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-ch2rb\" (UID: \"b3a94675-bd9d-4bdf-b73d-074135ab92e3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-ch2rb"
Jul 28 17:35:05.771834 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:35:05.771734    2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 17:35:05.771834 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:35:05.771743    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 17:35:05.771834 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:35:05.771766    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75cf897b8-llxcl: secret "image-registry-tls" not found
Jul 28 17:35:05.771834 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:35:05.771780    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3a94675-bd9d-4bdf-b73d-074135ab92e3-samples-operator-tls podName:b3a94675-bd9d-4bdf-b73d-074135ab92e3 nodeName:}" failed. No retries permitted until 2026-07-28 17:35:13.771767537 +0000 UTC m=+52.432923870 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b3a94675-bd9d-4bdf-b73d-074135ab92e3-samples-operator-tls") pod "cluster-samples-operator-6748489656-ch2rb" (UID: "b3a94675-bd9d-4bdf-b73d-074135ab92e3") : secret "samples-operator-tls" not found
Jul 28 17:35:05.771834 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:35:05.771821    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/23a671ef-805b-4481-8950-689b84637ec5-registry-tls podName:23a671ef-805b-4481-8950-689b84637ec5 nodeName:}" failed. No retries permitted until 2026-07-28 17:35:13.771807759 +0000 UTC m=+52.432964096 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/23a671ef-805b-4481-8950-689b84637ec5-registry-tls") pod "image-registry-75cf897b8-llxcl" (UID: "23a671ef-805b-4481-8950-689b84637ec5") : secret "image-registry-tls" not found
Jul 28 17:35:05.870115 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:05.870086    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:05.973949 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:05.973915    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dda2a096-8121-4e4d-a074-1565b2bf0872-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mj678\" (UID: \"dda2a096-8121-4e4d-a074-1565b2bf0872\") " pod="openshift-insights/insights-runtime-extractor-mj678"
Jul 28 17:35:05.974097 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:35:05.974037    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 17:35:05.974097 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:05.974083    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dda2a096-8121-4e4d-a074-1565b2bf0872-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mj678\" (UID: \"dda2a096-8121-4e4d-a074-1565b2bf0872\") " pod="openshift-insights/insights-runtime-extractor-mj678"
Jul 28 17:35:05.974168 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:35:05.974101    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dda2a096-8121-4e4d-a074-1565b2bf0872-kube-rbac-proxy-cm podName:dda2a096-8121-4e4d-a074-1565b2bf0872 nodeName:}" failed. No retries permitted until 2026-07-28 17:35:06.974084596 +0000 UTC m=+45.635240944 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dda2a096-8121-4e4d-a074-1565b2bf0872-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mj678" (UID: "dda2a096-8121-4e4d-a074-1565b2bf0872") : configmap "kube-rbac-proxy" not found
Jul 28 17:35:05.974207 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:35:05.974178    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 17:35:05.974277 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:35:05.974224    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dda2a096-8121-4e4d-a074-1565b2bf0872-insights-runtime-extractor-tls podName:dda2a096-8121-4e4d-a074-1565b2bf0872 nodeName:}" failed. No retries permitted until 2026-07-28 17:35:06.974212523 +0000 UTC m=+45.635368857 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dda2a096-8121-4e4d-a074-1565b2bf0872-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mj678" (UID: "dda2a096-8121-4e4d-a074-1565b2bf0872") : secret "insights-runtime-extractor-tls" not found
Jul 28 17:35:06.982940 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:06.982901    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dda2a096-8121-4e4d-a074-1565b2bf0872-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mj678\" (UID: \"dda2a096-8121-4e4d-a074-1565b2bf0872\") " pod="openshift-insights/insights-runtime-extractor-mj678"
Jul 28 17:35:06.983396 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:06.982988    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dda2a096-8121-4e4d-a074-1565b2bf0872-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mj678\" (UID: \"dda2a096-8121-4e4d-a074-1565b2bf0872\") " pod="openshift-insights/insights-runtime-extractor-mj678"
Jul 28 17:35:06.983396 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:35:06.983040    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 17:35:06.983396 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:35:06.983082    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 17:35:06.983396 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:35:06.983108    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dda2a096-8121-4e4d-a074-1565b2bf0872-kube-rbac-proxy-cm podName:dda2a096-8121-4e4d-a074-1565b2bf0872 nodeName:}" failed. No retries permitted until 2026-07-28 17:35:08.983092674 +0000 UTC m=+47.644249008 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dda2a096-8121-4e4d-a074-1565b2bf0872-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mj678" (UID: "dda2a096-8121-4e4d-a074-1565b2bf0872") : configmap "kube-rbac-proxy" not found
Jul 28 17:35:06.983396 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:35:06.983125    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dda2a096-8121-4e4d-a074-1565b2bf0872-insights-runtime-extractor-tls podName:dda2a096-8121-4e4d-a074-1565b2bf0872 nodeName:}" failed. No retries permitted until 2026-07-28 17:35:08.983116051 +0000 UTC m=+47.644272384 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dda2a096-8121-4e4d-a074-1565b2bf0872-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mj678" (UID: "dda2a096-8121-4e4d-a074-1565b2bf0872") : secret "insights-runtime-extractor-tls" not found
Jul 28 17:35:07.628955 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:07.628929    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-j4wkp_654d1f40-f526-4325-b40c-fb544079bff4/dns-node-resolver/0.log"
Jul 28 17:35:08.315397 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:08.315364    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:08.413949 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:08.413923    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:08.829076 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:08.829055    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-z8fhq_d6004102-aca2-4f56-8ddb-94374d3bc900/node-ca/0.log"
Jul 28 17:35:08.887360 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:08.887330    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:08.998707 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:08.998679    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dda2a096-8121-4e4d-a074-1565b2bf0872-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mj678\" (UID: \"dda2a096-8121-4e4d-a074-1565b2bf0872\") " pod="openshift-insights/insights-runtime-extractor-mj678"
Jul 28 17:35:08.998835 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:08.998809    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dda2a096-8121-4e4d-a074-1565b2bf0872-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mj678\" (UID: \"dda2a096-8121-4e4d-a074-1565b2bf0872\") " pod="openshift-insights/insights-runtime-extractor-mj678"
Jul 28 17:35:08.998878 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:35:08.998850    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 17:35:08.998917 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:35:08.998908    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dda2a096-8121-4e4d-a074-1565b2bf0872-insights-runtime-extractor-tls podName:dda2a096-8121-4e4d-a074-1565b2bf0872 nodeName:}" failed. No retries permitted until 2026-07-28 17:35:12.998894033 +0000 UTC m=+51.660050366 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dda2a096-8121-4e4d-a074-1565b2bf0872-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mj678" (UID: "dda2a096-8121-4e4d-a074-1565b2bf0872") : secret "insights-runtime-extractor-tls" not found
Jul 28 17:35:08.998963 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:35:08.998922    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 17:35:08.999001 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:35:08.998975    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dda2a096-8121-4e4d-a074-1565b2bf0872-kube-rbac-proxy-cm podName:dda2a096-8121-4e4d-a074-1565b2bf0872 nodeName:}" failed. No retries permitted until 2026-07-28 17:35:12.998959621 +0000 UTC m=+51.660115954 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dda2a096-8121-4e4d-a074-1565b2bf0872-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mj678" (UID: "dda2a096-8121-4e4d-a074-1565b2bf0872") : configmap "kube-rbac-proxy" not found
Jul 28 17:35:09.430980 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:09.430928    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-lpwn9_093b4d76-2af2-4586-8421-68b7047ff74e/serve-healthcheck-canary/0.log"
Jul 28 17:35:09.905341 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:09.905301    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:10.004019 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:10.003980    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:10.030190 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:10.030165    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-25vvm_ea3e1303-f4dc-4511-abbc-9a7c0e789164/kube-storage-version-migrator-operator/0.log"
Jul 28 17:35:10.104939 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:10.104910    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:10.205387 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:10.205314    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:10.267896 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:10.267869    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:10.595012 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:10.594985    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:11.798863 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:11.798835    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:11.862196 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:11.862153    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:11.997915 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:11.997887    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:12.203059 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:12.203035    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:12.405488 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:12.405456    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:12.607365 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:12.607328    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:13.034639 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:13.034561    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dda2a096-8121-4e4d-a074-1565b2bf0872-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mj678\" (UID: \"dda2a096-8121-4e4d-a074-1565b2bf0872\") " pod="openshift-insights/insights-runtime-extractor-mj678"
Jul 28 17:35:13.034639 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:13.034628    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dda2a096-8121-4e4d-a074-1565b2bf0872-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mj678\" (UID: \"dda2a096-8121-4e4d-a074-1565b2bf0872\") " pod="openshift-insights/insights-runtime-extractor-mj678"
Jul 28 17:35:13.035162 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:13.034656    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/897c5c1b-7fb1-40eb-8b4a-46e737d8e991-metrics-tls\") pod \"dns-default-psd2g\" (UID: \"897c5c1b-7fb1-40eb-8b4a-46e737d8e991\") " pod="openshift-dns/dns-default-psd2g"
Jul 28 17:35:13.035162 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:35:13.034788    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 17:35:13.035162 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:35:13.034844    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 17:35:13.035162 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:35:13.034850    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dda2a096-8121-4e4d-a074-1565b2bf0872-insights-runtime-extractor-tls podName:dda2a096-8121-4e4d-a074-1565b2bf0872 nodeName:}" failed. No retries permitted until 2026-07-28 17:35:21.034835532 +0000 UTC m=+59.695991868 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dda2a096-8121-4e4d-a074-1565b2bf0872-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mj678" (UID: "dda2a096-8121-4e4d-a074-1565b2bf0872") : secret "insights-runtime-extractor-tls" not found
Jul 28 17:35:13.035162 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:35:13.034905    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/897c5c1b-7fb1-40eb-8b4a-46e737d8e991-metrics-tls podName:897c5c1b-7fb1-40eb-8b4a-46e737d8e991 nodeName:}" failed. No retries permitted until 2026-07-28 17:35:29.034890507 +0000 UTC m=+67.696046843 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/897c5c1b-7fb1-40eb-8b4a-46e737d8e991-metrics-tls") pod "dns-default-psd2g" (UID: "897c5c1b-7fb1-40eb-8b4a-46e737d8e991") : secret "dns-default-metrics-tls" not found
Jul 28 17:35:13.035360 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:13.035189    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dda2a096-8121-4e4d-a074-1565b2bf0872-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mj678\" (UID: \"dda2a096-8121-4e4d-a074-1565b2bf0872\") " pod="openshift-insights/insights-runtime-extractor-mj678"
Jul 28 17:35:13.566979 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:13.566940    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:13.627820 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:13.627773    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:13.639844 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:13.639823    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7393a17e-a626-450c-bcb9-04ea1774e002-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-jqft4\" (UID: \"7393a17e-a626-450c-bcb9-04ea1774e002\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-jqft4"
Jul 28 17:35:13.639979 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:35:13.639965    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 17:35:13.640017 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:13.639974    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0641104e-1805-4a32-97fd-e9ee7c550152-service-ca-bundle\") pod \"router-default-7c67d86669-hftv7\" (UID: \"0641104e-1805-4a32-97fd-e9ee7c550152\") " pod="openshift-ingress/router-default-7c67d86669-hftv7"
Jul 28 17:35:13.640017 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:13.639997    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0641104e-1805-4a32-97fd-e9ee7c550152-metrics-certs\") pod \"router-default-7c67d86669-hftv7\" (UID: \"0641104e-1805-4a32-97fd-e9ee7c550152\") " pod="openshift-ingress/router-default-7c67d86669-hftv7"
Jul 28 17:35:13.640080 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:35:13.640022    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7393a17e-a626-450c-bcb9-04ea1774e002-cluster-monitoring-operator-tls podName:7393a17e-a626-450c-bcb9-04ea1774e002 nodeName:}" failed. No retries permitted until 2026-07-28 17:35:29.640008755 +0000 UTC m=+68.301165088 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7393a17e-a626-450c-bcb9-04ea1774e002-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-jqft4" (UID: "7393a17e-a626-450c-bcb9-04ea1774e002") : secret "cluster-monitoring-operator-tls" not found
Jul 28 17:35:13.640080 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:35:13.640075    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 17:35:13.640149 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:35:13.640114    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0641104e-1805-4a32-97fd-e9ee7c550152-service-ca-bundle podName:0641104e-1805-4a32-97fd-e9ee7c550152 nodeName:}" failed. No retries permitted until 2026-07-28 17:35:29.640098661 +0000 UTC m=+68.301255003 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0641104e-1805-4a32-97fd-e9ee7c550152-service-ca-bundle") pod "router-default-7c67d86669-hftv7" (UID: "0641104e-1805-4a32-97fd-e9ee7c550152") : configmap references non-existent config key: service-ca.crt
Jul 28 17:35:13.640149 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:35:13.640131    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0641104e-1805-4a32-97fd-e9ee7c550152-metrics-certs podName:0641104e-1805-4a32-97fd-e9ee7c550152 nodeName:}" failed. No retries permitted until 2026-07-28 17:35:29.640124541 +0000 UTC m=+68.301280874 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0641104e-1805-4a32-97fd-e9ee7c550152-metrics-certs") pod "router-default-7c67d86669-hftv7" (UID: "0641104e-1805-4a32-97fd-e9ee7c550152") : secret "router-metrics-certs-default" not found
Jul 28 17:35:13.796915 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:13.796850    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:13.842206 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:13.842153    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/23a671ef-805b-4481-8950-689b84637ec5-registry-tls\") pod \"image-registry-75cf897b8-llxcl\" (UID: \"23a671ef-805b-4481-8950-689b84637ec5\") " pod="openshift-image-registry/image-registry-75cf897b8-llxcl"
Jul 28 17:35:13.842206 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:13.842180    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3a94675-bd9d-4bdf-b73d-074135ab92e3-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-ch2rb\" (UID: \"b3a94675-bd9d-4bdf-b73d-074135ab92e3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-ch2rb"
Jul 28 17:35:13.844575 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:13.844556    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/23a671ef-805b-4481-8950-689b84637ec5-registry-tls\") pod \"image-registry-75cf897b8-llxcl\" (UID: \"23a671ef-805b-4481-8950-689b84637ec5\") " pod="openshift-image-registry/image-registry-75cf897b8-llxcl"
Jul 28 17:35:13.844662 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:13.844625    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3a94675-bd9d-4bdf-b73d-074135ab92e3-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-ch2rb\" (UID: \"b3a94675-bd9d-4bdf-b73d-074135ab92e3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-ch2rb"
Jul 28 17:35:13.883246 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:13.883220    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-ch2rb"
Jul 28 17:35:13.888393 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:13.888371    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75cf897b8-llxcl"
Jul 28 17:35:13.999765 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:13.999715    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:14.034199 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:14.034098    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-ch2rb"]
Jul 28 17:35:14.054180 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:14.054151    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-75cf897b8-llxcl"]
Jul 28 17:35:14.058543 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:35:14.058512    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23a671ef_805b_4481_8950_689b84637ec5.slice/crio-13f9cc0c5e48d62087412a887662b77aa6aab2c381efbbaadcde86954c962a03 WatchSource:0}: Error finding container 13f9cc0c5e48d62087412a887662b77aa6aab2c381efbbaadcde86954c962a03: Status 404 returned error can't find the container with id 13f9cc0c5e48d62087412a887662b77aa6aab2c381efbbaadcde86954c962a03
Jul 28 17:35:14.065190 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:14.065168    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75cf897b8-llxcl" event={"ID":"23a671ef-805b-4481-8950-689b84637ec5","Type":"ContainerStarted","Data":"13f9cc0c5e48d62087412a887662b77aa6aab2c381efbbaadcde86954c962a03"}
Jul 28 17:35:14.199740 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:14.199698    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:14.399970 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:14.399883    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:14.460742 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:14.460711    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:14.599739 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:14.599705    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:14.896893 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:14.896856    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:15.069346 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:15.069311    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-ch2rb" event={"ID":"b3a94675-bd9d-4bdf-b73d-074135ab92e3","Type":"ContainerStarted","Data":"b3703c0139f516f5db5c33c37341a2fafddb4bde01bff66d9e82de35644222ce"}
Jul 28 17:35:15.071054 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:15.071030    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75cf897b8-llxcl" event={"ID":"23a671ef-805b-4481-8950-689b84637ec5","Type":"ContainerStarted","Data":"34078c2ff20bb00cdfbbb982c68a3e8d8dd78ddcf65d0610129c9b3198c42a0a"}
Jul 28 17:35:15.071190 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:15.071154    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-75cf897b8-llxcl"
Jul 28 17:35:15.093260 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:15.093206    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-75cf897b8-llxcl" podStartSLOduration=18.093188193 podStartE2EDuration="18.093188193s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:34:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:35:15.092703439 +0000 UTC m=+53.753859794" watchObservedRunningTime="2026-07-28 17:35:15.093188193 +0000 UTC m=+53.754344532"
Jul 28 17:35:15.611940 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:15.611898    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:15.999745 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:15.999713    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:16.076045 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:16.076014    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-ch2rb" event={"ID":"b3a94675-bd9d-4bdf-b73d-074135ab92e3","Type":"ContainerStarted","Data":"20237bebe08d4b2e114dbe7452c710ec81a5121a3f3551a1e6c003a66da8d249"}
Jul 28 17:35:16.076405 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:16.076062    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-ch2rb" event={"ID":"b3a94675-bd9d-4bdf-b73d-074135ab92e3","Type":"ContainerStarted","Data":"c6c95182fcf92721a0b1403dedf3c30a3a31f5ac0c2a5a99dad2a259e9f0c802"}
Jul 28 17:35:16.104395 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:16.104369    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:16.199359 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:16.199296    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:16.434997 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:16.434968    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:16.598481 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:16.598453    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:17.606172 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:17.606111    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:17.710595 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:17.710557    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:17.996360 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:17.996279    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:18.196696 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:18.196664    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:18.482058 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:18.482031    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:18.601358 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:18.601326    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:19.799125 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:19.799096    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:19.996700 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:19.996663    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:20.197760 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:20.197710    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:20.397485 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:20.397452    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:20.597686 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:20.597660    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:21.100120 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:21.100094    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dda2a096-8121-4e4d-a074-1565b2bf0872-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mj678\" (UID: \"dda2a096-8121-4e4d-a074-1565b2bf0872\") " pod="openshift-insights/insights-runtime-extractor-mj678"
Jul 28 17:35:21.102392 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:21.102369    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dda2a096-8121-4e4d-a074-1565b2bf0872-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mj678\" (UID: \"dda2a096-8121-4e4d-a074-1565b2bf0872\") " pod="openshift-insights/insights-runtime-extractor-mj678"
Jul 28 17:35:21.185711 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:21.185691    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mj678"
Jul 28 17:35:21.323078 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:21.323021    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-ch2rb" podStartSLOduration=22.527458254 podStartE2EDuration="24.323003888s" totalImagesPullingTime="1.795545634s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:34:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:35:16.093856815 +0000 UTC m=+54.755013170" watchObservedRunningTime="2026-07-28 17:35:21.323003888 +0000 UTC m=+59.984160242"
Jul 28 17:35:21.323783 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:21.323762    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mj678"]
Jul 28 17:35:21.327232 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:35:21.327209    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddda2a096_8121_4e4d_a074_1565b2bf0872.slice/crio-a383ccfc38fee75ae70676d44bd2a140e2e724ebdc02337332d546189281198f WatchSource:0}: Error finding container a383ccfc38fee75ae70676d44bd2a140e2e724ebdc02337332d546189281198f: Status 404 returned error can't find the container with id a383ccfc38fee75ae70676d44bd2a140e2e724ebdc02337332d546189281198f
Jul 28 17:35:22.005259 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:22.005220    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-krssz"
Jul 28 17:35:22.097655 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:22.097617    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mj678" event={"ID":"dda2a096-8121-4e4d-a074-1565b2bf0872","Type":"ContainerStarted","Data":"a383ccfc38fee75ae70676d44bd2a140e2e724ebdc02337332d546189281198f"}
Jul 28 17:35:23.373771 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:23.373708    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:23.473754 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:23.473717    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:26.862804 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:26.862764    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85fc7776b6-nkkfd"]
Jul 28 17:35:26.864816 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:26.864797    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85fc7776b6-nkkfd"
Jul 28 17:35:26.869054 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:26.869025    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 28 17:35:26.869054 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:26.869025    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-566n9\""
Jul 28 17:35:26.869240 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:26.869058    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 28 17:35:26.869240 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:26.869104    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 28 17:35:26.869240 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:26.869050    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 28 17:35:26.875723 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:26.875701    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85fc7776b6-nkkfd"]
Jul 28 17:35:26.883535 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:26.883513    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-75cf897b8-llxcl"]
Jul 28 17:35:26.916890 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:26.916860    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6b87dbb6b9-n25x2"]
Jul 28 17:35:26.918801 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:26.918780    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b87dbb6b9-n25x2"
Jul 28 17:35:26.932357 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:26.932327    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6b87dbb6b9-n25x2"]
Jul 28 17:35:26.953735 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:26.953706    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jmcc\" (UniqueName: \"kubernetes.io/projected/8cc9cc19-d07b-4184-b83b-544c48677a8d-kube-api-access-4jmcc\") pod \"managed-serviceaccount-addon-agent-85fc7776b6-nkkfd\" (UID: \"8cc9cc19-d07b-4184-b83b-544c48677a8d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85fc7776b6-nkkfd"
Jul 28 17:35:26.953908 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:26.953889    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8cc9cc19-d07b-4184-b83b-544c48677a8d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-85fc7776b6-nkkfd\" (UID: \"8cc9cc19-d07b-4184-b83b-544c48677a8d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85fc7776b6-nkkfd"
Jul 28 17:35:26.973728 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:26.973699    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:27.041959 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:27.041928    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:27.054639 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:27.054542    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/91955790-de58-4f37-b897-77c442ec1a64-ca-trust-extracted\") pod \"image-registry-6b87dbb6b9-n25x2\" (UID: \"91955790-de58-4f37-b897-77c442ec1a64\") " pod="openshift-image-registry/image-registry-6b87dbb6b9-n25x2"
Jul 28 17:35:27.054639 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:27.054593    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/91955790-de58-4f37-b897-77c442ec1a64-bound-sa-token\") pod \"image-registry-6b87dbb6b9-n25x2\" (UID: \"91955790-de58-4f37-b897-77c442ec1a64\") " pod="openshift-image-registry/image-registry-6b87dbb6b9-n25x2"
Jul 28 17:35:27.055022 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:27.054887    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/91955790-de58-4f37-b897-77c442ec1a64-image-registry-private-configuration\") pod \"image-registry-6b87dbb6b9-n25x2\" (UID: \"91955790-de58-4f37-b897-77c442ec1a64\") " pod="openshift-image-registry/image-registry-6b87dbb6b9-n25x2"
Jul 28 17:35:27.055022 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:27.054963    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8cc9cc19-d07b-4184-b83b-544c48677a8d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-85fc7776b6-nkkfd\" (UID: \"8cc9cc19-d07b-4184-b83b-544c48677a8d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85fc7776b6-nkkfd"
Jul 28 17:35:27.055457 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:27.055177    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/91955790-de58-4f37-b897-77c442ec1a64-registry-tls\") pod \"image-registry-6b87dbb6b9-n25x2\" (UID: \"91955790-de58-4f37-b897-77c442ec1a64\") " pod="openshift-image-registry/image-registry-6b87dbb6b9-n25x2"
Jul 28 17:35:27.055457 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:27.055263    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snjgg\" (UniqueName: \"kubernetes.io/projected/91955790-de58-4f37-b897-77c442ec1a64-kube-api-access-snjgg\") pod \"image-registry-6b87dbb6b9-n25x2\" (UID: \"91955790-de58-4f37-b897-77c442ec1a64\") " pod="openshift-image-registry/image-registry-6b87dbb6b9-n25x2"
Jul 28 17:35:27.055457 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:27.055293    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jmcc\" (UniqueName: \"kubernetes.io/projected/8cc9cc19-d07b-4184-b83b-544c48677a8d-kube-api-access-4jmcc\") pod \"managed-serviceaccount-addon-agent-85fc7776b6-nkkfd\" (UID: \"8cc9cc19-d07b-4184-b83b-544c48677a8d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85fc7776b6-nkkfd"
Jul 28 17:35:27.055457 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:27.055323    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/91955790-de58-4f37-b897-77c442ec1a64-trusted-ca\") pod \"image-registry-6b87dbb6b9-n25x2\" (UID: \"91955790-de58-4f37-b897-77c442ec1a64\") " pod="openshift-image-registry/image-registry-6b87dbb6b9-n25x2"
Jul 28 17:35:27.055457 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:27.055353    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/91955790-de58-4f37-b897-77c442ec1a64-registry-certificates\") pod \"image-registry-6b87dbb6b9-n25x2\" (UID: \"91955790-de58-4f37-b897-77c442ec1a64\") " pod="openshift-image-registry/image-registry-6b87dbb6b9-n25x2"
Jul 28 17:35:27.055457 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:27.055398    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/91955790-de58-4f37-b897-77c442ec1a64-installation-pull-secrets\") pod \"image-registry-6b87dbb6b9-n25x2\" (UID: \"91955790-de58-4f37-b897-77c442ec1a64\") " pod="openshift-image-registry/image-registry-6b87dbb6b9-n25x2"
Jul 28 17:35:27.062332 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:27.062306    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8cc9cc19-d07b-4184-b83b-544c48677a8d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-85fc7776b6-nkkfd\" (UID: \"8cc9cc19-d07b-4184-b83b-544c48677a8d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85fc7776b6-nkkfd"
Jul 28 17:35:27.067312 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:27.067288    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jmcc\" (UniqueName: \"kubernetes.io/projected/8cc9cc19-d07b-4184-b83b-544c48677a8d-kube-api-access-4jmcc\") pod \"managed-serviceaccount-addon-agent-85fc7776b6-nkkfd\" (UID: \"8cc9cc19-d07b-4184-b83b-544c48677a8d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85fc7776b6-nkkfd"
Jul 28 17:35:27.118305 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:27.118239    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mj678" event={"ID":"dda2a096-8121-4e4d-a074-1565b2bf0872","Type":"ContainerStarted","Data":"41fd57e2db871dba753dffc5340839e0a4d0fe3877bc0dfbcf3d914915ba6cdd"}
Jul 28 17:35:27.139814 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:27.139783    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:27.156819 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:27.156789    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snjgg\" (UniqueName: \"kubernetes.io/projected/91955790-de58-4f37-b897-77c442ec1a64-kube-api-access-snjgg\") pod \"image-registry-6b87dbb6b9-n25x2\" (UID: \"91955790-de58-4f37-b897-77c442ec1a64\") " pod="openshift-image-registry/image-registry-6b87dbb6b9-n25x2"
Jul 28 17:35:27.156975 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:27.156834    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/91955790-de58-4f37-b897-77c442ec1a64-trusted-ca\") pod \"image-registry-6b87dbb6b9-n25x2\" (UID: \"91955790-de58-4f37-b897-77c442ec1a64\") " pod="openshift-image-registry/image-registry-6b87dbb6b9-n25x2"
Jul 28 17:35:27.156975 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:27.156863    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/91955790-de58-4f37-b897-77c442ec1a64-registry-certificates\") pod \"image-registry-6b87dbb6b9-n25x2\" (UID: \"91955790-de58-4f37-b897-77c442ec1a64\") " pod="openshift-image-registry/image-registry-6b87dbb6b9-n25x2"
Jul 28 17:35:27.156975 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:27.156904    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/91955790-de58-4f37-b897-77c442ec1a64-installation-pull-secrets\") pod \"image-registry-6b87dbb6b9-n25x2\" (UID: \"91955790-de58-4f37-b897-77c442ec1a64\") " pod="openshift-image-registry/image-registry-6b87dbb6b9-n25x2"
Jul 28 17:35:27.156975 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:27.156967    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/91955790-de58-4f37-b897-77c442ec1a64-ca-trust-extracted\") pod \"image-registry-6b87dbb6b9-n25x2\" (UID: \"91955790-de58-4f37-b897-77c442ec1a64\") " pod="openshift-image-registry/image-registry-6b87dbb6b9-n25x2"
Jul 28 17:35:27.157180 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:27.157003    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/91955790-de58-4f37-b897-77c442ec1a64-bound-sa-token\") pod \"image-registry-6b87dbb6b9-n25x2\" (UID: \"91955790-de58-4f37-b897-77c442ec1a64\") " pod="openshift-image-registry/image-registry-6b87dbb6b9-n25x2"
Jul 28 17:35:27.157180 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:27.157032    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/91955790-de58-4f37-b897-77c442ec1a64-image-registry-private-configuration\") pod \"image-registry-6b87dbb6b9-n25x2\" (UID: \"91955790-de58-4f37-b897-77c442ec1a64\") " pod="openshift-image-registry/image-registry-6b87dbb6b9-n25x2"
Jul 28 17:35:27.157180 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:27.157085    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/91955790-de58-4f37-b897-77c442ec1a64-registry-tls\") pod \"image-registry-6b87dbb6b9-n25x2\" (UID: \"91955790-de58-4f37-b897-77c442ec1a64\") " pod="openshift-image-registry/image-registry-6b87dbb6b9-n25x2"
Jul 28 17:35:27.158784 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:27.158363    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/91955790-de58-4f37-b897-77c442ec1a64-ca-trust-extracted\") pod \"image-registry-6b87dbb6b9-n25x2\" (UID: \"91955790-de58-4f37-b897-77c442ec1a64\") " pod="openshift-image-registry/image-registry-6b87dbb6b9-n25x2"
Jul 28 17:35:27.158784 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:27.158699    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/91955790-de58-4f37-b897-77c442ec1a64-trusted-ca\") pod \"image-registry-6b87dbb6b9-n25x2\" (UID: \"91955790-de58-4f37-b897-77c442ec1a64\") " pod="openshift-image-registry/image-registry-6b87dbb6b9-n25x2"
Jul 28 17:35:27.158784 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:27.158770    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/91955790-de58-4f37-b897-77c442ec1a64-registry-certificates\") pod \"image-registry-6b87dbb6b9-n25x2\" (UID: \"91955790-de58-4f37-b897-77c442ec1a64\") " pod="openshift-image-registry/image-registry-6b87dbb6b9-n25x2"
Jul 28 17:35:27.161076 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:27.160700    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/91955790-de58-4f37-b897-77c442ec1a64-registry-tls\") pod \"image-registry-6b87dbb6b9-n25x2\" (UID: \"91955790-de58-4f37-b897-77c442ec1a64\") " pod="openshift-image-registry/image-registry-6b87dbb6b9-n25x2"
Jul 28 17:35:27.161193 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:27.161178    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/91955790-de58-4f37-b897-77c442ec1a64-image-registry-private-configuration\") pod \"image-registry-6b87dbb6b9-n25x2\" (UID: \"91955790-de58-4f37-b897-77c442ec1a64\") " pod="openshift-image-registry/image-registry-6b87dbb6b9-n25x2"
Jul 28 17:35:27.162024 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:27.161898    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/91955790-de58-4f37-b897-77c442ec1a64-installation-pull-secrets\") pod \"image-registry-6b87dbb6b9-n25x2\" (UID: \"91955790-de58-4f37-b897-77c442ec1a64\") " pod="openshift-image-registry/image-registry-6b87dbb6b9-n25x2"
Jul 28 17:35:27.166996 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:27.166972    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/91955790-de58-4f37-b897-77c442ec1a64-bound-sa-token\") pod \"image-registry-6b87dbb6b9-n25x2\" (UID: \"91955790-de58-4f37-b897-77c442ec1a64\") " pod="openshift-image-registry/image-registry-6b87dbb6b9-n25x2"
Jul 28 17:35:27.167513 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:27.167473    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-snjgg\" (UniqueName: \"kubernetes.io/projected/91955790-de58-4f37-b897-77c442ec1a64-kube-api-access-snjgg\") pod \"image-registry-6b87dbb6b9-n25x2\" (UID: \"91955790-de58-4f37-b897-77c442ec1a64\") " pod="openshift-image-registry/image-registry-6b87dbb6b9-n25x2"
Jul 28 17:35:27.183697 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:27.183678    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85fc7776b6-nkkfd"
Jul 28 17:35:27.227994 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:27.227759    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b87dbb6b9-n25x2"
Jul 28 17:35:27.342595 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:27.342561    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85fc7776b6-nkkfd"]
Jul 28 17:35:27.351594 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:27.350819    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:27.408143 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:27.408113    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6b87dbb6b9-n25x2"]
Jul 28 17:35:27.410950 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:35:27.410921    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91955790_de58_4f37_b897_77c442ec1a64.slice/crio-c66e70de264fc544b421d7fffb4b0447b6cef36de311cdea25ebf3fac4d54c9e WatchSource:0}: Error finding container c66e70de264fc544b421d7fffb4b0447b6cef36de311cdea25ebf3fac4d54c9e: Status 404 returned error can't find the container with id c66e70de264fc544b421d7fffb4b0447b6cef36de311cdea25ebf3fac4d54c9e
Jul 28 17:35:27.490545 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:27.490293    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:27.663320 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:27.663234    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3c4a388-fdcc-4744-b6e2-d52dc701f235-metrics-certs\") pod \"network-metrics-daemon-4tthg\" (UID: \"d3c4a388-fdcc-4744-b6e2-d52dc701f235\") " pod="openshift-multus/network-metrics-daemon-4tthg"
Jul 28 17:35:27.663513 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:27.663321    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8a8edb3a-34c9-4a66-81aa-49474fdfdf37-original-pull-secret\") pod \"global-pull-secret-syncer-z75m5\" (UID: \"8a8edb3a-34c9-4a66-81aa-49474fdfdf37\") " pod="kube-system/global-pull-secret-syncer-z75m5"
Jul 28 17:35:27.666101 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:27.666077    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 17:35:27.666212 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:27.666126    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 17:35:27.675933 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:27.675906    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3c4a388-fdcc-4744-b6e2-d52dc701f235-metrics-certs\") pod \"network-metrics-daemon-4tthg\" (UID: \"d3c4a388-fdcc-4744-b6e2-d52dc701f235\") " pod="openshift-multus/network-metrics-daemon-4tthg"
Jul 28 17:35:27.676184 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:27.676168    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8a8edb3a-34c9-4a66-81aa-49474fdfdf37-original-pull-secret\") pod \"global-pull-secret-syncer-z75m5\" (UID: \"8a8edb3a-34c9-4a66-81aa-49474fdfdf37\") " pod="kube-system/global-pull-secret-syncer-z75m5"
Jul 28 17:35:27.700195 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:27.700153    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:27.764512 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:27.764478    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gg7b\" (UniqueName: \"kubernetes.io/projected/41d54bc6-a997-48d3-9752-f78204bbdf43-kube-api-access-4gg7b\") pod \"network-check-target-74vbl\" (UID: \"41d54bc6-a997-48d3-9752-f78204bbdf43\") " pod="openshift-network-diagnostics/network-check-target-74vbl"
Jul 28 17:35:27.767747 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:27.767722    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 17:35:27.777029 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:27.777009    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 17:35:27.787915 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:27.787890    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gg7b\" (UniqueName: \"kubernetes.io/projected/41d54bc6-a997-48d3-9752-f78204bbdf43-kube-api-access-4gg7b\") pod \"network-check-target-74vbl\" (UID: \"41d54bc6-a997-48d3-9752-f78204bbdf43\") " pod="openshift-network-diagnostics/network-check-target-74vbl"
Jul 28 17:35:27.963414 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:27.963341    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-c22cz\""
Jul 28 17:35:27.966547 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:27.966527    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z75m5"
Jul 28 17:35:27.971702 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:27.971679    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-74vbl"
Jul 28 17:35:27.974149 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:27.974130    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-z5ccl\""
Jul 28 17:35:27.981859 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:27.981717    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tthg"
Jul 28 17:35:28.127512 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:28.126907    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b87dbb6b9-n25x2" event={"ID":"91955790-de58-4f37-b897-77c442ec1a64","Type":"ContainerStarted","Data":"3f3067a3896165709ad41b946f07b77c9bfd98054bdf88a9954a466e07ecdfba"}
Jul 28 17:35:28.127512 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:28.126965    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b87dbb6b9-n25x2" event={"ID":"91955790-de58-4f37-b897-77c442ec1a64","Type":"ContainerStarted","Data":"c66e70de264fc544b421d7fffb4b0447b6cef36de311cdea25ebf3fac4d54c9e"}
Jul 28 17:35:28.127512 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:28.127324    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6b87dbb6b9-n25x2"
Jul 28 17:35:28.129622 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:28.129566    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85fc7776b6-nkkfd" event={"ID":"8cc9cc19-d07b-4184-b83b-544c48677a8d","Type":"ContainerStarted","Data":"83b98048b43026c5ff16e41ed7196f30926b04daf8102306fd4d79347768bedc"}
Jul 28 17:35:28.161561 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:28.161482    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6b87dbb6b9-n25x2" podStartSLOduration=2.161459899 podStartE2EDuration="2.161459899s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:35:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:35:28.159599189 +0000 UTC m=+66.820755555" watchObservedRunningTime="2026-07-28 17:35:28.161459899 +0000 UTC m=+66.822616255"
Jul 28 17:35:28.164165 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:35:28.164141    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a8edb3a_34c9_4a66_81aa_49474fdfdf37.slice/crio-d06e7dd6adc17ef38527c78eb0aa68f1b8a5513b492751fb7d5fbc2b0ce0e367 WatchSource:0}: Error finding container d06e7dd6adc17ef38527c78eb0aa68f1b8a5513b492751fb7d5fbc2b0ce0e367: Status 404 returned error can't find the container with id d06e7dd6adc17ef38527c78eb0aa68f1b8a5513b492751fb7d5fbc2b0ce0e367
Jul 28 17:35:28.164305 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:28.164276    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-z75m5"]
Jul 28 17:35:28.173960 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:28.173917    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-74vbl"]
Jul 28 17:35:28.193391 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:28.193364    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4tthg"]
Jul 28 17:35:29.076615 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:29.076578    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/897c5c1b-7fb1-40eb-8b4a-46e737d8e991-metrics-tls\") pod \"dns-default-psd2g\" (UID: \"897c5c1b-7fb1-40eb-8b4a-46e737d8e991\") " pod="openshift-dns/dns-default-psd2g"
Jul 28 17:35:29.079094 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:29.079030    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/897c5c1b-7fb1-40eb-8b4a-46e737d8e991-metrics-tls\") pod \"dns-default-psd2g\" (UID: \"897c5c1b-7fb1-40eb-8b4a-46e737d8e991\") " pod="openshift-dns/dns-default-psd2g"
Jul 28 17:35:29.134259 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:29.134221    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-z75m5" event={"ID":"8a8edb3a-34c9-4a66-81aa-49474fdfdf37","Type":"ContainerStarted","Data":"d06e7dd6adc17ef38527c78eb0aa68f1b8a5513b492751fb7d5fbc2b0ce0e367"}
Jul 28 17:35:29.300040 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:29.300002    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lxmvx\""
Jul 28 17:35:29.308134 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:29.308100    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-psd2g"
Jul 28 17:35:29.683320 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:29.683241    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0641104e-1805-4a32-97fd-e9ee7c550152-service-ca-bundle\") pod \"router-default-7c67d86669-hftv7\" (UID: \"0641104e-1805-4a32-97fd-e9ee7c550152\") " pod="openshift-ingress/router-default-7c67d86669-hftv7"
Jul 28 17:35:29.683320 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:29.683288    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0641104e-1805-4a32-97fd-e9ee7c550152-metrics-certs\") pod \"router-default-7c67d86669-hftv7\" (UID: \"0641104e-1805-4a32-97fd-e9ee7c550152\") " pod="openshift-ingress/router-default-7c67d86669-hftv7"
Jul 28 17:35:29.683520 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:29.683343    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7393a17e-a626-450c-bcb9-04ea1774e002-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-jqft4\" (UID: \"7393a17e-a626-450c-bcb9-04ea1774e002\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-jqft4"
Jul 28 17:35:29.683950 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:29.683912    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0641104e-1805-4a32-97fd-e9ee7c550152-service-ca-bundle\") pod \"router-default-7c67d86669-hftv7\" (UID: \"0641104e-1805-4a32-97fd-e9ee7c550152\") " pod="openshift-ingress/router-default-7c67d86669-hftv7"
Jul 28 17:35:29.685890 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:29.685863    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0641104e-1805-4a32-97fd-e9ee7c550152-metrics-certs\") pod \"router-default-7c67d86669-hftv7\" (UID: \"0641104e-1805-4a32-97fd-e9ee7c550152\") " pod="openshift-ingress/router-default-7c67d86669-hftv7"
Jul 28 17:35:29.686014 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:29.685950    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7393a17e-a626-450c-bcb9-04ea1774e002-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-jqft4\" (UID: \"7393a17e-a626-450c-bcb9-04ea1774e002\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-jqft4"
Jul 28 17:35:29.686075 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:35:29.686019    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3c4a388_fdcc_4744_b6e2_d52dc701f235.slice/crio-55c83dfb1dde2c4aa5b30cb23a32d206c1d6f1eb334473556604ced457a1f7dd WatchSource:0}: Error finding container 55c83dfb1dde2c4aa5b30cb23a32d206c1d6f1eb334473556604ced457a1f7dd: Status 404 returned error can't find the container with id 55c83dfb1dde2c4aa5b30cb23a32d206c1d6f1eb334473556604ced457a1f7dd
Jul 28 17:35:29.886221 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:29.886185    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-bbm7j\""
Jul 28 17:35:29.890525 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:29.890500    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-7nq9j\""
Jul 28 17:35:29.893489 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:29.893465    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-jqft4"
Jul 28 17:35:29.899210 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:29.899084    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7c67d86669-hftv7"
Jul 28 17:35:30.078679 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:30.078644    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:30.139287 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:30.139252    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4tthg" event={"ID":"d3c4a388-fdcc-4744-b6e2-d52dc701f235","Type":"ContainerStarted","Data":"55c83dfb1dde2c4aa5b30cb23a32d206c1d6f1eb334473556604ced457a1f7dd"}
Jul 28 17:35:30.140698 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:30.140671    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-74vbl" event={"ID":"41d54bc6-a997-48d3-9752-f78204bbdf43","Type":"ContainerStarted","Data":"7465aba6cc302e1313af6b825c813062d58806784ec68d40f69717eb7b8fc7bb"}
Jul 28 17:35:30.152620 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:30.152592    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:32.589023 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:32.588943    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-psd2g"]
Jul 28 17:35:32.635786 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:32.635705    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-jqft4"]
Jul 28 17:35:34.290622 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:35:34.290585    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod897c5c1b_7fb1_40eb_8b4a_46e737d8e991.slice/crio-93e76f7c38fe579054d7037fc64f1739c9dfd7b4d36e52300ee1d371358df29d WatchSource:0}: Error finding container 93e76f7c38fe579054d7037fc64f1739c9dfd7b4d36e52300ee1d371358df29d: Status 404 returned error can't find the container with id 93e76f7c38fe579054d7037fc64f1739c9dfd7b4d36e52300ee1d371358df29d
Jul 28 17:35:34.291485 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:35:34.291315    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7393a17e_a626_450c_bcb9_04ea1774e002.slice/crio-303bd0aef9cc07227b325c5a42e2c78844f43a5d7cbc118e6ddc1a19877ae59e WatchSource:0}: Error finding container 303bd0aef9cc07227b325c5a42e2c78844f43a5d7cbc118e6ddc1a19877ae59e: Status 404 returned error can't find the container with id 303bd0aef9cc07227b325c5a42e2c78844f43a5d7cbc118e6ddc1a19877ae59e
Jul 28 17:35:34.419152 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:34.419091    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7c67d86669-hftv7"]
Jul 28 17:35:35.158780 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:35.158722    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-jqft4" event={"ID":"7393a17e-a626-450c-bcb9-04ea1774e002","Type":"ContainerStarted","Data":"303bd0aef9cc07227b325c5a42e2c78844f43a5d7cbc118e6ddc1a19877ae59e"}
Jul 28 17:35:35.159990 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:35.159957    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-psd2g" event={"ID":"897c5c1b-7fb1-40eb-8b4a-46e737d8e991","Type":"ContainerStarted","Data":"93e76f7c38fe579054d7037fc64f1739c9dfd7b4d36e52300ee1d371358df29d"}
Jul 28 17:35:35.264189 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:35:35.264154    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0641104e_1805_4a32_97fd_e9ee7c550152.slice/crio-c99eae1599fdeaa4a7359459d1d74fb7475df38050fc7d5edb05939af5c3136c WatchSource:0}: Error finding container c99eae1599fdeaa4a7359459d1d74fb7475df38050fc7d5edb05939af5c3136c: Status 404 returned error can't find the container with id c99eae1599fdeaa4a7359459d1d74fb7475df38050fc7d5edb05939af5c3136c
Jul 28 17:35:35.771716 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:35.771624    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:35.872720 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:35.872666    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:35.974126 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:35.974057    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:36.047505 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:36.047401    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:36.167215 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:36.167136    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mj678" event={"ID":"dda2a096-8121-4e4d-a074-1565b2bf0872","Type":"ContainerStarted","Data":"a3f486f5555d58c50b487b079c516f33e161502016e896dd5dca7eeba49aec4b"}
Jul 28 17:35:36.169230 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:36.169202    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-z75m5" event={"ID":"8a8edb3a-34c9-4a66-81aa-49474fdfdf37","Type":"ContainerStarted","Data":"1b2d00dff23bff3e3833c95babb5c82e49ce648ec1b6317847804634540ae081"}
Jul 28 17:35:36.171853 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:36.171768    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4tthg" event={"ID":"d3c4a388-fdcc-4744-b6e2-d52dc701f235","Type":"ContainerStarted","Data":"592274b1e8406893563a2509a54df6d960d1b3f50d96fc86097f06c098d2b1b6"}
Jul 28 17:35:36.171997 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:36.171819    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4tthg" event={"ID":"d3c4a388-fdcc-4744-b6e2-d52dc701f235","Type":"ContainerStarted","Data":"1a6256139357c69d180092572b33a637cb7928e3f24085e19aadd833475cc54a"}
Jul 28 17:35:36.175513 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:36.175490    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7c67d86669-hftv7" event={"ID":"0641104e-1805-4a32-97fd-e9ee7c550152","Type":"ContainerStarted","Data":"1ddc85f240d0efe468979ea268e66285e71b58000b6a4d319b36d8571349d7f9"}
Jul 28 17:35:36.175623 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:36.175549    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7c67d86669-hftv7" event={"ID":"0641104e-1805-4a32-97fd-e9ee7c550152","Type":"ContainerStarted","Data":"c99eae1599fdeaa4a7359459d1d74fb7475df38050fc7d5edb05939af5c3136c"}
Jul 28 17:35:36.178502 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:36.178107    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85fc7776b6-nkkfd" event={"ID":"8cc9cc19-d07b-4184-b83b-544c48677a8d","Type":"ContainerStarted","Data":"38031c00b58128bcab8ccb59796ddf81231f5f19d61cd5b349579c836e627a1a"}
Jul 28 17:35:36.180234 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:36.180140    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-74vbl" event={"ID":"41d54bc6-a997-48d3-9752-f78204bbdf43","Type":"ContainerStarted","Data":"ebf597c6f760edc38864a8f7002e4031a71c788be127b1d55e6ae936370bb182"}
Jul 28 17:35:36.180349 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:36.180320    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-74vbl"
Jul 28 17:35:36.187961 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:36.187918    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-z75m5" podStartSLOduration=68.055164498 podStartE2EDuration="1m15.187899505s" totalImagesPullingTime="7.132735007s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:34:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:35:36.186030707 +0000 UTC m=+74.847187063" watchObservedRunningTime="2026-07-28 17:35:36.187899505 +0000 UTC m=+74.849055860"
Jul 28 17:35:36.210521 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:36.209539    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-7c67d86669-hftv7" podStartSLOduration=39.209521147 podStartE2EDuration="39.209521147s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:34:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:35:36.208647593 +0000 UTC m=+74.869803950" watchObservedRunningTime="2026-07-28 17:35:36.209521147 +0000 UTC m=+74.870677504"
Jul 28 17:35:36.224170 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:36.224123    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85fc7776b6-nkkfd" podStartSLOduration=2.338550368 podStartE2EDuration="10.224112107s" totalImagesPullingTime="7.885561739s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:35:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:35:36.223249604 +0000 UTC m=+74.884405954" watchObservedRunningTime="2026-07-28 17:35:36.224112107 +0000 UTC m=+74.885268463"
Jul 28 17:35:36.248193 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:36.247800    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-74vbl" podStartSLOduration=69.643906297 podStartE2EDuration="1m15.247783305s" totalImagesPullingTime="5.603877008s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:34:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:35:36.246780417 +0000 UTC m=+74.907936772" watchObservedRunningTime="2026-07-28 17:35:36.247783305 +0000 UTC m=+74.908939663"
Jul 28 17:35:36.272632 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:36.270165    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-4tthg" podStartSLOduration=69.675777581 podStartE2EDuration="1m15.27014624s" totalImagesPullingTime="5.594368659s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:34:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:35:36.264665104 +0000 UTC m=+74.925821460" watchObservedRunningTime="2026-07-28 17:35:36.27014624 +0000 UTC m=+74.931302599"
Jul 28 17:35:36.891493 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:36.891453    2563 patch_prober.go:28] interesting pod/image-registry-75cf897b8-llxcl container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 28 17:35:36.891930 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:36.891521    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-75cf897b8-llxcl" podUID="23a671ef-805b-4481-8950-689b84637ec5" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 17:35:36.899481 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:36.899321    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7c67d86669-hftv7"
Jul 28 17:35:36.902453 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:36.902406    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7c67d86669-hftv7"
Jul 28 17:35:36.927019 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:36.926980    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:37.185732 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:37.185599    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7c67d86669-hftv7"
Jul 28 17:35:37.186885 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:37.186850    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7c67d86669-hftv7"
Jul 28 17:35:37.323571 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:37.323542    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:37.423900 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:37.423844    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:37.671132 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:37.671087    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:37.771236 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:37.771161    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:37.847020 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:37.846982    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:37.922001 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:37.921966    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:38.118088 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:38.118061    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:38.190843 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:38.190810    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-jqft4" event={"ID":"7393a17e-a626-450c-bcb9-04ea1774e002","Type":"ContainerStarted","Data":"5742a2ad47ec3198e1fff5da91e20b1b29e0505371c753b77327c6d881659861"}
Jul 28 17:35:38.192995 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:38.192965    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-psd2g" event={"ID":"897c5c1b-7fb1-40eb-8b4a-46e737d8e991","Type":"ContainerStarted","Data":"a8b84e0f3c6b35350e8013693e7d65e979f96d8f055aa280f02e47a4ef79acaf"}
Jul 28 17:35:38.193129 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:38.193020    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-psd2g" event={"ID":"897c5c1b-7fb1-40eb-8b4a-46e737d8e991","Type":"ContainerStarted","Data":"0c2931a5894065f2cdfddc7a965119f4bd74591d51b34c835862dd307550d306"}
Jul 28 17:35:38.208835 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:38.208787    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-jqft4" podStartSLOduration=38.189387779 podStartE2EDuration="41.208771284s" totalImagesPullingTime="3.019383505s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:34:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:35:38.208726029 +0000 UTC m=+76.869882385" watchObservedRunningTime="2026-07-28 17:35:38.208771284 +0000 UTC m=+76.869927642"
Jul 28 17:35:38.219478 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:38.219450    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:38.231405 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:38.231359    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-psd2g" podStartSLOduration=38.212949502 podStartE2EDuration="41.231340131s" totalImagesPullingTime="3.018390629s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:34:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:35:38.230033692 +0000 UTC m=+76.891190074" watchObservedRunningTime="2026-07-28 17:35:38.231340131 +0000 UTC m=+76.892496490"
Jul 28 17:35:38.539774 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:38.539694    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:38.600170 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:38.600139    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:39.197277 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:39.197246    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-psd2g"
Jul 28 17:35:39.524915 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:39.524836    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:39.918398 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:39.918360    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:40.020013 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:40.019984    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:40.206367 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:40.206277    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mj678" event={"ID":"dda2a096-8121-4e4d-a074-1565b2bf0872","Type":"ContainerStarted","Data":"408a18f9af7230e4187dcfe7e60e6c18ab73792e765bef1664ef92d240c9cb92"}
Jul 28 17:35:40.226996 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:40.226952    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-mj678" podStartSLOduration=17.509271882 podStartE2EDuration="35.226938737s" totalImagesPullingTime="17.717666855s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:35:05 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:35:40.224973103 +0000 UTC m=+78.886129457" watchObservedRunningTime="2026-07-28 17:35:40.226938737 +0000 UTC m=+78.888095091"
Jul 28 17:35:40.890654 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:40.890623    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:40.967839 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:40.967810    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:41.157120 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:41.157058    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:42.079864 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:42.079832    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:42.179763 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:42.179740    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:42.539057 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:42.538979    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:42.599757 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:42.599729    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:42.735869 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:42.735841    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:42.810121 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:42.810051    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:43.132902 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:43.132794    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:43.233062 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:43.233034    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:43.535735 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:43.535671    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:43.636923 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:43.636897    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:43.747486 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:43.747428    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:44.014890 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:44.014862    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:44.072010 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:44.071987    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:44.145850 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:44.145821    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:44.439259 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:44.439232    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:45.361496 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:45.361463    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:46.043112 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:46.043079    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:46.277240 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:46.277216    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:46.306892 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:46.306828    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-jplhg"]
Jul 28 17:35:46.309907 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:46.309891    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jplhg"
Jul 28 17:35:46.312549 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:46.312525    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 28 17:35:46.312720 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:46.312673    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 28 17:35:46.312811 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:46.312739    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-p6zg4\""
Jul 28 17:35:46.312811 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:46.312776    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 28 17:35:46.313584 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:46.313571    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 28 17:35:46.380962 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:46.380940    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:46.413494 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:46.413473    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3573a7fb-6b9a-4b2b-99f0-99a7b4301719-node-exporter-textfile\") pod \"node-exporter-jplhg\" (UID: \"3573a7fb-6b9a-4b2b-99f0-99a7b4301719\") " pod="openshift-monitoring/node-exporter-jplhg"
Jul 28 17:35:46.413613 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:46.413506    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3573a7fb-6b9a-4b2b-99f0-99a7b4301719-metrics-client-ca\") pod \"node-exporter-jplhg\" (UID: \"3573a7fb-6b9a-4b2b-99f0-99a7b4301719\") " pod="openshift-monitoring/node-exporter-jplhg"
Jul 28 17:35:46.413613 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:46.413530    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3573a7fb-6b9a-4b2b-99f0-99a7b4301719-node-exporter-wtmp\") pod \"node-exporter-jplhg\" (UID: \"3573a7fb-6b9a-4b2b-99f0-99a7b4301719\") " pod="openshift-monitoring/node-exporter-jplhg"
Jul 28 17:35:46.413613 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:46.413549    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3573a7fb-6b9a-4b2b-99f0-99a7b4301719-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jplhg\" (UID: \"3573a7fb-6b9a-4b2b-99f0-99a7b4301719\") " pod="openshift-monitoring/node-exporter-jplhg"
Jul 28 17:35:46.413613 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:46.413600    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3573a7fb-6b9a-4b2b-99f0-99a7b4301719-sys\") pod \"node-exporter-jplhg\" (UID: \"3573a7fb-6b9a-4b2b-99f0-99a7b4301719\") " pod="openshift-monitoring/node-exporter-jplhg"
Jul 28 17:35:46.413811 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:46.413658    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkmpk\" (UniqueName: \"kubernetes.io/projected/3573a7fb-6b9a-4b2b-99f0-99a7b4301719-kube-api-access-nkmpk\") pod \"node-exporter-jplhg\" (UID: \"3573a7fb-6b9a-4b2b-99f0-99a7b4301719\") " pod="openshift-monitoring/node-exporter-jplhg"
Jul 28 17:35:46.413811 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:46.413707    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3573a7fb-6b9a-4b2b-99f0-99a7b4301719-root\") pod \"node-exporter-jplhg\" (UID: \"3573a7fb-6b9a-4b2b-99f0-99a7b4301719\") " pod="openshift-monitoring/node-exporter-jplhg"
Jul 28 17:35:46.413811 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:46.413747    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3573a7fb-6b9a-4b2b-99f0-99a7b4301719-node-exporter-tls\") pod \"node-exporter-jplhg\" (UID: \"3573a7fb-6b9a-4b2b-99f0-99a7b4301719\") " pod="openshift-monitoring/node-exporter-jplhg"
Jul 28 17:35:46.413811 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:46.413792    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3573a7fb-6b9a-4b2b-99f0-99a7b4301719-node-exporter-accelerators-collector-config\") pod \"node-exporter-jplhg\" (UID: \"3573a7fb-6b9a-4b2b-99f0-99a7b4301719\") " pod="openshift-monitoring/node-exporter-jplhg"
Jul 28 17:35:46.484341 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:46.484313    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:46.514465 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:46.514422    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3573a7fb-6b9a-4b2b-99f0-99a7b4301719-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jplhg\" (UID: \"3573a7fb-6b9a-4b2b-99f0-99a7b4301719\") " pod="openshift-monitoring/node-exporter-jplhg"
Jul 28 17:35:46.514572 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:46.514473    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3573a7fb-6b9a-4b2b-99f0-99a7b4301719-sys\") pod \"node-exporter-jplhg\" (UID: \"3573a7fb-6b9a-4b2b-99f0-99a7b4301719\") " pod="openshift-monitoring/node-exporter-jplhg"
Jul 28 17:35:46.514572 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:46.514499    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkmpk\" (UniqueName: \"kubernetes.io/projected/3573a7fb-6b9a-4b2b-99f0-99a7b4301719-kube-api-access-nkmpk\") pod \"node-exporter-jplhg\" (UID: \"3573a7fb-6b9a-4b2b-99f0-99a7b4301719\") " pod="openshift-monitoring/node-exporter-jplhg"
Jul 28 17:35:46.514572 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:46.514523    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3573a7fb-6b9a-4b2b-99f0-99a7b4301719-root\") pod \"node-exporter-jplhg\" (UID: \"3573a7fb-6b9a-4b2b-99f0-99a7b4301719\") " pod="openshift-monitoring/node-exporter-jplhg"
Jul 28 17:35:46.514572 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:46.514541    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3573a7fb-6b9a-4b2b-99f0-99a7b4301719-node-exporter-tls\") pod \"node-exporter-jplhg\" (UID: \"3573a7fb-6b9a-4b2b-99f0-99a7b4301719\") " pod="openshift-monitoring/node-exporter-jplhg"
Jul 28 17:35:46.514773 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:46.514600    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3573a7fb-6b9a-4b2b-99f0-99a7b4301719-sys\") pod \"node-exporter-jplhg\" (UID: \"3573a7fb-6b9a-4b2b-99f0-99a7b4301719\") " pod="openshift-monitoring/node-exporter-jplhg"
Jul 28 17:35:46.514773 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:35:46.514611    2563 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 28 17:35:46.514773 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:46.514618    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3573a7fb-6b9a-4b2b-99f0-99a7b4301719-root\") pod \"node-exporter-jplhg\" (UID: \"3573a7fb-6b9a-4b2b-99f0-99a7b4301719\") " pod="openshift-monitoring/node-exporter-jplhg"
Jul 28 17:35:46.514773 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:46.514624    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3573a7fb-6b9a-4b2b-99f0-99a7b4301719-node-exporter-accelerators-collector-config\") pod \"node-exporter-jplhg\" (UID: \"3573a7fb-6b9a-4b2b-99f0-99a7b4301719\") " pod="openshift-monitoring/node-exporter-jplhg"
Jul 28 17:35:46.514773 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:35:46.514683    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3573a7fb-6b9a-4b2b-99f0-99a7b4301719-node-exporter-tls podName:3573a7fb-6b9a-4b2b-99f0-99a7b4301719 nodeName:}" failed. No retries permitted until 2026-07-28 17:35:47.014663676 +0000 UTC m=+85.675820023 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/3573a7fb-6b9a-4b2b-99f0-99a7b4301719-node-exporter-tls") pod "node-exporter-jplhg" (UID: "3573a7fb-6b9a-4b2b-99f0-99a7b4301719") : secret "node-exporter-tls" not found
Jul 28 17:35:46.514773 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:46.514749    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3573a7fb-6b9a-4b2b-99f0-99a7b4301719-node-exporter-textfile\") pod \"node-exporter-jplhg\" (UID: \"3573a7fb-6b9a-4b2b-99f0-99a7b4301719\") " pod="openshift-monitoring/node-exporter-jplhg"
Jul 28 17:35:46.514773 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:46.514771    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3573a7fb-6b9a-4b2b-99f0-99a7b4301719-metrics-client-ca\") pod \"node-exporter-jplhg\" (UID: \"3573a7fb-6b9a-4b2b-99f0-99a7b4301719\") " pod="openshift-monitoring/node-exporter-jplhg"
Jul 28 17:35:46.515039 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:46.514796    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3573a7fb-6b9a-4b2b-99f0-99a7b4301719-node-exporter-wtmp\") pod \"node-exporter-jplhg\" (UID: \"3573a7fb-6b9a-4b2b-99f0-99a7b4301719\") " pod="openshift-monitoring/node-exporter-jplhg"
Jul 28 17:35:46.515039 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:46.514950    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3573a7fb-6b9a-4b2b-99f0-99a7b4301719-node-exporter-wtmp\") pod \"node-exporter-jplhg\" (UID: \"3573a7fb-6b9a-4b2b-99f0-99a7b4301719\") " pod="openshift-monitoring/node-exporter-jplhg"
Jul 28 17:35:46.515112 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:46.515087    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3573a7fb-6b9a-4b2b-99f0-99a7b4301719-node-exporter-textfile\") pod \"node-exporter-jplhg\" (UID: \"3573a7fb-6b9a-4b2b-99f0-99a7b4301719\") " pod="openshift-monitoring/node-exporter-jplhg"
Jul 28 17:35:46.515187 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:46.515170    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3573a7fb-6b9a-4b2b-99f0-99a7b4301719-node-exporter-accelerators-collector-config\") pod \"node-exporter-jplhg\" (UID: \"3573a7fb-6b9a-4b2b-99f0-99a7b4301719\") " pod="openshift-monitoring/node-exporter-jplhg"
Jul 28 17:35:46.515256 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:46.515241    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3573a7fb-6b9a-4b2b-99f0-99a7b4301719-metrics-client-ca\") pod \"node-exporter-jplhg\" (UID: \"3573a7fb-6b9a-4b2b-99f0-99a7b4301719\") " pod="openshift-monitoring/node-exporter-jplhg"
Jul 28 17:35:46.516754 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:46.516735    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3573a7fb-6b9a-4b2b-99f0-99a7b4301719-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jplhg\" (UID: \"3573a7fb-6b9a-4b2b-99f0-99a7b4301719\") " pod="openshift-monitoring/node-exporter-jplhg"
Jul 28 17:35:46.523116 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:46.523095    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkmpk\" (UniqueName: \"kubernetes.io/projected/3573a7fb-6b9a-4b2b-99f0-99a7b4301719-kube-api-access-nkmpk\") pod \"node-exporter-jplhg\" (UID: \"3573a7fb-6b9a-4b2b-99f0-99a7b4301719\") " pod="openshift-monitoring/node-exporter-jplhg"
Jul 28 17:35:46.584445 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:46.584414    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:46.681358 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:46.681327    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:46.888594 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:46.888521    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-75cf897b8-llxcl"
Jul 28 17:35:47.019699 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:47.019664    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3573a7fb-6b9a-4b2b-99f0-99a7b4301719-node-exporter-tls\") pod \"node-exporter-jplhg\" (UID: \"3573a7fb-6b9a-4b2b-99f0-99a7b4301719\") " pod="openshift-monitoring/node-exporter-jplhg"
Jul 28 17:35:47.021950 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:47.021926    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3573a7fb-6b9a-4b2b-99f0-99a7b4301719-node-exporter-tls\") pod \"node-exporter-jplhg\" (UID: \"3573a7fb-6b9a-4b2b-99f0-99a7b4301719\") " pod="openshift-monitoring/node-exporter-jplhg"
Jul 28 17:35:47.079918 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:47.079884    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:47.219277 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:47.219204    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jplhg"
Jul 28 17:35:47.230680 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:35:47.230639    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3573a7fb_6b9a_4b2b_99f0_99a7b4301719.slice/crio-346d0b34af5ef95ed02b8b11a8323106f2a74d1fe8a03e88cd5280a49aff63f5 WatchSource:0}: Error finding container 346d0b34af5ef95ed02b8b11a8323106f2a74d1fe8a03e88cd5280a49aff63f5: Status 404 returned error can't find the container with id 346d0b34af5ef95ed02b8b11a8323106f2a74d1fe8a03e88cd5280a49aff63f5
Jul 28 17:35:47.230851 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:47.230827    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:47.232985 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:47.232957    2563 patch_prober.go:28] interesting pod/image-registry-6b87dbb6b9-n25x2 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 28 17:35:47.233104 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:47.232997    2563 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6b87dbb6b9-n25x2" podUID="91955790-de58-4f37-b897-77c442ec1a64" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 17:35:47.328697 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:47.328661    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:47.409761 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:47.409729    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:48.231962 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:48.231938    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jplhg" event={"ID":"3573a7fb-6b9a-4b2b-99f0-99a7b4301719","Type":"ContainerStarted","Data":"346d0b34af5ef95ed02b8b11a8323106f2a74d1fe8a03e88cd5280a49aff63f5"}
Jul 28 17:35:48.769587 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:48.769553    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:48.869095 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:48.869066    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:49.139029 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:49.139005    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6b87dbb6b9-n25x2"
Jul 28 17:35:49.208693 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:49.208669    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-psd2g"
Jul 28 17:35:49.236847 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:49.236801    2563 generic.go:356] "Generic (PLEG): container finished" podID="3573a7fb-6b9a-4b2b-99f0-99a7b4301719" containerID="227da0d6f8a56903a88e6edd2673be498c957a25947521cfd3a89afd3f085f29" exitCode=0
Jul 28 17:35:49.237008 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:49.236929    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jplhg" event={"ID":"3573a7fb-6b9a-4b2b-99f0-99a7b4301719","Type":"ContainerDied","Data":"227da0d6f8a56903a88e6edd2673be498c957a25947521cfd3a89afd3f085f29"}
Jul 28 17:35:49.337294 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:49.337212    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:49.437211 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:49.437148    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:49.496922 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:49.496896    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:49.610858 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:49.610831    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:50.243060 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:50.243030    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jplhg" event={"ID":"3573a7fb-6b9a-4b2b-99f0-99a7b4301719","Type":"ContainerStarted","Data":"44d889f32576cce0fe47b515dd47d69c72c71f538f22f1bf2b7c4eaedaad0c7b"}
Jul 28 17:35:50.243501 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:50.243075    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jplhg" event={"ID":"3573a7fb-6b9a-4b2b-99f0-99a7b4301719","Type":"ContainerStarted","Data":"1d5ab57bd8da19bc79eeaf5c9c56ad38ba5d55c2ae93d9187d71b6311abfbc36"}
Jul 28 17:35:50.265991 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:50.265942    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-jplhg" podStartSLOduration=3.245687363 podStartE2EDuration="4.265926319s" totalImagesPullingTime="958.23496ms" totalInitContainerRuntime="62.003996ms" isStatefulPod=true podCreationTimestamp="2026-07-28 17:35:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:35:50.263959827 +0000 UTC m=+88.925116183" watchObservedRunningTime="2026-07-28 17:35:50.265926319 +0000 UTC m=+88.927082673"
Jul 28 17:35:50.968534 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:50.968501    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:51.070807 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:51.070783    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:51.174920 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:51.174893    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:51.275232 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:51.275152    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:51.901904 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:51.901865    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-75cf897b8-llxcl" podUID="23a671ef-805b-4481-8950-689b84637ec5" containerName="registry" containerID="cri-o://34078c2ff20bb00cdfbbb982c68a3e8d8dd78ddcf65d0610129c9b3198c42a0a" gracePeriod=30
Jul 28 17:35:52.138757 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:52.138736    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75cf897b8-llxcl"
Jul 28 17:35:52.255033 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:52.254945    2563 generic.go:356] "Generic (PLEG): container finished" podID="23a671ef-805b-4481-8950-689b84637ec5" containerID="34078c2ff20bb00cdfbbb982c68a3e8d8dd78ddcf65d0610129c9b3198c42a0a" exitCode=0
Jul 28 17:35:52.255033 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:52.254994    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75cf897b8-llxcl"
Jul 28 17:35:52.255033 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:52.255011    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75cf897b8-llxcl" event={"ID":"23a671ef-805b-4481-8950-689b84637ec5","Type":"ContainerDied","Data":"34078c2ff20bb00cdfbbb982c68a3e8d8dd78ddcf65d0610129c9b3198c42a0a"}
Jul 28 17:35:52.255230 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:52.255074    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75cf897b8-llxcl" event={"ID":"23a671ef-805b-4481-8950-689b84637ec5","Type":"ContainerDied","Data":"13f9cc0c5e48d62087412a887662b77aa6aab2c381efbbaadcde86954c962a03"}
Jul 28 17:35:52.255685 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:52.255672    2563 scope.go:117] "RemoveContainer" containerID="34078c2ff20bb00cdfbbb982c68a3e8d8dd78ddcf65d0610129c9b3198c42a0a"
Jul 28 17:35:52.257858 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:52.257839    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/23a671ef-805b-4481-8950-689b84637ec5-registry-tls\") pod \"23a671ef-805b-4481-8950-689b84637ec5\" (UID: \"23a671ef-805b-4481-8950-689b84637ec5\") "
Jul 28 17:35:52.257970 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:52.257876    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/23a671ef-805b-4481-8950-689b84637ec5-trusted-ca\") pod \"23a671ef-805b-4481-8950-689b84637ec5\" (UID: \"23a671ef-805b-4481-8950-689b84637ec5\") "
Jul 28 17:35:52.257970 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:52.257896    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/23a671ef-805b-4481-8950-689b84637ec5-image-registry-private-configuration\") pod \"23a671ef-805b-4481-8950-689b84637ec5\" (UID: \"23a671ef-805b-4481-8950-689b84637ec5\") "
Jul 28 17:35:52.257970 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:52.257924    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/23a671ef-805b-4481-8950-689b84637ec5-bound-sa-token\") pod \"23a671ef-805b-4481-8950-689b84637ec5\" (UID: \"23a671ef-805b-4481-8950-689b84637ec5\") "
Jul 28 17:35:52.257970 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:52.257963    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/23a671ef-805b-4481-8950-689b84637ec5-ca-trust-extracted\") pod \"23a671ef-805b-4481-8950-689b84637ec5\" (UID: \"23a671ef-805b-4481-8950-689b84637ec5\") "
Jul 28 17:35:52.258173 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:52.257985    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/23a671ef-805b-4481-8950-689b84637ec5-registry-certificates\") pod \"23a671ef-805b-4481-8950-689b84637ec5\" (UID: \"23a671ef-805b-4481-8950-689b84637ec5\") "
Jul 28 17:35:52.258173 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:52.258007    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/23a671ef-805b-4481-8950-689b84637ec5-installation-pull-secrets\") pod \"23a671ef-805b-4481-8950-689b84637ec5\" (UID: \"23a671ef-805b-4481-8950-689b84637ec5\") "
Jul 28 17:35:52.258173 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:52.258036    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v4qb6\" (UniqueName: \"kubernetes.io/projected/23a671ef-805b-4481-8950-689b84637ec5-kube-api-access-v4qb6\") pod \"23a671ef-805b-4481-8950-689b84637ec5\" (UID: \"23a671ef-805b-4481-8950-689b84637ec5\") "
Jul 28 17:35:52.258319 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:52.258282    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23a671ef-805b-4481-8950-689b84637ec5-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "23a671ef-805b-4481-8950-689b84637ec5" (UID: "23a671ef-805b-4481-8950-689b84637ec5"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:35:52.258568 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:52.258539    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23a671ef-805b-4481-8950-689b84637ec5-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "23a671ef-805b-4481-8950-689b84637ec5" (UID: "23a671ef-805b-4481-8950-689b84637ec5"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:35:52.260863 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:52.260814    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23a671ef-805b-4481-8950-689b84637ec5-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "23a671ef-805b-4481-8950-689b84637ec5" (UID: "23a671ef-805b-4481-8950-689b84637ec5"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:35:52.260967 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:52.260902    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23a671ef-805b-4481-8950-689b84637ec5-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "23a671ef-805b-4481-8950-689b84637ec5" (UID: "23a671ef-805b-4481-8950-689b84637ec5"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:35:52.261135 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:52.261104    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23a671ef-805b-4481-8950-689b84637ec5-kube-api-access-v4qb6" (OuterVolumeSpecName: "kube-api-access-v4qb6") pod "23a671ef-805b-4481-8950-689b84637ec5" (UID: "23a671ef-805b-4481-8950-689b84637ec5"). InnerVolumeSpecName "kube-api-access-v4qb6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:35:52.261238 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:52.261184    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23a671ef-805b-4481-8950-689b84637ec5-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "23a671ef-805b-4481-8950-689b84637ec5" (UID: "23a671ef-805b-4481-8950-689b84637ec5"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 17:35:52.261356 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:52.261332    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23a671ef-805b-4481-8950-689b84637ec5-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "23a671ef-805b-4481-8950-689b84637ec5" (UID: "23a671ef-805b-4481-8950-689b84637ec5"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 17:35:52.267404 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:52.267189    2563 scope.go:117] "RemoveContainer" containerID="34078c2ff20bb00cdfbbb982c68a3e8d8dd78ddcf65d0610129c9b3198c42a0a"
Jul 28 17:35:52.267877 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:52.267717    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23a671ef-805b-4481-8950-689b84637ec5-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "23a671ef-805b-4481-8950-689b84637ec5" (UID: "23a671ef-805b-4481-8950-689b84637ec5"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 17:35:52.267877 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:35:52.267784    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34078c2ff20bb00cdfbbb982c68a3e8d8dd78ddcf65d0610129c9b3198c42a0a\": container with ID starting with 34078c2ff20bb00cdfbbb982c68a3e8d8dd78ddcf65d0610129c9b3198c42a0a not found: ID does not exist" containerID="34078c2ff20bb00cdfbbb982c68a3e8d8dd78ddcf65d0610129c9b3198c42a0a"
Jul 28 17:35:52.267877 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:52.267831    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34078c2ff20bb00cdfbbb982c68a3e8d8dd78ddcf65d0610129c9b3198c42a0a"} err="failed to get container status \"34078c2ff20bb00cdfbbb982c68a3e8d8dd78ddcf65d0610129c9b3198c42a0a\": rpc error: code = NotFound desc = could not find container \"34078c2ff20bb00cdfbbb982c68a3e8d8dd78ddcf65d0610129c9b3198c42a0a\": container with ID starting with 34078c2ff20bb00cdfbbb982c68a3e8d8dd78ddcf65d0610129c9b3198c42a0a not found: ID does not exist"
Jul 28 17:35:52.359517 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:52.359484    2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/23a671ef-805b-4481-8950-689b84637ec5-trusted-ca\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:35:52.359517 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:52.359514    2563 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/23a671ef-805b-4481-8950-689b84637ec5-image-registry-private-configuration\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:35:52.359517 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:52.359525    2563 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/23a671ef-805b-4481-8950-689b84637ec5-bound-sa-token\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:35:52.359943 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:52.359534    2563 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/23a671ef-805b-4481-8950-689b84637ec5-ca-trust-extracted\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:35:52.359943 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:52.359544    2563 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/23a671ef-805b-4481-8950-689b84637ec5-registry-certificates\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:35:52.359943 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:52.359553    2563 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/23a671ef-805b-4481-8950-689b84637ec5-installation-pull-secrets\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:35:52.359943 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:52.359561    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v4qb6\" (UniqueName: \"kubernetes.io/projected/23a671ef-805b-4481-8950-689b84637ec5-kube-api-access-v4qb6\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:35:52.359943 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:52.359570    2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/23a671ef-805b-4481-8950-689b84637ec5-registry-tls\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:35:52.576983 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:52.576955    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-75cf897b8-llxcl"]
Jul 28 17:35:52.585576 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:52.585553    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-75cf897b8-llxcl"]
Jul 28 17:35:52.689091 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:52.689060    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:53.050941 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:53.050903    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:53.150695 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:53.150670    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:53.309946 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:53.309867    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:53.387635 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:53.387607    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:53.388146 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:53.388126    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6f67bfbc78-s8xwx"]
Jul 28 17:35:53.388472 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:53.388454    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23a671ef-805b-4481-8950-689b84637ec5" containerName="registry"
Jul 28 17:35:53.388472 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:53.388473    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="23a671ef-805b-4481-8950-689b84637ec5" containerName="registry"
Jul 28 17:35:53.388618 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:53.388535    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="23a671ef-805b-4481-8950-689b84637ec5" containerName="registry"
Jul 28 17:35:53.391723 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:53.391704    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f67bfbc78-s8xwx"
Jul 28 17:35:53.394713 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:53.394695    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 28 17:35:53.395760 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:53.395744    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 28 17:35:53.395875 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:53.395764    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 28 17:35:53.396250 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:53.396225    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 28 17:35:53.396334 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:53.396307    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 28 17:35:53.399656 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:53.399639    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-zr967\""
Jul 28 17:35:53.399773 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:53.399757    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 28 17:35:53.399967 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:53.399953    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 28 17:35:53.400978 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:53.400958    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 28 17:35:53.413656 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:53.413637    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f67bfbc78-s8xwx"]
Jul 28 17:35:53.456541 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:53.456519    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:53.567861 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:53.567832    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e5d031d2-b7fc-43b2-9684-0aac70f85f98-console-oauth-config\") pod \"console-6f67bfbc78-s8xwx\" (UID: \"e5d031d2-b7fc-43b2-9684-0aac70f85f98\") " pod="openshift-console/console-6f67bfbc78-s8xwx"
Jul 28 17:35:53.568031 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:53.567867    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e5d031d2-b7fc-43b2-9684-0aac70f85f98-service-ca\") pod \"console-6f67bfbc78-s8xwx\" (UID: \"e5d031d2-b7fc-43b2-9684-0aac70f85f98\") " pod="openshift-console/console-6f67bfbc78-s8xwx"
Jul 28 17:35:53.568031 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:53.567890    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e5d031d2-b7fc-43b2-9684-0aac70f85f98-console-serving-cert\") pod \"console-6f67bfbc78-s8xwx\" (UID: \"e5d031d2-b7fc-43b2-9684-0aac70f85f98\") " pod="openshift-console/console-6f67bfbc78-s8xwx"
Jul 28 17:35:53.568031 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:53.567944    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e5d031d2-b7fc-43b2-9684-0aac70f85f98-console-config\") pod \"console-6f67bfbc78-s8xwx\" (UID: \"e5d031d2-b7fc-43b2-9684-0aac70f85f98\") " pod="openshift-console/console-6f67bfbc78-s8xwx"
Jul 28 17:35:53.568031 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:53.567979    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e5d031d2-b7fc-43b2-9684-0aac70f85f98-oauth-serving-cert\") pod \"console-6f67bfbc78-s8xwx\" (UID: \"e5d031d2-b7fc-43b2-9684-0aac70f85f98\") " pod="openshift-console/console-6f67bfbc78-s8xwx"
Jul 28 17:35:53.568248 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:53.568032    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tt7r8\" (UniqueName: \"kubernetes.io/projected/e5d031d2-b7fc-43b2-9684-0aac70f85f98-kube-api-access-tt7r8\") pod \"console-6f67bfbc78-s8xwx\" (UID: \"e5d031d2-b7fc-43b2-9684-0aac70f85f98\") " pod="openshift-console/console-6f67bfbc78-s8xwx"
Jul 28 17:35:53.568248 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:53.568080    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e5d031d2-b7fc-43b2-9684-0aac70f85f98-trusted-ca-bundle\") pod \"console-6f67bfbc78-s8xwx\" (UID: \"e5d031d2-b7fc-43b2-9684-0aac70f85f98\") " pod="openshift-console/console-6f67bfbc78-s8xwx"
Jul 28 17:35:53.669249 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:53.669223    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e5d031d2-b7fc-43b2-9684-0aac70f85f98-trusted-ca-bundle\") pod \"console-6f67bfbc78-s8xwx\" (UID: \"e5d031d2-b7fc-43b2-9684-0aac70f85f98\") " pod="openshift-console/console-6f67bfbc78-s8xwx"
Jul 28 17:35:53.669427 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:53.669269    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e5d031d2-b7fc-43b2-9684-0aac70f85f98-console-oauth-config\") pod \"console-6f67bfbc78-s8xwx\" (UID: \"e5d031d2-b7fc-43b2-9684-0aac70f85f98\") " pod="openshift-console/console-6f67bfbc78-s8xwx"
Jul 28 17:35:53.669427 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:53.669291    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e5d031d2-b7fc-43b2-9684-0aac70f85f98-service-ca\") pod \"console-6f67bfbc78-s8xwx\" (UID: \"e5d031d2-b7fc-43b2-9684-0aac70f85f98\") " pod="openshift-console/console-6f67bfbc78-s8xwx"
Jul 28 17:35:53.669427 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:53.669312    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e5d031d2-b7fc-43b2-9684-0aac70f85f98-console-serving-cert\") pod \"console-6f67bfbc78-s8xwx\" (UID: \"e5d031d2-b7fc-43b2-9684-0aac70f85f98\") " pod="openshift-console/console-6f67bfbc78-s8xwx"
Jul 28 17:35:53.669427 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:53.669330    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e5d031d2-b7fc-43b2-9684-0aac70f85f98-console-config\") pod \"console-6f67bfbc78-s8xwx\" (UID: \"e5d031d2-b7fc-43b2-9684-0aac70f85f98\") " pod="openshift-console/console-6f67bfbc78-s8xwx"
Jul 28 17:35:53.669427 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:53.669346    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e5d031d2-b7fc-43b2-9684-0aac70f85f98-oauth-serving-cert\") pod \"console-6f67bfbc78-s8xwx\" (UID: \"e5d031d2-b7fc-43b2-9684-0aac70f85f98\") " pod="openshift-console/console-6f67bfbc78-s8xwx"
Jul 28 17:35:53.669427 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:53.669377    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tt7r8\" (UniqueName: \"kubernetes.io/projected/e5d031d2-b7fc-43b2-9684-0aac70f85f98-kube-api-access-tt7r8\") pod \"console-6f67bfbc78-s8xwx\" (UID: \"e5d031d2-b7fc-43b2-9684-0aac70f85f98\") " pod="openshift-console/console-6f67bfbc78-s8xwx"
Jul 28 17:35:53.670195 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:53.670173    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e5d031d2-b7fc-43b2-9684-0aac70f85f98-trusted-ca-bundle\") pod \"console-6f67bfbc78-s8xwx\" (UID: \"e5d031d2-b7fc-43b2-9684-0aac70f85f98\") " pod="openshift-console/console-6f67bfbc78-s8xwx"
Jul 28 17:35:53.670195 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:53.670181    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e5d031d2-b7fc-43b2-9684-0aac70f85f98-console-config\") pod \"console-6f67bfbc78-s8xwx\" (UID: \"e5d031d2-b7fc-43b2-9684-0aac70f85f98\") " pod="openshift-console/console-6f67bfbc78-s8xwx"
Jul 28 17:35:53.670195 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:53.670190    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e5d031d2-b7fc-43b2-9684-0aac70f85f98-service-ca\") pod \"console-6f67bfbc78-s8xwx\" (UID: \"e5d031d2-b7fc-43b2-9684-0aac70f85f98\") " pod="openshift-console/console-6f67bfbc78-s8xwx"
Jul 28 17:35:53.670348 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:53.670300    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e5d031d2-b7fc-43b2-9684-0aac70f85f98-oauth-serving-cert\") pod \"console-6f67bfbc78-s8xwx\" (UID: \"e5d031d2-b7fc-43b2-9684-0aac70f85f98\") " pod="openshift-console/console-6f67bfbc78-s8xwx"
Jul 28 17:35:53.671759 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:53.671741    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e5d031d2-b7fc-43b2-9684-0aac70f85f98-console-serving-cert\") pod \"console-6f67bfbc78-s8xwx\" (UID: \"e5d031d2-b7fc-43b2-9684-0aac70f85f98\") " pod="openshift-console/console-6f67bfbc78-s8xwx"
Jul 28 17:35:53.671870 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:53.671854    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e5d031d2-b7fc-43b2-9684-0aac70f85f98-console-oauth-config\") pod \"console-6f67bfbc78-s8xwx\" (UID: \"e5d031d2-b7fc-43b2-9684-0aac70f85f98\") " pod="openshift-console/console-6f67bfbc78-s8xwx"
Jul 28 17:35:53.678937 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:53.678919    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tt7r8\" (UniqueName: \"kubernetes.io/projected/e5d031d2-b7fc-43b2-9684-0aac70f85f98-kube-api-access-tt7r8\") pod \"console-6f67bfbc78-s8xwx\" (UID: \"e5d031d2-b7fc-43b2-9684-0aac70f85f98\") " pod="openshift-console/console-6f67bfbc78-s8xwx"
Jul 28 17:35:53.701097 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:53.701068    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f67bfbc78-s8xwx"
Jul 28 17:35:53.830098 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:53.830018    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f67bfbc78-s8xwx"]
Jul 28 17:35:53.833574 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:35:53.833548    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5d031d2_b7fc_43b2_9684_0aac70f85f98.slice/crio-0f73365bd0ac8fbd9004b8c10a87e77cd73ff940abae2a4cfbaf75650d31d112 WatchSource:0}: Error finding container 0f73365bd0ac8fbd9004b8c10a87e77cd73ff940abae2a4cfbaf75650d31d112: Status 404 returned error can't find the container with id 0f73365bd0ac8fbd9004b8c10a87e77cd73ff940abae2a4cfbaf75650d31d112
Jul 28 17:35:53.851935 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:53.851911    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23a671ef-805b-4481-8950-689b84637ec5" path="/var/lib/kubelet/pods/23a671ef-805b-4481-8950-689b84637ec5/volumes"
Jul 28 17:35:54.266293 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:54.266219    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f67bfbc78-s8xwx" event={"ID":"e5d031d2-b7fc-43b2-9684-0aac70f85f98","Type":"ContainerStarted","Data":"0f73365bd0ac8fbd9004b8c10a87e77cd73ff940abae2a4cfbaf75650d31d112"}
Jul 28 17:35:57.279126 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:57.279047    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f67bfbc78-s8xwx" event={"ID":"e5d031d2-b7fc-43b2-9684-0aac70f85f98","Type":"ContainerStarted","Data":"0bdb61f566502974af3fcac19543cc9481bf27ab85a78f731357f72acc9c20cf"}
Jul 28 17:35:57.299677 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:57.299625    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6f67bfbc78-s8xwx" podStartSLOduration=1.167244088 podStartE2EDuration="4.299609661s" totalImagesPullingTime="3.132365573s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:35:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:35:57.298724566 +0000 UTC m=+95.959880921" watchObservedRunningTime="2026-07-28 17:35:57.299609661 +0000 UTC m=+95.960766015"
Jul 28 17:35:57.393506 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:57.393471    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:57.493672 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:57.493642    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:59.326232 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:59.326204    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:35:59.382696 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:35:59.382667    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:36:01.662126 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:36:01.662093    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:36:01.763943 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:36:01.763916    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:36:01.864481 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:36:01.864448    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:36:01.986268 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:36:01.986190    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:36:03.702107 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:36:03.702077    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6f67bfbc78-s8xwx"
Jul 28 17:36:03.702681 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:36:03.702130    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6f67bfbc78-s8xwx"
Jul 28 17:36:03.706965 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:36:03.706943    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6f67bfbc78-s8xwx"
Jul 28 17:36:04.308335 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:36:04.308307    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6f67bfbc78-s8xwx"
Jul 28 17:36:05.771410 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:36:05.771378    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:36:05.872049 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:36:05.872023    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:36:07.174719 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:36:07.174664    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:36:07.187211 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:36:07.187188    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-74vbl"
Jul 28 17:36:07.275201 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:36:07.275158    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:36:07.315242 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:36:07.315209    2563 generic.go:356] "Generic (PLEG): container finished" podID="ea3e1303-f4dc-4511-abbc-9a7c0e789164" containerID="443ade54813b098490369137f585a28f1f7c2bdcdf1f90332338040fba1ad27e" exitCode=0
Jul 28 17:36:07.315426 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:36:07.315287    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-25vvm" event={"ID":"ea3e1303-f4dc-4511-abbc-9a7c0e789164","Type":"ContainerDied","Data":"443ade54813b098490369137f585a28f1f7c2bdcdf1f90332338040fba1ad27e"}
Jul 28 17:36:07.315665 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:36:07.315649    2563 scope.go:117] "RemoveContainer" containerID="443ade54813b098490369137f585a28f1f7c2bdcdf1f90332338040fba1ad27e"
Jul 28 17:36:07.773671 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:36:07.773579    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:36:07.872474 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:36:07.872443    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:36:08.309347 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:36:08.309314    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:36:08.320752 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:36:08.320723    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-25vvm" event={"ID":"ea3e1303-f4dc-4511-abbc-9a7c0e789164","Type":"ContainerStarted","Data":"e9de7bd26b900dde92fa19182a053286fd0545ff0da54b1ea469c8aef20dce0a"}
Jul 28 17:36:08.412310 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:36:08.412279    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:36:08.474655 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:36:08.474624    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:36:13.173967 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:36:13.173931    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:36:13.229797 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:36:13.229766    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:36:23.305288 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:36:23.305254    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:36:23.373158 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:36:23.373127    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:36:27.383692 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:36:27.383604    2563 generic.go:356] "Generic (PLEG): container finished" podID="dfad001a-8bdb-474e-9c4a-cfed601c7748" containerID="af915131b3badf50622caca677b736516d40ea6df5b14c802f5fa3f331ae9cb6" exitCode=0
Jul 28 17:36:27.384173 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:36:27.383680    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-zc5p9" event={"ID":"dfad001a-8bdb-474e-9c4a-cfed601c7748","Type":"ContainerDied","Data":"af915131b3badf50622caca677b736516d40ea6df5b14c802f5fa3f331ae9cb6"}
Jul 28 17:36:27.384173 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:36:27.384039    2563 scope.go:117] "RemoveContainer" containerID="af915131b3badf50622caca677b736516d40ea6df5b14c802f5fa3f331ae9cb6"
Jul 28 17:36:28.388783 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:36:28.388746    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-zc5p9" event={"ID":"dfad001a-8bdb-474e-9c4a-cfed601c7748","Type":"ContainerStarted","Data":"25e36ccf09fe08507deeea0894f09d3c797ecf7ba497d4e473afd66f5c83f9b2"}
Jul 28 17:36:29.405088 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:36:29.405058    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:36:39.600349 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:36:39.600317    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:37:05.629715 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:05.629683    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:37:05.836963 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:05.836936    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:37:14.258010 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:14.257929    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5fcdb49c66-9292g"]
Jul 28 17:37:14.260157 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:14.260135    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fcdb49c66-9292g"
Jul 28 17:37:14.271952 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:14.271928    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5fcdb49c66-9292g"]
Jul 28 17:37:14.314052 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:14.314027    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:37:14.350162 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:14.350137    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e49548dd-b175-4765-9d32-6453b472ebf6-oauth-serving-cert\") pod \"console-5fcdb49c66-9292g\" (UID: \"e49548dd-b175-4765-9d32-6453b472ebf6\") " pod="openshift-console/console-5fcdb49c66-9292g"
Jul 28 17:37:14.350229 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:14.350167    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e49548dd-b175-4765-9d32-6453b472ebf6-trusted-ca-bundle\") pod \"console-5fcdb49c66-9292g\" (UID: \"e49548dd-b175-4765-9d32-6453b472ebf6\") " pod="openshift-console/console-5fcdb49c66-9292g"
Jul 28 17:37:14.350229 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:14.350185    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e49548dd-b175-4765-9d32-6453b472ebf6-console-oauth-config\") pod \"console-5fcdb49c66-9292g\" (UID: \"e49548dd-b175-4765-9d32-6453b472ebf6\") " pod="openshift-console/console-5fcdb49c66-9292g"
Jul 28 17:37:14.350301 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:14.350271    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e49548dd-b175-4765-9d32-6453b472ebf6-console-serving-cert\") pod \"console-5fcdb49c66-9292g\" (UID: \"e49548dd-b175-4765-9d32-6453b472ebf6\") " pod="openshift-console/console-5fcdb49c66-9292g"
Jul 28 17:37:14.350334 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:14.350298    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e49548dd-b175-4765-9d32-6453b472ebf6-console-config\") pod \"console-5fcdb49c66-9292g\" (UID: \"e49548dd-b175-4765-9d32-6453b472ebf6\") " pod="openshift-console/console-5fcdb49c66-9292g"
Jul 28 17:37:14.350334 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:14.350320    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e49548dd-b175-4765-9d32-6453b472ebf6-service-ca\") pod \"console-5fcdb49c66-9292g\" (UID: \"e49548dd-b175-4765-9d32-6453b472ebf6\") " pod="openshift-console/console-5fcdb49c66-9292g"
Jul 28 17:37:14.350394 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:14.350351    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wn5fz\" (UniqueName: \"kubernetes.io/projected/e49548dd-b175-4765-9d32-6453b472ebf6-kube-api-access-wn5fz\") pod \"console-5fcdb49c66-9292g\" (UID: \"e49548dd-b175-4765-9d32-6453b472ebf6\") " pod="openshift-console/console-5fcdb49c66-9292g"
Jul 28 17:37:14.405143 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:14.405118    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:37:14.451233 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:14.451210    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e49548dd-b175-4765-9d32-6453b472ebf6-service-ca\") pod \"console-5fcdb49c66-9292g\" (UID: \"e49548dd-b175-4765-9d32-6453b472ebf6\") " pod="openshift-console/console-5fcdb49c66-9292g"
Jul 28 17:37:14.451329 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:14.451246    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wn5fz\" (UniqueName: \"kubernetes.io/projected/e49548dd-b175-4765-9d32-6453b472ebf6-kube-api-access-wn5fz\") pod \"console-5fcdb49c66-9292g\" (UID: \"e49548dd-b175-4765-9d32-6453b472ebf6\") " pod="openshift-console/console-5fcdb49c66-9292g"
Jul 28 17:37:14.451329 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:14.451274    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e49548dd-b175-4765-9d32-6453b472ebf6-oauth-serving-cert\") pod \"console-5fcdb49c66-9292g\" (UID: \"e49548dd-b175-4765-9d32-6453b472ebf6\") " pod="openshift-console/console-5fcdb49c66-9292g"
Jul 28 17:37:14.451329 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:14.451291    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e49548dd-b175-4765-9d32-6453b472ebf6-trusted-ca-bundle\") pod \"console-5fcdb49c66-9292g\" (UID: \"e49548dd-b175-4765-9d32-6453b472ebf6\") " pod="openshift-console/console-5fcdb49c66-9292g"
Jul 28 17:37:14.451329 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:14.451307    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e49548dd-b175-4765-9d32-6453b472ebf6-console-oauth-config\") pod \"console-5fcdb49c66-9292g\" (UID: \"e49548dd-b175-4765-9d32-6453b472ebf6\") " pod="openshift-console/console-5fcdb49c66-9292g"
Jul 28 17:37:14.451562 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:14.451338    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e49548dd-b175-4765-9d32-6453b472ebf6-console-serving-cert\") pod \"console-5fcdb49c66-9292g\" (UID: \"e49548dd-b175-4765-9d32-6453b472ebf6\") " pod="openshift-console/console-5fcdb49c66-9292g"
Jul 28 17:37:14.451562 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:14.451379    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e49548dd-b175-4765-9d32-6453b472ebf6-console-config\") pod \"console-5fcdb49c66-9292g\" (UID: \"e49548dd-b175-4765-9d32-6453b472ebf6\") " pod="openshift-console/console-5fcdb49c66-9292g"
Jul 28 17:37:14.452014 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:14.451985    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e49548dd-b175-4765-9d32-6453b472ebf6-service-ca\") pod \"console-5fcdb49c66-9292g\" (UID: \"e49548dd-b175-4765-9d32-6453b472ebf6\") " pod="openshift-console/console-5fcdb49c66-9292g"
Jul 28 17:37:14.452122 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:14.452062    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e49548dd-b175-4765-9d32-6453b472ebf6-oauth-serving-cert\") pod \"console-5fcdb49c66-9292g\" (UID: \"e49548dd-b175-4765-9d32-6453b472ebf6\") " pod="openshift-console/console-5fcdb49c66-9292g"
Jul 28 17:37:14.452122 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:14.452094    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e49548dd-b175-4765-9d32-6453b472ebf6-console-config\") pod \"console-5fcdb49c66-9292g\" (UID: \"e49548dd-b175-4765-9d32-6453b472ebf6\") " pod="openshift-console/console-5fcdb49c66-9292g"
Jul 28 17:37:14.452247 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:14.452227    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e49548dd-b175-4765-9d32-6453b472ebf6-trusted-ca-bundle\") pod \"console-5fcdb49c66-9292g\" (UID: \"e49548dd-b175-4765-9d32-6453b472ebf6\") " pod="openshift-console/console-5fcdb49c66-9292g"
Jul 28 17:37:14.453843 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:14.453825    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e49548dd-b175-4765-9d32-6453b472ebf6-console-oauth-config\") pod \"console-5fcdb49c66-9292g\" (UID: \"e49548dd-b175-4765-9d32-6453b472ebf6\") " pod="openshift-console/console-5fcdb49c66-9292g"
Jul 28 17:37:14.453933 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:14.453914    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e49548dd-b175-4765-9d32-6453b472ebf6-console-serving-cert\") pod \"console-5fcdb49c66-9292g\" (UID: \"e49548dd-b175-4765-9d32-6453b472ebf6\") " pod="openshift-console/console-5fcdb49c66-9292g"
Jul 28 17:37:14.460881 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:14.460857    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wn5fz\" (UniqueName: \"kubernetes.io/projected/e49548dd-b175-4765-9d32-6453b472ebf6-kube-api-access-wn5fz\") pod \"console-5fcdb49c66-9292g\" (UID: \"e49548dd-b175-4765-9d32-6453b472ebf6\") " pod="openshift-console/console-5fcdb49c66-9292g"
Jul 28 17:37:14.569406 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:14.569383    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fcdb49c66-9292g"
Jul 28 17:37:14.692456 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:14.692405    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5fcdb49c66-9292g"]
Jul 28 17:37:14.695485 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:37:14.695459    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode49548dd_b175_4765_9d32_6453b472ebf6.slice/crio-b86451134ca8eb30b7a0312a3130f8e7078fcdc8637b2eafb389751d4c7a6c75 WatchSource:0}: Error finding container b86451134ca8eb30b7a0312a3130f8e7078fcdc8637b2eafb389751d4c7a6c75: Status 404 returned error can't find the container with id b86451134ca8eb30b7a0312a3130f8e7078fcdc8637b2eafb389751d4c7a6c75
Jul 28 17:37:15.550316 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:15.550274    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fcdb49c66-9292g" event={"ID":"e49548dd-b175-4765-9d32-6453b472ebf6","Type":"ContainerStarted","Data":"9f3b7253e2085969560b59749a00871f76c2d5ead114626142def9f68a6e7d00"}
Jul 28 17:37:15.550740 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:15.550327    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fcdb49c66-9292g" event={"ID":"e49548dd-b175-4765-9d32-6453b472ebf6","Type":"ContainerStarted","Data":"b86451134ca8eb30b7a0312a3130f8e7078fcdc8637b2eafb389751d4c7a6c75"}
Jul 28 17:37:15.569501 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:15.569426    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5fcdb49c66-9292g" podStartSLOduration=1.569408533 podStartE2EDuration="1.569408533s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:37:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:37:15.56911037 +0000 UTC m=+174.230266724" watchObservedRunningTime="2026-07-28 17:37:15.569408533 +0000 UTC m=+174.230564889"
Jul 28 17:37:24.570004 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:24.569960    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5fcdb49c66-9292g"
Jul 28 17:37:24.570513 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:24.570035    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5fcdb49c66-9292g"
Jul 28 17:37:24.574714 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:24.574688    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5fcdb49c66-9292g"
Jul 28 17:37:24.589331 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:24.589301    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5fcdb49c66-9292g"
Jul 28 17:37:24.642626 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:24.642598    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f67bfbc78-s8xwx"]
Jul 28 17:37:24.759501 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:24.759473    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:37:24.857402 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:24.857324    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:37:24.919008 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:24.918979    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:37:49.662272 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:49.662212    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6f67bfbc78-s8xwx" podUID="e5d031d2-b7fc-43b2-9684-0aac70f85f98" containerName="console" containerID="cri-o://0bdb61f566502974af3fcac19543cc9481bf27ab85a78f731357f72acc9c20cf" gracePeriod=15
Jul 28 17:37:49.893227 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:49.893206    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f67bfbc78-s8xwx"
Jul 28 17:37:49.907414 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:49.907387    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tt7r8\" (UniqueName: \"kubernetes.io/projected/e5d031d2-b7fc-43b2-9684-0aac70f85f98-kube-api-access-tt7r8\") pod \"e5d031d2-b7fc-43b2-9684-0aac70f85f98\" (UID: \"e5d031d2-b7fc-43b2-9684-0aac70f85f98\") "
Jul 28 17:37:49.907526 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:49.907423    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e5d031d2-b7fc-43b2-9684-0aac70f85f98-console-oauth-config\") pod \"e5d031d2-b7fc-43b2-9684-0aac70f85f98\" (UID: \"e5d031d2-b7fc-43b2-9684-0aac70f85f98\") "
Jul 28 17:37:49.907526 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:49.907472    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e5d031d2-b7fc-43b2-9684-0aac70f85f98-console-serving-cert\") pod \"e5d031d2-b7fc-43b2-9684-0aac70f85f98\" (UID: \"e5d031d2-b7fc-43b2-9684-0aac70f85f98\") "
Jul 28 17:37:49.907526 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:49.907498    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e5d031d2-b7fc-43b2-9684-0aac70f85f98-trusted-ca-bundle\") pod \"e5d031d2-b7fc-43b2-9684-0aac70f85f98\" (UID: \"e5d031d2-b7fc-43b2-9684-0aac70f85f98\") "
Jul 28 17:37:49.907678 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:49.907526    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e5d031d2-b7fc-43b2-9684-0aac70f85f98-service-ca\") pod \"e5d031d2-b7fc-43b2-9684-0aac70f85f98\" (UID: \"e5d031d2-b7fc-43b2-9684-0aac70f85f98\") "
Jul 28 17:37:49.907678 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:49.907551    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e5d031d2-b7fc-43b2-9684-0aac70f85f98-console-config\") pod \"e5d031d2-b7fc-43b2-9684-0aac70f85f98\" (UID: \"e5d031d2-b7fc-43b2-9684-0aac70f85f98\") "
Jul 28 17:37:49.907678 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:49.907582    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e5d031d2-b7fc-43b2-9684-0aac70f85f98-oauth-serving-cert\") pod \"e5d031d2-b7fc-43b2-9684-0aac70f85f98\" (UID: \"e5d031d2-b7fc-43b2-9684-0aac70f85f98\") "
Jul 28 17:37:49.908134 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:49.908071    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5d031d2-b7fc-43b2-9684-0aac70f85f98-console-config" (OuterVolumeSpecName: "console-config") pod "e5d031d2-b7fc-43b2-9684-0aac70f85f98" (UID: "e5d031d2-b7fc-43b2-9684-0aac70f85f98"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:37:49.908134 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:49.908099    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5d031d2-b7fc-43b2-9684-0aac70f85f98-service-ca" (OuterVolumeSpecName: "service-ca") pod "e5d031d2-b7fc-43b2-9684-0aac70f85f98" (UID: "e5d031d2-b7fc-43b2-9684-0aac70f85f98"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:37:49.908260 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:49.908174    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5d031d2-b7fc-43b2-9684-0aac70f85f98-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e5d031d2-b7fc-43b2-9684-0aac70f85f98" (UID: "e5d031d2-b7fc-43b2-9684-0aac70f85f98"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:37:49.908332 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:49.908308    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5d031d2-b7fc-43b2-9684-0aac70f85f98-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e5d031d2-b7fc-43b2-9684-0aac70f85f98" (UID: "e5d031d2-b7fc-43b2-9684-0aac70f85f98"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:37:49.909710 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:49.909688    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5d031d2-b7fc-43b2-9684-0aac70f85f98-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e5d031d2-b7fc-43b2-9684-0aac70f85f98" (UID: "e5d031d2-b7fc-43b2-9684-0aac70f85f98"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 17:37:49.909848 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:49.909825    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5d031d2-b7fc-43b2-9684-0aac70f85f98-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e5d031d2-b7fc-43b2-9684-0aac70f85f98" (UID: "e5d031d2-b7fc-43b2-9684-0aac70f85f98"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 17:37:49.909895 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:49.909862    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5d031d2-b7fc-43b2-9684-0aac70f85f98-kube-api-access-tt7r8" (OuterVolumeSpecName: "kube-api-access-tt7r8") pod "e5d031d2-b7fc-43b2-9684-0aac70f85f98" (UID: "e5d031d2-b7fc-43b2-9684-0aac70f85f98"). InnerVolumeSpecName "kube-api-access-tt7r8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:37:50.008336 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:50.008310    2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e5d031d2-b7fc-43b2-9684-0aac70f85f98-service-ca\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:37:50.008336 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:50.008336    2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e5d031d2-b7fc-43b2-9684-0aac70f85f98-console-config\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:37:50.008336 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:50.008349    2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e5d031d2-b7fc-43b2-9684-0aac70f85f98-oauth-serving-cert\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:37:50.008563 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:50.008363    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tt7r8\" (UniqueName: \"kubernetes.io/projected/e5d031d2-b7fc-43b2-9684-0aac70f85f98-kube-api-access-tt7r8\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:37:50.008563 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:50.008375    2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e5d031d2-b7fc-43b2-9684-0aac70f85f98-console-oauth-config\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:37:50.008563 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:50.008386    2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e5d031d2-b7fc-43b2-9684-0aac70f85f98-console-serving-cert\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:37:50.008563 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:50.008398    2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e5d031d2-b7fc-43b2-9684-0aac70f85f98-trusted-ca-bundle\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:37:50.676211 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:50.676177    2563 generic.go:356] "Generic (PLEG): container finished" podID="e5d031d2-b7fc-43b2-9684-0aac70f85f98" containerID="0bdb61f566502974af3fcac19543cc9481bf27ab85a78f731357f72acc9c20cf" exitCode=0
Jul 28 17:37:50.676706 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:50.676219    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f67bfbc78-s8xwx" event={"ID":"e5d031d2-b7fc-43b2-9684-0aac70f85f98","Type":"ContainerDied","Data":"0bdb61f566502974af3fcac19543cc9481bf27ab85a78f731357f72acc9c20cf"}
Jul 28 17:37:50.676706 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:50.676230    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f67bfbc78-s8xwx"
Jul 28 17:37:50.676706 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:50.676256    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f67bfbc78-s8xwx" event={"ID":"e5d031d2-b7fc-43b2-9684-0aac70f85f98","Type":"ContainerDied","Data":"0f73365bd0ac8fbd9004b8c10a87e77cd73ff940abae2a4cfbaf75650d31d112"}
Jul 28 17:37:50.676706 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:50.676279    2563 scope.go:117] "RemoveContainer" containerID="0bdb61f566502974af3fcac19543cc9481bf27ab85a78f731357f72acc9c20cf"
Jul 28 17:37:50.684582 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:50.684567    2563 scope.go:117] "RemoveContainer" containerID="0bdb61f566502974af3fcac19543cc9481bf27ab85a78f731357f72acc9c20cf"
Jul 28 17:37:50.684818 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:37:50.684803    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0bdb61f566502974af3fcac19543cc9481bf27ab85a78f731357f72acc9c20cf\": container with ID starting with 0bdb61f566502974af3fcac19543cc9481bf27ab85a78f731357f72acc9c20cf not found: ID does not exist" containerID="0bdb61f566502974af3fcac19543cc9481bf27ab85a78f731357f72acc9c20cf"
Jul 28 17:37:50.684854 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:50.684825    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bdb61f566502974af3fcac19543cc9481bf27ab85a78f731357f72acc9c20cf"} err="failed to get container status \"0bdb61f566502974af3fcac19543cc9481bf27ab85a78f731357f72acc9c20cf\": rpc error: code = NotFound desc = could not find container \"0bdb61f566502974af3fcac19543cc9481bf27ab85a78f731357f72acc9c20cf\": container with ID starting with 0bdb61f566502974af3fcac19543cc9481bf27ab85a78f731357f72acc9c20cf not found: ID does not exist"
Jul 28 17:37:50.697964 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:50.697943    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f67bfbc78-s8xwx"]
Jul 28 17:37:50.702405 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:50.702388    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6f67bfbc78-s8xwx"]
Jul 28 17:37:51.853623 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:51.853591    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5d031d2-b7fc-43b2-9684-0aac70f85f98" path="/var/lib/kubelet/pods/e5d031d2-b7fc-43b2-9684-0aac70f85f98/volumes"
Jul 28 17:37:56.548951 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:56.548922    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:37:56.613705 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:37:56.613667    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:38:20.922697 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:38:20.922661    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:38:20.990088 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:38:20.990046    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:38:32.992741 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:38:32.992703    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:38:33.070314 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:38:33.070281    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:38:37.059910 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:38:37.059875    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-75c6f46c7d-rfnq9"]
Jul 28 17:38:37.060358 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:38:37.060164    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5d031d2-b7fc-43b2-9684-0aac70f85f98" containerName="console"
Jul 28 17:38:37.060358 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:38:37.060180    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5d031d2-b7fc-43b2-9684-0aac70f85f98" containerName="console"
Jul 28 17:38:37.060358 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:38:37.060237    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5d031d2-b7fc-43b2-9684-0aac70f85f98" containerName="console"
Jul 28 17:38:37.062769 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:38:37.062747    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75c6f46c7d-rfnq9"
Jul 28 17:38:37.079174 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:38:37.079150    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75c6f46c7d-rfnq9"]
Jul 28 17:38:37.121786 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:38:37.121755    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:38:37.149335 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:38:37.149301    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqtwd\" (UniqueName: \"kubernetes.io/projected/67825101-0dcd-4022-8526-451950973cdd-kube-api-access-fqtwd\") pod \"console-75c6f46c7d-rfnq9\" (UID: \"67825101-0dcd-4022-8526-451950973cdd\") " pod="openshift-console/console-75c6f46c7d-rfnq9"
Jul 28 17:38:37.149531 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:38:37.149347    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67825101-0dcd-4022-8526-451950973cdd-trusted-ca-bundle\") pod \"console-75c6f46c7d-rfnq9\" (UID: \"67825101-0dcd-4022-8526-451950973cdd\") " pod="openshift-console/console-75c6f46c7d-rfnq9"
Jul 28 17:38:37.149531 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:38:37.149388    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/67825101-0dcd-4022-8526-451950973cdd-console-config\") pod \"console-75c6f46c7d-rfnq9\" (UID: \"67825101-0dcd-4022-8526-451950973cdd\") " pod="openshift-console/console-75c6f46c7d-rfnq9"
Jul 28 17:38:37.149531 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:38:37.149410    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/67825101-0dcd-4022-8526-451950973cdd-service-ca\") pod \"console-75c6f46c7d-rfnq9\" (UID: \"67825101-0dcd-4022-8526-451950973cdd\") " pod="openshift-console/console-75c6f46c7d-rfnq9"
Jul 28 17:38:37.149531 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:38:37.149518    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/67825101-0dcd-4022-8526-451950973cdd-console-oauth-config\") pod \"console-75c6f46c7d-rfnq9\" (UID: \"67825101-0dcd-4022-8526-451950973cdd\") " pod="openshift-console/console-75c6f46c7d-rfnq9"
Jul 28 17:38:37.149699 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:38:37.149555    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/67825101-0dcd-4022-8526-451950973cdd-oauth-serving-cert\") pod \"console-75c6f46c7d-rfnq9\" (UID: \"67825101-0dcd-4022-8526-451950973cdd\") " pod="openshift-console/console-75c6f46c7d-rfnq9"
Jul 28 17:38:37.149699 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:38:37.149583    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/67825101-0dcd-4022-8526-451950973cdd-console-serving-cert\") pod \"console-75c6f46c7d-rfnq9\" (UID: \"67825101-0dcd-4022-8526-451950973cdd\") " pod="openshift-console/console-75c6f46c7d-rfnq9"
Jul 28 17:38:37.222493 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:38:37.222460    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:38:37.250342 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:38:37.250308    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqtwd\" (UniqueName: \"kubernetes.io/projected/67825101-0dcd-4022-8526-451950973cdd-kube-api-access-fqtwd\") pod \"console-75c6f46c7d-rfnq9\" (UID: \"67825101-0dcd-4022-8526-451950973cdd\") " pod="openshift-console/console-75c6f46c7d-rfnq9"
Jul 28 17:38:37.250342 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:38:37.250350    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67825101-0dcd-4022-8526-451950973cdd-trusted-ca-bundle\") pod \"console-75c6f46c7d-rfnq9\" (UID: \"67825101-0dcd-4022-8526-451950973cdd\") " pod="openshift-console/console-75c6f46c7d-rfnq9"
Jul 28 17:38:37.250620 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:38:37.250386    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/67825101-0dcd-4022-8526-451950973cdd-console-config\") pod \"console-75c6f46c7d-rfnq9\" (UID: \"67825101-0dcd-4022-8526-451950973cdd\") " pod="openshift-console/console-75c6f46c7d-rfnq9"
Jul 28 17:38:37.250620 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:38:37.250400    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/67825101-0dcd-4022-8526-451950973cdd-service-ca\") pod \"console-75c6f46c7d-rfnq9\" (UID: \"67825101-0dcd-4022-8526-451950973cdd\") " pod="openshift-console/console-75c6f46c7d-rfnq9"
Jul 28 17:38:37.250620 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:38:37.250426    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/67825101-0dcd-4022-8526-451950973cdd-console-oauth-config\") pod \"console-75c6f46c7d-rfnq9\" (UID: \"67825101-0dcd-4022-8526-451950973cdd\") " pod="openshift-console/console-75c6f46c7d-rfnq9"
Jul 28 17:38:37.250620 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:38:37.250471    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/67825101-0dcd-4022-8526-451950973cdd-oauth-serving-cert\") pod \"console-75c6f46c7d-rfnq9\" (UID: \"67825101-0dcd-4022-8526-451950973cdd\") " pod="openshift-console/console-75c6f46c7d-rfnq9"
Jul 28 17:38:37.250620 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:38:37.250504    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/67825101-0dcd-4022-8526-451950973cdd-console-serving-cert\") pod \"console-75c6f46c7d-rfnq9\" (UID: \"67825101-0dcd-4022-8526-451950973cdd\") " pod="openshift-console/console-75c6f46c7d-rfnq9"
Jul 28 17:38:37.251245 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:38:37.251217    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/67825101-0dcd-4022-8526-451950973cdd-console-config\") pod \"console-75c6f46c7d-rfnq9\" (UID: \"67825101-0dcd-4022-8526-451950973cdd\") " pod="openshift-console/console-75c6f46c7d-rfnq9"
Jul 28 17:38:37.251245 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:38:37.251234    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/67825101-0dcd-4022-8526-451950973cdd-oauth-serving-cert\") pod \"console-75c6f46c7d-rfnq9\" (UID: \"67825101-0dcd-4022-8526-451950973cdd\") " pod="openshift-console/console-75c6f46c7d-rfnq9"
Jul 28 17:38:37.251394 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:38:37.251217    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/67825101-0dcd-4022-8526-451950973cdd-service-ca\") pod \"console-75c6f46c7d-rfnq9\" (UID: \"67825101-0dcd-4022-8526-451950973cdd\") " pod="openshift-console/console-75c6f46c7d-rfnq9"
Jul 28 17:38:37.251394 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:38:37.251383    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67825101-0dcd-4022-8526-451950973cdd-trusted-ca-bundle\") pod \"console-75c6f46c7d-rfnq9\" (UID: \"67825101-0dcd-4022-8526-451950973cdd\") " pod="openshift-console/console-75c6f46c7d-rfnq9"
Jul 28 17:38:37.253019 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:38:37.252998    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/67825101-0dcd-4022-8526-451950973cdd-console-serving-cert\") pod \"console-75c6f46c7d-rfnq9\" (UID: \"67825101-0dcd-4022-8526-451950973cdd\") " pod="openshift-console/console-75c6f46c7d-rfnq9"
Jul 28 17:38:37.253115 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:38:37.253039    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/67825101-0dcd-4022-8526-451950973cdd-console-oauth-config\") pod \"console-75c6f46c7d-rfnq9\" (UID: \"67825101-0dcd-4022-8526-451950973cdd\") " pod="openshift-console/console-75c6f46c7d-rfnq9"
Jul 28 17:38:37.260868 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:38:37.260837    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqtwd\" (UniqueName: \"kubernetes.io/projected/67825101-0dcd-4022-8526-451950973cdd-kube-api-access-fqtwd\") pod \"console-75c6f46c7d-rfnq9\" (UID: \"67825101-0dcd-4022-8526-451950973cdd\") " pod="openshift-console/console-75c6f46c7d-rfnq9"
Jul 28 17:38:37.292398 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:38:37.292366    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:38:37.371388 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:38:37.371351    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75c6f46c7d-rfnq9"
Jul 28 17:38:37.399460 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:38:37.399413    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:38:37.500046 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:38:37.500022    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75c6f46c7d-rfnq9"]
Jul 28 17:38:37.502570 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:38:37.502543    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67825101_0dcd_4022_8526_451950973cdd.slice/crio-d6b7826e0f0f62c95b773c78183d7882cbc3d4062b223034c7ca901ca02fa47e WatchSource:0}: Error finding container d6b7826e0f0f62c95b773c78183d7882cbc3d4062b223034c7ca901ca02fa47e: Status 404 returned error can't find the container with id d6b7826e0f0f62c95b773c78183d7882cbc3d4062b223034c7ca901ca02fa47e
Jul 28 17:38:37.838015 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:38:37.837981    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75c6f46c7d-rfnq9" event={"ID":"67825101-0dcd-4022-8526-451950973cdd","Type":"ContainerStarted","Data":"b1532beeebd4542c905f48b2053baf752ce21743e8bdf534200a5f7faaa4a85a"}
Jul 28 17:38:37.838182 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:38:37.838034    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75c6f46c7d-rfnq9" event={"ID":"67825101-0dcd-4022-8526-451950973cdd","Type":"ContainerStarted","Data":"d6b7826e0f0f62c95b773c78183d7882cbc3d4062b223034c7ca901ca02fa47e"}
Jul 28 17:38:37.858859 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:38:37.858815    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-75c6f46c7d-rfnq9" podStartSLOduration=0.858799976 podStartE2EDuration="858.799976ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:38:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:38:37.85661299 +0000 UTC m=+256.517769345" watchObservedRunningTime="2026-07-28 17:38:37.858799976 +0000 UTC m=+256.519956330"
Jul 28 17:38:37.862087 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:38:37.862064    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:38:37.925959 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:38:37.925928    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:38:38.337581 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:38:38.337551    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:38:38.437974 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:38:38.437941    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:38:47.372010 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:38:47.371934    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-75c6f46c7d-rfnq9"
Jul 28 17:38:47.372010 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:38:47.372005    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-75c6f46c7d-rfnq9"
Jul 28 17:38:47.376845 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:38:47.376823    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-75c6f46c7d-rfnq9"
Jul 28 17:38:47.875986 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:38:47.875960    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-75c6f46c7d-rfnq9"
Jul 28 17:38:47.933389 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:38:47.933359    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5fcdb49c66-9292g"]
Jul 28 17:38:48.064048 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:38:48.064019    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:38:48.162474 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:38:48.162356    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:38:48.244162 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:38:48.244127    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:38:48.317865 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:38:48.317839    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:39:01.276280 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:01.276243    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:39:01.347804 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:01.347774    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:39:12.954790 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:12.954732    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5fcdb49c66-9292g" podUID="e49548dd-b175-4765-9d32-6453b472ebf6" containerName="console" containerID="cri-o://9f3b7253e2085969560b59749a00871f76c2d5ead114626142def9f68a6e7d00" gracePeriod=15
Jul 28 17:39:13.199891 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:13.199871    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fcdb49c66-9292g"
Jul 28 17:39:13.203157 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:13.203139    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wn5fz\" (UniqueName: \"kubernetes.io/projected/e49548dd-b175-4765-9d32-6453b472ebf6-kube-api-access-wn5fz\") pod \"e49548dd-b175-4765-9d32-6453b472ebf6\" (UID: \"e49548dd-b175-4765-9d32-6453b472ebf6\") "
Jul 28 17:39:13.203219 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:13.203173    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e49548dd-b175-4765-9d32-6453b472ebf6-console-serving-cert\") pod \"e49548dd-b175-4765-9d32-6453b472ebf6\" (UID: \"e49548dd-b175-4765-9d32-6453b472ebf6\") "
Jul 28 17:39:13.203219 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:13.203194    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e49548dd-b175-4765-9d32-6453b472ebf6-service-ca\") pod \"e49548dd-b175-4765-9d32-6453b472ebf6\" (UID: \"e49548dd-b175-4765-9d32-6453b472ebf6\") "
Jul 28 17:39:13.203219 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:13.203217    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e49548dd-b175-4765-9d32-6453b472ebf6-console-oauth-config\") pod \"e49548dd-b175-4765-9d32-6453b472ebf6\" (UID: \"e49548dd-b175-4765-9d32-6453b472ebf6\") "
Jul 28 17:39:13.203369 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:13.203242    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e49548dd-b175-4765-9d32-6453b472ebf6-trusted-ca-bundle\") pod \"e49548dd-b175-4765-9d32-6453b472ebf6\" (UID: \"e49548dd-b175-4765-9d32-6453b472ebf6\") "
Jul 28 17:39:13.203369 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:13.203257    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e49548dd-b175-4765-9d32-6453b472ebf6-console-config\") pod \"e49548dd-b175-4765-9d32-6453b472ebf6\" (UID: \"e49548dd-b175-4765-9d32-6453b472ebf6\") "
Jul 28 17:39:13.203369 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:13.203297    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e49548dd-b175-4765-9d32-6453b472ebf6-oauth-serving-cert\") pod \"e49548dd-b175-4765-9d32-6453b472ebf6\" (UID: \"e49548dd-b175-4765-9d32-6453b472ebf6\") "
Jul 28 17:39:13.203738 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:13.203600    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e49548dd-b175-4765-9d32-6453b472ebf6-service-ca" (OuterVolumeSpecName: "service-ca") pod "e49548dd-b175-4765-9d32-6453b472ebf6" (UID: "e49548dd-b175-4765-9d32-6453b472ebf6"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:39:13.203822 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:13.203737    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e49548dd-b175-4765-9d32-6453b472ebf6-console-config" (OuterVolumeSpecName: "console-config") pod "e49548dd-b175-4765-9d32-6453b472ebf6" (UID: "e49548dd-b175-4765-9d32-6453b472ebf6"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:39:13.203822 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:13.203762    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e49548dd-b175-4765-9d32-6453b472ebf6-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e49548dd-b175-4765-9d32-6453b472ebf6" (UID: "e49548dd-b175-4765-9d32-6453b472ebf6"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:39:13.203822 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:13.203758    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e49548dd-b175-4765-9d32-6453b472ebf6-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e49548dd-b175-4765-9d32-6453b472ebf6" (UID: "e49548dd-b175-4765-9d32-6453b472ebf6"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:39:13.205393 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:13.205343    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e49548dd-b175-4765-9d32-6453b472ebf6-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e49548dd-b175-4765-9d32-6453b472ebf6" (UID: "e49548dd-b175-4765-9d32-6453b472ebf6"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 17:39:13.205393 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:13.205342    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e49548dd-b175-4765-9d32-6453b472ebf6-kube-api-access-wn5fz" (OuterVolumeSpecName: "kube-api-access-wn5fz") pod "e49548dd-b175-4765-9d32-6453b472ebf6" (UID: "e49548dd-b175-4765-9d32-6453b472ebf6"). InnerVolumeSpecName "kube-api-access-wn5fz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:39:13.205393 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:13.205363    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e49548dd-b175-4765-9d32-6453b472ebf6-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e49548dd-b175-4765-9d32-6453b472ebf6" (UID: "e49548dd-b175-4765-9d32-6453b472ebf6"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 17:39:13.304122 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:13.304094    2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e49548dd-b175-4765-9d32-6453b472ebf6-console-oauth-config\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:39:13.304122 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:13.304116    2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e49548dd-b175-4765-9d32-6453b472ebf6-trusted-ca-bundle\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:39:13.304122 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:13.304126    2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e49548dd-b175-4765-9d32-6453b472ebf6-console-config\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:39:13.304320 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:13.304136    2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e49548dd-b175-4765-9d32-6453b472ebf6-oauth-serving-cert\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:39:13.304320 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:13.304144    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wn5fz\" (UniqueName: \"kubernetes.io/projected/e49548dd-b175-4765-9d32-6453b472ebf6-kube-api-access-wn5fz\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:39:13.304320 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:13.304152    2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e49548dd-b175-4765-9d32-6453b472ebf6-console-serving-cert\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:39:13.304320 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:13.304161    2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e49548dd-b175-4765-9d32-6453b472ebf6-service-ca\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:39:13.960240 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:13.960210    2563 generic.go:356] "Generic (PLEG): container finished" podID="e49548dd-b175-4765-9d32-6453b472ebf6" containerID="9f3b7253e2085969560b59749a00871f76c2d5ead114626142def9f68a6e7d00" exitCode=0
Jul 28 17:39:13.960601 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:13.960257    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fcdb49c66-9292g"
Jul 28 17:39:13.960601 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:13.960257    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fcdb49c66-9292g" event={"ID":"e49548dd-b175-4765-9d32-6453b472ebf6","Type":"ContainerDied","Data":"9f3b7253e2085969560b59749a00871f76c2d5ead114626142def9f68a6e7d00"}
Jul 28 17:39:13.960601 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:13.960372    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fcdb49c66-9292g" event={"ID":"e49548dd-b175-4765-9d32-6453b472ebf6","Type":"ContainerDied","Data":"b86451134ca8eb30b7a0312a3130f8e7078fcdc8637b2eafb389751d4c7a6c75"}
Jul 28 17:39:13.960601 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:13.960399    2563 scope.go:117] "RemoveContainer" containerID="9f3b7253e2085969560b59749a00871f76c2d5ead114626142def9f68a6e7d00"
Jul 28 17:39:13.968170 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:13.968153    2563 scope.go:117] "RemoveContainer" containerID="9f3b7253e2085969560b59749a00871f76c2d5ead114626142def9f68a6e7d00"
Jul 28 17:39:13.968461 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:39:13.968412    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f3b7253e2085969560b59749a00871f76c2d5ead114626142def9f68a6e7d00\": container with ID starting with 9f3b7253e2085969560b59749a00871f76c2d5ead114626142def9f68a6e7d00 not found: ID does not exist" containerID="9f3b7253e2085969560b59749a00871f76c2d5ead114626142def9f68a6e7d00"
Jul 28 17:39:13.968540 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:13.968454    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f3b7253e2085969560b59749a00871f76c2d5ead114626142def9f68a6e7d00"} err="failed to get container status \"9f3b7253e2085969560b59749a00871f76c2d5ead114626142def9f68a6e7d00\": rpc error: code = NotFound desc = could not find container \"9f3b7253e2085969560b59749a00871f76c2d5ead114626142def9f68a6e7d00\": container with ID starting with 9f3b7253e2085969560b59749a00871f76c2d5ead114626142def9f68a6e7d00 not found: ID does not exist"
Jul 28 17:39:13.978199 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:13.978179    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5fcdb49c66-9292g"]
Jul 28 17:39:13.982502 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:13.982482    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5fcdb49c66-9292g"]
Jul 28 17:39:15.851937 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:15.851905    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e49548dd-b175-4765-9d32-6453b472ebf6" path="/var/lib/kubelet/pods/e49548dd-b175-4765-9d32-6453b472ebf6/volumes"
Jul 28 17:39:21.730147 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:21.730120    2563 kubelet.go:1648] "Image garbage collection succeeded"
Jul 28 17:39:36.228214 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:36.228181    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:39:36.945153 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:36.945118    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c2flf"]
Jul 28 17:39:36.945367 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:36.945356    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e49548dd-b175-4765-9d32-6453b472ebf6" containerName="console"
Jul 28 17:39:36.945425 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:36.945368    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e49548dd-b175-4765-9d32-6453b472ebf6" containerName="console"
Jul 28 17:39:36.945425 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:36.945420    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e49548dd-b175-4765-9d32-6453b472ebf6" containerName="console"
Jul 28 17:39:36.948314 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:36.948298    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c2flf"
Jul 28 17:39:36.951132 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:36.951102    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 28 17:39:36.951246 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:36.951102    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 28 17:39:36.951246 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:36.951173    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-9q9hb\""
Jul 28 17:39:36.959279 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:36.959259    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c2flf"]
Jul 28 17:39:36.973241 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:36.973220    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bdeac6d7-7a35-4b38-91b5-fbebca22dd58-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c2flf\" (UID: \"bdeac6d7-7a35-4b38-91b5-fbebca22dd58\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c2flf"
Jul 28 17:39:36.973347 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:36.973276    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bdeac6d7-7a35-4b38-91b5-fbebca22dd58-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c2flf\" (UID: \"bdeac6d7-7a35-4b38-91b5-fbebca22dd58\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c2flf"
Jul 28 17:39:36.973347 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:36.973306    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pq8kd\" (UniqueName: \"kubernetes.io/projected/bdeac6d7-7a35-4b38-91b5-fbebca22dd58-kube-api-access-pq8kd\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c2flf\" (UID: \"bdeac6d7-7a35-4b38-91b5-fbebca22dd58\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c2flf"
Jul 28 17:39:37.074344 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:37.074309    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bdeac6d7-7a35-4b38-91b5-fbebca22dd58-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c2flf\" (UID: \"bdeac6d7-7a35-4b38-91b5-fbebca22dd58\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c2flf"
Jul 28 17:39:37.074548 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:37.074353    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pq8kd\" (UniqueName: \"kubernetes.io/projected/bdeac6d7-7a35-4b38-91b5-fbebca22dd58-kube-api-access-pq8kd\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c2flf\" (UID: \"bdeac6d7-7a35-4b38-91b5-fbebca22dd58\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c2flf"
Jul 28 17:39:37.074548 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:37.074376    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bdeac6d7-7a35-4b38-91b5-fbebca22dd58-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c2flf\" (UID: \"bdeac6d7-7a35-4b38-91b5-fbebca22dd58\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c2flf"
Jul 28 17:39:37.074744 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:37.074724    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bdeac6d7-7a35-4b38-91b5-fbebca22dd58-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c2flf\" (UID: \"bdeac6d7-7a35-4b38-91b5-fbebca22dd58\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c2flf"
Jul 28 17:39:37.074780 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:37.074733    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bdeac6d7-7a35-4b38-91b5-fbebca22dd58-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c2flf\" (UID: \"bdeac6d7-7a35-4b38-91b5-fbebca22dd58\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c2flf"
Jul 28 17:39:37.085360 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:37.085331    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pq8kd\" (UniqueName: \"kubernetes.io/projected/bdeac6d7-7a35-4b38-91b5-fbebca22dd58-kube-api-access-pq8kd\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c2flf\" (UID: \"bdeac6d7-7a35-4b38-91b5-fbebca22dd58\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c2flf"
Jul 28 17:39:37.256411 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:37.256321    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c2flf"
Jul 28 17:39:37.387798 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:37.387768    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c2flf"]
Jul 28 17:39:37.390530 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:39:37.390502    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdeac6d7_7a35_4b38_91b5_fbebca22dd58.slice/crio-56d71431a1761128a3342f8b34eb45e91f790d668e0b158126b8289183cee1ca WatchSource:0}: Error finding container 56d71431a1761128a3342f8b34eb45e91f790d668e0b158126b8289183cee1ca: Status 404 returned error can't find the container with id 56d71431a1761128a3342f8b34eb45e91f790d668e0b158126b8289183cee1ca
Jul 28 17:39:37.392609 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:37.392591    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 17:39:37.594751 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:37.594724    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:39:38.039998 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:38.039910    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c2flf" event={"ID":"bdeac6d7-7a35-4b38-91b5-fbebca22dd58","Type":"ContainerStarted","Data":"56d71431a1761128a3342f8b34eb45e91f790d668e0b158126b8289183cee1ca"}
Jul 28 17:39:44.062072 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:44.062047    2563 generic.go:356] "Generic (PLEG): container finished" podID="bdeac6d7-7a35-4b38-91b5-fbebca22dd58" containerID="432e86c3b06ecc8e97f957bd8d2cf733f38606981dcc5ab7f461dc179c72f3bb" exitCode=0
Jul 28 17:39:44.062419 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:44.062122    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c2flf" event={"ID":"bdeac6d7-7a35-4b38-91b5-fbebca22dd58","Type":"ContainerDied","Data":"432e86c3b06ecc8e97f957bd8d2cf733f38606981dcc5ab7f461dc179c72f3bb"}
Jul 28 17:39:47.075165 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:47.075129    2563 generic.go:356] "Generic (PLEG): container finished" podID="bdeac6d7-7a35-4b38-91b5-fbebca22dd58" containerID="f6276270145e23553e54ee77970ecd928a964c1885382660087d1b1527cdc242" exitCode=0
Jul 28 17:39:47.075579 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:47.075184    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c2flf" event={"ID":"bdeac6d7-7a35-4b38-91b5-fbebca22dd58","Type":"ContainerDied","Data":"f6276270145e23553e54ee77970ecd928a964c1885382660087d1b1527cdc242"}
Jul 28 17:39:54.104628 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:54.104594    2563 generic.go:356] "Generic (PLEG): container finished" podID="bdeac6d7-7a35-4b38-91b5-fbebca22dd58" containerID="eaddc82f7a003b3b37d9c473e9caaac2317d2a308396890c731a1942783b67ee" exitCode=0
Jul 28 17:39:54.105025 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:54.104665    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c2flf" event={"ID":"bdeac6d7-7a35-4b38-91b5-fbebca22dd58","Type":"ContainerDied","Data":"eaddc82f7a003b3b37d9c473e9caaac2317d2a308396890c731a1942783b67ee"}
Jul 28 17:39:55.227259 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:55.227236    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c2flf"
Jul 28 17:39:55.323088 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:55.323060    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bdeac6d7-7a35-4b38-91b5-fbebca22dd58-bundle\") pod \"bdeac6d7-7a35-4b38-91b5-fbebca22dd58\" (UID: \"bdeac6d7-7a35-4b38-91b5-fbebca22dd58\") "
Jul 28 17:39:55.323224 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:55.323120    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pq8kd\" (UniqueName: \"kubernetes.io/projected/bdeac6d7-7a35-4b38-91b5-fbebca22dd58-kube-api-access-pq8kd\") pod \"bdeac6d7-7a35-4b38-91b5-fbebca22dd58\" (UID: \"bdeac6d7-7a35-4b38-91b5-fbebca22dd58\") "
Jul 28 17:39:55.323224 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:55.323148    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bdeac6d7-7a35-4b38-91b5-fbebca22dd58-util\") pod \"bdeac6d7-7a35-4b38-91b5-fbebca22dd58\" (UID: \"bdeac6d7-7a35-4b38-91b5-fbebca22dd58\") "
Jul 28 17:39:55.323739 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:55.323709    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdeac6d7-7a35-4b38-91b5-fbebca22dd58-bundle" (OuterVolumeSpecName: "bundle") pod "bdeac6d7-7a35-4b38-91b5-fbebca22dd58" (UID: "bdeac6d7-7a35-4b38-91b5-fbebca22dd58"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 17:39:55.325218 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:55.325190    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdeac6d7-7a35-4b38-91b5-fbebca22dd58-kube-api-access-pq8kd" (OuterVolumeSpecName: "kube-api-access-pq8kd") pod "bdeac6d7-7a35-4b38-91b5-fbebca22dd58" (UID: "bdeac6d7-7a35-4b38-91b5-fbebca22dd58"). InnerVolumeSpecName "kube-api-access-pq8kd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:39:55.327268 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:55.327243    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdeac6d7-7a35-4b38-91b5-fbebca22dd58-util" (OuterVolumeSpecName: "util") pod "bdeac6d7-7a35-4b38-91b5-fbebca22dd58" (UID: "bdeac6d7-7a35-4b38-91b5-fbebca22dd58"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 17:39:55.423678 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:55.423616    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bdeac6d7-7a35-4b38-91b5-fbebca22dd58-bundle\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:39:55.423678 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:55.423638    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pq8kd\" (UniqueName: \"kubernetes.io/projected/bdeac6d7-7a35-4b38-91b5-fbebca22dd58-kube-api-access-pq8kd\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:39:55.423678 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:55.423648    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bdeac6d7-7a35-4b38-91b5-fbebca22dd58-util\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:39:56.113019 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:56.112944    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c2flf"
Jul 28 17:39:56.113152 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:56.112945    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c2flf" event={"ID":"bdeac6d7-7a35-4b38-91b5-fbebca22dd58","Type":"ContainerDied","Data":"56d71431a1761128a3342f8b34eb45e91f790d668e0b158126b8289183cee1ca"}
Jul 28 17:39:56.113152 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:56.113060    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="56d71431a1761128a3342f8b34eb45e91f790d668e0b158126b8289183cee1ca"
Jul 28 17:39:57.985729 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:57.985690    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:39:58.785700 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:39:58.785671    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:40:00.026107 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:00.026075    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-kmrw6"]
Jul 28 17:40:00.026556 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:00.026323    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bdeac6d7-7a35-4b38-91b5-fbebca22dd58" containerName="util"
Jul 28 17:40:00.026556 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:00.026333    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="bdeac6d7-7a35-4b38-91b5-fbebca22dd58" containerName="util"
Jul 28 17:40:00.026556 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:00.026353    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bdeac6d7-7a35-4b38-91b5-fbebca22dd58" containerName="extract"
Jul 28 17:40:00.026556 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:00.026358    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="bdeac6d7-7a35-4b38-91b5-fbebca22dd58" containerName="extract"
Jul 28 17:40:00.026556 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:00.026365    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bdeac6d7-7a35-4b38-91b5-fbebca22dd58" containerName="pull"
Jul 28 17:40:00.026556 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:00.026370    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="bdeac6d7-7a35-4b38-91b5-fbebca22dd58" containerName="pull"
Jul 28 17:40:00.026556 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:00.026409    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="bdeac6d7-7a35-4b38-91b5-fbebca22dd58" containerName="extract"
Jul 28 17:40:00.031554 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:00.031535    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-kmrw6"
Jul 28 17:40:00.034563 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:00.034534    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 28 17:40:00.034699 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:00.034583    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 28 17:40:00.034699 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:00.034642    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-mvv25\""
Jul 28 17:40:00.037162 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:00.037142    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-kmrw6"]
Jul 28 17:40:00.161775 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:00.161750    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ffzs\" (UniqueName: \"kubernetes.io/projected/c2f38ae3-4eb6-4225-8347-6a30123b0354-kube-api-access-6ffzs\") pod \"cert-manager-operator-controller-manager-55c444bf86-kmrw6\" (UID: \"c2f38ae3-4eb6-4225-8347-6a30123b0354\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-kmrw6"
Jul 28 17:40:00.161899 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:00.161785    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c2f38ae3-4eb6-4225-8347-6a30123b0354-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-kmrw6\" (UID: \"c2f38ae3-4eb6-4225-8347-6a30123b0354\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-kmrw6"
Jul 28 17:40:00.262208 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:00.262177    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ffzs\" (UniqueName: \"kubernetes.io/projected/c2f38ae3-4eb6-4225-8347-6a30123b0354-kube-api-access-6ffzs\") pod \"cert-manager-operator-controller-manager-55c444bf86-kmrw6\" (UID: \"c2f38ae3-4eb6-4225-8347-6a30123b0354\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-kmrw6"
Jul 28 17:40:00.262345 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:00.262219    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c2f38ae3-4eb6-4225-8347-6a30123b0354-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-kmrw6\" (UID: \"c2f38ae3-4eb6-4225-8347-6a30123b0354\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-kmrw6"
Jul 28 17:40:00.262581 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:00.262565    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c2f38ae3-4eb6-4225-8347-6a30123b0354-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-kmrw6\" (UID: \"c2f38ae3-4eb6-4225-8347-6a30123b0354\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-kmrw6"
Jul 28 17:40:00.271760 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:00.271743    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ffzs\" (UniqueName: \"kubernetes.io/projected/c2f38ae3-4eb6-4225-8347-6a30123b0354-kube-api-access-6ffzs\") pod \"cert-manager-operator-controller-manager-55c444bf86-kmrw6\" (UID: \"c2f38ae3-4eb6-4225-8347-6a30123b0354\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-kmrw6"
Jul 28 17:40:00.341199 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:00.341180    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-kmrw6"
Jul 28 17:40:00.490630 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:00.490605    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-kmrw6"]
Jul 28 17:40:00.491773 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:40:00.491750    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2f38ae3_4eb6_4225_8347_6a30123b0354.slice/crio-0e23eba027f7c8aae80048cb6eb636c88f66c0a28facd06fe3fb1ff8d7a52eec WatchSource:0}: Error finding container 0e23eba027f7c8aae80048cb6eb636c88f66c0a28facd06fe3fb1ff8d7a52eec: Status 404 returned error can't find the container with id 0e23eba027f7c8aae80048cb6eb636c88f66c0a28facd06fe3fb1ff8d7a52eec
Jul 28 17:40:01.132715 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:01.132681    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-kmrw6" event={"ID":"c2f38ae3-4eb6-4225-8347-6a30123b0354","Type":"ContainerStarted","Data":"0e23eba027f7c8aae80048cb6eb636c88f66c0a28facd06fe3fb1ff8d7a52eec"}
Jul 28 17:40:01.855547 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:01.855508    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:40:01.925848 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:01.925815    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:40:03.141973 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:03.141936    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-kmrw6" event={"ID":"c2f38ae3-4eb6-4225-8347-6a30123b0354","Type":"ContainerStarted","Data":"bcec56594bf4af5b82017d9865cd971f662067d1f7c823c362e3e246c7063a19"}
Jul 28 17:40:03.164772 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:03.164726    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-kmrw6" podStartSLOduration=1.095412399 podStartE2EDuration="3.164711108s" totalImagesPullingTime="2.069298709s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:40:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:40:03.16314925 +0000 UTC m=+341.824305604" watchObservedRunningTime="2026-07-28 17:40:03.164711108 +0000 UTC m=+341.825867463"
Jul 28 17:40:03.376182 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:03.376148    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:40:03.475601 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:03.475529    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:40:03.580490 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:03.580454    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:40:04.228724 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:04.228690    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frsjsj"]
Jul 28 17:40:04.241628 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:04.241609    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frsjsj"]
Jul 28 17:40:04.241756 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:04.241691    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frsjsj"
Jul 28 17:40:04.244469 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:04.244445    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 28 17:40:04.245577 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:04.245553    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-9q9hb\""
Jul 28 17:40:04.245687 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:04.245553    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 28 17:40:04.392120 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:04.392093    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bb31150c-2753-475b-a117-043f161ac922-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frsjsj\" (UID: \"bb31150c-2753-475b-a117-043f161ac922\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frsjsj"
Jul 28 17:40:04.392295 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:04.392128    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bb31150c-2753-475b-a117-043f161ac922-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frsjsj\" (UID: \"bb31150c-2753-475b-a117-043f161ac922\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frsjsj"
Jul 28 17:40:04.392295 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:04.392166    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nj5h9\" (UniqueName: \"kubernetes.io/projected/bb31150c-2753-475b-a117-043f161ac922-kube-api-access-nj5h9\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frsjsj\" (UID: \"bb31150c-2753-475b-a117-043f161ac922\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frsjsj"
Jul 28 17:40:04.497488 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:04.493789    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nj5h9\" (UniqueName: \"kubernetes.io/projected/bb31150c-2753-475b-a117-043f161ac922-kube-api-access-nj5h9\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frsjsj\" (UID: \"bb31150c-2753-475b-a117-043f161ac922\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frsjsj"
Jul 28 17:40:04.497488 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:04.493869    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bb31150c-2753-475b-a117-043f161ac922-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frsjsj\" (UID: \"bb31150c-2753-475b-a117-043f161ac922\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frsjsj"
Jul 28 17:40:04.497488 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:04.493914    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bb31150c-2753-475b-a117-043f161ac922-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frsjsj\" (UID: \"bb31150c-2753-475b-a117-043f161ac922\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frsjsj"
Jul 28 17:40:04.497488 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:04.494362    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bb31150c-2753-475b-a117-043f161ac922-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frsjsj\" (UID: \"bb31150c-2753-475b-a117-043f161ac922\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frsjsj"
Jul 28 17:40:04.497488 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:04.494984    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bb31150c-2753-475b-a117-043f161ac922-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frsjsj\" (UID: \"bb31150c-2753-475b-a117-043f161ac922\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frsjsj"
Jul 28 17:40:04.504605 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:04.504584    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nj5h9\" (UniqueName: \"kubernetes.io/projected/bb31150c-2753-475b-a117-043f161ac922-kube-api-access-nj5h9\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frsjsj\" (UID: \"bb31150c-2753-475b-a117-043f161ac922\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frsjsj"
Jul 28 17:40:04.551591 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:04.551567    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frsjsj"
Jul 28 17:40:04.701576 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:04.701549    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frsjsj"]
Jul 28 17:40:04.704011 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:40:04.703990    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb31150c_2753_475b_a117_043f161ac922.slice/crio-d5a4d0160ddbb986f05107450709f14fbb58c7634e15b39778d3fced03d3ab3b WatchSource:0}: Error finding container d5a4d0160ddbb986f05107450709f14fbb58c7634e15b39778d3fced03d3ab3b: Status 404 returned error can't find the container with id d5a4d0160ddbb986f05107450709f14fbb58c7634e15b39778d3fced03d3ab3b
Jul 28 17:40:05.151970 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:05.151938    2563 generic.go:356] "Generic (PLEG): container finished" podID="bb31150c-2753-475b-a117-043f161ac922" containerID="5aafffb73b3cfd23c6292edb4c4e1d8634a0df7a031061f4c01eb19d9ac7f469" exitCode=0
Jul 28 17:40:05.152101 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:05.152024    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frsjsj" event={"ID":"bb31150c-2753-475b-a117-043f161ac922","Type":"ContainerDied","Data":"5aafffb73b3cfd23c6292edb4c4e1d8634a0df7a031061f4c01eb19d9ac7f469"}
Jul 28 17:40:05.152101 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:05.152063    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frsjsj" event={"ID":"bb31150c-2753-475b-a117-043f161ac922","Type":"ContainerStarted","Data":"d5a4d0160ddbb986f05107450709f14fbb58c7634e15b39778d3fced03d3ab3b"}
Jul 28 17:40:05.797902 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:05.797803    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:40:06.213874 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:06.213841    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:40:06.233668 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:06.233639    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-r2dw4"]
Jul 28 17:40:06.256247 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:06.256225    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-r2dw4"]
Jul 28 17:40:06.256419 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:06.256306    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-r2dw4"
Jul 28 17:40:06.258905 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:06.258878    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 28 17:40:06.259011 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:06.258960    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 28 17:40:06.260141 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:06.260114    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-zwqhn\""
Jul 28 17:40:06.408330 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:06.408287    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flgcf\" (UniqueName: \"kubernetes.io/projected/8a6ede65-ca3b-4676-b662-3c3ca55a0ce6-kube-api-access-flgcf\") pod \"cert-manager-cainjector-559cdd8c87-r2dw4\" (UID: \"8a6ede65-ca3b-4676-b662-3c3ca55a0ce6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-r2dw4"
Jul 28 17:40:06.408539 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:06.408419    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8a6ede65-ca3b-4676-b662-3c3ca55a0ce6-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-r2dw4\" (UID: \"8a6ede65-ca3b-4676-b662-3c3ca55a0ce6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-r2dw4"
Jul 28 17:40:06.509283 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:06.509206    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8a6ede65-ca3b-4676-b662-3c3ca55a0ce6-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-r2dw4\" (UID: \"8a6ede65-ca3b-4676-b662-3c3ca55a0ce6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-r2dw4"
Jul 28 17:40:06.509283 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:06.509246    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flgcf\" (UniqueName: \"kubernetes.io/projected/8a6ede65-ca3b-4676-b662-3c3ca55a0ce6-kube-api-access-flgcf\") pod \"cert-manager-cainjector-559cdd8c87-r2dw4\" (UID: \"8a6ede65-ca3b-4676-b662-3c3ca55a0ce6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-r2dw4"
Jul 28 17:40:06.518204 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:06.518177    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8a6ede65-ca3b-4676-b662-3c3ca55a0ce6-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-r2dw4\" (UID: \"8a6ede65-ca3b-4676-b662-3c3ca55a0ce6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-r2dw4"
Jul 28 17:40:06.518495 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:06.518479    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-flgcf\" (UniqueName: \"kubernetes.io/projected/8a6ede65-ca3b-4676-b662-3c3ca55a0ce6-kube-api-access-flgcf\") pod \"cert-manager-cainjector-559cdd8c87-r2dw4\" (UID: \"8a6ede65-ca3b-4676-b662-3c3ca55a0ce6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-r2dw4"
Jul 28 17:40:06.566424 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:06.566402    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-r2dw4"
Jul 28 17:40:07.549915 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:07.549876    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-r2dw4"]
Jul 28 17:40:07.553429 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:40:07.553393    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8a6ede65_ca3b_4676_b662_3c3ca55a0ce6.slice/crio-800acd90e501845add68b1727aa490a03a5a4ca2afe88680f559be83373e8ccd WatchSource:0}: Error finding container 800acd90e501845add68b1727aa490a03a5a4ca2afe88680f559be83373e8ccd: Status 404 returned error can't find the container with id 800acd90e501845add68b1727aa490a03a5a4ca2afe88680f559be83373e8ccd
Jul 28 17:40:08.167805 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:08.167767    2563 generic.go:356] "Generic (PLEG): container finished" podID="bb31150c-2753-475b-a117-043f161ac922" containerID="3ae5eca3e9c0b5003b9b3e89f31a210d2c81caf8751014369c5b736abfcb201e" exitCode=0
Jul 28 17:40:08.167987 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:08.167856    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frsjsj" event={"ID":"bb31150c-2753-475b-a117-043f161ac922","Type":"ContainerDied","Data":"3ae5eca3e9c0b5003b9b3e89f31a210d2c81caf8751014369c5b736abfcb201e"}
Jul 28 17:40:08.169162 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:08.169131    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-r2dw4" event={"ID":"8a6ede65-ca3b-4676-b662-3c3ca55a0ce6","Type":"ContainerStarted","Data":"800acd90e501845add68b1727aa490a03a5a4ca2afe88680f559be83373e8ccd"}
Jul 28 17:40:08.602102 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:08.602056    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:40:08.663158 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:08.663127    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:40:09.176245 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:09.176211    2563 generic.go:356] "Generic (PLEG): container finished" podID="bb31150c-2753-475b-a117-043f161ac922" containerID="17f6cf16288a7d81f3de522aba068b2d1114dcf0fe6b03ccfac602401470fafd" exitCode=0
Jul 28 17:40:09.176414 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:09.176277    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frsjsj" event={"ID":"bb31150c-2753-475b-a117-043f161ac922","Type":"ContainerDied","Data":"17f6cf16288a7d81f3de522aba068b2d1114dcf0fe6b03ccfac602401470fafd"}
Jul 28 17:40:10.182052 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:10.182003    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-r2dw4" event={"ID":"8a6ede65-ca3b-4676-b662-3c3ca55a0ce6","Type":"ContainerStarted","Data":"e1e4ba4d82499a20de2272190c39e1cf28c0230fdeee95576ab2e508ca88c414"}
Jul 28 17:40:10.202095 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:10.202045    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-r2dw4" podStartSLOduration=1.775277982 podStartE2EDuration="4.202029636s" totalImagesPullingTime="2.426751654s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:40:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:40:10.200704191 +0000 UTC m=+348.861860545" watchObservedRunningTime="2026-07-28 17:40:10.202029636 +0000 UTC m=+348.863185991"
Jul 28 17:40:10.311222 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:10.311198    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frsjsj"
Jul 28 17:40:10.444821 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:10.444793    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nj5h9\" (UniqueName: \"kubernetes.io/projected/bb31150c-2753-475b-a117-043f161ac922-kube-api-access-nj5h9\") pod \"bb31150c-2753-475b-a117-043f161ac922\" (UID: \"bb31150c-2753-475b-a117-043f161ac922\") "
Jul 28 17:40:10.444968 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:10.444875    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bb31150c-2753-475b-a117-043f161ac922-bundle\") pod \"bb31150c-2753-475b-a117-043f161ac922\" (UID: \"bb31150c-2753-475b-a117-043f161ac922\") "
Jul 28 17:40:10.445012 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:10.444978    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bb31150c-2753-475b-a117-043f161ac922-util\") pod \"bb31150c-2753-475b-a117-043f161ac922\" (UID: \"bb31150c-2753-475b-a117-043f161ac922\") "
Jul 28 17:40:10.445207 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:10.445183    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb31150c-2753-475b-a117-043f161ac922-bundle" (OuterVolumeSpecName: "bundle") pod "bb31150c-2753-475b-a117-043f161ac922" (UID: "bb31150c-2753-475b-a117-043f161ac922"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 17:40:10.446854 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:10.446833    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb31150c-2753-475b-a117-043f161ac922-kube-api-access-nj5h9" (OuterVolumeSpecName: "kube-api-access-nj5h9") pod "bb31150c-2753-475b-a117-043f161ac922" (UID: "bb31150c-2753-475b-a117-043f161ac922"). InnerVolumeSpecName "kube-api-access-nj5h9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:40:10.449095 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:10.449073    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb31150c-2753-475b-a117-043f161ac922-util" (OuterVolumeSpecName: "util") pod "bb31150c-2753-475b-a117-043f161ac922" (UID: "bb31150c-2753-475b-a117-043f161ac922"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 17:40:10.545787 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:10.545737    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bb31150c-2753-475b-a117-043f161ac922-bundle\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:40:10.545787 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:10.545757    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bb31150c-2753-475b-a117-043f161ac922-util\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:40:10.545787 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:10.545766    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nj5h9\" (UniqueName: \"kubernetes.io/projected/bb31150c-2753-475b-a117-043f161ac922-kube-api-access-nj5h9\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:40:11.188052 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:11.188025    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frsjsj"
Jul 28 17:40:11.188527 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:11.188064    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frsjsj" event={"ID":"bb31150c-2753-475b-a117-043f161ac922","Type":"ContainerDied","Data":"d5a4d0160ddbb986f05107450709f14fbb58c7634e15b39778d3fced03d3ab3b"}
Jul 28 17:40:11.188527 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:11.188124    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d5a4d0160ddbb986f05107450709f14fbb58c7634e15b39778d3fced03d3ab3b"
Jul 28 17:40:13.002851 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:13.002819    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:40:14.632455 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:14.632406    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:40:14.731895 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:14.731864    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:40:21.297302 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:21.297272    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:40:21.397118 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:21.397090    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:40:23.019860 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:23.019808    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:40:24.504542 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:24.504506    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzgrtf"]
Jul 28 17:40:24.505001 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:24.504873    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb31150c-2753-475b-a117-043f161ac922" containerName="extract"
Jul 28 17:40:24.505001 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:24.504892    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb31150c-2753-475b-a117-043f161ac922" containerName="extract"
Jul 28 17:40:24.505001 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:24.504918    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb31150c-2753-475b-a117-043f161ac922" containerName="pull"
Jul 28 17:40:24.505001 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:24.504928    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb31150c-2753-475b-a117-043f161ac922" containerName="pull"
Jul 28 17:40:24.505001 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:24.504943    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb31150c-2753-475b-a117-043f161ac922" containerName="util"
Jul 28 17:40:24.505001 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:24.504952    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb31150c-2753-475b-a117-043f161ac922" containerName="util"
Jul 28 17:40:24.505293 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:24.505020    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb31150c-2753-475b-a117-043f161ac922" containerName="extract"
Jul 28 17:40:24.507347 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:24.507326    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzgrtf"
Jul 28 17:40:24.510016 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:24.509996    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-9q9hb\""
Jul 28 17:40:24.510142 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:24.510003    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 28 17:40:24.510142 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:24.510003    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 28 17:40:24.516056 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:24.516035    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzgrtf"]
Jul 28 17:40:24.646468 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:24.646419    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1ace9357-d6bf-40e6-b521-578b7eb81bfc-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzgrtf\" (UID: \"1ace9357-d6bf-40e6-b521-578b7eb81bfc\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzgrtf"
Jul 28 17:40:24.646631 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:24.646556    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1ace9357-d6bf-40e6-b521-578b7eb81bfc-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzgrtf\" (UID: \"1ace9357-d6bf-40e6-b521-578b7eb81bfc\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzgrtf"
Jul 28 17:40:24.646631 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:24.646593    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqkff\" (UniqueName: \"kubernetes.io/projected/1ace9357-d6bf-40e6-b521-578b7eb81bfc-kube-api-access-xqkff\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzgrtf\" (UID: \"1ace9357-d6bf-40e6-b521-578b7eb81bfc\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzgrtf"
Jul 28 17:40:24.747656 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:24.747629    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1ace9357-d6bf-40e6-b521-578b7eb81bfc-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzgrtf\" (UID: \"1ace9357-d6bf-40e6-b521-578b7eb81bfc\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzgrtf"
Jul 28 17:40:24.747656 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:24.747660    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqkff\" (UniqueName: \"kubernetes.io/projected/1ace9357-d6bf-40e6-b521-578b7eb81bfc-kube-api-access-xqkff\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzgrtf\" (UID: \"1ace9357-d6bf-40e6-b521-578b7eb81bfc\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzgrtf"
Jul 28 17:40:24.747847 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:24.747693    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1ace9357-d6bf-40e6-b521-578b7eb81bfc-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzgrtf\" (UID: \"1ace9357-d6bf-40e6-b521-578b7eb81bfc\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzgrtf"
Jul 28 17:40:24.747988 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:24.747969    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1ace9357-d6bf-40e6-b521-578b7eb81bfc-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzgrtf\" (UID: \"1ace9357-d6bf-40e6-b521-578b7eb81bfc\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzgrtf"
Jul 28 17:40:24.748044 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:24.748029    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1ace9357-d6bf-40e6-b521-578b7eb81bfc-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzgrtf\" (UID: \"1ace9357-d6bf-40e6-b521-578b7eb81bfc\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzgrtf"
Jul 28 17:40:24.756861 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:24.756812    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqkff\" (UniqueName: \"kubernetes.io/projected/1ace9357-d6bf-40e6-b521-578b7eb81bfc-kube-api-access-xqkff\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzgrtf\" (UID: \"1ace9357-d6bf-40e6-b521-578b7eb81bfc\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzgrtf"
Jul 28 17:40:24.816668 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:24.816631    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzgrtf"
Jul 28 17:40:24.941716 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:24.941575    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzgrtf"]
Jul 28 17:40:24.944188 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:40:24.944154    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ace9357_d6bf_40e6_b521_578b7eb81bfc.slice/crio-a6b2731bf4e2a416509947551f2d876ee7f0226c58b4dee37e0d91bb5fe5eaee WatchSource:0}: Error finding container a6b2731bf4e2a416509947551f2d876ee7f0226c58b4dee37e0d91bb5fe5eaee: Status 404 returned error can't find the container with id a6b2731bf4e2a416509947551f2d876ee7f0226c58b4dee37e0d91bb5fe5eaee
Jul 28 17:40:25.243743 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:25.243714    2563 generic.go:356] "Generic (PLEG): container finished" podID="1ace9357-d6bf-40e6-b521-578b7eb81bfc" containerID="a087fff3733a0df06b4bcc6bf8cd427c5044b5b37771837854e93b14c0a2c2e3" exitCode=0
Jul 28 17:40:25.243895 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:25.243778    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzgrtf" event={"ID":"1ace9357-d6bf-40e6-b521-578b7eb81bfc","Type":"ContainerDied","Data":"a087fff3733a0df06b4bcc6bf8cd427c5044b5b37771837854e93b14c0a2c2e3"}
Jul 28 17:40:25.243895 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:25.243826    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzgrtf" event={"ID":"1ace9357-d6bf-40e6-b521-578b7eb81bfc","Type":"ContainerStarted","Data":"a6b2731bf4e2a416509947551f2d876ee7f0226c58b4dee37e0d91bb5fe5eaee"}
Jul 28 17:40:26.249654 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:26.249619    2563 generic.go:356] "Generic (PLEG): container finished" podID="1ace9357-d6bf-40e6-b521-578b7eb81bfc" containerID="5dfb7a3bf446b3ee8bc63510af008ad6708b3a3c721a736c2b455f4e6d2ddaf2" exitCode=0
Jul 28 17:40:26.249654 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:26.249653    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzgrtf" event={"ID":"1ace9357-d6bf-40e6-b521-578b7eb81bfc","Type":"ContainerDied","Data":"5dfb7a3bf446b3ee8bc63510af008ad6708b3a3c721a736c2b455f4e6d2ddaf2"}
Jul 28 17:40:27.255738 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:27.255701    2563 generic.go:356] "Generic (PLEG): container finished" podID="1ace9357-d6bf-40e6-b521-578b7eb81bfc" containerID="21a890732698caa7d5732351aa129254f2ab72253b457a82aa40a61db46e028d" exitCode=0
Jul 28 17:40:27.256110 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:27.255798    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzgrtf" event={"ID":"1ace9357-d6bf-40e6-b521-578b7eb81bfc","Type":"ContainerDied","Data":"21a890732698caa7d5732351aa129254f2ab72253b457a82aa40a61db46e028d"}
Jul 28 17:40:28.381732 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:28.381710    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzgrtf"
Jul 28 17:40:28.474728 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:28.474699    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xqkff\" (UniqueName: \"kubernetes.io/projected/1ace9357-d6bf-40e6-b521-578b7eb81bfc-kube-api-access-xqkff\") pod \"1ace9357-d6bf-40e6-b521-578b7eb81bfc\" (UID: \"1ace9357-d6bf-40e6-b521-578b7eb81bfc\") "
Jul 28 17:40:28.474845 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:28.474742    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1ace9357-d6bf-40e6-b521-578b7eb81bfc-bundle\") pod \"1ace9357-d6bf-40e6-b521-578b7eb81bfc\" (UID: \"1ace9357-d6bf-40e6-b521-578b7eb81bfc\") "
Jul 28 17:40:28.474845 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:28.474785    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1ace9357-d6bf-40e6-b521-578b7eb81bfc-util\") pod \"1ace9357-d6bf-40e6-b521-578b7eb81bfc\" (UID: \"1ace9357-d6bf-40e6-b521-578b7eb81bfc\") "
Jul 28 17:40:28.475451 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:28.475408    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ace9357-d6bf-40e6-b521-578b7eb81bfc-bundle" (OuterVolumeSpecName: "bundle") pod "1ace9357-d6bf-40e6-b521-578b7eb81bfc" (UID: "1ace9357-d6bf-40e6-b521-578b7eb81bfc"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 17:40:28.476744 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:28.476719    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ace9357-d6bf-40e6-b521-578b7eb81bfc-kube-api-access-xqkff" (OuterVolumeSpecName: "kube-api-access-xqkff") pod "1ace9357-d6bf-40e6-b521-578b7eb81bfc" (UID: "1ace9357-d6bf-40e6-b521-578b7eb81bfc"). InnerVolumeSpecName "kube-api-access-xqkff". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:40:28.480235 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:28.480216    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ace9357-d6bf-40e6-b521-578b7eb81bfc-util" (OuterVolumeSpecName: "util") pod "1ace9357-d6bf-40e6-b521-578b7eb81bfc" (UID: "1ace9357-d6bf-40e6-b521-578b7eb81bfc"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 17:40:28.575213 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:28.575193    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1ace9357-d6bf-40e6-b521-578b7eb81bfc-bundle\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:40:28.575213 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:28.575211    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1ace9357-d6bf-40e6-b521-578b7eb81bfc-util\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:40:28.575324 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:28.575220    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xqkff\" (UniqueName: \"kubernetes.io/projected/1ace9357-d6bf-40e6-b521-578b7eb81bfc-kube-api-access-xqkff\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:40:29.266022 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:29.265983    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzgrtf" event={"ID":"1ace9357-d6bf-40e6-b521-578b7eb81bfc","Type":"ContainerDied","Data":"a6b2731bf4e2a416509947551f2d876ee7f0226c58b4dee37e0d91bb5fe5eaee"}
Jul 28 17:40:29.266022 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:29.266018    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzgrtf"
Jul 28 17:40:29.266246 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:29.266040    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a6b2731bf4e2a416509947551f2d876ee7f0226c58b4dee37e0d91bb5fe5eaee"
Jul 28 17:40:30.951977 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:30.951942    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:40:31.053020 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:31.052990    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:40:36.496234 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:36.496202    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:40:36.599843 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:36.599812    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:40:37.035176 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:37.035140    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:40:37.392793 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:37.392757    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:40:37.502379 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:37.502349    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:40:37.531713 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:37.531686    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5c7c5665b5-9g8wm"]
Jul 28 17:40:37.531971 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:37.531959    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ace9357-d6bf-40e6-b521-578b7eb81bfc" containerName="extract"
Jul 28 17:40:37.532014 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:37.531974    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ace9357-d6bf-40e6-b521-578b7eb81bfc" containerName="extract"
Jul 28 17:40:37.532014 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:37.531986    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ace9357-d6bf-40e6-b521-578b7eb81bfc" containerName="util"
Jul 28 17:40:37.532014 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:37.531992    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ace9357-d6bf-40e6-b521-578b7eb81bfc" containerName="util"
Jul 28 17:40:37.532014 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:37.532001    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ace9357-d6bf-40e6-b521-578b7eb81bfc" containerName="pull"
Jul 28 17:40:37.532014 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:37.532006    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ace9357-d6bf-40e6-b521-578b7eb81bfc" containerName="pull"
Jul 28 17:40:37.532155 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:37.532054    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ace9357-d6bf-40e6-b521-578b7eb81bfc" containerName="extract"
Jul 28 17:40:37.535152 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:37.535133    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5c7c5665b5-9g8wm"
Jul 28 17:40:37.537708 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:37.537685    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 28 17:40:37.537883 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:37.537725    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 28 17:40:37.537883 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:37.537785    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 28 17:40:37.538003 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:37.537988    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-djnth\""
Jul 28 17:40:37.538164 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:37.538153    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 28 17:40:37.579596 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:37.579570    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5c7c5665b5-9g8wm"]
Jul 28 17:40:37.632576 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:37.632542    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/091c1bab-94c3-43f8-92c9-6c4937a172ce-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5c7c5665b5-9g8wm\" (UID: \"091c1bab-94c3-43f8-92c9-6c4937a172ce\") " pod="opendatahub/opendatahub-operator-controller-manager-5c7c5665b5-9g8wm"
Jul 28 17:40:37.632576 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:37.632574    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/091c1bab-94c3-43f8-92c9-6c4937a172ce-manifests\") pod \"opendatahub-operator-controller-manager-5c7c5665b5-9g8wm\" (UID: \"091c1bab-94c3-43f8-92c9-6c4937a172ce\") " pod="opendatahub/opendatahub-operator-controller-manager-5c7c5665b5-9g8wm"
Jul 28 17:40:37.632789 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:37.632598    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/091c1bab-94c3-43f8-92c9-6c4937a172ce-tmp\") pod \"opendatahub-operator-controller-manager-5c7c5665b5-9g8wm\" (UID: \"091c1bab-94c3-43f8-92c9-6c4937a172ce\") " pod="opendatahub/opendatahub-operator-controller-manager-5c7c5665b5-9g8wm"
Jul 28 17:40:37.632789 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:37.632655    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/091c1bab-94c3-43f8-92c9-6c4937a172ce-webhook-cert\") pod \"opendatahub-operator-controller-manager-5c7c5665b5-9g8wm\" (UID: \"091c1bab-94c3-43f8-92c9-6c4937a172ce\") " pod="opendatahub/opendatahub-operator-controller-manager-5c7c5665b5-9g8wm"
Jul 28 17:40:37.632789 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:37.632705    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmdwj\" (UniqueName: \"kubernetes.io/projected/091c1bab-94c3-43f8-92c9-6c4937a172ce-kube-api-access-lmdwj\") pod \"opendatahub-operator-controller-manager-5c7c5665b5-9g8wm\" (UID: \"091c1bab-94c3-43f8-92c9-6c4937a172ce\") " pod="opendatahub/opendatahub-operator-controller-manager-5c7c5665b5-9g8wm"
Jul 28 17:40:37.733077 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:37.732990    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/091c1bab-94c3-43f8-92c9-6c4937a172ce-tmp\") pod \"opendatahub-operator-controller-manager-5c7c5665b5-9g8wm\" (UID: \"091c1bab-94c3-43f8-92c9-6c4937a172ce\") " pod="opendatahub/opendatahub-operator-controller-manager-5c7c5665b5-9g8wm"
Jul 28 17:40:37.733077 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:37.733029    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/091c1bab-94c3-43f8-92c9-6c4937a172ce-webhook-cert\") pod \"opendatahub-operator-controller-manager-5c7c5665b5-9g8wm\" (UID: \"091c1bab-94c3-43f8-92c9-6c4937a172ce\") " pod="opendatahub/opendatahub-operator-controller-manager-5c7c5665b5-9g8wm"
Jul 28 17:40:37.733077 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:37.733059    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmdwj\" (UniqueName: \"kubernetes.io/projected/091c1bab-94c3-43f8-92c9-6c4937a172ce-kube-api-access-lmdwj\") pod \"opendatahub-operator-controller-manager-5c7c5665b5-9g8wm\" (UID: \"091c1bab-94c3-43f8-92c9-6c4937a172ce\") " pod="opendatahub/opendatahub-operator-controller-manager-5c7c5665b5-9g8wm"
Jul 28 17:40:37.733346 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:37.733142    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/091c1bab-94c3-43f8-92c9-6c4937a172ce-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5c7c5665b5-9g8wm\" (UID: \"091c1bab-94c3-43f8-92c9-6c4937a172ce\") " pod="opendatahub/opendatahub-operator-controller-manager-5c7c5665b5-9g8wm"
Jul 28 17:40:37.733346 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:37.733213    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/091c1bab-94c3-43f8-92c9-6c4937a172ce-manifests\") pod \"opendatahub-operator-controller-manager-5c7c5665b5-9g8wm\" (UID: \"091c1bab-94c3-43f8-92c9-6c4937a172ce\") " pod="opendatahub/opendatahub-operator-controller-manager-5c7c5665b5-9g8wm"
Jul 28 17:40:37.733599 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:37.733567    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/091c1bab-94c3-43f8-92c9-6c4937a172ce-manifests\") pod \"opendatahub-operator-controller-manager-5c7c5665b5-9g8wm\" (UID: \"091c1bab-94c3-43f8-92c9-6c4937a172ce\") " pod="opendatahub/opendatahub-operator-controller-manager-5c7c5665b5-9g8wm"
Jul 28 17:40:37.735368 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:37.735340    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/091c1bab-94c3-43f8-92c9-6c4937a172ce-tmp\") pod \"opendatahub-operator-controller-manager-5c7c5665b5-9g8wm\" (UID: \"091c1bab-94c3-43f8-92c9-6c4937a172ce\") " pod="opendatahub/opendatahub-operator-controller-manager-5c7c5665b5-9g8wm"
Jul 28 17:40:37.735589 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:37.735573    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/091c1bab-94c3-43f8-92c9-6c4937a172ce-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5c7c5665b5-9g8wm\" (UID: \"091c1bab-94c3-43f8-92c9-6c4937a172ce\") " pod="opendatahub/opendatahub-operator-controller-manager-5c7c5665b5-9g8wm"
Jul 28 17:40:37.735637 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:37.735618    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/091c1bab-94c3-43f8-92c9-6c4937a172ce-webhook-cert\") pod \"opendatahub-operator-controller-manager-5c7c5665b5-9g8wm\" (UID: \"091c1bab-94c3-43f8-92c9-6c4937a172ce\") " pod="opendatahub/opendatahub-operator-controller-manager-5c7c5665b5-9g8wm"
Jul 28 17:40:37.743386 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:37.743362    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmdwj\" (UniqueName: \"kubernetes.io/projected/091c1bab-94c3-43f8-92c9-6c4937a172ce-kube-api-access-lmdwj\") pod \"opendatahub-operator-controller-manager-5c7c5665b5-9g8wm\" (UID: \"091c1bab-94c3-43f8-92c9-6c4937a172ce\") " pod="opendatahub/opendatahub-operator-controller-manager-5c7c5665b5-9g8wm"
Jul 28 17:40:37.748160 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:37.748136    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:40:37.845126 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:37.845096    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5c7c5665b5-9g8wm"
Jul 28 17:40:37.855940 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:37.855918    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:40:37.966481 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:37.966449    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:40:37.987164 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:37.987144    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5c7c5665b5-9g8wm"]
Jul 28 17:40:37.989611 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:40:37.989586    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod091c1bab_94c3_43f8_92c9_6c4937a172ce.slice/crio-d401e241bfed05aad061244dfd81faebad73c531ff50d8cfaa5d49c1f16ccef8 WatchSource:0}: Error finding container d401e241bfed05aad061244dfd81faebad73c531ff50d8cfaa5d49c1f16ccef8: Status 404 returned error can't find the container with id d401e241bfed05aad061244dfd81faebad73c531ff50d8cfaa5d49c1f16ccef8
Jul 28 17:40:38.090078 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:38.090046    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:40:38.188701 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:38.188673    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:40:38.289465 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:38.289365    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:40:38.304906 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:38.304875    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c7c5665b5-9g8wm" event={"ID":"091c1bab-94c3-43f8-92c9-6c4937a172ce","Type":"ContainerStarted","Data":"d401e241bfed05aad061244dfd81faebad73c531ff50d8cfaa5d49c1f16ccef8"}
Jul 28 17:40:38.399285 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:38.399254    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:40:41.320867 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:41.320838    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c7c5665b5-9g8wm" event={"ID":"091c1bab-94c3-43f8-92c9-6c4937a172ce","Type":"ContainerStarted","Data":"ec1b0d838d789586d47c0d3c4de2f76475b7dfed222dd4ff66c82f8dcb8aa326"}
Jul 28 17:40:42.327242 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:42.327210    2563 generic.go:356] "Generic (PLEG): container finished" podID="091c1bab-94c3-43f8-92c9-6c4937a172ce" containerID="ec1b0d838d789586d47c0d3c4de2f76475b7dfed222dd4ff66c82f8dcb8aa326" exitCode=0
Jul 28 17:40:42.327703 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:42.327256    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c7c5665b5-9g8wm" event={"ID":"091c1bab-94c3-43f8-92c9-6c4937a172ce","Type":"ContainerDied","Data":"ec1b0d838d789586d47c0d3c4de2f76475b7dfed222dd4ff66c82f8dcb8aa326"}
Jul 28 17:40:43.334033 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:43.333996    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c7c5665b5-9g8wm" event={"ID":"091c1bab-94c3-43f8-92c9-6c4937a172ce","Type":"ContainerStarted","Data":"e0476589b4c80964c33eb783ad9af79885d63378460558c3133f08e1f15c11e7"}
Jul 28 17:40:43.334401 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:43.334138    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5c7c5665b5-9g8wm"
Jul 28 17:40:43.396545 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:43.396496    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5c7c5665b5-9g8wm" podStartSLOduration=2.017636257 podStartE2EDuration="6.396481526s" totalImagesPullingTime="3.309432162s" totalInitContainerRuntime="1.069413107s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:40:37 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:40:43.394888534 +0000 UTC m=+382.056044889" watchObservedRunningTime="2026-07-28 17:40:43.396481526 +0000 UTC m=+382.057637881"
Jul 28 17:40:47.428489 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:47.428430    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:40:47.537701 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:47.537671    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:40:48.593188 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:48.593147    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:40:48.696269 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:48.696243    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:40:50.657521 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:50.657487    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:40:50.759993 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:50.759959    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:40:51.140879 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:51.140849    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:40:51.242650 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:51.242624    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:40:54.340498 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:54.340471    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5c7c5665b5-9g8wm"
Jul 28 17:40:56.251773 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:56.251739    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:40:56.367521 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:56.367491    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:40:56.931164 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:56.931127    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-fn4nb"]
Jul 28 17:40:56.938022 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:56.937998    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-fn4nb"
Jul 28 17:40:56.940718 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:56.940688    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 28 17:40:56.940840 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:56.940769    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-mjw7g\""
Jul 28 17:40:56.944586 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:56.944549    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-fn4nb"]
Jul 28 17:40:56.968421 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:56.968364    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnrlq\" (UniqueName: \"kubernetes.io/projected/7b824351-3974-48c2-a51d-279feba50bb2-kube-api-access-bnrlq\") pod \"odh-model-controller-7998c67dc4-fn4nb\" (UID: \"7b824351-3974-48c2-a51d-279feba50bb2\") " pod="opendatahub/odh-model-controller-7998c67dc4-fn4nb"
Jul 28 17:40:56.968583 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:56.968514    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7b824351-3974-48c2-a51d-279feba50bb2-cert\") pod \"odh-model-controller-7998c67dc4-fn4nb\" (UID: \"7b824351-3974-48c2-a51d-279feba50bb2\") " pod="opendatahub/odh-model-controller-7998c67dc4-fn4nb"
Jul 28 17:40:57.006665 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:57.006636    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:40:57.069185 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:57.069127    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7b824351-3974-48c2-a51d-279feba50bb2-cert\") pod \"odh-model-controller-7998c67dc4-fn4nb\" (UID: \"7b824351-3974-48c2-a51d-279feba50bb2\") " pod="opendatahub/odh-model-controller-7998c67dc4-fn4nb"
Jul 28 17:40:57.069185 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:57.069161    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnrlq\" (UniqueName: \"kubernetes.io/projected/7b824351-3974-48c2-a51d-279feba50bb2-kube-api-access-bnrlq\") pod \"odh-model-controller-7998c67dc4-fn4nb\" (UID: \"7b824351-3974-48c2-a51d-279feba50bb2\") " pod="opendatahub/odh-model-controller-7998c67dc4-fn4nb"
Jul 28 17:40:57.069512 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:40:57.069292    2563 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 28 17:40:57.069512 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:40:57.069370    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b824351-3974-48c2-a51d-279feba50bb2-cert podName:7b824351-3974-48c2-a51d-279feba50bb2 nodeName:}" failed. No retries permitted until 2026-07-28 17:40:57.569351004 +0000 UTC m=+396.230507339 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7b824351-3974-48c2-a51d-279feba50bb2-cert") pod "odh-model-controller-7998c67dc4-fn4nb" (UID: "7b824351-3974-48c2-a51d-279feba50bb2") : secret "odh-model-controller-webhook-cert" not found
Jul 28 17:40:57.087380 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:57.087352    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnrlq\" (UniqueName: \"kubernetes.io/projected/7b824351-3974-48c2-a51d-279feba50bb2-kube-api-access-bnrlq\") pod \"odh-model-controller-7998c67dc4-fn4nb\" (UID: \"7b824351-3974-48c2-a51d-279feba50bb2\") " pod="opendatahub/odh-model-controller-7998c67dc4-fn4nb"
Jul 28 17:40:57.112209 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:57.112180    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:40:57.572663 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:57.572633    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-ng7x4"]
Jul 28 17:40:57.574159 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:57.574130    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7b824351-3974-48c2-a51d-279feba50bb2-cert\") pod \"odh-model-controller-7998c67dc4-fn4nb\" (UID: \"7b824351-3974-48c2-a51d-279feba50bb2\") " pod="opendatahub/odh-model-controller-7998c67dc4-fn4nb"
Jul 28 17:40:57.576238 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:57.576215    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-ng7x4"
Jul 28 17:40:57.576627 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:57.576610    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7b824351-3974-48c2-a51d-279feba50bb2-cert\") pod \"odh-model-controller-7998c67dc4-fn4nb\" (UID: \"7b824351-3974-48c2-a51d-279feba50bb2\") " pod="opendatahub/odh-model-controller-7998c67dc4-fn4nb"
Jul 28 17:40:57.578557 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:57.578540    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 28 17:40:57.589123 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:57.589099    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-ng7x4"]
Jul 28 17:40:57.595690 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:57.595662    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:40:57.674803 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:57.674761    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ae086ab-fa26-4ea1-94a2-b35920c32d3b-tls-certs\") pod \"model-serving-api-54d89bd5fc-ng7x4\" (UID: \"0ae086ab-fa26-4ea1-94a2-b35920c32d3b\") " pod="opendatahub/model-serving-api-54d89bd5fc-ng7x4"
Jul 28 17:40:57.674963 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:57.674817    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zg9c\" (UniqueName: \"kubernetes.io/projected/0ae086ab-fa26-4ea1-94a2-b35920c32d3b-kube-api-access-6zg9c\") pod \"model-serving-api-54d89bd5fc-ng7x4\" (UID: \"0ae086ab-fa26-4ea1-94a2-b35920c32d3b\") " pod="opendatahub/model-serving-api-54d89bd5fc-ng7x4"
Jul 28 17:40:57.692792 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:57.692760    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:40:57.775889 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:57.775856    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ae086ab-fa26-4ea1-94a2-b35920c32d3b-tls-certs\") pod \"model-serving-api-54d89bd5fc-ng7x4\" (UID: \"0ae086ab-fa26-4ea1-94a2-b35920c32d3b\") " pod="opendatahub/model-serving-api-54d89bd5fc-ng7x4"
Jul 28 17:40:57.776041 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:57.775895    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zg9c\" (UniqueName: \"kubernetes.io/projected/0ae086ab-fa26-4ea1-94a2-b35920c32d3b-kube-api-access-6zg9c\") pod \"model-serving-api-54d89bd5fc-ng7x4\" (UID: \"0ae086ab-fa26-4ea1-94a2-b35920c32d3b\") " pod="opendatahub/model-serving-api-54d89bd5fc-ng7x4"
Jul 28 17:40:57.778228 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:57.778203    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ae086ab-fa26-4ea1-94a2-b35920c32d3b-tls-certs\") pod \"model-serving-api-54d89bd5fc-ng7x4\" (UID: \"0ae086ab-fa26-4ea1-94a2-b35920c32d3b\") " pod="opendatahub/model-serving-api-54d89bd5fc-ng7x4"
Jul 28 17:40:57.785454 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:57.785413    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zg9c\" (UniqueName: \"kubernetes.io/projected/0ae086ab-fa26-4ea1-94a2-b35920c32d3b-kube-api-access-6zg9c\") pod \"model-serving-api-54d89bd5fc-ng7x4\" (UID: \"0ae086ab-fa26-4ea1-94a2-b35920c32d3b\") " pod="opendatahub/model-serving-api-54d89bd5fc-ng7x4"
Jul 28 17:40:57.850206 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:57.850138    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-fn4nb"
Jul 28 17:40:57.892954 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:57.892929    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-ng7x4"
Jul 28 17:40:57.999632 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:57.999593    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-fn4nb"]
Jul 28 17:40:58.002279 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:40:58.002247    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b824351_3974_48c2_a51d_279feba50bb2.slice/crio-7436174c28d113167ce90fc7426b2c629e86a5fcba2f2f58d5b7824d722afd7f WatchSource:0}: Error finding container 7436174c28d113167ce90fc7426b2c629e86a5fcba2f2f58d5b7824d722afd7f: Status 404 returned error can't find the container with id 7436174c28d113167ce90fc7426b2c629e86a5fcba2f2f58d5b7824d722afd7f
Jul 28 17:40:58.033579 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:58.033500    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-ng7x4"]
Jul 28 17:40:58.035672 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:40:58.035649    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ae086ab_fa26_4ea1_94a2_b35920c32d3b.slice/crio-3b595f0a93d4695191ca616f5025f73379bdd005049b1f14041be10ce4812137 WatchSource:0}: Error finding container 3b595f0a93d4695191ca616f5025f73379bdd005049b1f14041be10ce4812137: Status 404 returned error can't find the container with id 3b595f0a93d4695191ca616f5025f73379bdd005049b1f14041be10ce4812137
Jul 28 17:40:58.397299 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:58.397266    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-fn4nb" event={"ID":"7b824351-3974-48c2-a51d-279feba50bb2","Type":"ContainerStarted","Data":"7436174c28d113167ce90fc7426b2c629e86a5fcba2f2f58d5b7824d722afd7f"}
Jul 28 17:40:58.398261 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:40:58.398236    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-ng7x4" event={"ID":"0ae086ab-fa26-4ea1-94a2-b35920c32d3b","Type":"ContainerStarted","Data":"3b595f0a93d4695191ca616f5025f73379bdd005049b1f14041be10ce4812137"}
Jul 28 17:41:01.413693 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:01.413589    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-ng7x4" event={"ID":"0ae086ab-fa26-4ea1-94a2-b35920c32d3b","Type":"ContainerStarted","Data":"f8067616a6ab592bb09df657936387387f9a9e2b139560a267b243a417c40f63"}
Jul 28 17:41:01.414114 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:01.413774    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-ng7x4"
Jul 28 17:41:01.420552 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:01.420520    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-fn4nb" event={"ID":"7b824351-3974-48c2-a51d-279feba50bb2","Type":"ContainerStarted","Data":"d613e92d393e94dd021814832f68c654516c0b23c3730ff51c26ff9b497143d5"}
Jul 28 17:41:01.420709 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:01.420692    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-fn4nb"
Jul 28 17:41:01.434568 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:01.434523    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-ng7x4" podStartSLOduration=1.403204741 podStartE2EDuration="4.434507444s" totalImagesPullingTime="3.031302703s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:40:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:41:01.432866797 +0000 UTC m=+400.094023186" watchObservedRunningTime="2026-07-28 17:41:01.434507444 +0000 UTC m=+400.095663777"
Jul 28 17:41:01.452212 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:01.452159    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-fn4nb" podStartSLOduration=2.3773172799999998 podStartE2EDuration="5.45214067s" totalImagesPullingTime="3.07482339s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:40:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:41:01.449960325 +0000 UTC m=+400.111116681" watchObservedRunningTime="2026-07-28 17:41:01.45214067 +0000 UTC m=+400.113297028"
Jul 28 17:41:07.013491 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:07.013455    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:41:07.117404 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:07.117366    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-5bfrd"]
Jul 28 17:41:07.120419 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:07.120398    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-zghk7"]
Jul 28 17:41:07.120556 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:07.120518    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-5bfrd"
Jul 28 17:41:07.123443 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:07.123408    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zghk7"
Jul 28 17:41:07.125697 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:07.125670    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-27qqb\""
Jul 28 17:41:07.125697 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:07.125678    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 28 17:41:07.130757 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:07.130739    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 28 17:41:07.131448 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:07.131418    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-nbgmh\""
Jul 28 17:41:07.148818 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:07.148791    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-zghk7"]
Jul 28 17:41:07.150202 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:07.150174    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d1058246-3d40-4ead-83f6-bed96bf7777c-cert\") pod \"kserve-controller-manager-d6f4b65b8-5bfrd\" (UID: \"d1058246-3d40-4ead-83f6-bed96bf7777c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-5bfrd"
Jul 28 17:41:07.150341 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:07.150262    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8647c\" (UniqueName: \"kubernetes.io/projected/76cb19a8-73c6-4277-a484-608429e16222-kube-api-access-8647c\") pod \"llmisvc-controller-manager-777c6bd7d5-zghk7\" (UID: \"76cb19a8-73c6-4277-a484-608429e16222\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zghk7"
Jul 28 17:41:07.150341 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:07.150308    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/76cb19a8-73c6-4277-a484-608429e16222-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-zghk7\" (UID: \"76cb19a8-73c6-4277-a484-608429e16222\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zghk7"
Jul 28 17:41:07.150341 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:07.150333    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dpvg\" (UniqueName: \"kubernetes.io/projected/d1058246-3d40-4ead-83f6-bed96bf7777c-kube-api-access-5dpvg\") pod \"kserve-controller-manager-d6f4b65b8-5bfrd\" (UID: \"d1058246-3d40-4ead-83f6-bed96bf7777c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-5bfrd"
Jul 28 17:41:07.155264 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:07.155238    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-5bfrd"]
Jul 28 17:41:07.165039 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:07.165011    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:41:07.230657 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:07.230624    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6b8f9cc6c-4mggp"]
Jul 28 17:41:07.233913 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:07.233886    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6b8f9cc6c-4mggp"
Jul 28 17:41:07.236664 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:07.236636    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 28 17:41:07.236821 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:07.236644    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 28 17:41:07.251505 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:07.251478    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d1058246-3d40-4ead-83f6-bed96bf7777c-cert\") pod \"kserve-controller-manager-d6f4b65b8-5bfrd\" (UID: \"d1058246-3d40-4ead-83f6-bed96bf7777c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-5bfrd"
Jul 28 17:41:07.251679 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:07.251526    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-258tx\" (UniqueName: \"kubernetes.io/projected/a977444b-6bdf-4448-9859-266756ff657f-kube-api-access-258tx\") pod \"kube-auth-proxy-6b8f9cc6c-4mggp\" (UID: \"a977444b-6bdf-4448-9859-266756ff657f\") " pod="openshift-ingress/kube-auth-proxy-6b8f9cc6c-4mggp"
Jul 28 17:41:07.251679 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:07.251567    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a977444b-6bdf-4448-9859-266756ff657f-tls-certs\") pod \"kube-auth-proxy-6b8f9cc6c-4mggp\" (UID: \"a977444b-6bdf-4448-9859-266756ff657f\") " pod="openshift-ingress/kube-auth-proxy-6b8f9cc6c-4mggp"
Jul 28 17:41:07.251679 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:07.251605    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8647c\" (UniqueName: \"kubernetes.io/projected/76cb19a8-73c6-4277-a484-608429e16222-kube-api-access-8647c\") pod \"llmisvc-controller-manager-777c6bd7d5-zghk7\" (UID: \"76cb19a8-73c6-4277-a484-608429e16222\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zghk7"
Jul 28 17:41:07.251785 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:07.251681    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/76cb19a8-73c6-4277-a484-608429e16222-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-zghk7\" (UID: \"76cb19a8-73c6-4277-a484-608429e16222\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zghk7"
Jul 28 17:41:07.251785 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:07.251713    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dpvg\" (UniqueName: \"kubernetes.io/projected/d1058246-3d40-4ead-83f6-bed96bf7777c-kube-api-access-5dpvg\") pod \"kserve-controller-manager-d6f4b65b8-5bfrd\" (UID: \"d1058246-3d40-4ead-83f6-bed96bf7777c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-5bfrd"
Jul 28 17:41:07.251785 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:07.251734    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a977444b-6bdf-4448-9859-266756ff657f-tmp\") pod \"kube-auth-proxy-6b8f9cc6c-4mggp\" (UID: \"a977444b-6bdf-4448-9859-266756ff657f\") " pod="openshift-ingress/kube-auth-proxy-6b8f9cc6c-4mggp"
Jul 28 17:41:07.258503 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:07.258474    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d1058246-3d40-4ead-83f6-bed96bf7777c-cert\") pod \"kserve-controller-manager-d6f4b65b8-5bfrd\" (UID: \"d1058246-3d40-4ead-83f6-bed96bf7777c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-5bfrd"
Jul 28 17:41:07.259500 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:07.259469    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6b8f9cc6c-4mggp"]
Jul 28 17:41:07.263457 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:07.263402    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/76cb19a8-73c6-4277-a484-608429e16222-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-zghk7\" (UID: \"76cb19a8-73c6-4277-a484-608429e16222\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zghk7"
Jul 28 17:41:07.268779 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:07.268722    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8647c\" (UniqueName: \"kubernetes.io/projected/76cb19a8-73c6-4277-a484-608429e16222-kube-api-access-8647c\") pod \"llmisvc-controller-manager-777c6bd7d5-zghk7\" (UID: \"76cb19a8-73c6-4277-a484-608429e16222\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zghk7"
Jul 28 17:41:07.268779 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:07.268755    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dpvg\" (UniqueName: \"kubernetes.io/projected/d1058246-3d40-4ead-83f6-bed96bf7777c-kube-api-access-5dpvg\") pod \"kserve-controller-manager-d6f4b65b8-5bfrd\" (UID: \"d1058246-3d40-4ead-83f6-bed96bf7777c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-5bfrd"
Jul 28 17:41:07.287615 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:07.287590    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:41:07.352940 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:07.352906    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a977444b-6bdf-4448-9859-266756ff657f-tmp\") pod \"kube-auth-proxy-6b8f9cc6c-4mggp\" (UID: \"a977444b-6bdf-4448-9859-266756ff657f\") " pod="openshift-ingress/kube-auth-proxy-6b8f9cc6c-4mggp"
Jul 28 17:41:07.353204 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:07.352958    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-258tx\" (UniqueName: \"kubernetes.io/projected/a977444b-6bdf-4448-9859-266756ff657f-kube-api-access-258tx\") pod \"kube-auth-proxy-6b8f9cc6c-4mggp\" (UID: \"a977444b-6bdf-4448-9859-266756ff657f\") " pod="openshift-ingress/kube-auth-proxy-6b8f9cc6c-4mggp"
Jul 28 17:41:07.353204 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:07.352987    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a977444b-6bdf-4448-9859-266756ff657f-tls-certs\") pod \"kube-auth-proxy-6b8f9cc6c-4mggp\" (UID: \"a977444b-6bdf-4448-9859-266756ff657f\") " pod="openshift-ingress/kube-auth-proxy-6b8f9cc6c-4mggp"
Jul 28 17:41:07.353204 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:41:07.353076    2563 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 28 17:41:07.353204 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:41:07.353154    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a977444b-6bdf-4448-9859-266756ff657f-tls-certs podName:a977444b-6bdf-4448-9859-266756ff657f nodeName:}" failed. No retries permitted until 2026-07-28 17:41:07.853139586 +0000 UTC m=+406.514295919 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/a977444b-6bdf-4448-9859-266756ff657f-tls-certs") pod "kube-auth-proxy-6b8f9cc6c-4mggp" (UID: "a977444b-6bdf-4448-9859-266756ff657f") : secret "kube-auth-proxy-tls" not found
Jul 28 17:41:07.355226 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:07.355204    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a977444b-6bdf-4448-9859-266756ff657f-tmp\") pod \"kube-auth-proxy-6b8f9cc6c-4mggp\" (UID: \"a977444b-6bdf-4448-9859-266756ff657f\") " pod="openshift-ingress/kube-auth-proxy-6b8f9cc6c-4mggp"
Jul 28 17:41:07.366643 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:07.366613    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-258tx\" (UniqueName: \"kubernetes.io/projected/a977444b-6bdf-4448-9859-266756ff657f-kube-api-access-258tx\") pod \"kube-auth-proxy-6b8f9cc6c-4mggp\" (UID: \"a977444b-6bdf-4448-9859-266756ff657f\") " pod="openshift-ingress/kube-auth-proxy-6b8f9cc6c-4mggp"
Jul 28 17:41:07.393041 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:07.393006    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:41:07.431859 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:07.431830    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-5bfrd"
Jul 28 17:41:07.437730 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:07.437715    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zghk7"
Jul 28 17:41:07.600974 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:07.600952    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-5bfrd"]
Jul 28 17:41:07.603068 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:41:07.603042    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1058246_3d40_4ead_83f6_bed96bf7777c.slice/crio-47a9034a1bea0e6f68e3981dc66fd8e91d2c4ff9d789039028af6156921b5717 WatchSource:0}: Error finding container 47a9034a1bea0e6f68e3981dc66fd8e91d2c4ff9d789039028af6156921b5717: Status 404 returned error can't find the container with id 47a9034a1bea0e6f68e3981dc66fd8e91d2c4ff9d789039028af6156921b5717
Jul 28 17:41:07.606917 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:07.606893    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-zghk7"]
Jul 28 17:41:07.609195 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:41:07.609173    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod76cb19a8_73c6_4277_a484_608429e16222.slice/crio-d885b55a22980526a09799b2c14f5b23f172ad8f8b29dd25879bc5f39a1462b8 WatchSource:0}: Error finding container d885b55a22980526a09799b2c14f5b23f172ad8f8b29dd25879bc5f39a1462b8: Status 404 returned error can't find the container with id d885b55a22980526a09799b2c14f5b23f172ad8f8b29dd25879bc5f39a1462b8
Jul 28 17:41:07.673267 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:07.673222    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:41:07.857404 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:07.857372    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a977444b-6bdf-4448-9859-266756ff657f-tls-certs\") pod \"kube-auth-proxy-6b8f9cc6c-4mggp\" (UID: \"a977444b-6bdf-4448-9859-266756ff657f\") " pod="openshift-ingress/kube-auth-proxy-6b8f9cc6c-4mggp"
Jul 28 17:41:07.859726 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:07.859711    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a977444b-6bdf-4448-9859-266756ff657f-tls-certs\") pod \"kube-auth-proxy-6b8f9cc6c-4mggp\" (UID: \"a977444b-6bdf-4448-9859-266756ff657f\") " pod="openshift-ingress/kube-auth-proxy-6b8f9cc6c-4mggp"
Jul 28 17:41:08.144216 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:08.144135    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6b8f9cc6c-4mggp"
Jul 28 17:41:08.281732 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:41:08.281697    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda977444b_6bdf_4448_9859_266756ff657f.slice/crio-dc38c3a3cfcd26e98f64ff8a89943f82375fa7f05a88f86592b072cc2ee64c16 WatchSource:0}: Error finding container dc38c3a3cfcd26e98f64ff8a89943f82375fa7f05a88f86592b072cc2ee64c16: Status 404 returned error can't find the container with id dc38c3a3cfcd26e98f64ff8a89943f82375fa7f05a88f86592b072cc2ee64c16
Jul 28 17:41:08.286635 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:08.286609    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6b8f9cc6c-4mggp"]
Jul 28 17:41:08.364749 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:08.364713    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:41:08.457651 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:08.457134    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6b8f9cc6c-4mggp" event={"ID":"a977444b-6bdf-4448-9859-266756ff657f","Type":"ContainerStarted","Data":"dc38c3a3cfcd26e98f64ff8a89943f82375fa7f05a88f86592b072cc2ee64c16"}
Jul 28 17:41:08.459357 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:08.459324    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zghk7" event={"ID":"76cb19a8-73c6-4277-a484-608429e16222","Type":"ContainerStarted","Data":"d885b55a22980526a09799b2c14f5b23f172ad8f8b29dd25879bc5f39a1462b8"}
Jul 28 17:41:08.460910 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:08.460884    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-5bfrd" event={"ID":"d1058246-3d40-4ead-83f6-bed96bf7777c","Type":"ContainerStarted","Data":"47a9034a1bea0e6f68e3981dc66fd8e91d2c4ff9d789039028af6156921b5717"}
Jul 28 17:41:08.480945 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:08.480920    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:41:08.585243 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:08.585193    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:41:09.153714 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:09.153609    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:41:09.172467 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:09.171795    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-8pm6v"]
Jul 28 17:41:09.175712 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:09.175688    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8pm6v"
Jul 28 17:41:09.178696 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:09.178672    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 28 17:41:09.178902 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:09.178887    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 28 17:41:09.179098 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:09.179083    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 28 17:41:09.179332 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:09.179314    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 28 17:41:09.179644 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:09.179610    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-x9j29\""
Jul 28 17:41:09.191559 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:09.191537    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-8pm6v"]
Jul 28 17:41:09.269802 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:09.269759    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/05df90ce-13cd-478f-864f-103c7cfa9689-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-8pm6v\" (UID: \"05df90ce-13cd-478f-864f-103c7cfa9689\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8pm6v"
Jul 28 17:41:09.269979 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:09.269829    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/05df90ce-13cd-478f-864f-103c7cfa9689-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-8pm6v\" (UID: \"05df90ce-13cd-478f-864f-103c7cfa9689\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8pm6v"
Jul 28 17:41:09.269979 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:09.269870    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/05df90ce-13cd-478f-864f-103c7cfa9689-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-8pm6v\" (UID: \"05df90ce-13cd-478f-864f-103c7cfa9689\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8pm6v"
Jul 28 17:41:09.269979 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:09.269898    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05df90ce-13cd-478f-864f-103c7cfa9689-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-8pm6v\" (UID: \"05df90ce-13cd-478f-864f-103c7cfa9689\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8pm6v"
Jul 28 17:41:09.270108 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:09.269984    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/05df90ce-13cd-478f-864f-103c7cfa9689-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-8pm6v\" (UID: \"05df90ce-13cd-478f-864f-103c7cfa9689\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8pm6v"
Jul 28 17:41:09.270108 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:09.270020    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kd7qh\" (UniqueName: \"kubernetes.io/projected/05df90ce-13cd-478f-864f-103c7cfa9689-kube-api-access-kd7qh\") pod \"istiod-openshift-gateway-5677ccb49b-8pm6v\" (UID: \"05df90ce-13cd-478f-864f-103c7cfa9689\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8pm6v"
Jul 28 17:41:09.270108 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:09.270053    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/05df90ce-13cd-478f-864f-103c7cfa9689-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-8pm6v\" (UID: \"05df90ce-13cd-478f-864f-103c7cfa9689\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8pm6v"
Jul 28 17:41:09.272900 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:09.272846    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:41:09.371428 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:09.370758    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/05df90ce-13cd-478f-864f-103c7cfa9689-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-8pm6v\" (UID: \"05df90ce-13cd-478f-864f-103c7cfa9689\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8pm6v"
Jul 28 17:41:09.371428 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:09.370831    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/05df90ce-13cd-478f-864f-103c7cfa9689-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-8pm6v\" (UID: \"05df90ce-13cd-478f-864f-103c7cfa9689\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8pm6v"
Jul 28 17:41:09.371428 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:09.370866    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05df90ce-13cd-478f-864f-103c7cfa9689-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-8pm6v\" (UID: \"05df90ce-13cd-478f-864f-103c7cfa9689\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8pm6v"
Jul 28 17:41:09.371428 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:09.370920    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/05df90ce-13cd-478f-864f-103c7cfa9689-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-8pm6v\" (UID: \"05df90ce-13cd-478f-864f-103c7cfa9689\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8pm6v"
Jul 28 17:41:09.371428 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:09.370968    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kd7qh\" (UniqueName: \"kubernetes.io/projected/05df90ce-13cd-478f-864f-103c7cfa9689-kube-api-access-kd7qh\") pod \"istiod-openshift-gateway-5677ccb49b-8pm6v\" (UID: \"05df90ce-13cd-478f-864f-103c7cfa9689\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8pm6v"
Jul 28 17:41:09.373842 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:09.373358    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/05df90ce-13cd-478f-864f-103c7cfa9689-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-8pm6v\" (UID: \"05df90ce-13cd-478f-864f-103c7cfa9689\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8pm6v"
Jul 28 17:41:09.373842 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:09.373396    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/05df90ce-13cd-478f-864f-103c7cfa9689-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-8pm6v\" (UID: \"05df90ce-13cd-478f-864f-103c7cfa9689\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8pm6v"
Jul 28 17:41:09.374293 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:09.374248    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/05df90ce-13cd-478f-864f-103c7cfa9689-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-8pm6v\" (UID: \"05df90ce-13cd-478f-864f-103c7cfa9689\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8pm6v"
Jul 28 17:41:09.379644 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:09.379620    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/05df90ce-13cd-478f-864f-103c7cfa9689-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-8pm6v\" (UID: \"05df90ce-13cd-478f-864f-103c7cfa9689\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8pm6v"
Jul 28 17:41:09.380396 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:09.380203    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/05df90ce-13cd-478f-864f-103c7cfa9689-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-8pm6v\" (UID: \"05df90ce-13cd-478f-864f-103c7cfa9689\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8pm6v"
Jul 28 17:41:09.380396 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:09.380358    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/05df90ce-13cd-478f-864f-103c7cfa9689-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-8pm6v\" (UID: \"05df90ce-13cd-478f-864f-103c7cfa9689\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8pm6v"
Jul 28 17:41:09.381131 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:09.381094    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/05df90ce-13cd-478f-864f-103c7cfa9689-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-8pm6v\" (UID: \"05df90ce-13cd-478f-864f-103c7cfa9689\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8pm6v"
Jul 28 17:41:09.385291 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:09.385252    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05df90ce-13cd-478f-864f-103c7cfa9689-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-8pm6v\" (UID: \"05df90ce-13cd-478f-864f-103c7cfa9689\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8pm6v"
Jul 28 17:41:09.385990 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:09.385950    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kd7qh\" (UniqueName: \"kubernetes.io/projected/05df90ce-13cd-478f-864f-103c7cfa9689-kube-api-access-kd7qh\") pod \"istiod-openshift-gateway-5677ccb49b-8pm6v\" (UID: \"05df90ce-13cd-478f-864f-103c7cfa9689\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8pm6v"
Jul 28 17:41:09.490647 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:09.490573    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8pm6v"
Jul 28 17:41:09.646852 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:09.646700    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-8pm6v"]
Jul 28 17:41:09.649375 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:41:09.649342    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05df90ce_13cd_478f_864f_103c7cfa9689.slice/crio-39e07c9b8e2f9589e721ae53b0127a8437c7c32be17da43b98080ea3bd44b22d WatchSource:0}: Error finding container 39e07c9b8e2f9589e721ae53b0127a8437c7c32be17da43b98080ea3bd44b22d: Status 404 returned error can't find the container with id 39e07c9b8e2f9589e721ae53b0127a8437c7c32be17da43b98080ea3bd44b22d
Jul 28 17:41:10.473068 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:10.473039    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8pm6v" event={"ID":"05df90ce-13cd-478f-864f-103c7cfa9689","Type":"ContainerStarted","Data":"39e07c9b8e2f9589e721ae53b0127a8437c7c32be17da43b98080ea3bd44b22d"}
Jul 28 17:41:10.570623 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:10.570591    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:41:11.482264 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:11.481990    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-5bfrd" event={"ID":"d1058246-3d40-4ead-83f6-bed96bf7777c","Type":"ContainerStarted","Data":"a0ba70c04fbfefd32243575b0ef20ece5c5b82575f721dad6f8ccc7648840ca0"}
Jul 28 17:41:11.482744 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:11.482315    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-5bfrd"
Jul 28 17:41:11.504242 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:11.504193    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-5bfrd" podStartSLOduration=1.659792172 podStartE2EDuration="4.504176223s" totalImagesPullingTime="2.844384051s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:41:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:41:11.50159655 +0000 UTC m=+410.162752917" watchObservedRunningTime="2026-07-28 17:41:11.504176223 +0000 UTC m=+410.165332582"
Jul 28 17:41:12.427495 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:12.427379    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-fn4nb"
Jul 28 17:41:12.430498 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:12.430472    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-ng7x4"
Jul 28 17:41:13.482418 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:13.482361    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:41:13.582691 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:13.582666    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:41:13.612277 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:13.612240    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 17:41:13.612348 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:13.612326    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 17:41:14.497066 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:14.497035    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zghk7" event={"ID":"76cb19a8-73c6-4277-a484-608429e16222","Type":"ContainerStarted","Data":"cb16d2752b25e950edc042769d2b5dc2dfa0bec8aab0488570b41a6773ff916c"}
Jul 28 17:41:14.497488 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:14.497200    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zghk7"
Jul 28 17:41:14.498578 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:14.498552    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6b8f9cc6c-4mggp" event={"ID":"a977444b-6bdf-4448-9859-266756ff657f","Type":"ContainerStarted","Data":"fae058e3c7fa2492ce1257921623e1816c1d1ca38abb994c2955a137d93bbb34"}
Jul 28 17:41:14.500031 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:14.500006    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8pm6v" event={"ID":"05df90ce-13cd-478f-864f-103c7cfa9689","Type":"ContainerStarted","Data":"63a1222adc87af422a959f26a8538fabadfa584425a1b89b7750d0bfcf406f76"}
Jul 28 17:41:14.500217 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:14.500201    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8pm6v"
Jul 28 17:41:14.501923 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:14.501900    2563 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-8pm6v container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 28 17:41:14.502003 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:14.501952    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8pm6v" podUID="05df90ce-13cd-478f-864f-103c7cfa9689" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 17:41:14.517002 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:14.516962    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zghk7" podStartSLOduration=1.5491615840000001 podStartE2EDuration="7.516947155s" totalImagesPullingTime="5.967785571s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:41:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:41:14.515333125 +0000 UTC m=+413.176489471" watchObservedRunningTime="2026-07-28 17:41:14.516947155 +0000 UTC m=+413.178103509"
Jul 28 17:41:14.535777 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:14.535735    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6b8f9cc6c-4mggp" podStartSLOduration=2.210029768 podStartE2EDuration="7.535726828s" totalImagesPullingTime="5.32569706s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:41:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:41:14.534179573 +0000 UTC m=+413.195335928" watchObservedRunningTime="2026-07-28 17:41:14.535726828 +0000 UTC m=+413.196883183"
Jul 28 17:41:14.555079 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:14.555021    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8pm6v" podStartSLOduration=1.594983112 podStartE2EDuration="5.555003578s" totalImagesPullingTime="3.960020466s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:41:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:41:14.55324915 +0000 UTC m=+413.214405508" watchObservedRunningTime="2026-07-28 17:41:14.555003578 +0000 UTC m=+413.216159934"
Jul 28 17:41:15.184045 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:15.184015    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:41:15.286457 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:15.286403    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:41:15.505180 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:15.505103    2563 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-8pm6v container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 28 17:41:15.505180 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:15.505151    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8pm6v" podUID="05df90ce-13cd-478f-864f-103c7cfa9689" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 17:41:16.070826 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:16.070784    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:41:16.308449 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:16.308394    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:41:16.699695 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:16.699659    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:41:18.505733 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:18.505706    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8pm6v"
Jul 28 17:41:25.738304 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:25.738269    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:41:25.826292 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:25.826259    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:41:27.833510 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:27.833475    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:41:27.940566 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:27.940533    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:41:38.989600 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:38.989558    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:41:39.089749 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:39.089715    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:41:42.492885 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:42.492858    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-5bfrd"
Jul 28 17:41:45.506744 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:45.506673    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zghk7"
Jul 28 17:41:57.841098 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:57.841057    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:41:57.933832 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:57.933804    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:41:58.032782 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:41:58.032752    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:42:13.173167 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:13.173133    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pnv6x"]
Jul 28 17:42:13.175690 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:13.175674    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pnv6x"
Jul 28 17:42:13.178241 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:13.178223    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-x977p\""
Jul 28 17:42:13.178321 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:13.178227    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 28 17:42:13.179255 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:13.179238    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 28 17:42:13.185481 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:13.185461    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pnv6x"]
Jul 28 17:42:13.304944 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:13.304913    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pnv6x\" (UID: \"fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pnv6x"
Jul 28 17:42:13.305075 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:13.304971    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5zrp\" (UniqueName: \"kubernetes.io/projected/fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa-kube-api-access-c5zrp\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pnv6x\" (UID: \"fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pnv6x"
Jul 28 17:42:13.305075 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:13.304994    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pnv6x\" (UID: \"fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pnv6x"
Jul 28 17:42:13.405670 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:13.405642    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pnv6x\" (UID: \"fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pnv6x"
Jul 28 17:42:13.405769 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:13.405738    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5zrp\" (UniqueName: \"kubernetes.io/projected/fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa-kube-api-access-c5zrp\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pnv6x\" (UID: \"fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pnv6x"
Jul 28 17:42:13.405812 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:13.405782    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pnv6x\" (UID: \"fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pnv6x"
Jul 28 17:42:13.405979 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:13.405963    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pnv6x\" (UID: \"fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pnv6x"
Jul 28 17:42:13.406039 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:13.406024    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pnv6x\" (UID: \"fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pnv6x"
Jul 28 17:42:13.415208 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:13.415181    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5zrp\" (UniqueName: \"kubernetes.io/projected/fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa-kube-api-access-c5zrp\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pnv6x\" (UID: \"fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pnv6x"
Jul 28 17:42:13.485184 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:13.485135    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pnv6x"
Jul 28 17:42:13.611963 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:13.611941    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pnv6x"]
Jul 28 17:42:13.613788 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:42:13.613762    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdcdf9cb_e5b6_4ab8_9867_d81ce71b67aa.slice/crio-cefbbdb47aaca46641c17affe988c09e9045e15cda9d0e4d1c2d1c46af3ac3bf WatchSource:0}: Error finding container cefbbdb47aaca46641c17affe988c09e9045e15cda9d0e4d1c2d1c46af3ac3bf: Status 404 returned error can't find the container with id cefbbdb47aaca46641c17affe988c09e9045e15cda9d0e4d1c2d1c46af3ac3bf
Jul 28 17:42:13.754430 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:13.754367    2563 generic.go:356] "Generic (PLEG): container finished" podID="fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa" containerID="4b8652e591f91f74855f7b2c8fdd11bcf95229562d39ffcb13b0cb4dd54d1c07" exitCode=0
Jul 28 17:42:13.754430 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:13.754415    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pnv6x" event={"ID":"fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa","Type":"ContainerDied","Data":"4b8652e591f91f74855f7b2c8fdd11bcf95229562d39ffcb13b0cb4dd54d1c07"}
Jul 28 17:42:13.754430 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:13.754515    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pnv6x" event={"ID":"fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa","Type":"ContainerStarted","Data":"cefbbdb47aaca46641c17affe988c09e9045e15cda9d0e4d1c2d1c46af3ac3bf"}
Jul 28 17:42:13.775207 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:13.775183    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1scs6r"]
Jul 28 17:42:13.778004 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:13.777989    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1scs6r"
Jul 28 17:42:13.786534 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:13.786515    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1scs6r"]
Jul 28 17:42:13.908961 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:13.908937    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3f67ec8b-7142-4b63-9df8-ed079ca8a9f7-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1scs6r\" (UID: \"3f67ec8b-7142-4b63-9df8-ed079ca8a9f7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1scs6r"
Jul 28 17:42:13.909113 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:13.909086    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3f67ec8b-7142-4b63-9df8-ed079ca8a9f7-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1scs6r\" (UID: \"3f67ec8b-7142-4b63-9df8-ed079ca8a9f7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1scs6r"
Jul 28 17:42:13.909218 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:13.909149    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhgfp\" (UniqueName: \"kubernetes.io/projected/3f67ec8b-7142-4b63-9df8-ed079ca8a9f7-kube-api-access-rhgfp\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1scs6r\" (UID: \"3f67ec8b-7142-4b63-9df8-ed079ca8a9f7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1scs6r"
Jul 28 17:42:14.009827 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:14.009773    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3f67ec8b-7142-4b63-9df8-ed079ca8a9f7-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1scs6r\" (UID: \"3f67ec8b-7142-4b63-9df8-ed079ca8a9f7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1scs6r"
Jul 28 17:42:14.009827 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:14.009809    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3f67ec8b-7142-4b63-9df8-ed079ca8a9f7-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1scs6r\" (UID: \"3f67ec8b-7142-4b63-9df8-ed079ca8a9f7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1scs6r"
Jul 28 17:42:14.009962 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:14.009828    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhgfp\" (UniqueName: \"kubernetes.io/projected/3f67ec8b-7142-4b63-9df8-ed079ca8a9f7-kube-api-access-rhgfp\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1scs6r\" (UID: \"3f67ec8b-7142-4b63-9df8-ed079ca8a9f7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1scs6r"
Jul 28 17:42:14.010113 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:14.010095    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3f67ec8b-7142-4b63-9df8-ed079ca8a9f7-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1scs6r\" (UID: \"3f67ec8b-7142-4b63-9df8-ed079ca8a9f7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1scs6r"
Jul 28 17:42:14.010165 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:14.010149    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3f67ec8b-7142-4b63-9df8-ed079ca8a9f7-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1scs6r\" (UID: \"3f67ec8b-7142-4b63-9df8-ed079ca8a9f7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1scs6r"
Jul 28 17:42:14.019460 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:14.019424    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhgfp\" (UniqueName: \"kubernetes.io/projected/3f67ec8b-7142-4b63-9df8-ed079ca8a9f7-kube-api-access-rhgfp\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1scs6r\" (UID: \"3f67ec8b-7142-4b63-9df8-ed079ca8a9f7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1scs6r"
Jul 28 17:42:14.086505 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:14.086485    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1scs6r"
Jul 28 17:42:14.177644 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:14.177612    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nvggl"]
Jul 28 17:42:14.180743 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:14.180723    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nvggl"
Jul 28 17:42:14.189515 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:14.189493    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nvggl"]
Jul 28 17:42:14.216543 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:14.216486    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1scs6r"]
Jul 28 17:42:14.218633 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:42:14.218608    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f67ec8b_7142_4b63_9df8_ed079ca8a9f7.slice/crio-d1e104bbcc591ab1e6ad119617e06aa1db1f08209ff071835fcd93dfa29361a8 WatchSource:0}: Error finding container d1e104bbcc591ab1e6ad119617e06aa1db1f08209ff071835fcd93dfa29361a8: Status 404 returned error can't find the container with id d1e104bbcc591ab1e6ad119617e06aa1db1f08209ff071835fcd93dfa29361a8
Jul 28 17:42:14.312804 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:14.312780    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zskwf\" (UniqueName: \"kubernetes.io/projected/cd6773b9-1881-431a-bd08-757ec5e03c00-kube-api-access-zskwf\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nvggl\" (UID: \"cd6773b9-1881-431a-bd08-757ec5e03c00\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nvggl"
Jul 28 17:42:14.312901 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:14.312823    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cd6773b9-1881-431a-bd08-757ec5e03c00-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nvggl\" (UID: \"cd6773b9-1881-431a-bd08-757ec5e03c00\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nvggl"
Jul 28 17:42:14.312901 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:14.312880    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cd6773b9-1881-431a-bd08-757ec5e03c00-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nvggl\" (UID: \"cd6773b9-1881-431a-bd08-757ec5e03c00\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nvggl"
Jul 28 17:42:14.413997 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:14.413974    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zskwf\" (UniqueName: \"kubernetes.io/projected/cd6773b9-1881-431a-bd08-757ec5e03c00-kube-api-access-zskwf\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nvggl\" (UID: \"cd6773b9-1881-431a-bd08-757ec5e03c00\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nvggl"
Jul 28 17:42:14.414115 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:14.414034    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cd6773b9-1881-431a-bd08-757ec5e03c00-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nvggl\" (UID: \"cd6773b9-1881-431a-bd08-757ec5e03c00\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nvggl"
Jul 28 17:42:14.414115 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:14.414073    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cd6773b9-1881-431a-bd08-757ec5e03c00-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nvggl\" (UID: \"cd6773b9-1881-431a-bd08-757ec5e03c00\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nvggl"
Jul 28 17:42:14.414382 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:14.414361    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cd6773b9-1881-431a-bd08-757ec5e03c00-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nvggl\" (UID: \"cd6773b9-1881-431a-bd08-757ec5e03c00\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nvggl"
Jul 28 17:42:14.414382 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:14.414371    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cd6773b9-1881-431a-bd08-757ec5e03c00-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nvggl\" (UID: \"cd6773b9-1881-431a-bd08-757ec5e03c00\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nvggl"
Jul 28 17:42:14.423999 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:14.423972    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zskwf\" (UniqueName: \"kubernetes.io/projected/cd6773b9-1881-431a-bd08-757ec5e03c00-kube-api-access-zskwf\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nvggl\" (UID: \"cd6773b9-1881-431a-bd08-757ec5e03c00\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nvggl"
Jul 28 17:42:14.493330 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:14.493304    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nvggl"
Jul 28 17:42:14.572816 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:14.572787    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlgc2"]
Jul 28 17:42:14.575541 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:14.575519    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlgc2"
Jul 28 17:42:14.588549 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:14.588488    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlgc2"]
Jul 28 17:42:14.616327 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:14.616293    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/010eb983-47d5-4c20-a9d4-f8f27c32d136-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlgc2\" (UID: \"010eb983-47d5-4c20-a9d4-f8f27c32d136\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlgc2"
Jul 28 17:42:14.616484 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:14.616445    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/010eb983-47d5-4c20-a9d4-f8f27c32d136-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlgc2\" (UID: \"010eb983-47d5-4c20-a9d4-f8f27c32d136\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlgc2"
Jul 28 17:42:14.616540 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:14.616519    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzmv2\" (UniqueName: \"kubernetes.io/projected/010eb983-47d5-4c20-a9d4-f8f27c32d136-kube-api-access-qzmv2\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlgc2\" (UID: \"010eb983-47d5-4c20-a9d4-f8f27c32d136\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlgc2"
Jul 28 17:42:14.635418 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:14.635388    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nvggl"]
Jul 28 17:42:14.650189 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:42:14.650164    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd6773b9_1881_431a_bd08_757ec5e03c00.slice/crio-11d1ae81c1edaf4d6f0b302d425ead85fb22d8b28a81a73e4ea0c040da02f3bf WatchSource:0}: Error finding container 11d1ae81c1edaf4d6f0b302d425ead85fb22d8b28a81a73e4ea0c040da02f3bf: Status 404 returned error can't find the container with id 11d1ae81c1edaf4d6f0b302d425ead85fb22d8b28a81a73e4ea0c040da02f3bf
Jul 28 17:42:14.716889 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:14.716867    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/010eb983-47d5-4c20-a9d4-f8f27c32d136-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlgc2\" (UID: \"010eb983-47d5-4c20-a9d4-f8f27c32d136\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlgc2"
Jul 28 17:42:14.716992 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:14.716928    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/010eb983-47d5-4c20-a9d4-f8f27c32d136-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlgc2\" (UID: \"010eb983-47d5-4c20-a9d4-f8f27c32d136\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlgc2"
Jul 28 17:42:14.716992 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:14.716962    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzmv2\" (UniqueName: \"kubernetes.io/projected/010eb983-47d5-4c20-a9d4-f8f27c32d136-kube-api-access-qzmv2\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlgc2\" (UID: \"010eb983-47d5-4c20-a9d4-f8f27c32d136\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlgc2"
Jul 28 17:42:14.717365 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:14.717337    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/010eb983-47d5-4c20-a9d4-f8f27c32d136-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlgc2\" (UID: \"010eb983-47d5-4c20-a9d4-f8f27c32d136\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlgc2"
Jul 28 17:42:14.717475 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:14.717361    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/010eb983-47d5-4c20-a9d4-f8f27c32d136-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlgc2\" (UID: \"010eb983-47d5-4c20-a9d4-f8f27c32d136\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlgc2"
Jul 28 17:42:14.726627 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:14.726600    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzmv2\" (UniqueName: \"kubernetes.io/projected/010eb983-47d5-4c20-a9d4-f8f27c32d136-kube-api-access-qzmv2\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlgc2\" (UID: \"010eb983-47d5-4c20-a9d4-f8f27c32d136\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlgc2"
Jul 28 17:42:14.760902 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:14.760871    2563 generic.go:356] "Generic (PLEG): container finished" podID="3f67ec8b-7142-4b63-9df8-ed079ca8a9f7" containerID="c8af72a0202096e1107dda5cfb3a2e59582318b7ffd9ee768743cd1176f37c49" exitCode=0
Jul 28 17:42:14.761024 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:14.760952    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1scs6r" event={"ID":"3f67ec8b-7142-4b63-9df8-ed079ca8a9f7","Type":"ContainerDied","Data":"c8af72a0202096e1107dda5cfb3a2e59582318b7ffd9ee768743cd1176f37c49"}
Jul 28 17:42:14.761024 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:14.761008    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1scs6r" event={"ID":"3f67ec8b-7142-4b63-9df8-ed079ca8a9f7","Type":"ContainerStarted","Data":"d1e104bbcc591ab1e6ad119617e06aa1db1f08209ff071835fcd93dfa29361a8"}
Jul 28 17:42:14.768724 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:14.768696    2563 generic.go:356] "Generic (PLEG): container finished" podID="fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa" containerID="5dd96aa522873cea9a288d57156527ad1e4278912b42d61d349c57ee47de3597" exitCode=0
Jul 28 17:42:14.768831 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:14.768780    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pnv6x" event={"ID":"fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa","Type":"ContainerDied","Data":"5dd96aa522873cea9a288d57156527ad1e4278912b42d61d349c57ee47de3597"}
Jul 28 17:42:14.770410 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:14.770374    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nvggl" event={"ID":"cd6773b9-1881-431a-bd08-757ec5e03c00","Type":"ContainerStarted","Data":"cb8bd706d173a3b62119781fb89e3bdf70190b26b08d03c539a9d6de0122113c"}
Jul 28 17:42:14.770537 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:14.770426    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nvggl" event={"ID":"cd6773b9-1881-431a-bd08-757ec5e03c00","Type":"ContainerStarted","Data":"11d1ae81c1edaf4d6f0b302d425ead85fb22d8b28a81a73e4ea0c040da02f3bf"}
Jul 28 17:42:14.890698 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:14.890667    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlgc2"
Jul 28 17:42:15.019417 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:15.019277    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlgc2"]
Jul 28 17:42:15.021336 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:42:15.021313    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod010eb983_47d5_4c20_a9d4_f8f27c32d136.slice/crio-6df549901e9e823ee3529ce82642c6499825711c42098074928dc5441dd13b9f WatchSource:0}: Error finding container 6df549901e9e823ee3529ce82642c6499825711c42098074928dc5441dd13b9f: Status 404 returned error can't find the container with id 6df549901e9e823ee3529ce82642c6499825711c42098074928dc5441dd13b9f
Jul 28 17:42:15.777338 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:15.777260    2563 generic.go:356] "Generic (PLEG): container finished" podID="fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa" containerID="32c6d08c23307bba159542bdcc67bd2ba50adec35cdbf7e33036a79bd5297ab7" exitCode=0
Jul 28 17:42:15.777759 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:15.777341    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pnv6x" event={"ID":"fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa","Type":"ContainerDied","Data":"32c6d08c23307bba159542bdcc67bd2ba50adec35cdbf7e33036a79bd5297ab7"}
Jul 28 17:42:15.778996 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:15.778975    2563 generic.go:356] "Generic (PLEG): container finished" podID="cd6773b9-1881-431a-bd08-757ec5e03c00" containerID="cb8bd706d173a3b62119781fb89e3bdf70190b26b08d03c539a9d6de0122113c" exitCode=0
Jul 28 17:42:15.778996 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:15.778994    2563 generic.go:356] "Generic (PLEG): container finished" podID="cd6773b9-1881-431a-bd08-757ec5e03c00" containerID="a3a1f96e78ecdcf6cde6af83d4c867c107d56445ddfdf46a0e4768f27bffb427" exitCode=0
Jul 28 17:42:15.779137 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:15.779060    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nvggl" event={"ID":"cd6773b9-1881-431a-bd08-757ec5e03c00","Type":"ContainerDied","Data":"cb8bd706d173a3b62119781fb89e3bdf70190b26b08d03c539a9d6de0122113c"}
Jul 28 17:42:15.779137 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:15.779099    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nvggl" event={"ID":"cd6773b9-1881-431a-bd08-757ec5e03c00","Type":"ContainerDied","Data":"a3a1f96e78ecdcf6cde6af83d4c867c107d56445ddfdf46a0e4768f27bffb427"}
Jul 28 17:42:15.780702 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:15.780596    2563 generic.go:356] "Generic (PLEG): container finished" podID="010eb983-47d5-4c20-a9d4-f8f27c32d136" containerID="d0e704afd9b60d51fcf1f0aa07dd4c2e3795df15f94e77f45993a03319ea6b72" exitCode=0
Jul 28 17:42:15.780702 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:15.780671    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlgc2" event={"ID":"010eb983-47d5-4c20-a9d4-f8f27c32d136","Type":"ContainerDied","Data":"d0e704afd9b60d51fcf1f0aa07dd4c2e3795df15f94e77f45993a03319ea6b72"}
Jul 28 17:42:15.780810 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:15.780713    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlgc2" event={"ID":"010eb983-47d5-4c20-a9d4-f8f27c32d136","Type":"ContainerStarted","Data":"6df549901e9e823ee3529ce82642c6499825711c42098074928dc5441dd13b9f"}
Jul 28 17:42:15.782620 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:15.782588    2563 generic.go:356] "Generic (PLEG): container finished" podID="3f67ec8b-7142-4b63-9df8-ed079ca8a9f7" containerID="1a27f17c7c95c1a61ac0bc2159f3912f468df1a073db7935c455411d8aa77bc1" exitCode=0
Jul 28 17:42:15.782723 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:15.782640    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1scs6r" event={"ID":"3f67ec8b-7142-4b63-9df8-ed079ca8a9f7","Type":"ContainerDied","Data":"1a27f17c7c95c1a61ac0bc2159f3912f468df1a073db7935c455411d8aa77bc1"}
Jul 28 17:42:16.790575 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:16.790482    2563 generic.go:356] "Generic (PLEG): container finished" podID="010eb983-47d5-4c20-a9d4-f8f27c32d136" containerID="8e8d18023310fe9d37a9300ae6bd06fca300c8d2bcab20b12a439fa807cfa7c3" exitCode=0
Jul 28 17:42:16.791033 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:16.790572    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlgc2" event={"ID":"010eb983-47d5-4c20-a9d4-f8f27c32d136","Type":"ContainerDied","Data":"8e8d18023310fe9d37a9300ae6bd06fca300c8d2bcab20b12a439fa807cfa7c3"}
Jul 28 17:42:16.792714 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:16.792693    2563 generic.go:356] "Generic (PLEG): container finished" podID="3f67ec8b-7142-4b63-9df8-ed079ca8a9f7" containerID="be53e163cdb359ad532aaecfe1f2542c72cc101a57c8b2d17ecaf7c30d56a3df" exitCode=0
Jul 28 17:42:16.792810 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:16.792777    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1scs6r" event={"ID":"3f67ec8b-7142-4b63-9df8-ed079ca8a9f7","Type":"ContainerDied","Data":"be53e163cdb359ad532aaecfe1f2542c72cc101a57c8b2d17ecaf7c30d56a3df"}
Jul 28 17:42:16.794747 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:16.794729    2563 generic.go:356] "Generic (PLEG): container finished" podID="cd6773b9-1881-431a-bd08-757ec5e03c00" containerID="2ee853aba6a419f07e4bd5b0729c0bb98ed0a51959736d2145a8262f2547070b" exitCode=0
Jul 28 17:42:16.794850 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:16.794807    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nvggl" event={"ID":"cd6773b9-1881-431a-bd08-757ec5e03c00","Type":"ContainerDied","Data":"2ee853aba6a419f07e4bd5b0729c0bb98ed0a51959736d2145a8262f2547070b"}
Jul 28 17:42:16.929256 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:16.929230    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pnv6x"
Jul 28 17:42:17.037699 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:17.037671    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa-bundle\") pod \"fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa\" (UID: \"fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa\") "
Jul 28 17:42:17.037842 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:17.037735    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa-util\") pod \"fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa\" (UID: \"fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa\") "
Jul 28 17:42:17.037842 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:17.037770    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c5zrp\" (UniqueName: \"kubernetes.io/projected/fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa-kube-api-access-c5zrp\") pod \"fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa\" (UID: \"fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa\") "
Jul 28 17:42:17.038190 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:17.038164    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa-bundle" (OuterVolumeSpecName: "bundle") pod "fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa" (UID: "fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 17:42:17.039941 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:17.039918    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa-kube-api-access-c5zrp" (OuterVolumeSpecName: "kube-api-access-c5zrp") pod "fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa" (UID: "fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa"). InnerVolumeSpecName "kube-api-access-c5zrp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:42:17.043106 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:17.043063    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa-util" (OuterVolumeSpecName: "util") pod "fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa" (UID: "fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 17:42:17.138512 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:17.138479    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa-bundle\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:42:17.138512 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:17.138506    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa-util\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:42:17.138512 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:17.138515    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c5zrp\" (UniqueName: \"kubernetes.io/projected/fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa-kube-api-access-c5zrp\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:42:17.801898 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:17.801822    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pnv6x"
Jul 28 17:42:17.801898 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:17.801858    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pnv6x" event={"ID":"fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa","Type":"ContainerDied","Data":"cefbbdb47aaca46641c17affe988c09e9045e15cda9d0e4d1c2d1c46af3ac3bf"}
Jul 28 17:42:17.802329 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:17.801904    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cefbbdb47aaca46641c17affe988c09e9045e15cda9d0e4d1c2d1c46af3ac3bf"
Jul 28 17:42:17.804452 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:17.804394    2563 generic.go:356] "Generic (PLEG): container finished" podID="010eb983-47d5-4c20-a9d4-f8f27c32d136" containerID="84531ab503e563983d9f96b5a1d33f4cc31aa298458fefae62fa2a5d0ba8db58" exitCode=0
Jul 28 17:42:17.804967 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:17.804932    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlgc2" event={"ID":"010eb983-47d5-4c20-a9d4-f8f27c32d136","Type":"ContainerDied","Data":"84531ab503e563983d9f96b5a1d33f4cc31aa298458fefae62fa2a5d0ba8db58"}
Jul 28 17:42:17.960371 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:17.960351    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nvggl"
Jul 28 17:42:17.963102 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:17.963082    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1scs6r"
Jul 28 17:42:18.045184 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:18.045153    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rhgfp\" (UniqueName: \"kubernetes.io/projected/3f67ec8b-7142-4b63-9df8-ed079ca8a9f7-kube-api-access-rhgfp\") pod \"3f67ec8b-7142-4b63-9df8-ed079ca8a9f7\" (UID: \"3f67ec8b-7142-4b63-9df8-ed079ca8a9f7\") "
Jul 28 17:42:18.045184 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:18.045190    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zskwf\" (UniqueName: \"kubernetes.io/projected/cd6773b9-1881-431a-bd08-757ec5e03c00-kube-api-access-zskwf\") pod \"cd6773b9-1881-431a-bd08-757ec5e03c00\" (UID: \"cd6773b9-1881-431a-bd08-757ec5e03c00\") "
Jul 28 17:42:18.045418 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:18.045212    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cd6773b9-1881-431a-bd08-757ec5e03c00-bundle\") pod \"cd6773b9-1881-431a-bd08-757ec5e03c00\" (UID: \"cd6773b9-1881-431a-bd08-757ec5e03c00\") "
Jul 28 17:42:18.045418 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:18.045245    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cd6773b9-1881-431a-bd08-757ec5e03c00-util\") pod \"cd6773b9-1881-431a-bd08-757ec5e03c00\" (UID: \"cd6773b9-1881-431a-bd08-757ec5e03c00\") "
Jul 28 17:42:18.045418 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:18.045269    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3f67ec8b-7142-4b63-9df8-ed079ca8a9f7-bundle\") pod \"3f67ec8b-7142-4b63-9df8-ed079ca8a9f7\" (UID: \"3f67ec8b-7142-4b63-9df8-ed079ca8a9f7\") "
Jul 28 17:42:18.045418 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:18.045292    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3f67ec8b-7142-4b63-9df8-ed079ca8a9f7-util\") pod \"3f67ec8b-7142-4b63-9df8-ed079ca8a9f7\" (UID: \"3f67ec8b-7142-4b63-9df8-ed079ca8a9f7\") "
Jul 28 17:42:18.045884 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:18.045838    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd6773b9-1881-431a-bd08-757ec5e03c00-bundle" (OuterVolumeSpecName: "bundle") pod "cd6773b9-1881-431a-bd08-757ec5e03c00" (UID: "cd6773b9-1881-431a-bd08-757ec5e03c00"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 17:42:18.045884 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:18.045844    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f67ec8b-7142-4b63-9df8-ed079ca8a9f7-bundle" (OuterVolumeSpecName: "bundle") pod "3f67ec8b-7142-4b63-9df8-ed079ca8a9f7" (UID: "3f67ec8b-7142-4b63-9df8-ed079ca8a9f7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 17:42:18.047372 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:18.047347    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd6773b9-1881-431a-bd08-757ec5e03c00-kube-api-access-zskwf" (OuterVolumeSpecName: "kube-api-access-zskwf") pod "cd6773b9-1881-431a-bd08-757ec5e03c00" (UID: "cd6773b9-1881-431a-bd08-757ec5e03c00"). InnerVolumeSpecName "kube-api-access-zskwf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:42:18.047818 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:18.047799    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f67ec8b-7142-4b63-9df8-ed079ca8a9f7-kube-api-access-rhgfp" (OuterVolumeSpecName: "kube-api-access-rhgfp") pod "3f67ec8b-7142-4b63-9df8-ed079ca8a9f7" (UID: "3f67ec8b-7142-4b63-9df8-ed079ca8a9f7"). InnerVolumeSpecName "kube-api-access-rhgfp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:42:18.051351 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:18.051316    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd6773b9-1881-431a-bd08-757ec5e03c00-util" (OuterVolumeSpecName: "util") pod "cd6773b9-1881-431a-bd08-757ec5e03c00" (UID: "cd6773b9-1881-431a-bd08-757ec5e03c00"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 17:42:18.052196 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:18.052136    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f67ec8b-7142-4b63-9df8-ed079ca8a9f7-util" (OuterVolumeSpecName: "util") pod "3f67ec8b-7142-4b63-9df8-ed079ca8a9f7" (UID: "3f67ec8b-7142-4b63-9df8-ed079ca8a9f7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 17:42:18.146368 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:18.146331    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rhgfp\" (UniqueName: \"kubernetes.io/projected/3f67ec8b-7142-4b63-9df8-ed079ca8a9f7-kube-api-access-rhgfp\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:42:18.146368 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:18.146362    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zskwf\" (UniqueName: \"kubernetes.io/projected/cd6773b9-1881-431a-bd08-757ec5e03c00-kube-api-access-zskwf\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:42:18.146368 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:18.146373    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cd6773b9-1881-431a-bd08-757ec5e03c00-bundle\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:42:18.146608 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:18.146382    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cd6773b9-1881-431a-bd08-757ec5e03c00-util\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:42:18.146608 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:18.146392    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3f67ec8b-7142-4b63-9df8-ed079ca8a9f7-bundle\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:42:18.146608 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:18.146398    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3f67ec8b-7142-4b63-9df8-ed079ca8a9f7-util\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:42:18.811330 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:18.811293    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1scs6r"
Jul 28 17:42:18.811330 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:18.811307    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1scs6r" event={"ID":"3f67ec8b-7142-4b63-9df8-ed079ca8a9f7","Type":"ContainerDied","Data":"d1e104bbcc591ab1e6ad119617e06aa1db1f08209ff071835fcd93dfa29361a8"}
Jul 28 17:42:18.811817 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:18.811351    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d1e104bbcc591ab1e6ad119617e06aa1db1f08209ff071835fcd93dfa29361a8"
Jul 28 17:42:18.813282 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:18.813258    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nvggl" event={"ID":"cd6773b9-1881-431a-bd08-757ec5e03c00","Type":"ContainerDied","Data":"11d1ae81c1edaf4d6f0b302d425ead85fb22d8b28a81a73e4ea0c040da02f3bf"}
Jul 28 17:42:18.813405 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:18.813301    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="11d1ae81c1edaf4d6f0b302d425ead85fb22d8b28a81a73e4ea0c040da02f3bf"
Jul 28 17:42:18.813467 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:18.813405    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nvggl"
Jul 28 17:42:18.936684 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:18.936663    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlgc2"
Jul 28 17:42:19.053881 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:19.053859    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/010eb983-47d5-4c20-a9d4-f8f27c32d136-bundle\") pod \"010eb983-47d5-4c20-a9d4-f8f27c32d136\" (UID: \"010eb983-47d5-4c20-a9d4-f8f27c32d136\") "
Jul 28 17:42:19.054020 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:19.053887    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qzmv2\" (UniqueName: \"kubernetes.io/projected/010eb983-47d5-4c20-a9d4-f8f27c32d136-kube-api-access-qzmv2\") pod \"010eb983-47d5-4c20-a9d4-f8f27c32d136\" (UID: \"010eb983-47d5-4c20-a9d4-f8f27c32d136\") "
Jul 28 17:42:19.054020 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:19.053928    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/010eb983-47d5-4c20-a9d4-f8f27c32d136-util\") pod \"010eb983-47d5-4c20-a9d4-f8f27c32d136\" (UID: \"010eb983-47d5-4c20-a9d4-f8f27c32d136\") "
Jul 28 17:42:19.054429 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:19.054401    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/010eb983-47d5-4c20-a9d4-f8f27c32d136-bundle" (OuterVolumeSpecName: "bundle") pod "010eb983-47d5-4c20-a9d4-f8f27c32d136" (UID: "010eb983-47d5-4c20-a9d4-f8f27c32d136"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 17:42:19.055805 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:19.055778    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/010eb983-47d5-4c20-a9d4-f8f27c32d136-kube-api-access-qzmv2" (OuterVolumeSpecName: "kube-api-access-qzmv2") pod "010eb983-47d5-4c20-a9d4-f8f27c32d136" (UID: "010eb983-47d5-4c20-a9d4-f8f27c32d136"). InnerVolumeSpecName "kube-api-access-qzmv2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:42:19.059188 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:19.059154    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/010eb983-47d5-4c20-a9d4-f8f27c32d136-util" (OuterVolumeSpecName: "util") pod "010eb983-47d5-4c20-a9d4-f8f27c32d136" (UID: "010eb983-47d5-4c20-a9d4-f8f27c32d136"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 17:42:19.154929 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:19.154903    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/010eb983-47d5-4c20-a9d4-f8f27c32d136-bundle\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:42:19.154929 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:19.154926    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qzmv2\" (UniqueName: \"kubernetes.io/projected/010eb983-47d5-4c20-a9d4-f8f27c32d136-kube-api-access-qzmv2\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:42:19.154929 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:19.154935    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/010eb983-47d5-4c20-a9d4-f8f27c32d136-util\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:42:19.819298 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:19.819264    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlgc2" event={"ID":"010eb983-47d5-4c20-a9d4-f8f27c32d136","Type":"ContainerDied","Data":"6df549901e9e823ee3529ce82642c6499825711c42098074928dc5441dd13b9f"}
Jul 28 17:42:19.819703 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:19.819312    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6df549901e9e823ee3529ce82642c6499825711c42098074928dc5441dd13b9f"
Jul 28 17:42:19.819703 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:19.819310    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlgc2"
Jul 28 17:42:21.300393 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.300362    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-669956765d-fgv4m"]
Jul 28 17:42:21.300746 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.300672    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd6773b9-1881-431a-bd08-757ec5e03c00" containerName="pull"
Jul 28 17:42:21.300746 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.300683    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd6773b9-1881-431a-bd08-757ec5e03c00" containerName="pull"
Jul 28 17:42:21.300746 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.300692    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="010eb983-47d5-4c20-a9d4-f8f27c32d136" containerName="pull"
Jul 28 17:42:21.300746 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.300697    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="010eb983-47d5-4c20-a9d4-f8f27c32d136" containerName="pull"
Jul 28 17:42:21.300746 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.300706    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f67ec8b-7142-4b63-9df8-ed079ca8a9f7" containerName="util"
Jul 28 17:42:21.300746 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.300712    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f67ec8b-7142-4b63-9df8-ed079ca8a9f7" containerName="util"
Jul 28 17:42:21.300746 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.300719    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd6773b9-1881-431a-bd08-757ec5e03c00" containerName="extract"
Jul 28 17:42:21.300746 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.300724    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd6773b9-1881-431a-bd08-757ec5e03c00" containerName="extract"
Jul 28 17:42:21.300746 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.300732    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="010eb983-47d5-4c20-a9d4-f8f27c32d136" containerName="extract"
Jul 28 17:42:21.300746 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.300737    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="010eb983-47d5-4c20-a9d4-f8f27c32d136" containerName="extract"
Jul 28 17:42:21.300746 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.300746    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd6773b9-1881-431a-bd08-757ec5e03c00" containerName="util"
Jul 28 17:42:21.300746 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.300751    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd6773b9-1881-431a-bd08-757ec5e03c00" containerName="util"
Jul 28 17:42:21.301122 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.300757    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="010eb983-47d5-4c20-a9d4-f8f27c32d136" containerName="util"
Jul 28 17:42:21.301122 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.300762    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="010eb983-47d5-4c20-a9d4-f8f27c32d136" containerName="util"
Jul 28 17:42:21.301122 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.300768    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f67ec8b-7142-4b63-9df8-ed079ca8a9f7" containerName="extract"
Jul 28 17:42:21.301122 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.300773    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f67ec8b-7142-4b63-9df8-ed079ca8a9f7" containerName="extract"
Jul 28 17:42:21.301122 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.300781    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa" containerName="pull"
Jul 28 17:42:21.301122 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.300785    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa" containerName="pull"
Jul 28 17:42:21.301122 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.300792    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa" containerName="util"
Jul 28 17:42:21.301122 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.300799    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa" containerName="util"
Jul 28 17:42:21.301122 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.300804    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa" containerName="extract"
Jul 28 17:42:21.301122 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.300809    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa" containerName="extract"
Jul 28 17:42:21.301122 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.300818    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f67ec8b-7142-4b63-9df8-ed079ca8a9f7" containerName="pull"
Jul 28 17:42:21.301122 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.300823    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f67ec8b-7142-4b63-9df8-ed079ca8a9f7" containerName="pull"
Jul 28 17:42:21.301122 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.300870    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd6773b9-1881-431a-bd08-757ec5e03c00" containerName="extract"
Jul 28 17:42:21.301122 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.300878    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f67ec8b-7142-4b63-9df8-ed079ca8a9f7" containerName="extract"
Jul 28 17:42:21.301122 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.300883    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="010eb983-47d5-4c20-a9d4-f8f27c32d136" containerName="extract"
Jul 28 17:42:21.301122 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.300889    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa" containerName="extract"
Jul 28 17:42:21.302927 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.302907    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-669956765d-fgv4m"
Jul 28 17:42:21.321161 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.321136    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-669956765d-fgv4m"]
Jul 28 17:42:21.368171 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.368146    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:42:21.370732 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.370716    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ecf6fe5c-8ae5-458e-9964-e5974adb0c83-console-oauth-config\") pod \"console-669956765d-fgv4m\" (UID: \"ecf6fe5c-8ae5-458e-9964-e5974adb0c83\") " pod="openshift-console/console-669956765d-fgv4m"
Jul 28 17:42:21.370784 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.370755    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ecf6fe5c-8ae5-458e-9964-e5974adb0c83-console-serving-cert\") pod \"console-669956765d-fgv4m\" (UID: \"ecf6fe5c-8ae5-458e-9964-e5974adb0c83\") " pod="openshift-console/console-669956765d-fgv4m"
Jul 28 17:42:21.370784 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.370775    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ecf6fe5c-8ae5-458e-9964-e5974adb0c83-trusted-ca-bundle\") pod \"console-669956765d-fgv4m\" (UID: \"ecf6fe5c-8ae5-458e-9964-e5974adb0c83\") " pod="openshift-console/console-669956765d-fgv4m"
Jul 28 17:42:21.370867 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.370846    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ecf6fe5c-8ae5-458e-9964-e5974adb0c83-console-config\") pod \"console-669956765d-fgv4m\" (UID: \"ecf6fe5c-8ae5-458e-9964-e5974adb0c83\") " pod="openshift-console/console-669956765d-fgv4m"
Jul 28 17:42:21.370904 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.370888    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ecf6fe5c-8ae5-458e-9964-e5974adb0c83-service-ca\") pod \"console-669956765d-fgv4m\" (UID: \"ecf6fe5c-8ae5-458e-9964-e5974adb0c83\") " pod="openshift-console/console-669956765d-fgv4m"
Jul 28 17:42:21.370937 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.370919    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ecf6fe5c-8ae5-458e-9964-e5974adb0c83-oauth-serving-cert\") pod \"console-669956765d-fgv4m\" (UID: \"ecf6fe5c-8ae5-458e-9964-e5974adb0c83\") " pod="openshift-console/console-669956765d-fgv4m"
Jul 28 17:42:21.370973 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.370936    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdfht\" (UniqueName: \"kubernetes.io/projected/ecf6fe5c-8ae5-458e-9964-e5974adb0c83-kube-api-access-jdfht\") pod \"console-669956765d-fgv4m\" (UID: \"ecf6fe5c-8ae5-458e-9964-e5974adb0c83\") " pod="openshift-console/console-669956765d-fgv4m"
Jul 28 17:42:21.471339 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.471310    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:42:21.471501 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.471312    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ecf6fe5c-8ae5-458e-9964-e5974adb0c83-console-oauth-config\") pod \"console-669956765d-fgv4m\" (UID: \"ecf6fe5c-8ae5-458e-9964-e5974adb0c83\") " pod="openshift-console/console-669956765d-fgv4m"
Jul 28 17:42:21.471501 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.471400    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ecf6fe5c-8ae5-458e-9964-e5974adb0c83-console-serving-cert\") pod \"console-669956765d-fgv4m\" (UID: \"ecf6fe5c-8ae5-458e-9964-e5974adb0c83\") " pod="openshift-console/console-669956765d-fgv4m"
Jul 28 17:42:21.471501 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.471421    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ecf6fe5c-8ae5-458e-9964-e5974adb0c83-trusted-ca-bundle\") pod \"console-669956765d-fgv4m\" (UID: \"ecf6fe5c-8ae5-458e-9964-e5974adb0c83\") " pod="openshift-console/console-669956765d-fgv4m"
Jul 28 17:42:21.471501 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.471460    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ecf6fe5c-8ae5-458e-9964-e5974adb0c83-console-config\") pod \"console-669956765d-fgv4m\" (UID: \"ecf6fe5c-8ae5-458e-9964-e5974adb0c83\") " pod="openshift-console/console-669956765d-fgv4m"
Jul 28 17:42:21.471501 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.471498    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ecf6fe5c-8ae5-458e-9964-e5974adb0c83-service-ca\") pod \"console-669956765d-fgv4m\" (UID: \"ecf6fe5c-8ae5-458e-9964-e5974adb0c83\") " pod="openshift-console/console-669956765d-fgv4m"
Jul 28 17:42:21.471745 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.471531    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ecf6fe5c-8ae5-458e-9964-e5974adb0c83-oauth-serving-cert\") pod \"console-669956765d-fgv4m\" (UID: \"ecf6fe5c-8ae5-458e-9964-e5974adb0c83\") " pod="openshift-console/console-669956765d-fgv4m"
Jul 28 17:42:21.471745 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.471547    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdfht\" (UniqueName: \"kubernetes.io/projected/ecf6fe5c-8ae5-458e-9964-e5974adb0c83-kube-api-access-jdfht\") pod \"console-669956765d-fgv4m\" (UID: \"ecf6fe5c-8ae5-458e-9964-e5974adb0c83\") " pod="openshift-console/console-669956765d-fgv4m"
Jul 28 17:42:21.472204 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.472182    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ecf6fe5c-8ae5-458e-9964-e5974adb0c83-console-config\") pod \"console-669956765d-fgv4m\" (UID: \"ecf6fe5c-8ae5-458e-9964-e5974adb0c83\") " pod="openshift-console/console-669956765d-fgv4m"
Jul 28 17:42:21.472361 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.472273    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ecf6fe5c-8ae5-458e-9964-e5974adb0c83-oauth-serving-cert\") pod \"console-669956765d-fgv4m\" (UID: \"ecf6fe5c-8ae5-458e-9964-e5974adb0c83\") " pod="openshift-console/console-669956765d-fgv4m"
Jul 28 17:42:21.472361 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.472285    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ecf6fe5c-8ae5-458e-9964-e5974adb0c83-trusted-ca-bundle\") pod \"console-669956765d-fgv4m\" (UID: \"ecf6fe5c-8ae5-458e-9964-e5974adb0c83\") " pod="openshift-console/console-669956765d-fgv4m"
Jul 28 17:42:21.472499 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.472358    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ecf6fe5c-8ae5-458e-9964-e5974adb0c83-service-ca\") pod \"console-669956765d-fgv4m\" (UID: \"ecf6fe5c-8ae5-458e-9964-e5974adb0c83\") " pod="openshift-console/console-669956765d-fgv4m"
Jul 28 17:42:21.473831 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.473808    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ecf6fe5c-8ae5-458e-9964-e5974adb0c83-console-serving-cert\") pod \"console-669956765d-fgv4m\" (UID: \"ecf6fe5c-8ae5-458e-9964-e5974adb0c83\") " pod="openshift-console/console-669956765d-fgv4m"
Jul 28 17:42:21.473831 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.473823    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ecf6fe5c-8ae5-458e-9964-e5974adb0c83-console-oauth-config\") pod \"console-669956765d-fgv4m\" (UID: \"ecf6fe5c-8ae5-458e-9964-e5974adb0c83\") " pod="openshift-console/console-669956765d-fgv4m"
Jul 28 17:42:21.485699 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.485651    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdfht\" (UniqueName: \"kubernetes.io/projected/ecf6fe5c-8ae5-458e-9964-e5974adb0c83-kube-api-access-jdfht\") pod \"console-669956765d-fgv4m\" (UID: \"ecf6fe5c-8ae5-458e-9964-e5974adb0c83\") " pod="openshift-console/console-669956765d-fgv4m"
Jul 28 17:42:21.568573 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.568541    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:42:21.612631 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.612599    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-669956765d-fgv4m"
Jul 28 17:42:21.744861 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.744835    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-669956765d-fgv4m"]
Jul 28 17:42:21.746293 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:42:21.746262    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecf6fe5c_8ae5_458e_9964_e5974adb0c83.slice/crio-6f9bf053b078d55c000f977605b3804e082a9716444f8f45d8f11fb6cceb594c WatchSource:0}: Error finding container 6f9bf053b078d55c000f977605b3804e082a9716444f8f45d8f11fb6cceb594c: Status 404 returned error can't find the container with id 6f9bf053b078d55c000f977605b3804e082a9716444f8f45d8f11fb6cceb594c
Jul 28 17:42:21.832677 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.832608    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-669956765d-fgv4m" event={"ID":"ecf6fe5c-8ae5-458e-9964-e5974adb0c83","Type":"ContainerStarted","Data":"ab7356e1fcf6f0dba1dfcbc2f8c1eb59f3d682fbb6c2d3dccdf96a4c85f5f028"}
Jul 28 17:42:21.832677 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.832672    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-669956765d-fgv4m" event={"ID":"ecf6fe5c-8ae5-458e-9964-e5974adb0c83","Type":"ContainerStarted","Data":"6f9bf053b078d55c000f977605b3804e082a9716444f8f45d8f11fb6cceb594c"}
Jul 28 17:42:21.854804 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:21.854743    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-669956765d-fgv4m" podStartSLOduration=0.854723654 podStartE2EDuration="854.723654ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:42:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:42:21.851401785 +0000 UTC m=+480.512558141" watchObservedRunningTime="2026-07-28 17:42:21.854723654 +0000 UTC m=+480.515880009"
Jul 28 17:42:25.453060 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:25.453023    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:42:27.569162 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:27.569129    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-w487q"]
Jul 28 17:42:27.571551 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:27.571536    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-w487q"
Jul 28 17:42:27.574160 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:27.574138    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 28 17:42:27.574280 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:27.574184    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 28 17:42:27.575234 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:27.575218    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-kjbhp\""
Jul 28 17:42:27.575320 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:27.575223    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 28 17:42:27.582600 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:27.582582    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-w487q"]
Jul 28 17:42:27.723150 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:27.723127    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdzvl\" (UniqueName: \"kubernetes.io/projected/f6365e9a-e607-4e3a-b0f9-d7a904d120d6-kube-api-access-mdzvl\") pod \"dns-operator-controller-manager-864dd78c4c-w487q\" (UID: \"f6365e9a-e607-4e3a-b0f9-d7a904d120d6\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-w487q"
Jul 28 17:42:27.824398 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:27.824337    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdzvl\" (UniqueName: \"kubernetes.io/projected/f6365e9a-e607-4e3a-b0f9-d7a904d120d6-kube-api-access-mdzvl\") pod \"dns-operator-controller-manager-864dd78c4c-w487q\" (UID: \"f6365e9a-e607-4e3a-b0f9-d7a904d120d6\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-w487q"
Jul 28 17:42:27.836943 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:27.836916    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdzvl\" (UniqueName: \"kubernetes.io/projected/f6365e9a-e607-4e3a-b0f9-d7a904d120d6-kube-api-access-mdzvl\") pod \"dns-operator-controller-manager-864dd78c4c-w487q\" (UID: \"f6365e9a-e607-4e3a-b0f9-d7a904d120d6\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-w487q"
Jul 28 17:42:27.881908 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:27.881888    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-w487q"
Jul 28 17:42:28.015213 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:28.015185    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-w487q"]
Jul 28 17:42:28.017346 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:42:28.017319    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6365e9a_e607_4e3a_b0f9_d7a904d120d6.slice/crio-d90bf3dd6315a835c1a7efe10d56562df9ca732896cced0b6a2b65b14178f0cc WatchSource:0}: Error finding container d90bf3dd6315a835c1a7efe10d56562df9ca732896cced0b6a2b65b14178f0cc: Status 404 returned error can't find the container with id d90bf3dd6315a835c1a7efe10d56562df9ca732896cced0b6a2b65b14178f0cc
Jul 28 17:42:28.051225 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:28.051198    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:42:28.524707 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:28.524679    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:42:28.685942 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:28.685912    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:42:28.821217 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:28.821136    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:42:28.868356 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:28.868322    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-w487q" event={"ID":"f6365e9a-e607-4e3a-b0f9-d7a904d120d6","Type":"ContainerStarted","Data":"d90bf3dd6315a835c1a7efe10d56562df9ca732896cced0b6a2b65b14178f0cc"}
Jul 28 17:42:30.293183 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:30.293143    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:42:30.878659 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:30.878614    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-w487q" event={"ID":"f6365e9a-e607-4e3a-b0f9-d7a904d120d6","Type":"ContainerStarted","Data":"4b04132bead317fd4cc0c6546f81129c72f45e4415177922ce70d4cc0956abf0"}
Jul 28 17:42:30.878823 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:30.878718    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-w487q"
Jul 28 17:42:30.901023 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:30.900976    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-w487q" podStartSLOduration=1.581361454 podStartE2EDuration="3.900962651s" totalImagesPullingTime="2.319601197s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:42:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:42:30.898671376 +0000 UTC m=+489.559827732" watchObservedRunningTime="2026-07-28 17:42:30.900962651 +0000 UTC m=+489.562119005"
Jul 28 17:42:31.613564 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:31.613528    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-669956765d-fgv4m"
Jul 28 17:42:31.613911 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:31.613613    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-669956765d-fgv4m"
Jul 28 17:42:31.618308 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:31.618288    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-669956765d-fgv4m"
Jul 28 17:42:31.887604 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:31.887520    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-669956765d-fgv4m"
Jul 28 17:42:31.993314 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:31.993284    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-75c6f46c7d-rfnq9"]
Jul 28 17:42:32.092548 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:32.092522    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:42:32.187371 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:32.187293    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:42:32.288407 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:32.288376    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:42:32.388244 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:32.388215    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:42:32.494591 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:32.494504    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:42:32.701275 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:32.701244    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:42:36.143545 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:36.143511    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:42:36.707443 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:36.707398    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-skbcf"]
Jul 28 17:42:36.710013 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:36.709995    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-skbcf"
Jul 28 17:42:36.714555 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:36.714538    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-z9zdn\""
Jul 28 17:42:36.725649 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:36.725626    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-skbcf"]
Jul 28 17:42:36.803417 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:36.803385    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/06bef2e5-7a67-4d73-aeb9-92190c75e0fd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-skbcf\" (UID: \"06bef2e5-7a67-4d73-aeb9-92190c75e0fd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-skbcf"
Jul 28 17:42:36.803578 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:36.803465    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95897\" (UniqueName: \"kubernetes.io/projected/06bef2e5-7a67-4d73-aeb9-92190c75e0fd-kube-api-access-95897\") pod \"kuadrant-operator-controller-manager-8684c9ff49-skbcf\" (UID: \"06bef2e5-7a67-4d73-aeb9-92190c75e0fd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-skbcf"
Jul 28 17:42:36.904065 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:36.904033    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/06bef2e5-7a67-4d73-aeb9-92190c75e0fd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-skbcf\" (UID: \"06bef2e5-7a67-4d73-aeb9-92190c75e0fd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-skbcf"
Jul 28 17:42:36.904249 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:36.904087    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95897\" (UniqueName: \"kubernetes.io/projected/06bef2e5-7a67-4d73-aeb9-92190c75e0fd-kube-api-access-95897\") pod \"kuadrant-operator-controller-manager-8684c9ff49-skbcf\" (UID: \"06bef2e5-7a67-4d73-aeb9-92190c75e0fd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-skbcf"
Jul 28 17:42:36.904461 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:36.904417    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/06bef2e5-7a67-4d73-aeb9-92190c75e0fd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-skbcf\" (UID: \"06bef2e5-7a67-4d73-aeb9-92190c75e0fd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-skbcf"
Jul 28 17:42:36.915524 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:36.915495    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-95897\" (UniqueName: \"kubernetes.io/projected/06bef2e5-7a67-4d73-aeb9-92190c75e0fd-kube-api-access-95897\") pod \"kuadrant-operator-controller-manager-8684c9ff49-skbcf\" (UID: \"06bef2e5-7a67-4d73-aeb9-92190c75e0fd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-skbcf"
Jul 28 17:42:37.020017 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:37.019938    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-skbcf"
Jul 28 17:42:37.157924 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:37.157892    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-skbcf"]
Jul 28 17:42:37.161403 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:42:37.161378    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06bef2e5_7a67_4d73_aeb9_92190c75e0fd.slice/crio-fed5025d50c7a18d1d5b32778212e2e732605266b56e448e6cc24f2267eead8f WatchSource:0}: Error finding container fed5025d50c7a18d1d5b32778212e2e732605266b56e448e6cc24f2267eead8f: Status 404 returned error can't find the container with id fed5025d50c7a18d1d5b32778212e2e732605266b56e448e6cc24f2267eead8f
Jul 28 17:42:37.912783 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:37.912749    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-skbcf" event={"ID":"06bef2e5-7a67-4d73-aeb9-92190c75e0fd","Type":"ContainerStarted","Data":"fed5025d50c7a18d1d5b32778212e2e732605266b56e448e6cc24f2267eead8f"}
Jul 28 17:42:38.065935 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:38.065904    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:42:38.167652 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:38.167569    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:42:40.466747 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:40.466707    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:42:40.570317 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:40.570285    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:42:41.885475 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:41.885421    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-w487q"
Jul 28 17:42:42.942234 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:42.942141    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-skbcf" event={"ID":"06bef2e5-7a67-4d73-aeb9-92190c75e0fd","Type":"ContainerStarted","Data":"f27d65718a9d6de74018688cca7ea834824e6aae7cd1c61c7f3dd09b07d6bef0"}
Jul 28 17:42:42.942234 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:42.942206    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-skbcf"
Jul 28 17:42:42.968570 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:42.968526    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-skbcf" podStartSLOduration=1.491337545 podStartE2EDuration="6.968510883s" totalImagesPullingTime="5.477173338s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:42:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:42:42.965641621 +0000 UTC m=+501.626797989" watchObservedRunningTime="2026-07-28 17:42:42.968510883 +0000 UTC m=+501.629667239"
Jul 28 17:42:45.407025 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:45.406990    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:42:45.975834 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:45.975803    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:42:46.090087 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:46.090059    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:42:46.193941 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:46.193913    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:42:46.296308 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:46.296244    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:42:46.392277 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:46.392252    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:42:46.493107 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:46.493078    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:42:50.029760 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:50.029729    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:42:50.127951 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:50.127924    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:42:53.949101 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:53.949074    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-skbcf"
Jul 28 17:42:55.949681 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:55.949651    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5lkm9"]
Jul 28 17:42:55.954731 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:55.954712    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5lkm9"
Jul 28 17:42:55.957977 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:55.957957    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-skbcf"]
Jul 28 17:42:55.958136 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:55.958113    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-skbcf" podUID="06bef2e5-7a67-4d73-aeb9-92190c75e0fd" containerName="manager" containerID="cri-o://f27d65718a9d6de74018688cca7ea834824e6aae7cd1c61c7f3dd09b07d6bef0" gracePeriod=2
Jul 28 17:42:55.971278 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:55.971253    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5lkm9"]
Jul 28 17:42:55.974691 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:55.974673    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-skbcf"]
Jul 28 17:42:55.988855 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:42:55.988827    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-skbcf\" is forbidden: User \"system:node:ip-10-0-136-126.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-126.ec2.internal' and this object" podUID="06bef2e5-7a67-4d73-aeb9-92190c75e0fd" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-skbcf"
Jul 28 17:42:56.023776 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:56.023753    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j56s6"]
Jul 28 17:42:56.024091 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:56.024077    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06bef2e5-7a67-4d73-aeb9-92190c75e0fd" containerName="manager"
Jul 28 17:42:56.024139 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:56.024092    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="06bef2e5-7a67-4d73-aeb9-92190c75e0fd" containerName="manager"
Jul 28 17:42:56.024182 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:56.024157    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="06bef2e5-7a67-4d73-aeb9-92190c75e0fd" containerName="manager"
Jul 28 17:42:56.027208 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:56.027190    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j56s6"
Jul 28 17:42:56.029703 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:42:56.029670    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-skbcf\" is forbidden: User \"system:node:ip-10-0-136-126.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-126.ec2.internal' and this object" podUID="06bef2e5-7a67-4d73-aeb9-92190c75e0fd" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-skbcf"
Jul 28 17:42:56.029849 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:56.029831    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-npwpf\""
Jul 28 17:42:56.040227 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:56.040206    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j56s6"]
Jul 28 17:42:56.067752 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:56.067725    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fd54\" (UniqueName: \"kubernetes.io/projected/0bb917a1-5d9d-4d4b-ae0e-3718aa32d030-kube-api-access-4fd54\") pod \"limitador-operator-controller-manager-859d7b78bd-j56s6\" (UID: \"0bb917a1-5d9d-4d4b-ae0e-3718aa32d030\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j56s6"
Jul 28 17:42:56.067860 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:56.067770    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2e6cbe21-6226-41a3-91a4-b120183e6662-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-5lkm9\" (UID: \"2e6cbe21-6226-41a3-91a4-b120183e6662\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5lkm9"
Jul 28 17:42:56.067860 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:56.067832    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdmv9\" (UniqueName: \"kubernetes.io/projected/2e6cbe21-6226-41a3-91a4-b120183e6662-kube-api-access-rdmv9\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-5lkm9\" (UID: \"2e6cbe21-6226-41a3-91a4-b120183e6662\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5lkm9"
Jul 28 17:42:56.168596 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:56.168569    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdmv9\" (UniqueName: \"kubernetes.io/projected/2e6cbe21-6226-41a3-91a4-b120183e6662-kube-api-access-rdmv9\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-5lkm9\" (UID: \"2e6cbe21-6226-41a3-91a4-b120183e6662\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5lkm9"
Jul 28 17:42:56.168742 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:56.168629    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fd54\" (UniqueName: \"kubernetes.io/projected/0bb917a1-5d9d-4d4b-ae0e-3718aa32d030-kube-api-access-4fd54\") pod \"limitador-operator-controller-manager-859d7b78bd-j56s6\" (UID: \"0bb917a1-5d9d-4d4b-ae0e-3718aa32d030\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j56s6"
Jul 28 17:42:56.168742 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:56.168665    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2e6cbe21-6226-41a3-91a4-b120183e6662-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-5lkm9\" (UID: \"2e6cbe21-6226-41a3-91a4-b120183e6662\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5lkm9"
Jul 28 17:42:56.168995 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:56.168977    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2e6cbe21-6226-41a3-91a4-b120183e6662-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-5lkm9\" (UID: \"2e6cbe21-6226-41a3-91a4-b120183e6662\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5lkm9"
Jul 28 17:42:56.179113 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:56.179093    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fd54\" (UniqueName: \"kubernetes.io/projected/0bb917a1-5d9d-4d4b-ae0e-3718aa32d030-kube-api-access-4fd54\") pod \"limitador-operator-controller-manager-859d7b78bd-j56s6\" (UID: \"0bb917a1-5d9d-4d4b-ae0e-3718aa32d030\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j56s6"
Jul 28 17:42:56.179225 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:56.179135    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdmv9\" (UniqueName: \"kubernetes.io/projected/2e6cbe21-6226-41a3-91a4-b120183e6662-kube-api-access-rdmv9\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-5lkm9\" (UID: \"2e6cbe21-6226-41a3-91a4-b120183e6662\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5lkm9"
Jul 28 17:42:56.198534 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:56.198515    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-skbcf"
Jul 28 17:42:56.200849 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:42:56.200793    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-skbcf\" is forbidden: User \"system:node:ip-10-0-136-126.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-126.ec2.internal' and this object" podUID="06bef2e5-7a67-4d73-aeb9-92190c75e0fd" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-skbcf"
Jul 28 17:42:56.265014 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:56.264983    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5lkm9"
Jul 28 17:42:56.269030 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:56.269011    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-95897\" (UniqueName: \"kubernetes.io/projected/06bef2e5-7a67-4d73-aeb9-92190c75e0fd-kube-api-access-95897\") pod \"06bef2e5-7a67-4d73-aeb9-92190c75e0fd\" (UID: \"06bef2e5-7a67-4d73-aeb9-92190c75e0fd\") "
Jul 28 17:42:56.269094 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:56.269076    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/06bef2e5-7a67-4d73-aeb9-92190c75e0fd-extensions-socket-volume\") pod \"06bef2e5-7a67-4d73-aeb9-92190c75e0fd\" (UID: \"06bef2e5-7a67-4d73-aeb9-92190c75e0fd\") "
Jul 28 17:42:56.269578 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:56.269546    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06bef2e5-7a67-4d73-aeb9-92190c75e0fd-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "06bef2e5-7a67-4d73-aeb9-92190c75e0fd" (UID: "06bef2e5-7a67-4d73-aeb9-92190c75e0fd"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 17:42:56.270932 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:56.270913    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06bef2e5-7a67-4d73-aeb9-92190c75e0fd-kube-api-access-95897" (OuterVolumeSpecName: "kube-api-access-95897") pod "06bef2e5-7a67-4d73-aeb9-92190c75e0fd" (UID: "06bef2e5-7a67-4d73-aeb9-92190c75e0fd"). InnerVolumeSpecName "kube-api-access-95897". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:42:56.337120 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:56.337094    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j56s6"
Jul 28 17:42:56.370242 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:56.370211    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-95897\" (UniqueName: \"kubernetes.io/projected/06bef2e5-7a67-4d73-aeb9-92190c75e0fd-kube-api-access-95897\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:42:56.370242 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:56.370242    2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/06bef2e5-7a67-4d73-aeb9-92190c75e0fd-extensions-socket-volume\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:42:56.412842 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:56.412813    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5lkm9"]
Jul 28 17:42:56.414149 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:42:56.414122    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e6cbe21_6226_41a3_91a4_b120183e6662.slice/crio-6b522ba967f62be686fa4bf1bdbdff0b5e4c7bc61ca882f3fcd2b1e70821dd25 WatchSource:0}: Error finding container 6b522ba967f62be686fa4bf1bdbdff0b5e4c7bc61ca882f3fcd2b1e70821dd25: Status 404 returned error can't find the container with id 6b522ba967f62be686fa4bf1bdbdff0b5e4c7bc61ca882f3fcd2b1e70821dd25
Jul 28 17:42:56.486737 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:56.486714    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j56s6"]
Jul 28 17:42:56.488772 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:42:56.488751    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bb917a1_5d9d_4d4b_ae0e_3718aa32d030.slice/crio-b7f374ca5c893265f0228c9de0a4c1c0ed7c715c5ed61e68b1f6dcaa34127ec6 WatchSource:0}: Error finding container b7f374ca5c893265f0228c9de0a4c1c0ed7c715c5ed61e68b1f6dcaa34127ec6: Status 404 returned error can't find the container with id b7f374ca5c893265f0228c9de0a4c1c0ed7c715c5ed61e68b1f6dcaa34127ec6
Jul 28 17:42:56.780443 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:56.780349    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:42:56.883367 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:56.883338    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:42:57.012631 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:57.012585    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-75c6f46c7d-rfnq9" podUID="67825101-0dcd-4022-8526-451950973cdd" containerName="console" containerID="cri-o://b1532beeebd4542c905f48b2053baf752ce21743e8bdf534200a5f7faaa4a85a" gracePeriod=15
Jul 28 17:42:57.014076 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:57.014045    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j56s6" event={"ID":"0bb917a1-5d9d-4d4b-ae0e-3718aa32d030","Type":"ContainerStarted","Data":"b7f374ca5c893265f0228c9de0a4c1c0ed7c715c5ed61e68b1f6dcaa34127ec6"}
Jul 28 17:42:57.015838 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:57.015811    2563 generic.go:356] "Generic (PLEG): container finished" podID="06bef2e5-7a67-4d73-aeb9-92190c75e0fd" containerID="f27d65718a9d6de74018688cca7ea834824e6aae7cd1c61c7f3dd09b07d6bef0" exitCode=0
Jul 28 17:42:57.015989 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:57.015974    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-skbcf"
Jul 28 17:42:57.016158 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:57.016124    2563 scope.go:117] "RemoveContainer" containerID="f27d65718a9d6de74018688cca7ea834824e6aae7cd1c61c7f3dd09b07d6bef0"
Jul 28 17:42:57.019214 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:57.018911    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5lkm9" event={"ID":"2e6cbe21-6226-41a3-91a4-b120183e6662","Type":"ContainerStarted","Data":"a7d02790856a3e4897bea5df3624c283feece95f3c94c5eaf6704ea79a9a88eb"}
Jul 28 17:42:57.019214 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:57.018970    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5lkm9" event={"ID":"2e6cbe21-6226-41a3-91a4-b120183e6662","Type":"ContainerStarted","Data":"6b522ba967f62be686fa4bf1bdbdff0b5e4c7bc61ca882f3fcd2b1e70821dd25"}
Jul 28 17:42:57.019214 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:57.019085    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5lkm9"
Jul 28 17:42:57.019214 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:42:57.019198    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-skbcf\" is forbidden: User \"system:node:ip-10-0-136-126.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-126.ec2.internal' and this object" podUID="06bef2e5-7a67-4d73-aeb9-92190c75e0fd" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-skbcf"
Jul 28 17:42:57.022171 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:42:57.022138    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-skbcf\" is forbidden: User \"system:node:ip-10-0-136-126.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-126.ec2.internal' and this object" podUID="06bef2e5-7a67-4d73-aeb9-92190c75e0fd" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-skbcf"
Jul 28 17:42:57.046881 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:57.046035    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5lkm9" podStartSLOduration=2.0460198529999998 podStartE2EDuration="2.046019853s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:42:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:42:57.044509472 +0000 UTC m=+515.705665826" watchObservedRunningTime="2026-07-28 17:42:57.046019853 +0000 UTC m=+515.707176208"
Jul 28 17:42:57.061410 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:42:57.061375    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-skbcf\" is forbidden: User \"system:node:ip-10-0-136-126.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-126.ec2.internal' and this object" podUID="06bef2e5-7a67-4d73-aeb9-92190c75e0fd" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-skbcf"
Jul 28 17:42:57.123481 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:57.123451    2563 scope.go:117] "RemoveContainer" containerID="f27d65718a9d6de74018688cca7ea834824e6aae7cd1c61c7f3dd09b07d6bef0"
Jul 28 17:42:57.123820 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:42:57.123792    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f27d65718a9d6de74018688cca7ea834824e6aae7cd1c61c7f3dd09b07d6bef0\": container with ID starting with f27d65718a9d6de74018688cca7ea834824e6aae7cd1c61c7f3dd09b07d6bef0 not found: ID does not exist" containerID="f27d65718a9d6de74018688cca7ea834824e6aae7cd1c61c7f3dd09b07d6bef0"
Jul 28 17:42:57.123921 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:57.123834    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f27d65718a9d6de74018688cca7ea834824e6aae7cd1c61c7f3dd09b07d6bef0"} err="failed to get container status \"f27d65718a9d6de74018688cca7ea834824e6aae7cd1c61c7f3dd09b07d6bef0\": rpc error: code = NotFound desc = could not find container \"f27d65718a9d6de74018688cca7ea834824e6aae7cd1c61c7f3dd09b07d6bef0\": container with ID starting with f27d65718a9d6de74018688cca7ea834824e6aae7cd1c61c7f3dd09b07d6bef0 not found: ID does not exist"
Jul 28 17:42:57.308503 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:57.308476    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75c6f46c7d-rfnq9"
Jul 28 17:42:57.344630 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:42:57.344593    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-skbcf\" is forbidden: User \"system:node:ip-10-0-136-126.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-126.ec2.internal' and this object" podUID="06bef2e5-7a67-4d73-aeb9-92190c75e0fd" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-skbcf"
Jul 28 17:42:57.380102 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:57.380072    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/67825101-0dcd-4022-8526-451950973cdd-console-serving-cert\") pod \"67825101-0dcd-4022-8526-451950973cdd\" (UID: \"67825101-0dcd-4022-8526-451950973cdd\") "
Jul 28 17:42:57.380289 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:57.380120    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/67825101-0dcd-4022-8526-451950973cdd-oauth-serving-cert\") pod \"67825101-0dcd-4022-8526-451950973cdd\" (UID: \"67825101-0dcd-4022-8526-451950973cdd\") "
Jul 28 17:42:57.380289 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:57.380156    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/67825101-0dcd-4022-8526-451950973cdd-console-oauth-config\") pod \"67825101-0dcd-4022-8526-451950973cdd\" (UID: \"67825101-0dcd-4022-8526-451950973cdd\") "
Jul 28 17:42:57.380455 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:57.380189    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fqtwd\" (UniqueName: \"kubernetes.io/projected/67825101-0dcd-4022-8526-451950973cdd-kube-api-access-fqtwd\") pod \"67825101-0dcd-4022-8526-451950973cdd\" (UID: \"67825101-0dcd-4022-8526-451950973cdd\") "
Jul 28 17:42:57.380591 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:57.380483    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67825101-0dcd-4022-8526-451950973cdd-trusted-ca-bundle\") pod \"67825101-0dcd-4022-8526-451950973cdd\" (UID: \"67825101-0dcd-4022-8526-451950973cdd\") "
Jul 28 17:42:57.380591 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:57.380530    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/67825101-0dcd-4022-8526-451950973cdd-console-config\") pod \"67825101-0dcd-4022-8526-451950973cdd\" (UID: \"67825101-0dcd-4022-8526-451950973cdd\") "
Jul 28 17:42:57.380713 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:57.380624    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67825101-0dcd-4022-8526-451950973cdd-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "67825101-0dcd-4022-8526-451950973cdd" (UID: "67825101-0dcd-4022-8526-451950973cdd"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:42:57.380713 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:57.380650    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/67825101-0dcd-4022-8526-451950973cdd-service-ca\") pod \"67825101-0dcd-4022-8526-451950973cdd\" (UID: \"67825101-0dcd-4022-8526-451950973cdd\") "
Jul 28 17:42:57.380923 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:57.380891    2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/67825101-0dcd-4022-8526-451950973cdd-oauth-serving-cert\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:42:57.381049 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:57.380904    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67825101-0dcd-4022-8526-451950973cdd-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "67825101-0dcd-4022-8526-451950973cdd" (UID: "67825101-0dcd-4022-8526-451950973cdd"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:42:57.381124 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:57.381076    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67825101-0dcd-4022-8526-451950973cdd-service-ca" (OuterVolumeSpecName: "service-ca") pod "67825101-0dcd-4022-8526-451950973cdd" (UID: "67825101-0dcd-4022-8526-451950973cdd"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:42:57.381220 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:57.381167    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67825101-0dcd-4022-8526-451950973cdd-console-config" (OuterVolumeSpecName: "console-config") pod "67825101-0dcd-4022-8526-451950973cdd" (UID: "67825101-0dcd-4022-8526-451950973cdd"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:42:57.382908 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:57.382650    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67825101-0dcd-4022-8526-451950973cdd-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "67825101-0dcd-4022-8526-451950973cdd" (UID: "67825101-0dcd-4022-8526-451950973cdd"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 17:42:57.382908 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:57.382871    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67825101-0dcd-4022-8526-451950973cdd-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "67825101-0dcd-4022-8526-451950973cdd" (UID: "67825101-0dcd-4022-8526-451950973cdd"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 17:42:57.383074 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:57.382964    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67825101-0dcd-4022-8526-451950973cdd-kube-api-access-fqtwd" (OuterVolumeSpecName: "kube-api-access-fqtwd") pod "67825101-0dcd-4022-8526-451950973cdd" (UID: "67825101-0dcd-4022-8526-451950973cdd"). InnerVolumeSpecName "kube-api-access-fqtwd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:42:57.481631 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:57.481602    2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/67825101-0dcd-4022-8526-451950973cdd-service-ca\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:42:57.481631 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:57.481628    2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/67825101-0dcd-4022-8526-451950973cdd-console-serving-cert\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:42:57.481631 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:57.481637    2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/67825101-0dcd-4022-8526-451950973cdd-console-oauth-config\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:42:57.481902 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:57.481646    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fqtwd\" (UniqueName: \"kubernetes.io/projected/67825101-0dcd-4022-8526-451950973cdd-kube-api-access-fqtwd\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:42:57.481902 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:57.481654    2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67825101-0dcd-4022-8526-451950973cdd-trusted-ca-bundle\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:42:57.481902 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:57.481663    2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/67825101-0dcd-4022-8526-451950973cdd-console-config\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:42:57.853084 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:57.853059    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06bef2e5-7a67-4d73-aeb9-92190c75e0fd" path="/var/lib/kubelet/pods/06bef2e5-7a67-4d73-aeb9-92190c75e0fd/volumes"
Jul 28 17:42:58.025638 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:58.025604    2563 generic.go:356] "Generic (PLEG): container finished" podID="67825101-0dcd-4022-8526-451950973cdd" containerID="b1532beeebd4542c905f48b2053baf752ce21743e8bdf534200a5f7faaa4a85a" exitCode=0
Jul 28 17:42:58.026023 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:58.025652    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75c6f46c7d-rfnq9"
Jul 28 17:42:58.026023 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:58.025687    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75c6f46c7d-rfnq9" event={"ID":"67825101-0dcd-4022-8526-451950973cdd","Type":"ContainerDied","Data":"b1532beeebd4542c905f48b2053baf752ce21743e8bdf534200a5f7faaa4a85a"}
Jul 28 17:42:58.026023 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:58.025731    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75c6f46c7d-rfnq9" event={"ID":"67825101-0dcd-4022-8526-451950973cdd","Type":"ContainerDied","Data":"d6b7826e0f0f62c95b773c78183d7882cbc3d4062b223034c7ca901ca02fa47e"}
Jul 28 17:42:58.026023 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:58.025753    2563 scope.go:117] "RemoveContainer" containerID="b1532beeebd4542c905f48b2053baf752ce21743e8bdf534200a5f7faaa4a85a"
Jul 28 17:42:58.027551 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:58.027523    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j56s6" event={"ID":"0bb917a1-5d9d-4d4b-ae0e-3718aa32d030","Type":"ContainerStarted","Data":"a0e22db25a91bd8923a760cbbd582b9303ecd204abe9667206af85c56bc0aa4b"}
Jul 28 17:42:58.028067 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:58.028010    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j56s6"
Jul 28 17:42:58.034651 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:58.034633    2563 scope.go:117] "RemoveContainer" containerID="b1532beeebd4542c905f48b2053baf752ce21743e8bdf534200a5f7faaa4a85a"
Jul 28 17:42:58.034912 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:42:58.034895    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1532beeebd4542c905f48b2053baf752ce21743e8bdf534200a5f7faaa4a85a\": container with ID starting with b1532beeebd4542c905f48b2053baf752ce21743e8bdf534200a5f7faaa4a85a not found: ID does not exist" containerID="b1532beeebd4542c905f48b2053baf752ce21743e8bdf534200a5f7faaa4a85a"
Jul 28 17:42:58.034979 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:58.034919    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1532beeebd4542c905f48b2053baf752ce21743e8bdf534200a5f7faaa4a85a"} err="failed to get container status \"b1532beeebd4542c905f48b2053baf752ce21743e8bdf534200a5f7faaa4a85a\": rpc error: code = NotFound desc = could not find container \"b1532beeebd4542c905f48b2053baf752ce21743e8bdf534200a5f7faaa4a85a\": container with ID starting with b1532beeebd4542c905f48b2053baf752ce21743e8bdf534200a5f7faaa4a85a not found: ID does not exist"
Jul 28 17:42:58.058104 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:58.058080    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-75c6f46c7d-rfnq9"]
Jul 28 17:42:58.061971 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:58.061953    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-75c6f46c7d-rfnq9"]
Jul 28 17:42:58.097785 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:58.097749    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j56s6" podStartSLOduration=1.808957156 podStartE2EDuration="3.097737845s" totalImagesPullingTime="1.288780689s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:42:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:42:58.096407191 +0000 UTC m=+516.757563546" watchObservedRunningTime="2026-07-28 17:42:58.097737845 +0000 UTC m=+516.758894199"
Jul 28 17:42:59.742562 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:59.742518    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:42:59.853728 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:59.853698    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:42:59.855701 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:42:59.855678    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67825101-0dcd-4022-8526-451950973cdd" path="/var/lib/kubelet/pods/67825101-0dcd-4022-8526-451950973cdd/volumes"
Jul 28 17:43:05.518643 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:05.518616    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:43:05.627564 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:05.627538    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:43:06.120288 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:06.120253    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:43:06.195569 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:06.195531    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:43:07.851373 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:07.851337    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:43:08.031817 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:08.031784    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5lkm9"
Jul 28 17:43:08.437409 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:08.437373    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:43:08.532905 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:08.532875    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:43:08.590615 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:08.590578    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j26kf"]
Jul 28 17:43:08.590976 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:08.590959    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67825101-0dcd-4022-8526-451950973cdd" containerName="console"
Jul 28 17:43:08.591076 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:08.590978    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="67825101-0dcd-4022-8526-451950973cdd" containerName="console"
Jul 28 17:43:08.591076 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:08.591065    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="67825101-0dcd-4022-8526-451950973cdd" containerName="console"
Jul 28 17:43:08.595478 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:08.595459    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j26kf"
Jul 28 17:43:08.606906 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:08.606884    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j26kf"]
Jul 28 17:43:08.672800 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:08.672719    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skk4w\" (UniqueName: \"kubernetes.io/projected/a81f2137-7e96-4a34-bcf8-febcc470dde4-kube-api-access-skk4w\") pod \"kuadrant-operator-controller-manager-86878c769f-j26kf\" (UID: \"a81f2137-7e96-4a34-bcf8-febcc470dde4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j26kf"
Jul 28 17:43:08.672800 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:08.672757    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a81f2137-7e96-4a34-bcf8-febcc470dde4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-j26kf\" (UID: \"a81f2137-7e96-4a34-bcf8-febcc470dde4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j26kf"
Jul 28 17:43:08.774201 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:08.774168    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skk4w\" (UniqueName: \"kubernetes.io/projected/a81f2137-7e96-4a34-bcf8-febcc470dde4-kube-api-access-skk4w\") pod \"kuadrant-operator-controller-manager-86878c769f-j26kf\" (UID: \"a81f2137-7e96-4a34-bcf8-febcc470dde4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j26kf"
Jul 28 17:43:08.774201 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:08.774202    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a81f2137-7e96-4a34-bcf8-febcc470dde4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-j26kf\" (UID: \"a81f2137-7e96-4a34-bcf8-febcc470dde4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j26kf"
Jul 28 17:43:08.774596 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:08.774578    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a81f2137-7e96-4a34-bcf8-febcc470dde4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-j26kf\" (UID: \"a81f2137-7e96-4a34-bcf8-febcc470dde4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j26kf"
Jul 28 17:43:08.790538 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:08.790515    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-skk4w\" (UniqueName: \"kubernetes.io/projected/a81f2137-7e96-4a34-bcf8-febcc470dde4-kube-api-access-skk4w\") pod \"kuadrant-operator-controller-manager-86878c769f-j26kf\" (UID: \"a81f2137-7e96-4a34-bcf8-febcc470dde4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j26kf"
Jul 28 17:43:08.905406 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:08.905365    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j26kf"
Jul 28 17:43:09.071984 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:09.071962    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j26kf"]
Jul 28 17:43:09.073654 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:43:09.073627    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda81f2137_7e96_4a34_bcf8_febcc470dde4.slice/crio-9c2e98a474ba33879522f361df782aa1696a736396f491fab12eac3048ece3a1 WatchSource:0}: Error finding container 9c2e98a474ba33879522f361df782aa1696a736396f491fab12eac3048ece3a1: Status 404 returned error can't find the container with id 9c2e98a474ba33879522f361df782aa1696a736396f491fab12eac3048ece3a1
Jul 28 17:43:09.087416 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:09.087392    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j26kf" event={"ID":"a81f2137-7e96-4a34-bcf8-febcc470dde4","Type":"ContainerStarted","Data":"9c2e98a474ba33879522f361df782aa1696a736396f491fab12eac3048ece3a1"}
Jul 28 17:43:10.042992 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:10.042959    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j56s6"
Jul 28 17:43:10.094960 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:10.094930    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j26kf" event={"ID":"a81f2137-7e96-4a34-bcf8-febcc470dde4","Type":"ContainerStarted","Data":"db31824b2905d606371a201ee02b1098789658a5c33662ff09f53a8cfa63f2a6"}
Jul 28 17:43:10.095119 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:10.095012    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j26kf"
Jul 28 17:43:10.115358 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:10.115315    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j26kf" podStartSLOduration=2.115298813 podStartE2EDuration="2.115298813s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:43:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:43:10.113961286 +0000 UTC m=+528.775117640" watchObservedRunningTime="2026-07-28 17:43:10.115298813 +0000 UTC m=+528.776455171"
Jul 28 17:43:14.129230 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:14.129153    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:43:21.102959 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:21.102930    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j26kf"
Jul 28 17:43:21.166858 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:21.166819    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5lkm9"]
Jul 28 17:43:21.167124 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:21.167078    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5lkm9" podUID="2e6cbe21-6226-41a3-91a4-b120183e6662" containerName="manager" containerID="cri-o://a7d02790856a3e4897bea5df3624c283feece95f3c94c5eaf6704ea79a9a88eb" gracePeriod=10
Jul 28 17:43:21.411517 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:21.411492    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5lkm9"
Jul 28 17:43:21.479786 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:21.479753    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rdmv9\" (UniqueName: \"kubernetes.io/projected/2e6cbe21-6226-41a3-91a4-b120183e6662-kube-api-access-rdmv9\") pod \"2e6cbe21-6226-41a3-91a4-b120183e6662\" (UID: \"2e6cbe21-6226-41a3-91a4-b120183e6662\") "
Jul 28 17:43:21.479968 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:21.479800    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2e6cbe21-6226-41a3-91a4-b120183e6662-extensions-socket-volume\") pod \"2e6cbe21-6226-41a3-91a4-b120183e6662\" (UID: \"2e6cbe21-6226-41a3-91a4-b120183e6662\") "
Jul 28 17:43:21.480176 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:21.480155    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e6cbe21-6226-41a3-91a4-b120183e6662-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "2e6cbe21-6226-41a3-91a4-b120183e6662" (UID: "2e6cbe21-6226-41a3-91a4-b120183e6662"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 17:43:21.481835 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:21.481804    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e6cbe21-6226-41a3-91a4-b120183e6662-kube-api-access-rdmv9" (OuterVolumeSpecName: "kube-api-access-rdmv9") pod "2e6cbe21-6226-41a3-91a4-b120183e6662" (UID: "2e6cbe21-6226-41a3-91a4-b120183e6662"). InnerVolumeSpecName "kube-api-access-rdmv9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:43:21.580764 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:21.580729    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rdmv9\" (UniqueName: \"kubernetes.io/projected/2e6cbe21-6226-41a3-91a4-b120183e6662-kube-api-access-rdmv9\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:43:21.580764 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:21.580759    2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2e6cbe21-6226-41a3-91a4-b120183e6662-extensions-socket-volume\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:43:22.159117 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:22.159081    2563 generic.go:356] "Generic (PLEG): container finished" podID="2e6cbe21-6226-41a3-91a4-b120183e6662" containerID="a7d02790856a3e4897bea5df3624c283feece95f3c94c5eaf6704ea79a9a88eb" exitCode=0
Jul 28 17:43:22.159654 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:22.159126    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5lkm9"
Jul 28 17:43:22.159654 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:22.159162    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5lkm9" event={"ID":"2e6cbe21-6226-41a3-91a4-b120183e6662","Type":"ContainerDied","Data":"a7d02790856a3e4897bea5df3624c283feece95f3c94c5eaf6704ea79a9a88eb"}
Jul 28 17:43:22.159654 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:22.159226    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5lkm9" event={"ID":"2e6cbe21-6226-41a3-91a4-b120183e6662","Type":"ContainerDied","Data":"6b522ba967f62be686fa4bf1bdbdff0b5e4c7bc61ca882f3fcd2b1e70821dd25"}
Jul 28 17:43:22.159654 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:22.159263    2563 scope.go:117] "RemoveContainer" containerID="a7d02790856a3e4897bea5df3624c283feece95f3c94c5eaf6704ea79a9a88eb"
Jul 28 17:43:22.168204 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:22.168014    2563 scope.go:117] "RemoveContainer" containerID="a7d02790856a3e4897bea5df3624c283feece95f3c94c5eaf6704ea79a9a88eb"
Jul 28 17:43:22.168303 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:43:22.168285    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7d02790856a3e4897bea5df3624c283feece95f3c94c5eaf6704ea79a9a88eb\": container with ID starting with a7d02790856a3e4897bea5df3624c283feece95f3c94c5eaf6704ea79a9a88eb not found: ID does not exist" containerID="a7d02790856a3e4897bea5df3624c283feece95f3c94c5eaf6704ea79a9a88eb"
Jul 28 17:43:22.168360 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:22.168315    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7d02790856a3e4897bea5df3624c283feece95f3c94c5eaf6704ea79a9a88eb"} err="failed to get container status \"a7d02790856a3e4897bea5df3624c283feece95f3c94c5eaf6704ea79a9a88eb\": rpc error: code = NotFound desc = could not find container \"a7d02790856a3e4897bea5df3624c283feece95f3c94c5eaf6704ea79a9a88eb\": container with ID starting with a7d02790856a3e4897bea5df3624c283feece95f3c94c5eaf6704ea79a9a88eb not found: ID does not exist"
Jul 28 17:43:22.181618 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:22.181594    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5lkm9"]
Jul 28 17:43:22.190055 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:22.190033    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5lkm9"]
Jul 28 17:43:23.853173 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:23.853140    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e6cbe21-6226-41a3-91a4-b120183e6662" path="/var/lib/kubelet/pods/2e6cbe21-6226-41a3-91a4-b120183e6662/volumes"
Jul 28 17:43:24.636031 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:24.635998    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:43:24.895760 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:24.895669    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:43:41.303129 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:41.303090    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:43:41.421073 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:41.421032    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:43:41.517721 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:41.517684    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:43:42.072327 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:42.072294    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:43:42.212762 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:42.212735    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:43:42.321802 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:42.321772    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:43:55.356499 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:55.356452    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:43:55.459302 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:43:55.459273    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:44:03.246266 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:44:03.246206    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:44:03.352361 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:44:03.352332    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:44:03.721975 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:44:03.721942    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:44:16.981706 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:44:16.981665    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:44:17.417427 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:44:17.417380    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:44:18.466036 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:44:18.466001    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:44:18.565932 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:44:18.565896    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:44:24.775480 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:44:24.775448    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:44:24.798075 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:44:24.798037    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-58ff9c49b8-kmzcb"]
Jul 28 17:44:24.798535 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:44:24.798515    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e6cbe21-6226-41a3-91a4-b120183e6662" containerName="manager"
Jul 28 17:44:24.798535 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:44:24.798535    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e6cbe21-6226-41a3-91a4-b120183e6662" containerName="manager"
Jul 28 17:44:24.798712 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:44:24.798629    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e6cbe21-6226-41a3-91a4-b120183e6662" containerName="manager"
Jul 28 17:44:24.802759 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:44:24.802738    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58ff9c49b8-kmzcb"
Jul 28 17:44:24.806795 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:44:24.806774    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 28 17:44:24.806795 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:44:24.806784    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 28 17:44:24.806976 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:44:24.806775    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-hrp6b\""
Jul 28 17:44:24.817502 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:44:24.817477    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-58ff9c49b8-kmzcb"]
Jul 28 17:44:24.880008 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:44:24.879972    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:44:24.909234 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:44:24.909201    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f71aca1c-820e-490d-a281-5d5829a4729e-webhook-cert\") pod \"maas-controller-58ff9c49b8-kmzcb\" (UID: \"f71aca1c-820e-490d-a281-5d5829a4729e\") " pod="opendatahub/maas-controller-58ff9c49b8-kmzcb"
Jul 28 17:44:24.909400 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:44:24.909285    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wt28m\" (UniqueName: \"kubernetes.io/projected/f71aca1c-820e-490d-a281-5d5829a4729e-kube-api-access-wt28m\") pod \"maas-controller-58ff9c49b8-kmzcb\" (UID: \"f71aca1c-820e-490d-a281-5d5829a4729e\") " pod="opendatahub/maas-controller-58ff9c49b8-kmzcb"
Jul 28 17:44:24.994960 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:44:24.994919    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:44:25.010069 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:44:25.010042    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wt28m\" (UniqueName: \"kubernetes.io/projected/f71aca1c-820e-490d-a281-5d5829a4729e-kube-api-access-wt28m\") pod \"maas-controller-58ff9c49b8-kmzcb\" (UID: \"f71aca1c-820e-490d-a281-5d5829a4729e\") " pod="opendatahub/maas-controller-58ff9c49b8-kmzcb"
Jul 28 17:44:25.010207 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:44:25.010099    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f71aca1c-820e-490d-a281-5d5829a4729e-webhook-cert\") pod \"maas-controller-58ff9c49b8-kmzcb\" (UID: \"f71aca1c-820e-490d-a281-5d5829a4729e\") " pod="opendatahub/maas-controller-58ff9c49b8-kmzcb"
Jul 28 17:44:25.012351 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:44:25.012332    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f71aca1c-820e-490d-a281-5d5829a4729e-webhook-cert\") pod \"maas-controller-58ff9c49b8-kmzcb\" (UID: \"f71aca1c-820e-490d-a281-5d5829a4729e\") " pod="opendatahub/maas-controller-58ff9c49b8-kmzcb"
Jul 28 17:44:25.019782 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:44:25.019758    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wt28m\" (UniqueName: \"kubernetes.io/projected/f71aca1c-820e-490d-a281-5d5829a4729e-kube-api-access-wt28m\") pod \"maas-controller-58ff9c49b8-kmzcb\" (UID: \"f71aca1c-820e-490d-a281-5d5829a4729e\") " pod="opendatahub/maas-controller-58ff9c49b8-kmzcb"
Jul 28 17:44:25.115962 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:44:25.115937    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58ff9c49b8-kmzcb"
Jul 28 17:44:25.244681 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:44:25.244657    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-58ff9c49b8-kmzcb"]
Jul 28 17:44:25.247133 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:44:25.247110    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf71aca1c_820e_490d_a281_5d5829a4729e.slice/crio-82ab8e270a704e65ee6bdcb1135abf660826a077021b8f62b553dee047b85653 WatchSource:0}: Error finding container 82ab8e270a704e65ee6bdcb1135abf660826a077021b8f62b553dee047b85653: Status 404 returned error can't find the container with id 82ab8e270a704e65ee6bdcb1135abf660826a077021b8f62b553dee047b85653
Jul 28 17:44:25.471543 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:44:25.471462    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58ff9c49b8-kmzcb" event={"ID":"f71aca1c-820e-490d-a281-5d5829a4729e","Type":"ContainerStarted","Data":"82ab8e270a704e65ee6bdcb1135abf660826a077021b8f62b553dee047b85653"}
Jul 28 17:44:27.949499 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:44:27.949456    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:44:28.043269 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:44:28.043198    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:44:28.489272 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:44:28.489237    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58ff9c49b8-kmzcb" event={"ID":"f71aca1c-820e-490d-a281-5d5829a4729e","Type":"ContainerStarted","Data":"89ecf70664beefb3446dc5af09cb5d6dd8412967bc3beb0a522b53064b065eeb"}
Jul 28 17:44:28.489471 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:44:28.489308    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-58ff9c49b8-kmzcb"
Jul 28 17:44:28.511770 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:44:28.511722    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-58ff9c49b8-kmzcb" podStartSLOduration=1.9706455950000001 podStartE2EDuration="4.511706725s" totalImagesPullingTime="2.54106113s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:44:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:44:28.509718764 +0000 UTC m=+607.170875129" watchObservedRunningTime="2026-07-28 17:44:28.511706725 +0000 UTC m=+607.172863081"
Jul 28 17:44:34.734050 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:44:34.734013    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:44:34.827605 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:44:34.827574    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:44:39.496966 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:44:39.496935    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-58ff9c49b8-kmzcb"
Jul 28 17:44:39.780158 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:44:39.780073    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-58ff9c49b8-kmzcb"]
Jul 28 17:44:39.780333 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:44:39.780307    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-58ff9c49b8-kmzcb" podUID="f71aca1c-820e-490d-a281-5d5829a4729e" containerName="manager" containerID="cri-o://89ecf70664beefb3446dc5af09cb5d6dd8412967bc3beb0a522b53064b065eeb" gracePeriod=30
Jul 28 17:44:40.025327 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:44:40.025303    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58ff9c49b8-kmzcb"
Jul 28 17:44:40.031663 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:44:40.031615    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f71aca1c-820e-490d-a281-5d5829a4729e-webhook-cert\") pod \"f71aca1c-820e-490d-a281-5d5829a4729e\" (UID: \"f71aca1c-820e-490d-a281-5d5829a4729e\") "
Jul 28 17:44:40.031759 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:44:40.031735    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wt28m\" (UniqueName: \"kubernetes.io/projected/f71aca1c-820e-490d-a281-5d5829a4729e-kube-api-access-wt28m\") pod \"f71aca1c-820e-490d-a281-5d5829a4729e\" (UID: \"f71aca1c-820e-490d-a281-5d5829a4729e\") "
Jul 28 17:44:40.033565 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:44:40.033536    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f71aca1c-820e-490d-a281-5d5829a4729e-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "f71aca1c-820e-490d-a281-5d5829a4729e" (UID: "f71aca1c-820e-490d-a281-5d5829a4729e"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 17:44:40.033663 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:44:40.033648    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f71aca1c-820e-490d-a281-5d5829a4729e-kube-api-access-wt28m" (OuterVolumeSpecName: "kube-api-access-wt28m") pod "f71aca1c-820e-490d-a281-5d5829a4729e" (UID: "f71aca1c-820e-490d-a281-5d5829a4729e"). InnerVolumeSpecName "kube-api-access-wt28m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:44:40.133221 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:44:40.133195    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wt28m\" (UniqueName: \"kubernetes.io/projected/f71aca1c-820e-490d-a281-5d5829a4729e-kube-api-access-wt28m\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:44:40.133221 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:44:40.133217    2563 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f71aca1c-820e-490d-a281-5d5829a4729e-webhook-cert\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:44:40.551042 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:44:40.551011    2563 generic.go:356] "Generic (PLEG): container finished" podID="f71aca1c-820e-490d-a281-5d5829a4729e" containerID="89ecf70664beefb3446dc5af09cb5d6dd8412967bc3beb0a522b53064b065eeb" exitCode=0
Jul 28 17:44:40.551517 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:44:40.551054    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58ff9c49b8-kmzcb"
Jul 28 17:44:40.551517 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:44:40.551094    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58ff9c49b8-kmzcb" event={"ID":"f71aca1c-820e-490d-a281-5d5829a4729e","Type":"ContainerDied","Data":"89ecf70664beefb3446dc5af09cb5d6dd8412967bc3beb0a522b53064b065eeb"}
Jul 28 17:44:40.551517 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:44:40.551151    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58ff9c49b8-kmzcb" event={"ID":"f71aca1c-820e-490d-a281-5d5829a4729e","Type":"ContainerDied","Data":"82ab8e270a704e65ee6bdcb1135abf660826a077021b8f62b553dee047b85653"}
Jul 28 17:44:40.551517 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:44:40.551187    2563 scope.go:117] "RemoveContainer" containerID="89ecf70664beefb3446dc5af09cb5d6dd8412967bc3beb0a522b53064b065eeb"
Jul 28 17:44:40.560419 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:44:40.560399    2563 scope.go:117] "RemoveContainer" containerID="89ecf70664beefb3446dc5af09cb5d6dd8412967bc3beb0a522b53064b065eeb"
Jul 28 17:44:40.560750 ip-10-0-136-126 kubenswrapper[2563]: E0728 17:44:40.560721    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89ecf70664beefb3446dc5af09cb5d6dd8412967bc3beb0a522b53064b065eeb\": container with ID starting with 89ecf70664beefb3446dc5af09cb5d6dd8412967bc3beb0a522b53064b065eeb not found: ID does not exist" containerID="89ecf70664beefb3446dc5af09cb5d6dd8412967bc3beb0a522b53064b065eeb"
Jul 28 17:44:40.560818 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:44:40.560761    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89ecf70664beefb3446dc5af09cb5d6dd8412967bc3beb0a522b53064b065eeb"} err="failed to get container status \"89ecf70664beefb3446dc5af09cb5d6dd8412967bc3beb0a522b53064b065eeb\": rpc error: code = NotFound desc = could not find container \"89ecf70664beefb3446dc5af09cb5d6dd8412967bc3beb0a522b53064b065eeb\": container with ID starting with 89ecf70664beefb3446dc5af09cb5d6dd8412967bc3beb0a522b53064b065eeb not found: ID does not exist"
Jul 28 17:44:40.575348 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:44:40.575323    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-58ff9c49b8-kmzcb"]
Jul 28 17:44:40.580364 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:44:40.580333    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-58ff9c49b8-kmzcb"]
Jul 28 17:44:41.852732 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:44:41.852699    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f71aca1c-820e-490d-a281-5d5829a4729e" path="/var/lib/kubelet/pods/f71aca1c-820e-490d-a281-5d5829a4729e/volumes"
Jul 28 17:44:43.467347 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:44:43.467266    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:45:01.093923 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:01.093882    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:45:01.189026 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:01.188998    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:45:01.805894 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:01.805861    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:45:01.911238 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:01.911204    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7d4cd97cdc-zdk6z"]
Jul 28 17:45:01.911616 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:01.911599    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f71aca1c-820e-490d-a281-5d5829a4729e" containerName="manager"
Jul 28 17:45:01.911715 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:01.911618    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f71aca1c-820e-490d-a281-5d5829a4729e" containerName="manager"
Jul 28 17:45:01.911772 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:01.911713    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f71aca1c-820e-490d-a281-5d5829a4729e" containerName="manager"
Jul 28 17:45:01.919586 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:01.919563    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7d4cd97cdc-zdk6z"
Jul 28 17:45:01.924514 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:01.924493    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 17:45:01.924834 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:01.924521    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 28 17:45:01.924977 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:01.924577    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-tct44\""
Jul 28 17:45:01.925105 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:01.924812    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 28 17:45:01.926850 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:01.926827    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7d4cd97cdc-zdk6z"]
Jul 28 17:45:02.016143 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:02.016110    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46fc5fc3-5d95-4878-b4ab-a3e73285a72b-openshift-service-ca\") pod \"maas-api-7d4cd97cdc-zdk6z\" (UID: \"46fc5fc3-5d95-4878-b4ab-a3e73285a72b\") " pod="odh-ai-gateway-infra/maas-api-7d4cd97cdc-zdk6z"
Jul 28 17:45:02.016143 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:02.016147    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2cm4\" (UniqueName: \"kubernetes.io/projected/46fc5fc3-5d95-4878-b4ab-a3e73285a72b-kube-api-access-g2cm4\") pod \"maas-api-7d4cd97cdc-zdk6z\" (UID: \"46fc5fc3-5d95-4878-b4ab-a3e73285a72b\") " pod="odh-ai-gateway-infra/maas-api-7d4cd97cdc-zdk6z"
Jul 28 17:45:02.016344 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:02.016206    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46fc5fc3-5d95-4878-b4ab-a3e73285a72b-maas-api-tls\") pod \"maas-api-7d4cd97cdc-zdk6z\" (UID: \"46fc5fc3-5d95-4878-b4ab-a3e73285a72b\") " pod="odh-ai-gateway-infra/maas-api-7d4cd97cdc-zdk6z"
Jul 28 17:45:02.082874 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:02.082847    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:45:02.116974 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:02.116944    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46fc5fc3-5d95-4878-b4ab-a3e73285a72b-openshift-service-ca\") pod \"maas-api-7d4cd97cdc-zdk6z\" (UID: \"46fc5fc3-5d95-4878-b4ab-a3e73285a72b\") " pod="odh-ai-gateway-infra/maas-api-7d4cd97cdc-zdk6z"
Jul 28 17:45:02.117300 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:02.116982    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2cm4\" (UniqueName: \"kubernetes.io/projected/46fc5fc3-5d95-4878-b4ab-a3e73285a72b-kube-api-access-g2cm4\") pod \"maas-api-7d4cd97cdc-zdk6z\" (UID: \"46fc5fc3-5d95-4878-b4ab-a3e73285a72b\") " pod="odh-ai-gateway-infra/maas-api-7d4cd97cdc-zdk6z"
Jul 28 17:45:02.117300 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:02.117010    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46fc5fc3-5d95-4878-b4ab-a3e73285a72b-maas-api-tls\") pod \"maas-api-7d4cd97cdc-zdk6z\" (UID: \"46fc5fc3-5d95-4878-b4ab-a3e73285a72b\") " pod="odh-ai-gateway-infra/maas-api-7d4cd97cdc-zdk6z"
Jul 28 17:45:02.117704 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:02.117682    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46fc5fc3-5d95-4878-b4ab-a3e73285a72b-openshift-service-ca\") pod \"maas-api-7d4cd97cdc-zdk6z\" (UID: \"46fc5fc3-5d95-4878-b4ab-a3e73285a72b\") " pod="odh-ai-gateway-infra/maas-api-7d4cd97cdc-zdk6z"
Jul 28 17:45:02.119560 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:02.119536    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46fc5fc3-5d95-4878-b4ab-a3e73285a72b-maas-api-tls\") pod \"maas-api-7d4cd97cdc-zdk6z\" (UID: \"46fc5fc3-5d95-4878-b4ab-a3e73285a72b\") " pod="odh-ai-gateway-infra/maas-api-7d4cd97cdc-zdk6z"
Jul 28 17:45:02.126685 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:02.126648    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2cm4\" (UniqueName: \"kubernetes.io/projected/46fc5fc3-5d95-4878-b4ab-a3e73285a72b-kube-api-access-g2cm4\") pod \"maas-api-7d4cd97cdc-zdk6z\" (UID: \"46fc5fc3-5d95-4878-b4ab-a3e73285a72b\") " pod="odh-ai-gateway-infra/maas-api-7d4cd97cdc-zdk6z"
Jul 28 17:45:02.232998 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:02.232964    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7d4cd97cdc-zdk6z"
Jul 28 17:45:02.372715 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:02.372689    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7d4cd97cdc-zdk6z"]
Jul 28 17:45:02.375239 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:45:02.375212    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46fc5fc3_5d95_4878_b4ab_a3e73285a72b.slice/crio-5f5735eed7e48c48e16da93bb264ed6dd8e2b394b6fad73d39376fa3bcf4a3c7 WatchSource:0}: Error finding container 5f5735eed7e48c48e16da93bb264ed6dd8e2b394b6fad73d39376fa3bcf4a3c7: Status 404 returned error can't find the container with id 5f5735eed7e48c48e16da93bb264ed6dd8e2b394b6fad73d39376fa3bcf4a3c7
Jul 28 17:45:02.377212 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:02.377190    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 17:45:02.420107 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:02.420075    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:45:02.520945 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:02.520916    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:45:02.674208 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:02.674102    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7d4cd97cdc-zdk6z" event={"ID":"46fc5fc3-5d95-4878-b4ab-a3e73285a72b","Type":"ContainerStarted","Data":"5f5735eed7e48c48e16da93bb264ed6dd8e2b394b6fad73d39376fa3bcf4a3c7"}
Jul 28 17:45:04.685863 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:04.685826    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7d4cd97cdc-zdk6z" event={"ID":"46fc5fc3-5d95-4878-b4ab-a3e73285a72b","Type":"ContainerStarted","Data":"032db57ea05a361aa802a69fa69ebb278440e6abb1c02355e81c1b33ac4ddb53"}
Jul 28 17:45:04.686242 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:04.685959    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7d4cd97cdc-zdk6z"
Jul 28 17:45:04.710417 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:04.710368    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7d4cd97cdc-zdk6z" podStartSLOduration=2.174853376 podStartE2EDuration="3.710352701s" totalImagesPullingTime="1.535499325s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:45:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:45:04.708489287 +0000 UTC m=+643.369645637" watchObservedRunningTime="2026-07-28 17:45:04.710352701 +0000 UTC m=+643.371509056"
Jul 28 17:45:10.695825 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:10.695791    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7d4cd97cdc-zdk6z"
Jul 28 17:45:11.785875 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:11.785840    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:45:11.884029 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:11.883996    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:45:24.434489 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:24.434451    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:45:36.349935 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:36.349901    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:45:44.704091 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:44.704062    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-g67gp"]
Jul 28 17:45:44.708952 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:44.708935    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g67gp"
Jul 28 17:45:44.712586 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:44.712566    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 28 17:45:44.713580 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:44.713560    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 28 17:45:44.713673 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:44.713638    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 28 17:45:44.713739 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:44.713722    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-dc42l\""
Jul 28 17:45:44.725005 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:44.724986    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-g67gp"]
Jul 28 17:45:44.732290 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:44.732267    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:45:44.766664 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:44.766636    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9efa933f-0829-4477-9d6b-b7ee749f3ef6-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-g67gp\" (UID: \"9efa933f-0829-4477-9d6b-b7ee749f3ef6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g67gp"
Jul 28 17:45:44.766796 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:44.766701    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9efa933f-0829-4477-9d6b-b7ee749f3ef6-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-g67gp\" (UID: \"9efa933f-0829-4477-9d6b-b7ee749f3ef6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g67gp"
Jul 28 17:45:44.766796 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:44.766760    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9efa933f-0829-4477-9d6b-b7ee749f3ef6-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-g67gp\" (UID: \"9efa933f-0829-4477-9d6b-b7ee749f3ef6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g67gp"
Jul 28 17:45:44.766796 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:44.766786    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9efa933f-0829-4477-9d6b-b7ee749f3ef6-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-g67gp\" (UID: \"9efa933f-0829-4477-9d6b-b7ee749f3ef6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g67gp"
Jul 28 17:45:44.766911 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:44.766808    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwcws\" (UniqueName: \"kubernetes.io/projected/9efa933f-0829-4477-9d6b-b7ee749f3ef6-kube-api-access-wwcws\") pod \"e2e-trlp-test-simulated-kserve-598466548d-g67gp\" (UID: \"9efa933f-0829-4477-9d6b-b7ee749f3ef6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g67gp"
Jul 28 17:45:44.766911 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:44.766862    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9efa933f-0829-4477-9d6b-b7ee749f3ef6-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-g67gp\" (UID: \"9efa933f-0829-4477-9d6b-b7ee749f3ef6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g67gp"
Jul 28 17:45:44.867894 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:44.867860    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9efa933f-0829-4477-9d6b-b7ee749f3ef6-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-g67gp\" (UID: \"9efa933f-0829-4477-9d6b-b7ee749f3ef6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g67gp"
Jul 28 17:45:44.867894 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:44.867898    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9efa933f-0829-4477-9d6b-b7ee749f3ef6-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-g67gp\" (UID: \"9efa933f-0829-4477-9d6b-b7ee749f3ef6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g67gp"
Jul 28 17:45:44.868162 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:44.867917    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwcws\" (UniqueName: \"kubernetes.io/projected/9efa933f-0829-4477-9d6b-b7ee749f3ef6-kube-api-access-wwcws\") pod \"e2e-trlp-test-simulated-kserve-598466548d-g67gp\" (UID: \"9efa933f-0829-4477-9d6b-b7ee749f3ef6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g67gp"
Jul 28 17:45:44.868162 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:44.867956    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9efa933f-0829-4477-9d6b-b7ee749f3ef6-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-g67gp\" (UID: \"9efa933f-0829-4477-9d6b-b7ee749f3ef6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g67gp"
Jul 28 17:45:44.868162 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:44.868094    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9efa933f-0829-4477-9d6b-b7ee749f3ef6-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-g67gp\" (UID: \"9efa933f-0829-4477-9d6b-b7ee749f3ef6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g67gp"
Jul 28 17:45:44.868631 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:44.868262    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9efa933f-0829-4477-9d6b-b7ee749f3ef6-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-g67gp\" (UID: \"9efa933f-0829-4477-9d6b-b7ee749f3ef6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g67gp"
Jul 28 17:45:44.868631 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:44.868329    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9efa933f-0829-4477-9d6b-b7ee749f3ef6-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-g67gp\" (UID: \"9efa933f-0829-4477-9d6b-b7ee749f3ef6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g67gp"
Jul 28 17:45:44.868631 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:44.868380    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9efa933f-0829-4477-9d6b-b7ee749f3ef6-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-g67gp\" (UID: \"9efa933f-0829-4477-9d6b-b7ee749f3ef6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g67gp"
Jul 28 17:45:44.868631 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:44.868622    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9efa933f-0829-4477-9d6b-b7ee749f3ef6-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-g67gp\" (UID: \"9efa933f-0829-4477-9d6b-b7ee749f3ef6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g67gp"
Jul 28 17:45:44.870281 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:44.870260    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9efa933f-0829-4477-9d6b-b7ee749f3ef6-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-g67gp\" (UID: \"9efa933f-0829-4477-9d6b-b7ee749f3ef6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g67gp"
Jul 28 17:45:44.870671 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:44.870654    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9efa933f-0829-4477-9d6b-b7ee749f3ef6-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-g67gp\" (UID: \"9efa933f-0829-4477-9d6b-b7ee749f3ef6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g67gp"
Jul 28 17:45:44.881247 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:44.881215    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwcws\" (UniqueName: \"kubernetes.io/projected/9efa933f-0829-4477-9d6b-b7ee749f3ef6-kube-api-access-wwcws\") pod \"e2e-trlp-test-simulated-kserve-598466548d-g67gp\" (UID: \"9efa933f-0829-4477-9d6b-b7ee749f3ef6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g67gp"
Jul 28 17:45:45.018369 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:45.018288    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g67gp"
Jul 28 17:45:45.152279 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:45.150459    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-g67gp"]
Jul 28 17:45:45.155079 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:45:45.155052    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9efa933f_0829_4477_9d6b_b7ee749f3ef6.slice/crio-a5a278ec8953ef70d2f9a781d79369ef956000565b84913350bd6fd3ce98c96b WatchSource:0}: Error finding container a5a278ec8953ef70d2f9a781d79369ef956000565b84913350bd6fd3ce98c96b: Status 404 returned error can't find the container with id a5a278ec8953ef70d2f9a781d79369ef956000565b84913350bd6fd3ce98c96b
Jul 28 17:45:45.895865 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:45.895830    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g67gp" event={"ID":"9efa933f-0829-4477-9d6b-b7ee749f3ef6","Type":"ContainerStarted","Data":"a5a278ec8953ef70d2f9a781d79369ef956000565b84913350bd6fd3ce98c96b"}
Jul 28 17:45:46.903057 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:46.903025    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g67gp" event={"ID":"9efa933f-0829-4477-9d6b-b7ee749f3ef6","Type":"ContainerStarted","Data":"62f56e77825e52eeca3097176f59b38ab840f290f886cb94f0ffb94358371692"}
Jul 28 17:45:46.929425 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:46.929374    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g67gp" podStartSLOduration=1.8681888450000002 podStartE2EDuration="2.929359343s" totalImagesPullingTime="1.061170498s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:45:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:45:46.927445569 +0000 UTC m=+685.588601916" watchObservedRunningTime="2026-07-28 17:45:46.929359343 +0000 UTC m=+685.590515695"
Jul 28 17:45:52.233187 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:52.233144    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:45:55.019157 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:55.019124    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g67gp"
Jul 28 17:45:55.019541 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:55.019179    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g67gp"
Jul 28 17:45:55.031647 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:55.031618    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g67gp"
Jul 28 17:45:55.957740 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:45:55.957711    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g67gp"
Jul 28 17:46:05.041244 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:46:05.041211    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:46:08.228936 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:46:08.228907    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:46:40.721644 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:46:40.721567    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:46:40.823279 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:46:40.823240    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:47:02.589282 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:47:02.589247    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:47:02.678917 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:47:02.678885    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:47:03.827596 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:47:03.827562    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:47:04.077574 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:47:04.077540    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:49:11.672207 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:49:11.672170    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:49:13.190894 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:49:13.190798    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:49:48.758380 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:49:48.758321    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:49:50.080904 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:49:50.080870    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:49:55.904267 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:49:55.904230    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:49:56.005692 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:49:56.005643    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:49:56.105616 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:49:56.105574    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:49:56.293936 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:49:56.293850    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:50:03.308367 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:50:03.308327    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:50:03.435896 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:50:03.435868    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:50:09.870356 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:50:09.870321    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:51:13.470746 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:51:13.470707    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:51:13.570163 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:51:13.570129    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:52:03.482276 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:52:03.482239    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:52:03.582081 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:52:03.582046    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:52:51.700456 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:52:51.700400    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:54:22.343353 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:54:22.343312    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j26kf"]
Jul 28 17:54:22.343892 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:54:22.343543    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j26kf" podUID="a81f2137-7e96-4a34-bcf8-febcc470dde4" containerName="manager" containerID="cri-o://db31824b2905d606371a201ee02b1098789658a5c33662ff09f53a8cfa63f2a6" gracePeriod=10
Jul 28 17:54:25.501546 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:54:25.501514    2563 generic.go:356] "Generic (PLEG): container finished" podID="a81f2137-7e96-4a34-bcf8-febcc470dde4" containerID="db31824b2905d606371a201ee02b1098789658a5c33662ff09f53a8cfa63f2a6" exitCode=0
Jul 28 17:54:25.501848 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:54:25.501586    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j26kf" event={"ID":"a81f2137-7e96-4a34-bcf8-febcc470dde4","Type":"ContainerDied","Data":"db31824b2905d606371a201ee02b1098789658a5c33662ff09f53a8cfa63f2a6"}
Jul 28 17:54:26.005286 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:54:26.005262    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j26kf"
Jul 28 17:54:26.088343 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:54:26.088317    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a81f2137-7e96-4a34-bcf8-febcc470dde4-extensions-socket-volume\") pod \"a81f2137-7e96-4a34-bcf8-febcc470dde4\" (UID: \"a81f2137-7e96-4a34-bcf8-febcc470dde4\") "
Jul 28 17:54:26.088517 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:54:26.088414    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-skk4w\" (UniqueName: \"kubernetes.io/projected/a81f2137-7e96-4a34-bcf8-febcc470dde4-kube-api-access-skk4w\") pod \"a81f2137-7e96-4a34-bcf8-febcc470dde4\" (UID: \"a81f2137-7e96-4a34-bcf8-febcc470dde4\") "
Jul 28 17:54:26.088769 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:54:26.088746    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a81f2137-7e96-4a34-bcf8-febcc470dde4-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a81f2137-7e96-4a34-bcf8-febcc470dde4" (UID: "a81f2137-7e96-4a34-bcf8-febcc470dde4"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 17:54:26.090368 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:54:26.090343    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a81f2137-7e96-4a34-bcf8-febcc470dde4-kube-api-access-skk4w" (OuterVolumeSpecName: "kube-api-access-skk4w") pod "a81f2137-7e96-4a34-bcf8-febcc470dde4" (UID: "a81f2137-7e96-4a34-bcf8-febcc470dde4"). InnerVolumeSpecName "kube-api-access-skk4w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:54:26.189813 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:54:26.189747    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-skk4w\" (UniqueName: \"kubernetes.io/projected/a81f2137-7e96-4a34-bcf8-febcc470dde4-kube-api-access-skk4w\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:54:26.189813 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:54:26.189768    2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a81f2137-7e96-4a34-bcf8-febcc470dde4-extensions-socket-volume\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 17:54:26.508151 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:54:26.508066    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j26kf"
Jul 28 17:54:26.508151 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:54:26.508104    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j26kf" event={"ID":"a81f2137-7e96-4a34-bcf8-febcc470dde4","Type":"ContainerDied","Data":"9c2e98a474ba33879522f361df782aa1696a736396f491fab12eac3048ece3a1"}
Jul 28 17:54:26.508627 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:54:26.508162    2563 scope.go:117] "RemoveContainer" containerID="db31824b2905d606371a201ee02b1098789658a5c33662ff09f53a8cfa63f2a6"
Jul 28 17:54:26.534248 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:54:26.534216    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j26kf"]
Jul 28 17:54:26.538843 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:54:26.538822    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j26kf"]
Jul 28 17:54:27.852753 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:54:27.852722    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a81f2137-7e96-4a34-bcf8-febcc470dde4" path="/var/lib/kubelet/pods/a81f2137-7e96-4a34-bcf8-febcc470dde4/volumes"
Jul 28 17:55:23.675449 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:55:23.675406    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6gbws"]
Jul 28 17:55:23.675833 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:55:23.675748    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a81f2137-7e96-4a34-bcf8-febcc470dde4" containerName="manager"
Jul 28 17:55:23.675833 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:55:23.675760    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a81f2137-7e96-4a34-bcf8-febcc470dde4" containerName="manager"
Jul 28 17:55:23.675833 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:55:23.675821    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a81f2137-7e96-4a34-bcf8-febcc470dde4" containerName="manager"
Jul 28 17:55:23.678931 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:55:23.678915    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6gbws"
Jul 28 17:55:23.681419 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:55:23.681399    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-z9zdn\""
Jul 28 17:55:23.690867 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:55:23.690839    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6gbws"]
Jul 28 17:55:23.764501 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:55:23.764460    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfgnt\" (UniqueName: \"kubernetes.io/projected/f61cb9c1-bef2-44df-b009-93d12dee3429-kube-api-access-nfgnt\") pod \"kuadrant-operator-controller-manager-86878c769f-6gbws\" (UID: \"f61cb9c1-bef2-44df-b009-93d12dee3429\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6gbws"
Jul 28 17:55:23.764658 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:55:23.764544    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f61cb9c1-bef2-44df-b009-93d12dee3429-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6gbws\" (UID: \"f61cb9c1-bef2-44df-b009-93d12dee3429\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6gbws"
Jul 28 17:55:23.864935 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:55:23.864906    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f61cb9c1-bef2-44df-b009-93d12dee3429-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6gbws\" (UID: \"f61cb9c1-bef2-44df-b009-93d12dee3429\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6gbws"
Jul 28 17:55:23.865160 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:55:23.865140    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfgnt\" (UniqueName: \"kubernetes.io/projected/f61cb9c1-bef2-44df-b009-93d12dee3429-kube-api-access-nfgnt\") pod \"kuadrant-operator-controller-manager-86878c769f-6gbws\" (UID: \"f61cb9c1-bef2-44df-b009-93d12dee3429\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6gbws"
Jul 28 17:55:23.865302 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:55:23.865282    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f61cb9c1-bef2-44df-b009-93d12dee3429-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6gbws\" (UID: \"f61cb9c1-bef2-44df-b009-93d12dee3429\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6gbws"
Jul 28 17:55:23.874879 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:55:23.874849    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfgnt\" (UniqueName: \"kubernetes.io/projected/f61cb9c1-bef2-44df-b009-93d12dee3429-kube-api-access-nfgnt\") pod \"kuadrant-operator-controller-manager-86878c769f-6gbws\" (UID: \"f61cb9c1-bef2-44df-b009-93d12dee3429\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6gbws"
Jul 28 17:55:23.989571 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:55:23.989509    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6gbws"
Jul 28 17:55:24.131726 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:55:24.131696    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6gbws"]
Jul 28 17:55:24.133708 ip-10-0-136-126 kubenswrapper[2563]: W0728 17:55:24.133680    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf61cb9c1_bef2_44df_b009_93d12dee3429.slice/crio-578e158e154c8cfa79cb184c6526e25b2234d1bf44dfd8593ac9b88bc835be2d WatchSource:0}: Error finding container 578e158e154c8cfa79cb184c6526e25b2234d1bf44dfd8593ac9b88bc835be2d: Status 404 returned error can't find the container with id 578e158e154c8cfa79cb184c6526e25b2234d1bf44dfd8593ac9b88bc835be2d
Jul 28 17:55:24.805730 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:55:24.805690    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6gbws" event={"ID":"f61cb9c1-bef2-44df-b009-93d12dee3429","Type":"ContainerStarted","Data":"276f6d618069ee41272c1c7cfc3d56e5065f323a9117d1993ef58a893b8f8705"}
Jul 28 17:55:24.806114 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:55:24.805760    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6gbws"
Jul 28 17:55:24.806114 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:55:24.805783    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6gbws" event={"ID":"f61cb9c1-bef2-44df-b009-93d12dee3429","Type":"ContainerStarted","Data":"578e158e154c8cfa79cb184c6526e25b2234d1bf44dfd8593ac9b88bc835be2d"}
Jul 28 17:55:24.831867 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:55:24.831801    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6gbws" podStartSLOduration=1.831784638 podStartE2EDuration="1.831784638s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:55:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:55:24.829179613 +0000 UTC m=+1263.490335968" watchObservedRunningTime="2026-07-28 17:55:24.831784638 +0000 UTC m=+1263.492940994"
Jul 28 17:55:35.813660 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:55:35.813629    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6gbws"
Jul 28 17:58:23.121766 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:58:23.121683    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:58:48.328545 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:58:48.328511    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:58:52.749134 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:58:52.749081    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:59:50.869371 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:59:50.869290    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:59:50.945559 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:59:50.945531    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 17:59:52.103608 ip-10-0-136-126 kubenswrapper[2563]: I0728 17:59:52.103544    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:00:00.155488 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:00:00.155430    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754360-n2m5s"]
Jul 28 18:00:00.158867 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:00:00.158849    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754360-n2m5s"
Jul 28 18:00:00.161554 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:00:00.161531    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9xcv2\""
Jul 28 18:00:00.168821 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:00:00.168796    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754360-n2m5s"]
Jul 28 18:00:00.169015 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:00:00.168998    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754360-n2m5s"
Jul 28 18:00:00.512635 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:00:00.512614    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754360-n2m5s"]
Jul 28 18:00:00.514029 ip-10-0-136-126 kubenswrapper[2563]: W0728 18:00:00.514002    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab4c837e_4522_4662_b6f9_60c6cb3566b5.slice/crio-083487dde3ca2aa8ef488d242f280510455b3ee18269a47ec165cf9f5823fd8d WatchSource:0}: Error finding container 083487dde3ca2aa8ef488d242f280510455b3ee18269a47ec165cf9f5823fd8d: Status 404 returned error can't find the container with id 083487dde3ca2aa8ef488d242f280510455b3ee18269a47ec165cf9f5823fd8d
Jul 28 18:00:00.515940 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:00:00.515924    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 18:00:01.242007 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:00:01.241974    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754360-n2m5s" event={"ID":"ab4c837e-4522-4662-b6f9-60c6cb3566b5","Type":"ContainerStarted","Data":"083487dde3ca2aa8ef488d242f280510455b3ee18269a47ec165cf9f5823fd8d"}
Jul 28 18:00:05.264956 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:00:05.264918    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754360-n2m5s" event={"ID":"ab4c837e-4522-4662-b6f9-60c6cb3566b5","Type":"ContainerStarted","Data":"48267e98fe543bd1c2e2751b9880e940707ab73895c3e599dda6be96f4ff94dc"}
Jul 28 18:00:05.286021 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:00:05.285980    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754360-n2m5s" podStartSLOduration=1.129260866 podStartE2EDuration="5.285967087s" totalImagesPullingTime="4.156706221s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:00:05.283381916 +0000 UTC m=+1543.944538272" watchObservedRunningTime="2026-07-28 18:00:05.285967087 +0000 UTC m=+1543.947123442"
Jul 28 18:00:06.271187 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:00:06.271150    2563 generic.go:356] "Generic (PLEG): container finished" podID="ab4c837e-4522-4662-b6f9-60c6cb3566b5" containerID="48267e98fe543bd1c2e2751b9880e940707ab73895c3e599dda6be96f4ff94dc" exitCode=0
Jul 28 18:00:06.271661 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:00:06.271235    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754360-n2m5s" event={"ID":"ab4c837e-4522-4662-b6f9-60c6cb3566b5","Type":"ContainerDied","Data":"48267e98fe543bd1c2e2751b9880e940707ab73895c3e599dda6be96f4ff94dc"}
Jul 28 18:00:07.413354 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:00:07.413329    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754360-n2m5s"
Jul 28 18:00:08.281999 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:00:08.281920    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754360-n2m5s"
Jul 28 18:00:08.281999 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:00:08.281963    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754360-n2m5s" event={"ID":"ab4c837e-4522-4662-b6f9-60c6cb3566b5","Type":"ContainerDied","Data":"083487dde3ca2aa8ef488d242f280510455b3ee18269a47ec165cf9f5823fd8d"}
Jul 28 18:00:08.282180 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:00:08.282010    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="083487dde3ca2aa8ef488d242f280510455b3ee18269a47ec165cf9f5823fd8d"
Jul 28 18:01:52.950786 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:01:52.950751    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:03:14.573253 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:03:14.573221    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:03:38.729729 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:03:38.729698    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:03:42.191032 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:03:42.190997    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:03:42.291416 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:03:42.291375    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:03:42.717529 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:03:42.717497    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:03:49.422558 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:03:49.422524    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:03:49.492133 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:03:49.492098    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:03:49.703570 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:03:49.703500    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:03:50.059742 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:03:50.059667    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:03:50.452661 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:03:50.452624    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:03:52.730096 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:03:52.730055    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:03:52.846242 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:03:52.846200    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:03:52.853013 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:03:52.852983    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-3fe65036-545c78b754-49xhg"]
Jul 28 18:03:52.853325 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:03:52.853314    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab4c837e-4522-4662-b6f9-60c6cb3566b5" containerName="cleanup"
Jul 28 18:03:52.853369 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:03:52.853327    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab4c837e-4522-4662-b6f9-60c6cb3566b5" containerName="cleanup"
Jul 28 18:03:52.853409 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:03:52.853398    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab4c837e-4522-4662-b6f9-60c6cb3566b5" containerName="cleanup"
Jul 28 18:03:52.856342 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:03:52.856324    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3fe65036-545c78b754-49xhg"
Jul 28 18:03:52.858766 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:03:52.858749    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-3fe65036-serving-cert\""
Jul 28 18:03:52.870949 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:03:52.870928    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-3fe65036-545c78b754-49xhg"]
Jul 28 18:03:52.880018 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:03:52.879992    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d23409b9-3fb3-4c3c-a70a-7938519dabf1-maas-api-tls\") pod \"maas-api-e2e-ait-3fe65036-545c78b754-49xhg\" (UID: \"d23409b9-3fb3-4c3c-a70a-7938519dabf1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3fe65036-545c78b754-49xhg"
Jul 28 18:03:52.880144 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:03:52.880055    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d23409b9-3fb3-4c3c-a70a-7938519dabf1-openshift-service-ca\") pod \"maas-api-e2e-ait-3fe65036-545c78b754-49xhg\" (UID: \"d23409b9-3fb3-4c3c-a70a-7938519dabf1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3fe65036-545c78b754-49xhg"
Jul 28 18:03:52.880144 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:03:52.880092    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9ckm\" (UniqueName: \"kubernetes.io/projected/d23409b9-3fb3-4c3c-a70a-7938519dabf1-kube-api-access-x9ckm\") pod \"maas-api-e2e-ait-3fe65036-545c78b754-49xhg\" (UID: \"d23409b9-3fb3-4c3c-a70a-7938519dabf1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3fe65036-545c78b754-49xhg"
Jul 28 18:03:52.932619 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:03:52.932578    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:03:52.981398 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:03:52.981311    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d23409b9-3fb3-4c3c-a70a-7938519dabf1-maas-api-tls\") pod \"maas-api-e2e-ait-3fe65036-545c78b754-49xhg\" (UID: \"d23409b9-3fb3-4c3c-a70a-7938519dabf1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3fe65036-545c78b754-49xhg"
Jul 28 18:03:52.981398 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:03:52.981362    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d23409b9-3fb3-4c3c-a70a-7938519dabf1-openshift-service-ca\") pod \"maas-api-e2e-ait-3fe65036-545c78b754-49xhg\" (UID: \"d23409b9-3fb3-4c3c-a70a-7938519dabf1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3fe65036-545c78b754-49xhg"
Jul 28 18:03:52.981398 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:03:52.981395    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9ckm\" (UniqueName: \"kubernetes.io/projected/d23409b9-3fb3-4c3c-a70a-7938519dabf1-kube-api-access-x9ckm\") pod \"maas-api-e2e-ait-3fe65036-545c78b754-49xhg\" (UID: \"d23409b9-3fb3-4c3c-a70a-7938519dabf1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3fe65036-545c78b754-49xhg"
Jul 28 18:03:52.981710 ip-10-0-136-126 kubenswrapper[2563]: E0728 18:03:52.981488    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-3fe65036-serving-cert: secret "maas-api-e2e-ait-3fe65036-serving-cert" not found
Jul 28 18:03:52.981710 ip-10-0-136-126 kubenswrapper[2563]: E0728 18:03:52.981555    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d23409b9-3fb3-4c3c-a70a-7938519dabf1-maas-api-tls podName:d23409b9-3fb3-4c3c-a70a-7938519dabf1 nodeName:}" failed. No retries permitted until 2026-07-28 18:03:53.481538741 +0000 UTC m=+1772.142695074 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d23409b9-3fb3-4c3c-a70a-7938519dabf1-maas-api-tls") pod "maas-api-e2e-ait-3fe65036-545c78b754-49xhg" (UID: "d23409b9-3fb3-4c3c-a70a-7938519dabf1") : secret "maas-api-e2e-ait-3fe65036-serving-cert" not found
Jul 28 18:03:52.982066 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:03:52.982048    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d23409b9-3fb3-4c3c-a70a-7938519dabf1-openshift-service-ca\") pod \"maas-api-e2e-ait-3fe65036-545c78b754-49xhg\" (UID: \"d23409b9-3fb3-4c3c-a70a-7938519dabf1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3fe65036-545c78b754-49xhg"
Jul 28 18:03:52.992113 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:03:52.992090    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9ckm\" (UniqueName: \"kubernetes.io/projected/d23409b9-3fb3-4c3c-a70a-7938519dabf1-kube-api-access-x9ckm\") pod \"maas-api-e2e-ait-3fe65036-545c78b754-49xhg\" (UID: \"d23409b9-3fb3-4c3c-a70a-7938519dabf1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3fe65036-545c78b754-49xhg"
Jul 28 18:03:53.485364 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:03:53.485334    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d23409b9-3fb3-4c3c-a70a-7938519dabf1-maas-api-tls\") pod \"maas-api-e2e-ait-3fe65036-545c78b754-49xhg\" (UID: \"d23409b9-3fb3-4c3c-a70a-7938519dabf1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3fe65036-545c78b754-49xhg"
Jul 28 18:03:53.485523 ip-10-0-136-126 kubenswrapper[2563]: E0728 18:03:53.485495    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-3fe65036-serving-cert: secret "maas-api-e2e-ait-3fe65036-serving-cert" not found
Jul 28 18:03:53.485577 ip-10-0-136-126 kubenswrapper[2563]: E0728 18:03:53.485562    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d23409b9-3fb3-4c3c-a70a-7938519dabf1-maas-api-tls podName:d23409b9-3fb3-4c3c-a70a-7938519dabf1 nodeName:}" failed. No retries permitted until 2026-07-28 18:03:54.485547154 +0000 UTC m=+1773.146703490 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d23409b9-3fb3-4c3c-a70a-7938519dabf1-maas-api-tls") pod "maas-api-e2e-ait-3fe65036-545c78b754-49xhg" (UID: "d23409b9-3fb3-4c3c-a70a-7938519dabf1") : secret "maas-api-e2e-ait-3fe65036-serving-cert" not found
Jul 28 18:03:53.642152 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:03:53.642121    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:03:53.738112 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:03:53.738037    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:03:53.858342 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:03:53.858314    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:03:54.495319 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:03:54.495290    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d23409b9-3fb3-4c3c-a70a-7938519dabf1-maas-api-tls\") pod \"maas-api-e2e-ait-3fe65036-545c78b754-49xhg\" (UID: \"d23409b9-3fb3-4c3c-a70a-7938519dabf1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3fe65036-545c78b754-49xhg"
Jul 28 18:03:54.497649 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:03:54.497631    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d23409b9-3fb3-4c3c-a70a-7938519dabf1-maas-api-tls\") pod \"maas-api-e2e-ait-3fe65036-545c78b754-49xhg\" (UID: \"d23409b9-3fb3-4c3c-a70a-7938519dabf1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3fe65036-545c78b754-49xhg"
Jul 28 18:03:54.667054 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:03:54.667017    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3fe65036-545c78b754-49xhg"
Jul 28 18:03:54.803312 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:03:54.803287    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-3fe65036-545c78b754-49xhg"]
Jul 28 18:03:54.805084 ip-10-0-136-126 kubenswrapper[2563]: W0728 18:03:54.805060    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd23409b9_3fb3_4c3c_a70a_7938519dabf1.slice/crio-f786cf71efbef10d6994ae490c7855466aa3219f63adf3009c3a0664343dd9cc WatchSource:0}: Error finding container f786cf71efbef10d6994ae490c7855466aa3219f63adf3009c3a0664343dd9cc: Status 404 returned error can't find the container with id f786cf71efbef10d6994ae490c7855466aa3219f63adf3009c3a0664343dd9cc
Jul 28 18:03:55.430583 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:03:55.430549    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3fe65036-545c78b754-49xhg" event={"ID":"d23409b9-3fb3-4c3c-a70a-7938519dabf1","Type":"ContainerStarted","Data":"f786cf71efbef10d6994ae490c7855466aa3219f63adf3009c3a0664343dd9cc"}
Jul 28 18:03:56.437025 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:03:56.436986    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3fe65036-545c78b754-49xhg" event={"ID":"d23409b9-3fb3-4c3c-a70a-7938519dabf1","Type":"ContainerStarted","Data":"786f49235f2033ee7835361e147fe53464963ddcb4b5439898bd5a15ea332347"}
Jul 28 18:03:56.437429 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:03:56.437074    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3fe65036-545c78b754-49xhg"
Jul 28 18:03:56.455873 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:03:56.455814    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3fe65036-545c78b754-49xhg" podStartSLOduration=3.819760715 podStartE2EDuration="4.455795726s" totalImagesPullingTime="636.035011ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:03:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:03:56.455487856 +0000 UTC m=+1775.116644212" watchObservedRunningTime="2026-07-28 18:03:56.455795726 +0000 UTC m=+1775.116952086"
Jul 28 18:04:02.448089 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:02.448062    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3fe65036-545c78b754-49xhg"
Jul 28 18:04:18.405918 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:18.405839    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fe65036-2rzjc"]
Jul 28 18:04:18.409646 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:18.409624    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fe65036-2rzjc"
Jul 28 18:04:18.412704 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:18.412686    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9xcv2\""
Jul 28 18:04:18.421145 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:18.421125    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fe65036-2rzjc"]
Jul 28 18:04:18.511176 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:18.511147    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ad84128-552a-4611-ab9e-ce34f275ee12-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fe65036-2rzjc\" (UID: \"1ad84128-552a-4611-ab9e-ce34f275ee12\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fe65036-2rzjc"
Jul 28 18:04:18.612413 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:18.612384    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ad84128-552a-4611-ab9e-ce34f275ee12-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fe65036-2rzjc\" (UID: \"1ad84128-552a-4611-ab9e-ce34f275ee12\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fe65036-2rzjc"
Jul 28 18:04:18.613022 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:18.613001    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ad84128-552a-4611-ab9e-ce34f275ee12-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fe65036-2rzjc\" (UID: \"1ad84128-552a-4611-ab9e-ce34f275ee12\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fe65036-2rzjc"
Jul 28 18:04:18.719497 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:18.719411    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fe65036-2rzjc"
Jul 28 18:04:18.849785 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:18.849753    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fe65036-2rzjc"]
Jul 28 18:04:18.851067 ip-10-0-136-126 kubenswrapper[2563]: W0728 18:04:18.851044    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ad84128_552a_4611_ab9e_ce34f275ee12.slice/crio-ae4a67f1bc3def51a5f394e8871074d821a998d7bbd8b5cca1655d9ba1214f44 WatchSource:0}: Error finding container ae4a67f1bc3def51a5f394e8871074d821a998d7bbd8b5cca1655d9ba1214f44: Status 404 returned error can't find the container with id ae4a67f1bc3def51a5f394e8871074d821a998d7bbd8b5cca1655d9ba1214f44
Jul 28 18:04:19.554453 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:19.554411    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fe65036-2rzjc" event={"ID":"1ad84128-552a-4611-ab9e-ce34f275ee12","Type":"ContainerStarted","Data":"a928d3de518aa8b78f8eaa12dd21189f05c085c3b574ba2d9b5365ac36a90e32"}
Jul 28 18:04:19.554771 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:19.554490    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fe65036-2rzjc" event={"ID":"1ad84128-552a-4611-ab9e-ce34f275ee12","Type":"ContainerStarted","Data":"ae4a67f1bc3def51a5f394e8871074d821a998d7bbd8b5cca1655d9ba1214f44"}
Jul 28 18:04:20.560946 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:20.560909    2563 generic.go:356] "Generic (PLEG): container finished" podID="1ad84128-552a-4611-ab9e-ce34f275ee12" containerID="a928d3de518aa8b78f8eaa12dd21189f05c085c3b574ba2d9b5365ac36a90e32" exitCode=0
Jul 28 18:04:20.561510 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:20.560995    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fe65036-2rzjc" event={"ID":"1ad84128-552a-4611-ab9e-ce34f275ee12","Type":"ContainerDied","Data":"a928d3de518aa8b78f8eaa12dd21189f05c085c3b574ba2d9b5365ac36a90e32"}
Jul 28 18:04:20.688278 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:20.688256    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fe65036-2rzjc"
Jul 28 18:04:20.831458 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:20.831406    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ad84128-552a-4611-ab9e-ce34f275ee12-maas-api-service-ca\") pod \"1ad84128-552a-4611-ab9e-ce34f275ee12\" (UID: \"1ad84128-552a-4611-ab9e-ce34f275ee12\") "
Jul 28 18:04:20.831766 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:20.831745    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ad84128-552a-4611-ab9e-ce34f275ee12-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1ad84128-552a-4611-ab9e-ce34f275ee12" (UID: "1ad84128-552a-4611-ab9e-ce34f275ee12"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:04:20.932898 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:20.932874    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ad84128-552a-4611-ab9e-ce34f275ee12-maas-api-service-ca\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:04:21.567863 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:21.567836    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fe65036-2rzjc"
Jul 28 18:04:21.568235 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:21.567836    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fe65036-2rzjc" event={"ID":"1ad84128-552a-4611-ab9e-ce34f275ee12","Type":"ContainerDied","Data":"ae4a67f1bc3def51a5f394e8871074d821a998d7bbd8b5cca1655d9ba1214f44"}
Jul 28 18:04:21.568235 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:21.567960    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ae4a67f1bc3def51a5f394e8871074d821a998d7bbd8b5cca1655d9ba1214f44"
Jul 28 18:04:23.787636 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:23.787600    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-3fe65036-545c78b754-49xhg"]
Jul 28 18:04:23.788381 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:23.788344    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3fe65036-545c78b754-49xhg" podUID="d23409b9-3fb3-4c3c-a70a-7938519dabf1" containerName="maas-api" containerID="cri-o://786f49235f2033ee7835361e147fe53464963ddcb4b5439898bd5a15ea332347" gracePeriod=30
Jul 28 18:04:23.826270 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:23.826239    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:04:23.981341 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:23.981308    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:04:24.039097 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:24.039032    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3fe65036-545c78b754-49xhg"
Jul 28 18:04:24.072969 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:24.072315    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:04:24.077736 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:24.077710    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fe65036-2rzjc"]
Jul 28 18:04:24.081034 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:24.081009    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fe65036-2rzjc"]
Jul 28 18:04:24.157643 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:24.157612    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x9ckm\" (UniqueName: \"kubernetes.io/projected/d23409b9-3fb3-4c3c-a70a-7938519dabf1-kube-api-access-x9ckm\") pod \"d23409b9-3fb3-4c3c-a70a-7938519dabf1\" (UID: \"d23409b9-3fb3-4c3c-a70a-7938519dabf1\") "
Jul 28 18:04:24.157803 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:24.157658    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d23409b9-3fb3-4c3c-a70a-7938519dabf1-openshift-service-ca\") pod \"d23409b9-3fb3-4c3c-a70a-7938519dabf1\" (UID: \"d23409b9-3fb3-4c3c-a70a-7938519dabf1\") "
Jul 28 18:04:24.157803 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:24.157691    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d23409b9-3fb3-4c3c-a70a-7938519dabf1-maas-api-tls\") pod \"d23409b9-3fb3-4c3c-a70a-7938519dabf1\" (UID: \"d23409b9-3fb3-4c3c-a70a-7938519dabf1\") "
Jul 28 18:04:24.158032 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:24.158007    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d23409b9-3fb3-4c3c-a70a-7938519dabf1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d23409b9-3fb3-4c3c-a70a-7938519dabf1" (UID: "d23409b9-3fb3-4c3c-a70a-7938519dabf1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:04:24.159950 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:24.159921    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d23409b9-3fb3-4c3c-a70a-7938519dabf1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d23409b9-3fb3-4c3c-a70a-7938519dabf1" (UID: "d23409b9-3fb3-4c3c-a70a-7938519dabf1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:04:24.159950 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:24.159930    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d23409b9-3fb3-4c3c-a70a-7938519dabf1-kube-api-access-x9ckm" (OuterVolumeSpecName: "kube-api-access-x9ckm") pod "d23409b9-3fb3-4c3c-a70a-7938519dabf1" (UID: "d23409b9-3fb3-4c3c-a70a-7938519dabf1"). InnerVolumeSpecName "kube-api-access-x9ckm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:04:24.259115 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:24.259073    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d23409b9-3fb3-4c3c-a70a-7938519dabf1-openshift-service-ca\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:04:24.259115 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:24.259108    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d23409b9-3fb3-4c3c-a70a-7938519dabf1-maas-api-tls\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:04:24.259115 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:24.259121    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x9ckm\" (UniqueName: \"kubernetes.io/projected/d23409b9-3fb3-4c3c-a70a-7938519dabf1-kube-api-access-x9ckm\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:04:24.327799 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:24.327763    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:04:24.481421 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:24.481393    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:04:24.585008 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:24.584932    2563 generic.go:356] "Generic (PLEG): container finished" podID="d23409b9-3fb3-4c3c-a70a-7938519dabf1" containerID="786f49235f2033ee7835361e147fe53464963ddcb4b5439898bd5a15ea332347" exitCode=0
Jul 28 18:04:24.585008 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:24.584984    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3fe65036-545c78b754-49xhg"
Jul 28 18:04:24.585197 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:24.585009    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3fe65036-545c78b754-49xhg" event={"ID":"d23409b9-3fb3-4c3c-a70a-7938519dabf1","Type":"ContainerDied","Data":"786f49235f2033ee7835361e147fe53464963ddcb4b5439898bd5a15ea332347"}
Jul 28 18:04:24.585197 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:24.585080    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3fe65036-545c78b754-49xhg" event={"ID":"d23409b9-3fb3-4c3c-a70a-7938519dabf1","Type":"ContainerDied","Data":"f786cf71efbef10d6994ae490c7855466aa3219f63adf3009c3a0664343dd9cc"}
Jul 28 18:04:24.585197 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:24.585102    2563 scope.go:117] "RemoveContainer" containerID="786f49235f2033ee7835361e147fe53464963ddcb4b5439898bd5a15ea332347"
Jul 28 18:04:24.594143 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:24.594126    2563 scope.go:117] "RemoveContainer" containerID="786f49235f2033ee7835361e147fe53464963ddcb4b5439898bd5a15ea332347"
Jul 28 18:04:24.594391 ip-10-0-136-126 kubenswrapper[2563]: E0728 18:04:24.594370    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"786f49235f2033ee7835361e147fe53464963ddcb4b5439898bd5a15ea332347\": container with ID starting with 786f49235f2033ee7835361e147fe53464963ddcb4b5439898bd5a15ea332347 not found: ID does not exist" containerID="786f49235f2033ee7835361e147fe53464963ddcb4b5439898bd5a15ea332347"
Jul 28 18:04:24.594461 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:24.594401    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"786f49235f2033ee7835361e147fe53464963ddcb4b5439898bd5a15ea332347"} err="failed to get container status \"786f49235f2033ee7835361e147fe53464963ddcb4b5439898bd5a15ea332347\": rpc error: code = NotFound desc = could not find container \"786f49235f2033ee7835361e147fe53464963ddcb4b5439898bd5a15ea332347\": container with ID starting with 786f49235f2033ee7835361e147fe53464963ddcb4b5439898bd5a15ea332347 not found: ID does not exist"
Jul 28 18:04:24.608333 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:24.608311    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-3fe65036-545c78b754-49xhg"]
Jul 28 18:04:24.613784 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:24.613765    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-3fe65036-545c78b754-49xhg"]
Jul 28 18:04:24.745117 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:24.745083    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:04:24.846451 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:24.846354    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:04:24.992138 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:24.992108    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:04:25.336807 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:25.336767    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:04:25.853414 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:25.853386    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ad84128-552a-4611-ab9e-ce34f275ee12" path="/var/lib/kubelet/pods/1ad84128-552a-4611-ab9e-ce34f275ee12/volumes"
Jul 28 18:04:25.853795 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:25.853781    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d23409b9-3fb3-4c3c-a70a-7938519dabf1" path="/var/lib/kubelet/pods/d23409b9-3fb3-4c3c-a70a-7938519dabf1/volumes"
Jul 28 18:04:26.018328 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:26.018299    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:04:28.062149 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:28.062116    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:04:28.221640 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:28.221609    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-6ad02c46-dbbbfdf47-x2l2q"]
Jul 28 18:04:28.221959 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:28.221947    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ad84128-552a-4611-ab9e-ce34f275ee12" containerName="revoke-keys"
Jul 28 18:04:28.222004 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:28.221960    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ad84128-552a-4611-ab9e-ce34f275ee12" containerName="revoke-keys"
Jul 28 18:04:28.222004 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:28.221974    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d23409b9-3fb3-4c3c-a70a-7938519dabf1" containerName="maas-api"
Jul 28 18:04:28.222004 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:28.221979    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d23409b9-3fb3-4c3c-a70a-7938519dabf1" containerName="maas-api"
Jul 28 18:04:28.222109 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:28.222027    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d23409b9-3fb3-4c3c-a70a-7938519dabf1" containerName="maas-api"
Jul 28 18:04:28.222109 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:28.222036    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ad84128-552a-4611-ab9e-ce34f275ee12" containerName="revoke-keys"
Jul 28 18:04:28.226316 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:28.226300    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6ad02c46-dbbbfdf47-x2l2q"
Jul 28 18:04:28.229628 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:28.229610    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-6ad02c46-serving-cert\""
Jul 28 18:04:28.262007 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:28.261981    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-6ad02c46-dbbbfdf47-x2l2q"]
Jul 28 18:04:28.270472 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:28.270447    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:04:28.396628 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:28.396599    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjr7f\" (UniqueName: \"kubernetes.io/projected/cab6e009-1e20-4392-9600-38bf679d58d0-kube-api-access-hjr7f\") pod \"maas-api-e2e-migrate-6ad02c46-dbbbfdf47-x2l2q\" (UID: \"cab6e009-1e20-4392-9600-38bf679d58d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6ad02c46-dbbbfdf47-x2l2q"
Jul 28 18:04:28.396765 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:28.396638    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cab6e009-1e20-4392-9600-38bf679d58d0-openshift-service-ca\") pod \"maas-api-e2e-migrate-6ad02c46-dbbbfdf47-x2l2q\" (UID: \"cab6e009-1e20-4392-9600-38bf679d58d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6ad02c46-dbbbfdf47-x2l2q"
Jul 28 18:04:28.396765 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:28.396745    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cab6e009-1e20-4392-9600-38bf679d58d0-maas-api-tls\") pod \"maas-api-e2e-migrate-6ad02c46-dbbbfdf47-x2l2q\" (UID: \"cab6e009-1e20-4392-9600-38bf679d58d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6ad02c46-dbbbfdf47-x2l2q"
Jul 28 18:04:28.497833 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:28.497802    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjr7f\" (UniqueName: \"kubernetes.io/projected/cab6e009-1e20-4392-9600-38bf679d58d0-kube-api-access-hjr7f\") pod \"maas-api-e2e-migrate-6ad02c46-dbbbfdf47-x2l2q\" (UID: \"cab6e009-1e20-4392-9600-38bf679d58d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6ad02c46-dbbbfdf47-x2l2q"
Jul 28 18:04:28.498041 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:28.497839    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cab6e009-1e20-4392-9600-38bf679d58d0-openshift-service-ca\") pod \"maas-api-e2e-migrate-6ad02c46-dbbbfdf47-x2l2q\" (UID: \"cab6e009-1e20-4392-9600-38bf679d58d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6ad02c46-dbbbfdf47-x2l2q"
Jul 28 18:04:28.498041 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:28.497889    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cab6e009-1e20-4392-9600-38bf679d58d0-maas-api-tls\") pod \"maas-api-e2e-migrate-6ad02c46-dbbbfdf47-x2l2q\" (UID: \"cab6e009-1e20-4392-9600-38bf679d58d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6ad02c46-dbbbfdf47-x2l2q"
Jul 28 18:04:28.498563 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:28.498541    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cab6e009-1e20-4392-9600-38bf679d58d0-openshift-service-ca\") pod \"maas-api-e2e-migrate-6ad02c46-dbbbfdf47-x2l2q\" (UID: \"cab6e009-1e20-4392-9600-38bf679d58d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6ad02c46-dbbbfdf47-x2l2q"
Jul 28 18:04:28.500302 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:28.500285    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cab6e009-1e20-4392-9600-38bf679d58d0-maas-api-tls\") pod \"maas-api-e2e-migrate-6ad02c46-dbbbfdf47-x2l2q\" (UID: \"cab6e009-1e20-4392-9600-38bf679d58d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6ad02c46-dbbbfdf47-x2l2q"
Jul 28 18:04:28.508925 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:28.508898    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjr7f\" (UniqueName: \"kubernetes.io/projected/cab6e009-1e20-4392-9600-38bf679d58d0-kube-api-access-hjr7f\") pod \"maas-api-e2e-migrate-6ad02c46-dbbbfdf47-x2l2q\" (UID: \"cab6e009-1e20-4392-9600-38bf679d58d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6ad02c46-dbbbfdf47-x2l2q"
Jul 28 18:04:28.522346 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:28.522327    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:04:28.536344 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:28.536320    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6ad02c46-dbbbfdf47-x2l2q"
Jul 28 18:04:28.880993 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:28.880967    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-6ad02c46-dbbbfdf47-x2l2q"]
Jul 28 18:04:28.882552 ip-10-0-136-126 kubenswrapper[2563]: W0728 18:04:28.882520    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcab6e009_1e20_4392_9600_38bf679d58d0.slice/crio-b71a5904aed18d75839d3860fe345889de3d9ce959f6f0b39a02ca3d4fc5479c WatchSource:0}: Error finding container b71a5904aed18d75839d3860fe345889de3d9ce959f6f0b39a02ca3d4fc5479c: Status 404 returned error can't find the container with id b71a5904aed18d75839d3860fe345889de3d9ce959f6f0b39a02ca3d4fc5479c
Jul 28 18:04:28.898049 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:28.898026    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:04:28.999363 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:28.999335    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:04:29.613203 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:29.613159    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6ad02c46-dbbbfdf47-x2l2q" event={"ID":"cab6e009-1e20-4392-9600-38bf679d58d0","Type":"ContainerStarted","Data":"9ededb97a552f93e75b9503a9d478cf2133b10d877485e5ff25d8d0b7afe9e2d"}
Jul 28 18:04:29.613773 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:29.613228    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6ad02c46-dbbbfdf47-x2l2q" event={"ID":"cab6e009-1e20-4392-9600-38bf679d58d0","Type":"ContainerStarted","Data":"b71a5904aed18d75839d3860fe345889de3d9ce959f6f0b39a02ca3d4fc5479c"}
Jul 28 18:04:29.613773 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:29.613348    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6ad02c46-dbbbfdf47-x2l2q"
Jul 28 18:04:29.633561 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:29.633513    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6ad02c46-dbbbfdf47-x2l2q" podStartSLOduration=1.322953179 podStartE2EDuration="1.633500331s" totalImagesPullingTime="310.547152ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:04:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:04:29.631915084 +0000 UTC m=+1808.293071441" watchObservedRunningTime="2026-07-28 18:04:29.633500331 +0000 UTC m=+1808.294656685"
Jul 28 18:04:35.624832 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:35.624803    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6ad02c46-dbbbfdf47-x2l2q"
Jul 28 18:04:53.143564 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:53.143532    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6ad02c46-g2nzs"]
Jul 28 18:04:53.147215 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:53.147198    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6ad02c46-g2nzs"
Jul 28 18:04:53.150536 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:53.150516    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9xcv2\""
Jul 28 18:04:53.163880 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:53.163858    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6ad02c46-g2nzs"]
Jul 28 18:04:53.202370 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:53.202345    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9dcbcd0-735f-4db9-8e2f-bf7f7e38858e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-6ad02c46-g2nzs\" (UID: \"f9dcbcd0-735f-4db9-8e2f-bf7f7e38858e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6ad02c46-g2nzs"
Jul 28 18:04:53.303719 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:53.303681    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9dcbcd0-735f-4db9-8e2f-bf7f7e38858e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-6ad02c46-g2nzs\" (UID: \"f9dcbcd0-735f-4db9-8e2f-bf7f7e38858e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6ad02c46-g2nzs"
Jul 28 18:04:53.304313 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:53.304291    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9dcbcd0-735f-4db9-8e2f-bf7f7e38858e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-6ad02c46-g2nzs\" (UID: \"f9dcbcd0-735f-4db9-8e2f-bf7f7e38858e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6ad02c46-g2nzs"
Jul 28 18:04:53.457243 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:53.457160    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6ad02c46-g2nzs"
Jul 28 18:04:53.587807 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:53.587783    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6ad02c46-g2nzs"]
Jul 28 18:04:53.590171 ip-10-0-136-126 kubenswrapper[2563]: W0728 18:04:53.590135    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9dcbcd0_735f_4db9_8e2f_bf7f7e38858e.slice/crio-81bb0cec4f273598d7f126f846993ad3a0e059a94e4f236cf63cbee859cf320f WatchSource:0}: Error finding container 81bb0cec4f273598d7f126f846993ad3a0e059a94e4f236cf63cbee859cf320f: Status 404 returned error can't find the container with id 81bb0cec4f273598d7f126f846993ad3a0e059a94e4f236cf63cbee859cf320f
Jul 28 18:04:53.740428 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:53.740349    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6ad02c46-g2nzs" event={"ID":"f9dcbcd0-735f-4db9-8e2f-bf7f7e38858e","Type":"ContainerStarted","Data":"81bb0cec4f273598d7f126f846993ad3a0e059a94e4f236cf63cbee859cf320f"}
Jul 28 18:04:54.746562 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:54.746531    2563 generic.go:356] "Generic (PLEG): container finished" podID="f9dcbcd0-735f-4db9-8e2f-bf7f7e38858e" containerID="fbdaafaf81f306363c0778f50007d5ed48d0cdf23b74989fa91695a615ec3e69" exitCode=0
Jul 28 18:04:54.746984 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:54.746566    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6ad02c46-g2nzs" event={"ID":"f9dcbcd0-735f-4db9-8e2f-bf7f7e38858e","Type":"ContainerDied","Data":"fbdaafaf81f306363c0778f50007d5ed48d0cdf23b74989fa91695a615ec3e69"}
Jul 28 18:04:55.888350 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:55.888326    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6ad02c46-g2nzs"
Jul 28 18:04:56.028200 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:56.028125    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9dcbcd0-735f-4db9-8e2f-bf7f7e38858e-maas-api-service-ca\") pod \"f9dcbcd0-735f-4db9-8e2f-bf7f7e38858e\" (UID: \"f9dcbcd0-735f-4db9-8e2f-bf7f7e38858e\") "
Jul 28 18:04:56.028476 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:56.028426    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9dcbcd0-735f-4db9-8e2f-bf7f7e38858e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f9dcbcd0-735f-4db9-8e2f-bf7f7e38858e" (UID: "f9dcbcd0-735f-4db9-8e2f-bf7f7e38858e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:04:56.129352 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:56.129328    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9dcbcd0-735f-4db9-8e2f-bf7f7e38858e-maas-api-service-ca\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:04:56.757629 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:56.757590    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6ad02c46-g2nzs" event={"ID":"f9dcbcd0-735f-4db9-8e2f-bf7f7e38858e","Type":"ContainerDied","Data":"81bb0cec4f273598d7f126f846993ad3a0e059a94e4f236cf63cbee859cf320f"}
Jul 28 18:04:56.757629 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:56.757626    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6ad02c46-g2nzs"
Jul 28 18:04:56.757847 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:56.757676    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="81bb0cec4f273598d7f126f846993ad3a0e059a94e4f236cf63cbee859cf320f"
Jul 28 18:04:57.857302 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:57.857276    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-6ad02c46-dbbbfdf47-x2l2q"]
Jul 28 18:04:57.857688 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:57.857463    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6ad02c46-dbbbfdf47-x2l2q" podUID="cab6e009-1e20-4392-9600-38bf679d58d0" containerName="maas-api" containerID="cri-o://9ededb97a552f93e75b9503a9d478cf2133b10d877485e5ff25d8d0b7afe9e2d" gracePeriod=30
Jul 28 18:04:57.887864 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:57.887840    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:04:57.984212 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:57.984184    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:04:58.081698 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:58.081665    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:04:58.097168 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:58.097148    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6ad02c46-dbbbfdf47-x2l2q"
Jul 28 18:04:58.152030 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:58.151960    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6ad02c46-g2nzs"]
Jul 28 18:04:58.158174 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:58.158152    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6ad02c46-g2nzs"]
Jul 28 18:04:58.178474 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:58.178424    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:04:58.254388 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:58.254364    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hjr7f\" (UniqueName: \"kubernetes.io/projected/cab6e009-1e20-4392-9600-38bf679d58d0-kube-api-access-hjr7f\") pod \"cab6e009-1e20-4392-9600-38bf679d58d0\" (UID: \"cab6e009-1e20-4392-9600-38bf679d58d0\") "
Jul 28 18:04:58.254571 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:58.254403    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cab6e009-1e20-4392-9600-38bf679d58d0-maas-api-tls\") pod \"cab6e009-1e20-4392-9600-38bf679d58d0\" (UID: \"cab6e009-1e20-4392-9600-38bf679d58d0\") "
Jul 28 18:04:58.254641 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:58.254601    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cab6e009-1e20-4392-9600-38bf679d58d0-openshift-service-ca\") pod \"cab6e009-1e20-4392-9600-38bf679d58d0\" (UID: \"cab6e009-1e20-4392-9600-38bf679d58d0\") "
Jul 28 18:04:58.254956 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:58.254922    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cab6e009-1e20-4392-9600-38bf679d58d0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cab6e009-1e20-4392-9600-38bf679d58d0" (UID: "cab6e009-1e20-4392-9600-38bf679d58d0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:04:58.255081 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:58.255063    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cab6e009-1e20-4392-9600-38bf679d58d0-openshift-service-ca\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:04:58.256645 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:58.256623    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cab6e009-1e20-4392-9600-38bf679d58d0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cab6e009-1e20-4392-9600-38bf679d58d0" (UID: "cab6e009-1e20-4392-9600-38bf679d58d0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:04:58.256742 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:58.256724    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cab6e009-1e20-4392-9600-38bf679d58d0-kube-api-access-hjr7f" (OuterVolumeSpecName: "kube-api-access-hjr7f") pod "cab6e009-1e20-4392-9600-38bf679d58d0" (UID: "cab6e009-1e20-4392-9600-38bf679d58d0"). InnerVolumeSpecName "kube-api-access-hjr7f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:04:58.342482 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:58.342427    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:04:58.356319 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:58.356295    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hjr7f\" (UniqueName: \"kubernetes.io/projected/cab6e009-1e20-4392-9600-38bf679d58d0-kube-api-access-hjr7f\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:04:58.356319 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:58.356316    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cab6e009-1e20-4392-9600-38bf679d58d0-maas-api-tls\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:04:58.579869 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:58.579842    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:04:58.768948 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:58.768914    2563 generic.go:356] "Generic (PLEG): container finished" podID="cab6e009-1e20-4392-9600-38bf679d58d0" containerID="9ededb97a552f93e75b9503a9d478cf2133b10d877485e5ff25d8d0b7afe9e2d" exitCode=0
Jul 28 18:04:58.769113 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:58.768971    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6ad02c46-dbbbfdf47-x2l2q"
Jul 28 18:04:58.769113 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:58.768965    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6ad02c46-dbbbfdf47-x2l2q" event={"ID":"cab6e009-1e20-4392-9600-38bf679d58d0","Type":"ContainerDied","Data":"9ededb97a552f93e75b9503a9d478cf2133b10d877485e5ff25d8d0b7afe9e2d"}
Jul 28 18:04:58.769113 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:58.769093    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6ad02c46-dbbbfdf47-x2l2q" event={"ID":"cab6e009-1e20-4392-9600-38bf679d58d0","Type":"ContainerDied","Data":"b71a5904aed18d75839d3860fe345889de3d9ce959f6f0b39a02ca3d4fc5479c"}
Jul 28 18:04:58.769279 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:58.769119    2563 scope.go:117] "RemoveContainer" containerID="9ededb97a552f93e75b9503a9d478cf2133b10d877485e5ff25d8d0b7afe9e2d"
Jul 28 18:04:58.778769 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:58.778753    2563 scope.go:117] "RemoveContainer" containerID="9ededb97a552f93e75b9503a9d478cf2133b10d877485e5ff25d8d0b7afe9e2d"
Jul 28 18:04:58.779050 ip-10-0-136-126 kubenswrapper[2563]: E0728 18:04:58.779025    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ededb97a552f93e75b9503a9d478cf2133b10d877485e5ff25d8d0b7afe9e2d\": container with ID starting with 9ededb97a552f93e75b9503a9d478cf2133b10d877485e5ff25d8d0b7afe9e2d not found: ID does not exist" containerID="9ededb97a552f93e75b9503a9d478cf2133b10d877485e5ff25d8d0b7afe9e2d"
Jul 28 18:04:58.779145 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:58.779058    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ededb97a552f93e75b9503a9d478cf2133b10d877485e5ff25d8d0b7afe9e2d"} err="failed to get container status \"9ededb97a552f93e75b9503a9d478cf2133b10d877485e5ff25d8d0b7afe9e2d\": rpc error: code = NotFound desc = could not find container \"9ededb97a552f93e75b9503a9d478cf2133b10d877485e5ff25d8d0b7afe9e2d\": container with ID starting with 9ededb97a552f93e75b9503a9d478cf2133b10d877485e5ff25d8d0b7afe9e2d not found: ID does not exist"
Jul 28 18:04:58.805856 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:58.805836    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-6ad02c46-dbbbfdf47-x2l2q"]
Jul 28 18:04:58.818729 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:58.818710    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-6ad02c46-dbbbfdf47-x2l2q"]
Jul 28 18:04:58.986021 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:58.985953    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:04:59.230509 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:59.230480    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:04:59.687839 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:59.687807    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:04:59.853984 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:59.853950    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cab6e009-1e20-4392-9600-38bf679d58d0" path="/var/lib/kubelet/pods/cab6e009-1e20-4392-9600-38bf679d58d0/volumes"
Jul 28 18:04:59.854394 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:04:59.854373    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9dcbcd0-735f-4db9-8e2f-bf7f7e38858e" path="/var/lib/kubelet/pods/f9dcbcd0-735f-4db9-8e2f-bf7f7e38858e/volumes"
Jul 28 18:05:02.182211 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:02.182173    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:05:02.288325 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:02.288279    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:05:02.303186 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:02.303158    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-21c55cfa-f87b85b84-rrlsl"]
Jul 28 18:05:02.303580 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:02.303565    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cab6e009-1e20-4392-9600-38bf679d58d0" containerName="maas-api"
Jul 28 18:05:02.303580 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:02.303580    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="cab6e009-1e20-4392-9600-38bf679d58d0" containerName="maas-api"
Jul 28 18:05:02.303684 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:02.303590    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9dcbcd0-735f-4db9-8e2f-bf7f7e38858e" containerName="revoke-keys"
Jul 28 18:05:02.303684 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:02.303595    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9dcbcd0-735f-4db9-8e2f-bf7f7e38858e" containerName="revoke-keys"
Jul 28 18:05:02.303684 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:02.303661    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9dcbcd0-735f-4db9-8e2f-bf7f7e38858e" containerName="revoke-keys"
Jul 28 18:05:02.303684 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:02.303671    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="cab6e009-1e20-4392-9600-38bf679d58d0" containerName="maas-api"
Jul 28 18:05:02.308451 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:02.308421    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-21c55cfa-f87b85b84-rrlsl"
Jul 28 18:05:02.311455 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:02.311421    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-21c55cfa-serving-cert\""
Jul 28 18:05:02.321655 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:02.321636    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-21c55cfa-f87b85b84-rrlsl"]
Jul 28 18:05:02.381713 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:02.381686    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:05:02.395380 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:02.395355    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jndfz\" (UniqueName: \"kubernetes.io/projected/289fb6c9-1365-4a14-a5ea-b1a1f5c72cde-kube-api-access-jndfz\") pod \"maas-api-e2e-ait-21c55cfa-f87b85b84-rrlsl\" (UID: \"289fb6c9-1365-4a14-a5ea-b1a1f5c72cde\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-21c55cfa-f87b85b84-rrlsl"
Jul 28 18:05:02.395535 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:02.395394    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/289fb6c9-1365-4a14-a5ea-b1a1f5c72cde-openshift-service-ca\") pod \"maas-api-e2e-ait-21c55cfa-f87b85b84-rrlsl\" (UID: \"289fb6c9-1365-4a14-a5ea-b1a1f5c72cde\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-21c55cfa-f87b85b84-rrlsl"
Jul 28 18:05:02.395776 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:02.395756    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/289fb6c9-1365-4a14-a5ea-b1a1f5c72cde-maas-api-tls\") pod \"maas-api-e2e-ait-21c55cfa-f87b85b84-rrlsl\" (UID: \"289fb6c9-1365-4a14-a5ea-b1a1f5c72cde\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-21c55cfa-f87b85b84-rrlsl"
Jul 28 18:05:02.484238 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:02.484160    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:05:02.496813 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:02.496784    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/289fb6c9-1365-4a14-a5ea-b1a1f5c72cde-maas-api-tls\") pod \"maas-api-e2e-ait-21c55cfa-f87b85b84-rrlsl\" (UID: \"289fb6c9-1365-4a14-a5ea-b1a1f5c72cde\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-21c55cfa-f87b85b84-rrlsl"
Jul 28 18:05:02.496944 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:02.496819    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jndfz\" (UniqueName: \"kubernetes.io/projected/289fb6c9-1365-4a14-a5ea-b1a1f5c72cde-kube-api-access-jndfz\") pod \"maas-api-e2e-ait-21c55cfa-f87b85b84-rrlsl\" (UID: \"289fb6c9-1365-4a14-a5ea-b1a1f5c72cde\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-21c55cfa-f87b85b84-rrlsl"
Jul 28 18:05:02.496944 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:02.496844    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/289fb6c9-1365-4a14-a5ea-b1a1f5c72cde-openshift-service-ca\") pod \"maas-api-e2e-ait-21c55cfa-f87b85b84-rrlsl\" (UID: \"289fb6c9-1365-4a14-a5ea-b1a1f5c72cde\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-21c55cfa-f87b85b84-rrlsl"
Jul 28 18:05:02.497478 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:02.497461    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/289fb6c9-1365-4a14-a5ea-b1a1f5c72cde-openshift-service-ca\") pod \"maas-api-e2e-ait-21c55cfa-f87b85b84-rrlsl\" (UID: \"289fb6c9-1365-4a14-a5ea-b1a1f5c72cde\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-21c55cfa-f87b85b84-rrlsl"
Jul 28 18:05:02.499152 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:02.499134    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/289fb6c9-1365-4a14-a5ea-b1a1f5c72cde-maas-api-tls\") pod \"maas-api-e2e-ait-21c55cfa-f87b85b84-rrlsl\" (UID: \"289fb6c9-1365-4a14-a5ea-b1a1f5c72cde\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-21c55cfa-f87b85b84-rrlsl"
Jul 28 18:05:02.507071 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:02.507043    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jndfz\" (UniqueName: \"kubernetes.io/projected/289fb6c9-1365-4a14-a5ea-b1a1f5c72cde-kube-api-access-jndfz\") pod \"maas-api-e2e-ait-21c55cfa-f87b85b84-rrlsl\" (UID: \"289fb6c9-1365-4a14-a5ea-b1a1f5c72cde\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-21c55cfa-f87b85b84-rrlsl"
Jul 28 18:05:02.618201 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:02.618168    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-21c55cfa-f87b85b84-rrlsl"
Jul 28 18:05:02.759814 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:02.759784    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-21c55cfa-f87b85b84-rrlsl"]
Jul 28 18:05:02.761245 ip-10-0-136-126 kubenswrapper[2563]: W0728 18:05:02.761221    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod289fb6c9_1365_4a14_a5ea_b1a1f5c72cde.slice/crio-42e6a47642caaac756f91850fa045f1e202bdcf4de21e4fd1cc9e3170143a435 WatchSource:0}: Error finding container 42e6a47642caaac756f91850fa045f1e202bdcf4de21e4fd1cc9e3170143a435: Status 404 returned error can't find the container with id 42e6a47642caaac756f91850fa045f1e202bdcf4de21e4fd1cc9e3170143a435
Jul 28 18:05:02.762639 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:02.762621    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 18:05:02.791612 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:02.791581    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-21c55cfa-f87b85b84-rrlsl" event={"ID":"289fb6c9-1365-4a14-a5ea-b1a1f5c72cde","Type":"ContainerStarted","Data":"42e6a47642caaac756f91850fa045f1e202bdcf4de21e4fd1cc9e3170143a435"}
Jul 28 18:05:03.172737 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:03.172701    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:05:03.272856 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:03.272827    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:05:03.798758 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:03.798723    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-21c55cfa-f87b85b84-rrlsl" event={"ID":"289fb6c9-1365-4a14-a5ea-b1a1f5c72cde","Type":"ContainerStarted","Data":"b76f458bd47b23edac3e5bee1763e0490c639ed90af6cf7c0ef333d15295f220"}
Jul 28 18:05:03.798950 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:03.798784    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-21c55cfa-f87b85b84-rrlsl"
Jul 28 18:05:03.823691 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:03.823604    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-21c55cfa-f87b85b84-rrlsl" podStartSLOduration=1.019946168 podStartE2EDuration="1.823581691s" totalImagesPullingTime="803.635523ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:05:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:05:03.822576412 +0000 UTC m=+1842.483732779" watchObservedRunningTime="2026-07-28 18:05:03.823581691 +0000 UTC m=+1842.484738080"
Jul 28 18:05:09.809704 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:09.809674    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-21c55cfa-f87b85b84-rrlsl"
Jul 28 18:05:28.203730 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:28.203699    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-21c55cfa-zx4jw"]
Jul 28 18:05:28.207128 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:28.207109    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-21c55cfa-zx4jw"
Jul 28 18:05:28.211458 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:28.211425    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9xcv2\""
Jul 28 18:05:28.228562 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:28.228533    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-21c55cfa-zx4jw"]
Jul 28 18:05:28.300497 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:28.300476    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d17dca6-ca1b-4577-91b1-4cb9f926beff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-21c55cfa-zx4jw\" (UID: \"7d17dca6-ca1b-4577-91b1-4cb9f926beff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-21c55cfa-zx4jw"
Jul 28 18:05:28.401365 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:28.401338    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d17dca6-ca1b-4577-91b1-4cb9f926beff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-21c55cfa-zx4jw\" (UID: \"7d17dca6-ca1b-4577-91b1-4cb9f926beff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-21c55cfa-zx4jw"
Jul 28 18:05:28.401953 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:28.401934    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d17dca6-ca1b-4577-91b1-4cb9f926beff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-21c55cfa-zx4jw\" (UID: \"7d17dca6-ca1b-4577-91b1-4cb9f926beff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-21c55cfa-zx4jw"
Jul 28 18:05:28.517538 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:28.517474    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-21c55cfa-zx4jw"
Jul 28 18:05:28.650592 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:28.650563    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-21c55cfa-zx4jw"]
Jul 28 18:05:28.651903 ip-10-0-136-126 kubenswrapper[2563]: W0728 18:05:28.651878    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d17dca6_ca1b_4577_91b1_4cb9f926beff.slice/crio-58b05281f9e694394854f4141cc57b6b211e5e485665463d5bc3c79638252b03 WatchSource:0}: Error finding container 58b05281f9e694394854f4141cc57b6b211e5e485665463d5bc3c79638252b03: Status 404 returned error can't find the container with id 58b05281f9e694394854f4141cc57b6b211e5e485665463d5bc3c79638252b03
Jul 28 18:05:28.934797 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:28.934739    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-21c55cfa-zx4jw" event={"ID":"7d17dca6-ca1b-4577-91b1-4cb9f926beff","Type":"ContainerStarted","Data":"8710bba8e2537dbefe9b37b6f848977db9fffdb25aef0741fba0d2c04bca2a14"}
Jul 28 18:05:28.934911 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:28.934860    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-21c55cfa-zx4jw" event={"ID":"7d17dca6-ca1b-4577-91b1-4cb9f926beff","Type":"ContainerStarted","Data":"58b05281f9e694394854f4141cc57b6b211e5e485665463d5bc3c79638252b03"}
Jul 28 18:05:28.955575 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:28.955529    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-21c55cfa-zx4jw" podStartSLOduration=0.955512384 podStartE2EDuration="955.512384ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:05:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:05:28.953973147 +0000 UTC m=+1867.615129502" watchObservedRunningTime="2026-07-28 18:05:28.955512384 +0000 UTC m=+1867.616668738"
Jul 28 18:05:29.940627 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:29.940593    2563 generic.go:356] "Generic (PLEG): container finished" podID="7d17dca6-ca1b-4577-91b1-4cb9f926beff" containerID="8710bba8e2537dbefe9b37b6f848977db9fffdb25aef0741fba0d2c04bca2a14" exitCode=0
Jul 28 18:05:29.941002 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:29.940678    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-21c55cfa-zx4jw" event={"ID":"7d17dca6-ca1b-4577-91b1-4cb9f926beff","Type":"ContainerDied","Data":"8710bba8e2537dbefe9b37b6f848977db9fffdb25aef0741fba0d2c04bca2a14"}
Jul 28 18:05:31.076382 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:31.076360    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-21c55cfa-zx4jw"
Jul 28 18:05:31.120603 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:31.120578    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d17dca6-ca1b-4577-91b1-4cb9f926beff-maas-api-service-ca\") pod \"7d17dca6-ca1b-4577-91b1-4cb9f926beff\" (UID: \"7d17dca6-ca1b-4577-91b1-4cb9f926beff\") "
Jul 28 18:05:31.121016 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:31.120990    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d17dca6-ca1b-4577-91b1-4cb9f926beff-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7d17dca6-ca1b-4577-91b1-4cb9f926beff" (UID: "7d17dca6-ca1b-4577-91b1-4cb9f926beff"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:05:31.221350 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:31.221282    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d17dca6-ca1b-4577-91b1-4cb9f926beff-maas-api-service-ca\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:05:31.952856 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:31.952824    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-21c55cfa-zx4jw"
Jul 28 18:05:31.953021 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:31.952859    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-21c55cfa-zx4jw" event={"ID":"7d17dca6-ca1b-4577-91b1-4cb9f926beff","Type":"ContainerDied","Data":"58b05281f9e694394854f4141cc57b6b211e5e485665463d5bc3c79638252b03"}
Jul 28 18:05:31.953021 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:31.952906    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="58b05281f9e694394854f4141cc57b6b211e5e485665463d5bc3c79638252b03"
Jul 28 18:05:32.521727 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:32.521689    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-21c55cfa-f87b85b84-rrlsl"]
Jul 28 18:05:32.522146 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:32.521925    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-21c55cfa-f87b85b84-rrlsl" podUID="289fb6c9-1365-4a14-a5ea-b1a1f5c72cde" containerName="maas-api" containerID="cri-o://b76f458bd47b23edac3e5bee1763e0490c639ed90af6cf7c0ef333d15295f220" gracePeriod=30
Jul 28 18:05:32.541160 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:32.541124    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:05:32.644027 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:32.644000    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:05:32.745863 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:32.745833    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:05:32.776112 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:32.776058    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-21c55cfa-f87b85b84-rrlsl"
Jul 28 18:05:32.787277 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:32.787251    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-21c55cfa-zx4jw"]
Jul 28 18:05:32.796040 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:32.796018    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-21c55cfa-zx4jw"]
Jul 28 18:05:32.833319 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:32.833283    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:05:32.833614 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:32.833590    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jndfz\" (UniqueName: \"kubernetes.io/projected/289fb6c9-1365-4a14-a5ea-b1a1f5c72cde-kube-api-access-jndfz\") pod \"289fb6c9-1365-4a14-a5ea-b1a1f5c72cde\" (UID: \"289fb6c9-1365-4a14-a5ea-b1a1f5c72cde\") "
Jul 28 18:05:32.833766 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:32.833645    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/289fb6c9-1365-4a14-a5ea-b1a1f5c72cde-openshift-service-ca\") pod \"289fb6c9-1365-4a14-a5ea-b1a1f5c72cde\" (UID: \"289fb6c9-1365-4a14-a5ea-b1a1f5c72cde\") "
Jul 28 18:05:32.833766 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:32.833696    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/289fb6c9-1365-4a14-a5ea-b1a1f5c72cde-maas-api-tls\") pod \"289fb6c9-1365-4a14-a5ea-b1a1f5c72cde\" (UID: \"289fb6c9-1365-4a14-a5ea-b1a1f5c72cde\") "
Jul 28 18:05:32.834415 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:32.834387    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/289fb6c9-1365-4a14-a5ea-b1a1f5c72cde-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "289fb6c9-1365-4a14-a5ea-b1a1f5c72cde" (UID: "289fb6c9-1365-4a14-a5ea-b1a1f5c72cde"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:05:32.835853 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:32.835831    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/289fb6c9-1365-4a14-a5ea-b1a1f5c72cde-kube-api-access-jndfz" (OuterVolumeSpecName: "kube-api-access-jndfz") pod "289fb6c9-1365-4a14-a5ea-b1a1f5c72cde" (UID: "289fb6c9-1365-4a14-a5ea-b1a1f5c72cde"). InnerVolumeSpecName "kube-api-access-jndfz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:05:32.836144 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:32.836117    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/289fb6c9-1365-4a14-a5ea-b1a1f5c72cde-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "289fb6c9-1365-4a14-a5ea-b1a1f5c72cde" (UID: "289fb6c9-1365-4a14-a5ea-b1a1f5c72cde"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:05:32.934757 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:32.934723    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/289fb6c9-1365-4a14-a5ea-b1a1f5c72cde-maas-api-tls\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:05:32.934757 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:32.934750    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jndfz\" (UniqueName: \"kubernetes.io/projected/289fb6c9-1365-4a14-a5ea-b1a1f5c72cde-kube-api-access-jndfz\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:05:32.934757 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:32.934761    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/289fb6c9-1365-4a14-a5ea-b1a1f5c72cde-openshift-service-ca\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:05:32.962866 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:32.962831    2563 generic.go:356] "Generic (PLEG): container finished" podID="289fb6c9-1365-4a14-a5ea-b1a1f5c72cde" containerID="b76f458bd47b23edac3e5bee1763e0490c639ed90af6cf7c0ef333d15295f220" exitCode=0
Jul 28 18:05:32.963019 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:32.962875    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-21c55cfa-f87b85b84-rrlsl"
Jul 28 18:05:32.963019 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:32.962926    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-21c55cfa-f87b85b84-rrlsl" event={"ID":"289fb6c9-1365-4a14-a5ea-b1a1f5c72cde","Type":"ContainerDied","Data":"b76f458bd47b23edac3e5bee1763e0490c639ed90af6cf7c0ef333d15295f220"}
Jul 28 18:05:32.963019 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:32.962980    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-21c55cfa-f87b85b84-rrlsl" event={"ID":"289fb6c9-1365-4a14-a5ea-b1a1f5c72cde","Type":"ContainerDied","Data":"42e6a47642caaac756f91850fa045f1e202bdcf4de21e4fd1cc9e3170143a435"}
Jul 28 18:05:32.963019 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:32.963008    2563 scope.go:117] "RemoveContainer" containerID="b76f458bd47b23edac3e5bee1763e0490c639ed90af6cf7c0ef333d15295f220"
Jul 28 18:05:32.972187 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:32.972170    2563 scope.go:117] "RemoveContainer" containerID="b76f458bd47b23edac3e5bee1763e0490c639ed90af6cf7c0ef333d15295f220"
Jul 28 18:05:32.972566 ip-10-0-136-126 kubenswrapper[2563]: E0728 18:05:32.972428    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b76f458bd47b23edac3e5bee1763e0490c639ed90af6cf7c0ef333d15295f220\": container with ID starting with b76f458bd47b23edac3e5bee1763e0490c639ed90af6cf7c0ef333d15295f220 not found: ID does not exist" containerID="b76f458bd47b23edac3e5bee1763e0490c639ed90af6cf7c0ef333d15295f220"
Jul 28 18:05:32.972659 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:32.972580    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b76f458bd47b23edac3e5bee1763e0490c639ed90af6cf7c0ef333d15295f220"} err="failed to get container status \"b76f458bd47b23edac3e5bee1763e0490c639ed90af6cf7c0ef333d15295f220\": rpc error: code = NotFound desc = could not find container \"b76f458bd47b23edac3e5bee1763e0490c639ed90af6cf7c0ef333d15295f220\": container with ID starting with b76f458bd47b23edac3e5bee1763e0490c639ed90af6cf7c0ef333d15295f220 not found: ID does not exist"
Jul 28 18:05:32.990028 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:32.990002    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-21c55cfa-f87b85b84-rrlsl"]
Jul 28 18:05:32.995714 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:32.995692    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-21c55cfa-f87b85b84-rrlsl"]
Jul 28 18:05:33.854040 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:33.854009    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="289fb6c9-1365-4a14-a5ea-b1a1f5c72cde" path="/var/lib/kubelet/pods/289fb6c9-1365-4a14-a5ea-b1a1f5c72cde/volumes"
Jul 28 18:05:33.856780 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:33.854352    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d17dca6-ca1b-4577-91b1-4cb9f926beff" path="/var/lib/kubelet/pods/7d17dca6-ca1b-4577-91b1-4cb9f926beff/volumes"
Jul 28 18:05:34.130304 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:34.130221    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:05:34.439502 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:34.439411    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:05:34.651631 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:34.651595    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:05:35.259992 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:35.259962    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:05:35.772551 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:35.772521    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:05:37.568885 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:37.568851    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:05:37.735241 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:37.735209    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:05:37.741142 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:37.741112    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-55c1dce5-f9b79cbf6-7cnlp"]
Jul 28 18:05:37.741493 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:37.741478    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d17dca6-ca1b-4577-91b1-4cb9f926beff" containerName="revoke-keys"
Jul 28 18:05:37.741545 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:37.741495    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d17dca6-ca1b-4577-91b1-4cb9f926beff" containerName="revoke-keys"
Jul 28 18:05:37.741545 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:37.741517    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="289fb6c9-1365-4a14-a5ea-b1a1f5c72cde" containerName="maas-api"
Jul 28 18:05:37.741545 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:37.741523    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="289fb6c9-1365-4a14-a5ea-b1a1f5c72cde" containerName="maas-api"
Jul 28 18:05:37.741643 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:37.741586    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d17dca6-ca1b-4577-91b1-4cb9f926beff" containerName="revoke-keys"
Jul 28 18:05:37.741643 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:37.741594    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="289fb6c9-1365-4a14-a5ea-b1a1f5c72cde" containerName="maas-api"
Jul 28 18:05:37.744936 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:37.744919    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-55c1dce5-f9b79cbf6-7cnlp"
Jul 28 18:05:37.747857 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:37.747840    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-55c1dce5-serving-cert\""
Jul 28 18:05:37.757304 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:37.757283    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-55c1dce5-f9b79cbf6-7cnlp"]
Jul 28 18:05:37.771952 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:37.771925    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dbb5f33-bae4-4518-bcdc-689d4031290c-openshift-service-ca\") pod \"maas-api-e2e-derive-55c1dce5-f9b79cbf6-7cnlp\" (UID: \"5dbb5f33-bae4-4518-bcdc-689d4031290c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-55c1dce5-f9b79cbf6-7cnlp"
Jul 28 18:05:37.772095 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:37.771957    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5dbb5f33-bae4-4518-bcdc-689d4031290c-maas-api-tls\") pod \"maas-api-e2e-derive-55c1dce5-f9b79cbf6-7cnlp\" (UID: \"5dbb5f33-bae4-4518-bcdc-689d4031290c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-55c1dce5-f9b79cbf6-7cnlp"
Jul 28 18:05:37.772095 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:37.771988    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22tvb\" (UniqueName: \"kubernetes.io/projected/5dbb5f33-bae4-4518-bcdc-689d4031290c-kube-api-access-22tvb\") pod \"maas-api-e2e-derive-55c1dce5-f9b79cbf6-7cnlp\" (UID: \"5dbb5f33-bae4-4518-bcdc-689d4031290c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-55c1dce5-f9b79cbf6-7cnlp"
Jul 28 18:05:37.830031 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:37.829954    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:05:37.873265 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:37.873233    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dbb5f33-bae4-4518-bcdc-689d4031290c-openshift-service-ca\") pod \"maas-api-e2e-derive-55c1dce5-f9b79cbf6-7cnlp\" (UID: \"5dbb5f33-bae4-4518-bcdc-689d4031290c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-55c1dce5-f9b79cbf6-7cnlp"
Jul 28 18:05:37.873265 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:37.873269    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5dbb5f33-bae4-4518-bcdc-689d4031290c-maas-api-tls\") pod \"maas-api-e2e-derive-55c1dce5-f9b79cbf6-7cnlp\" (UID: \"5dbb5f33-bae4-4518-bcdc-689d4031290c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-55c1dce5-f9b79cbf6-7cnlp"
Jul 28 18:05:37.873520 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:37.873296    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22tvb\" (UniqueName: \"kubernetes.io/projected/5dbb5f33-bae4-4518-bcdc-689d4031290c-kube-api-access-22tvb\") pod \"maas-api-e2e-derive-55c1dce5-f9b79cbf6-7cnlp\" (UID: \"5dbb5f33-bae4-4518-bcdc-689d4031290c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-55c1dce5-f9b79cbf6-7cnlp"
Jul 28 18:05:37.873520 ip-10-0-136-126 kubenswrapper[2563]: E0728 18:05:37.873378    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-55c1dce5-serving-cert: secret "maas-api-e2e-derive-55c1dce5-serving-cert" not found
Jul 28 18:05:37.873520 ip-10-0-136-126 kubenswrapper[2563]: E0728 18:05:37.873465    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5dbb5f33-bae4-4518-bcdc-689d4031290c-maas-api-tls podName:5dbb5f33-bae4-4518-bcdc-689d4031290c nodeName:}" failed. No retries permitted until 2026-07-28 18:05:38.373445574 +0000 UTC m=+1877.034601920 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5dbb5f33-bae4-4518-bcdc-689d4031290c-maas-api-tls") pod "maas-api-e2e-derive-55c1dce5-f9b79cbf6-7cnlp" (UID: "5dbb5f33-bae4-4518-bcdc-689d4031290c") : secret "maas-api-e2e-derive-55c1dce5-serving-cert" not found
Jul 28 18:05:37.873845 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:37.873826    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dbb5f33-bae4-4518-bcdc-689d4031290c-openshift-service-ca\") pod \"maas-api-e2e-derive-55c1dce5-f9b79cbf6-7cnlp\" (UID: \"5dbb5f33-bae4-4518-bcdc-689d4031290c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-55c1dce5-f9b79cbf6-7cnlp"
Jul 28 18:05:37.883404 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:37.883381    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22tvb\" (UniqueName: \"kubernetes.io/projected/5dbb5f33-bae4-4518-bcdc-689d4031290c-kube-api-access-22tvb\") pod \"maas-api-e2e-derive-55c1dce5-f9b79cbf6-7cnlp\" (UID: \"5dbb5f33-bae4-4518-bcdc-689d4031290c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-55c1dce5-f9b79cbf6-7cnlp"
Jul 28 18:05:38.000019 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:37.999987    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:05:38.377883 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:38.377846    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5dbb5f33-bae4-4518-bcdc-689d4031290c-maas-api-tls\") pod \"maas-api-e2e-derive-55c1dce5-f9b79cbf6-7cnlp\" (UID: \"5dbb5f33-bae4-4518-bcdc-689d4031290c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-55c1dce5-f9b79cbf6-7cnlp"
Jul 28 18:05:38.380192 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:38.380168    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5dbb5f33-bae4-4518-bcdc-689d4031290c-maas-api-tls\") pod \"maas-api-e2e-derive-55c1dce5-f9b79cbf6-7cnlp\" (UID: \"5dbb5f33-bae4-4518-bcdc-689d4031290c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-55c1dce5-f9b79cbf6-7cnlp"
Jul 28 18:05:38.635520 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:38.635421    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:05:38.655460 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:38.655404    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-55c1dce5-f9b79cbf6-7cnlp"
Jul 28 18:05:38.816605 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:38.816571    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-55c1dce5-f9b79cbf6-7cnlp"]
Jul 28 18:05:38.824606 ip-10-0-136-126 kubenswrapper[2563]: W0728 18:05:38.824582    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dbb5f33_bae4_4518_bcdc_689d4031290c.slice/crio-eb7de81c1d582ba4409afc925cf13be644396ce3a9bd04dffdee8185f13e63fe WatchSource:0}: Error finding container eb7de81c1d582ba4409afc925cf13be644396ce3a9bd04dffdee8185f13e63fe: Status 404 returned error can't find the container with id eb7de81c1d582ba4409afc925cf13be644396ce3a9bd04dffdee8185f13e63fe
Jul 28 18:05:38.997013 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:38.996934    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-55c1dce5-f9b79cbf6-7cnlp" event={"ID":"5dbb5f33-bae4-4518-bcdc-689d4031290c","Type":"ContainerStarted","Data":"eb7de81c1d582ba4409afc925cf13be644396ce3a9bd04dffdee8185f13e63fe"}
Jul 28 18:05:39.080197 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:39.080164    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:05:40.003492 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:40.003396    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-55c1dce5-f9b79cbf6-7cnlp" event={"ID":"5dbb5f33-bae4-4518-bcdc-689d4031290c","Type":"ContainerStarted","Data":"d60928d7c4323da3ad8a89727ced4f49aa0423abbcac3be6b925e987c9700514"}
Jul 28 18:05:40.003827 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:40.003561    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-55c1dce5-f9b79cbf6-7cnlp"
Jul 28 18:05:46.014240 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:46.014156    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-55c1dce5-f9b79cbf6-7cnlp"
Jul 28 18:05:46.038812 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:46.038757    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-55c1dce5-f9b79cbf6-7cnlp" podStartSLOduration=8.124762643 podStartE2EDuration="9.038738616s" totalImagesPullingTime="913.975973ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:05:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:05:40.029947148 +0000 UTC m=+1878.691103497" watchObservedRunningTime="2026-07-28 18:05:46.038738616 +0000 UTC m=+1884.699894974"
Jul 28 18:05:51.031841 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:05:51.031806    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:06:02.563221 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:02.563189    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-55c1dce5-dqxlq"]
Jul 28 18:06:02.566671 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:02.566651    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-55c1dce5-dqxlq"
Jul 28 18:06:02.573002 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:02.572981    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9xcv2\""
Jul 28 18:06:02.595453 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:02.595226    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-55c1dce5-dqxlq"]
Jul 28 18:06:02.692249 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:02.692219    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f358d5e-cfcd-497e-a22f-4d10ca0b33b2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-55c1dce5-dqxlq\" (UID: \"5f358d5e-cfcd-497e-a22f-4d10ca0b33b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-55c1dce5-dqxlq"
Jul 28 18:06:02.793390 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:02.793347    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f358d5e-cfcd-497e-a22f-4d10ca0b33b2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-55c1dce5-dqxlq\" (UID: \"5f358d5e-cfcd-497e-a22f-4d10ca0b33b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-55c1dce5-dqxlq"
Jul 28 18:06:02.793972 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:02.793953    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f358d5e-cfcd-497e-a22f-4d10ca0b33b2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-55c1dce5-dqxlq\" (UID: \"5f358d5e-cfcd-497e-a22f-4d10ca0b33b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-55c1dce5-dqxlq"
Jul 28 18:06:02.892004 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:02.891970    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-55c1dce5-dqxlq"
Jul 28 18:06:03.023995 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:03.023964    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-55c1dce5-dqxlq"]
Jul 28 18:06:03.025597 ip-10-0-136-126 kubenswrapper[2563]: W0728 18:06:03.025567    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f358d5e_cfcd_497e_a22f_4d10ca0b33b2.slice/crio-1f8c4de4043a627793e348c1fa61d602de0e700a4c16c0079768c836e89f9df8 WatchSource:0}: Error finding container 1f8c4de4043a627793e348c1fa61d602de0e700a4c16c0079768c836e89f9df8: Status 404 returned error can't find the container with id 1f8c4de4043a627793e348c1fa61d602de0e700a4c16c0079768c836e89f9df8
Jul 28 18:06:03.127215 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:03.127175    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-55c1dce5-dqxlq" event={"ID":"5f358d5e-cfcd-497e-a22f-4d10ca0b33b2","Type":"ContainerStarted","Data":"1f8c4de4043a627793e348c1fa61d602de0e700a4c16c0079768c836e89f9df8"}
Jul 28 18:06:04.135121 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:04.135088    2563 generic.go:356] "Generic (PLEG): container finished" podID="5f358d5e-cfcd-497e-a22f-4d10ca0b33b2" containerID="4e7d3fe179322c4c1a32afd39801966677fad7bf4446131d7dfebd822e32886d" exitCode=0
Jul 28 18:06:04.135504 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:04.135158    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-55c1dce5-dqxlq" event={"ID":"5f358d5e-cfcd-497e-a22f-4d10ca0b33b2","Type":"ContainerDied","Data":"4e7d3fe179322c4c1a32afd39801966677fad7bf4446131d7dfebd822e32886d"}
Jul 28 18:06:05.269866 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:05.269842    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-55c1dce5-dqxlq"
Jul 28 18:06:05.415688 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:05.415619    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f358d5e-cfcd-497e-a22f-4d10ca0b33b2-maas-api-service-ca\") pod \"5f358d5e-cfcd-497e-a22f-4d10ca0b33b2\" (UID: \"5f358d5e-cfcd-497e-a22f-4d10ca0b33b2\") "
Jul 28 18:06:05.415945 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:05.415923    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f358d5e-cfcd-497e-a22f-4d10ca0b33b2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5f358d5e-cfcd-497e-a22f-4d10ca0b33b2" (UID: "5f358d5e-cfcd-497e-a22f-4d10ca0b33b2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:06:05.517086 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:05.517057    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f358d5e-cfcd-497e-a22f-4d10ca0b33b2-maas-api-service-ca\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:06:06.146250 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:06.146180    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-55c1dce5-dqxlq"
Jul 28 18:06:06.146388 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:06.146176    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-55c1dce5-dqxlq" event={"ID":"5f358d5e-cfcd-497e-a22f-4d10ca0b33b2","Type":"ContainerDied","Data":"1f8c4de4043a627793e348c1fa61d602de0e700a4c16c0079768c836e89f9df8"}
Jul 28 18:06:06.146388 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:06.146303    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1f8c4de4043a627793e348c1fa61d602de0e700a4c16c0079768c836e89f9df8"
Jul 28 18:06:07.287077 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:07.287043    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-55c1dce5-f9b79cbf6-7cnlp"]
Jul 28 18:06:07.287535 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:07.287250    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-55c1dce5-f9b79cbf6-7cnlp" podUID="5dbb5f33-bae4-4518-bcdc-689d4031290c" containerName="maas-api" containerID="cri-o://d60928d7c4323da3ad8a89727ced4f49aa0423abbcac3be6b925e987c9700514" gracePeriod=30
Jul 28 18:06:07.338726 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:07.338692    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:06:07.448195 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:07.448160    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:06:07.547787 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:07.547716    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-55c1dce5-f9b79cbf6-7cnlp"
Jul 28 18:06:07.561393 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:07.561368    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:06:07.625602 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:07.625564    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-55c1dce5-dqxlq"]
Jul 28 18:06:07.630004 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:07.629972    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-55c1dce5-dqxlq"]
Jul 28 18:06:07.636472 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:07.636419    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-22tvb\" (UniqueName: \"kubernetes.io/projected/5dbb5f33-bae4-4518-bcdc-689d4031290c-kube-api-access-22tvb\") pod \"5dbb5f33-bae4-4518-bcdc-689d4031290c\" (UID: \"5dbb5f33-bae4-4518-bcdc-689d4031290c\") "
Jul 28 18:06:07.636610 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:07.636568    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dbb5f33-bae4-4518-bcdc-689d4031290c-openshift-service-ca\") pod \"5dbb5f33-bae4-4518-bcdc-689d4031290c\" (UID: \"5dbb5f33-bae4-4518-bcdc-689d4031290c\") "
Jul 28 18:06:07.636679 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:07.636620    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5dbb5f33-bae4-4518-bcdc-689d4031290c-maas-api-tls\") pod \"5dbb5f33-bae4-4518-bcdc-689d4031290c\" (UID: \"5dbb5f33-bae4-4518-bcdc-689d4031290c\") "
Jul 28 18:06:07.636990 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:07.636953    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5dbb5f33-bae4-4518-bcdc-689d4031290c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5dbb5f33-bae4-4518-bcdc-689d4031290c" (UID: "5dbb5f33-bae4-4518-bcdc-689d4031290c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:06:07.638693 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:07.638666    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5dbb5f33-bae4-4518-bcdc-689d4031290c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5dbb5f33-bae4-4518-bcdc-689d4031290c" (UID: "5dbb5f33-bae4-4518-bcdc-689d4031290c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:06:07.638811 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:07.638734    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5dbb5f33-bae4-4518-bcdc-689d4031290c-kube-api-access-22tvb" (OuterVolumeSpecName: "kube-api-access-22tvb") pod "5dbb5f33-bae4-4518-bcdc-689d4031290c" (UID: "5dbb5f33-bae4-4518-bcdc-689d4031290c"). InnerVolumeSpecName "kube-api-access-22tvb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:06:07.685491 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:07.685455    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:06:07.737706 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:07.737677    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5dbb5f33-bae4-4518-bcdc-689d4031290c-maas-api-tls\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:06:07.737706 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:07.737704    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-22tvb\" (UniqueName: \"kubernetes.io/projected/5dbb5f33-bae4-4518-bcdc-689d4031290c-kube-api-access-22tvb\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:06:07.737706 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:07.737714    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dbb5f33-bae4-4518-bcdc-689d4031290c-openshift-service-ca\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:06:07.803568 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:07.803489    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:06:07.853120 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:07.853087    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f358d5e-cfcd-497e-a22f-4d10ca0b33b2" path="/var/lib/kubelet/pods/5f358d5e-cfcd-497e-a22f-4d10ca0b33b2/volumes"
Jul 28 18:06:08.096085 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:08.096055    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:06:08.156961 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:08.156932    2563 generic.go:356] "Generic (PLEG): container finished" podID="5dbb5f33-bae4-4518-bcdc-689d4031290c" containerID="d60928d7c4323da3ad8a89727ced4f49aa0423abbcac3be6b925e987c9700514" exitCode=0
Jul 28 18:06:08.157131 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:08.156993    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-55c1dce5-f9b79cbf6-7cnlp"
Jul 28 18:06:08.157131 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:08.157024    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-55c1dce5-f9b79cbf6-7cnlp" event={"ID":"5dbb5f33-bae4-4518-bcdc-689d4031290c","Type":"ContainerDied","Data":"d60928d7c4323da3ad8a89727ced4f49aa0423abbcac3be6b925e987c9700514"}
Jul 28 18:06:08.157131 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:08.157088    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-55c1dce5-f9b79cbf6-7cnlp" event={"ID":"5dbb5f33-bae4-4518-bcdc-689d4031290c","Type":"ContainerDied","Data":"eb7de81c1d582ba4409afc925cf13be644396ce3a9bd04dffdee8185f13e63fe"}
Jul 28 18:06:08.157131 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:08.157118    2563 scope.go:117] "RemoveContainer" containerID="d60928d7c4323da3ad8a89727ced4f49aa0423abbcac3be6b925e987c9700514"
Jul 28 18:06:08.165825 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:08.165809    2563 scope.go:117] "RemoveContainer" containerID="d60928d7c4323da3ad8a89727ced4f49aa0423abbcac3be6b925e987c9700514"
Jul 28 18:06:08.166065 ip-10-0-136-126 kubenswrapper[2563]: E0728 18:06:08.166047    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d60928d7c4323da3ad8a89727ced4f49aa0423abbcac3be6b925e987c9700514\": container with ID starting with d60928d7c4323da3ad8a89727ced4f49aa0423abbcac3be6b925e987c9700514 not found: ID does not exist" containerID="d60928d7c4323da3ad8a89727ced4f49aa0423abbcac3be6b925e987c9700514"
Jul 28 18:06:08.166132 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:08.166076    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d60928d7c4323da3ad8a89727ced4f49aa0423abbcac3be6b925e987c9700514"} err="failed to get container status \"d60928d7c4323da3ad8a89727ced4f49aa0423abbcac3be6b925e987c9700514\": rpc error: code = NotFound desc = could not find container \"d60928d7c4323da3ad8a89727ced4f49aa0423abbcac3be6b925e987c9700514\": container with ID starting with d60928d7c4323da3ad8a89727ced4f49aa0423abbcac3be6b925e987c9700514 not found: ID does not exist"
Jul 28 18:06:08.188926 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:08.188907    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-55c1dce5-f9b79cbf6-7cnlp"]
Jul 28 18:06:08.193660 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:08.193640    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-55c1dce5-f9b79cbf6-7cnlp"]
Jul 28 18:06:08.653605 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:08.653570    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:06:08.749526 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:08.749494    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:06:08.913404 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:08.913328    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:06:09.853366 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:09.853332    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5dbb5f33-bae4-4518-bcdc-689d4031290c" path="/var/lib/kubelet/pods/5dbb5f33-bae4-4518-bcdc-689d4031290c/volumes"
Jul 28 18:06:10.493609 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:10.493578    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:06:18.376682 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:18.376648    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:06:18.440751 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:18.440710    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:06:18.552795 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:18.552762    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:06:18.654911 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:18.654834    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:06:18.752115 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:18.752091    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:06:19.209244 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:19.209214    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:06:40.943676 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:40.943643    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:06:45.339946 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:45.339911    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:06:52.747016 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:52.746974    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:06:52.829056 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:52.829024    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:06:52.924273 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:06:52.924243    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:08.961643 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:07:08.961605    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:15.768718 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:07:15.768641    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:15.839136 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:07:15.839102    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:15.934457 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:07:15.934405    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:16.035769 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:07:16.035692    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:26.586634 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:07:26.586594    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:27.461662 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:07:27.461631    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:35.912492 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:07:35.912453    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:36.022843 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:07:36.022809    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:36.110494 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:07:36.110455    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:36.212656 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:07:36.212578    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:50.043099 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:07:50.043052    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:50.145211 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:07:50.145175    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:50.243694 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:07:50.243660    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:07:52.211951 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:07:52.211915    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:08:00.376652 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:00.376604    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:08:00.493695 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:00.493665    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:08:01.211725 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:01.211675    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:08:07.883612 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:07.883576    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:08:07.984842 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:07.984807    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:08:08.196906 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:08.196820    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:08:08.573258 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:08.573225    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:08:08.866553 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:08.866464    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:08:11.816764 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:11.816725    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:08:11.921705 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:11.921670    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:08:11.928700 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:11.928677    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-688e4a86-7dc7fcff79-jmbpj"]
Jul 28 18:08:11.929014 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:11.929001    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5dbb5f33-bae4-4518-bcdc-689d4031290c" containerName="maas-api"
Jul 28 18:08:11.929058 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:11.929016    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5dbb5f33-bae4-4518-bcdc-689d4031290c" containerName="maas-api"
Jul 28 18:08:11.929058 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:11.929026    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f358d5e-cfcd-497e-a22f-4d10ca0b33b2" containerName="revoke-keys"
Jul 28 18:08:11.929058 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:11.929032    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f358d5e-cfcd-497e-a22f-4d10ca0b33b2" containerName="revoke-keys"
Jul 28 18:08:11.929163 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:11.929091    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f358d5e-cfcd-497e-a22f-4d10ca0b33b2" containerName="revoke-keys"
Jul 28 18:08:11.929163 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:11.929100    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5dbb5f33-bae4-4518-bcdc-689d4031290c" containerName="maas-api"
Jul 28 18:08:11.932105 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:11.932083    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-688e4a86-7dc7fcff79-jmbpj"
Jul 28 18:08:11.936975 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:11.936957    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-688e4a86-serving-cert\""
Jul 28 18:08:11.948869 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:11.948842    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-688e4a86-7dc7fcff79-jmbpj"]
Jul 28 18:08:11.980995 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:11.980970    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxx6b\" (UniqueName: \"kubernetes.io/projected/d9456476-ecd0-4313-8a2c-ee1c1b6cd1a4-kube-api-access-xxx6b\") pod \"maas-api-e2e-mt-688e4a86-7dc7fcff79-jmbpj\" (UID: \"d9456476-ecd0-4313-8a2c-ee1c1b6cd1a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-688e4a86-7dc7fcff79-jmbpj"
Jul 28 18:08:11.981102 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:11.981013    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9456476-ecd0-4313-8a2c-ee1c1b6cd1a4-openshift-service-ca\") pod \"maas-api-e2e-mt-688e4a86-7dc7fcff79-jmbpj\" (UID: \"d9456476-ecd0-4313-8a2c-ee1c1b6cd1a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-688e4a86-7dc7fcff79-jmbpj"
Jul 28 18:08:11.981144 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:11.981096    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9456476-ecd0-4313-8a2c-ee1c1b6cd1a4-maas-api-tls\") pod \"maas-api-e2e-mt-688e4a86-7dc7fcff79-jmbpj\" (UID: \"d9456476-ecd0-4313-8a2c-ee1c1b6cd1a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-688e4a86-7dc7fcff79-jmbpj"
Jul 28 18:08:12.017783 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:12.017753    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:08:12.084461 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:12.082564    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9456476-ecd0-4313-8a2c-ee1c1b6cd1a4-openshift-service-ca\") pod \"maas-api-e2e-mt-688e4a86-7dc7fcff79-jmbpj\" (UID: \"d9456476-ecd0-4313-8a2c-ee1c1b6cd1a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-688e4a86-7dc7fcff79-jmbpj"
Jul 28 18:08:12.084461 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:12.082663    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9456476-ecd0-4313-8a2c-ee1c1b6cd1a4-maas-api-tls\") pod \"maas-api-e2e-mt-688e4a86-7dc7fcff79-jmbpj\" (UID: \"d9456476-ecd0-4313-8a2c-ee1c1b6cd1a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-688e4a86-7dc7fcff79-jmbpj"
Jul 28 18:08:12.084461 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:12.082758    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxx6b\" (UniqueName: \"kubernetes.io/projected/d9456476-ecd0-4313-8a2c-ee1c1b6cd1a4-kube-api-access-xxx6b\") pod \"maas-api-e2e-mt-688e4a86-7dc7fcff79-jmbpj\" (UID: \"d9456476-ecd0-4313-8a2c-ee1c1b6cd1a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-688e4a86-7dc7fcff79-jmbpj"
Jul 28 18:08:12.084461 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:12.083865    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9456476-ecd0-4313-8a2c-ee1c1b6cd1a4-openshift-service-ca\") pod \"maas-api-e2e-mt-688e4a86-7dc7fcff79-jmbpj\" (UID: \"d9456476-ecd0-4313-8a2c-ee1c1b6cd1a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-688e4a86-7dc7fcff79-jmbpj"
Jul 28 18:08:12.088459 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:12.086867    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9456476-ecd0-4313-8a2c-ee1c1b6cd1a4-maas-api-tls\") pod \"maas-api-e2e-mt-688e4a86-7dc7fcff79-jmbpj\" (UID: \"d9456476-ecd0-4313-8a2c-ee1c1b6cd1a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-688e4a86-7dc7fcff79-jmbpj"
Jul 28 18:08:12.103376 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:12.103350    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxx6b\" (UniqueName: \"kubernetes.io/projected/d9456476-ecd0-4313-8a2c-ee1c1b6cd1a4-kube-api-access-xxx6b\") pod \"maas-api-e2e-mt-688e4a86-7dc7fcff79-jmbpj\" (UID: \"d9456476-ecd0-4313-8a2c-ee1c1b6cd1a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-688e4a86-7dc7fcff79-jmbpj"
Jul 28 18:08:12.155383 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:12.155350    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:08:12.242889 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:12.242850    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-688e4a86-7dc7fcff79-jmbpj"
Jul 28 18:08:12.387214 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:12.387185    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-688e4a86-7dc7fcff79-jmbpj"]
Jul 28 18:08:12.779251 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:12.779170    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:08:12.806133 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:12.806093    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-688e4a86-7dc7fcff79-jmbpj" event={"ID":"d9456476-ecd0-4313-8a2c-ee1c1b6cd1a4","Type":"ContainerStarted","Data":"e627be904ff1821f05cb77ac8a616558e37435c487bd9e4bc35cdabc03241403"}
Jul 28 18:08:12.877618 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:12.877588    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:08:13.813481 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:13.813447    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-688e4a86-7dc7fcff79-jmbpj" event={"ID":"d9456476-ecd0-4313-8a2c-ee1c1b6cd1a4","Type":"ContainerStarted","Data":"5c27b959954ad3aa9779d83ed09a5d8d46260d810de98940ae687e358bd207a0"}
Jul 28 18:08:13.813676 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:13.813581    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-688e4a86-7dc7fcff79-jmbpj"
Jul 28 18:08:13.833911 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:13.833868    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-688e4a86-7dc7fcff79-jmbpj" podStartSLOduration=2.253581847 podStartE2EDuration="2.833852386s" totalImagesPullingTime="580.270539ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:08:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:08:13.833216834 +0000 UTC m=+2032.494373192" watchObservedRunningTime="2026-07-28 18:08:13.833852386 +0000 UTC m=+2032.495008742"
Jul 28 18:08:19.824166 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:19.824138    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-688e4a86-7dc7fcff79-jmbpj"
Jul 28 18:08:36.870474 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:36.870424    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:08:37.596956 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:37.596922    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:08:37.697839 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:37.697798    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:08:38.129378 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:38.129338    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:08:44.809844 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:44.809763    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:08:45.215832 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:45.215799    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-688e4a86-bpz9g"]
Jul 28 18:08:45.219417 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:45.219401    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-688e4a86-bpz9g"
Jul 28 18:08:45.223260 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:45.223238    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9xcv2\""
Jul 28 18:08:45.229274 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:45.229249    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-688e4a86-bpz9g"]
Jul 28 18:08:45.368573 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:45.368541    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4095a2d-f338-4e7d-b7c1-7edf3fed00ff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-688e4a86-bpz9g\" (UID: \"a4095a2d-f338-4e7d-b7c1-7edf3fed00ff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-688e4a86-bpz9g"
Jul 28 18:08:45.470079 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:45.469990    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4095a2d-f338-4e7d-b7c1-7edf3fed00ff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-688e4a86-bpz9g\" (UID: \"a4095a2d-f338-4e7d-b7c1-7edf3fed00ff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-688e4a86-bpz9g"
Jul 28 18:08:45.470675 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:45.470655    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4095a2d-f338-4e7d-b7c1-7edf3fed00ff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-688e4a86-bpz9g\" (UID: \"a4095a2d-f338-4e7d-b7c1-7edf3fed00ff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-688e4a86-bpz9g"
Jul 28 18:08:45.530779 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:45.530750    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-688e4a86-bpz9g"
Jul 28 18:08:45.662721 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:45.662689    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-688e4a86-bpz9g"]
Jul 28 18:08:45.663882 ip-10-0-136-126 kubenswrapper[2563]: W0728 18:08:45.663857    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4095a2d_f338_4e7d_b7c1_7edf3fed00ff.slice/crio-dbd706e0bdbffb6d1064c096b3a3b041fcb5a752b3283b09af48de75e61a0c0e WatchSource:0}: Error finding container dbd706e0bdbffb6d1064c096b3a3b041fcb5a752b3283b09af48de75e61a0c0e: Status 404 returned error can't find the container with id dbd706e0bdbffb6d1064c096b3a3b041fcb5a752b3283b09af48de75e61a0c0e
Jul 28 18:08:45.991817 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:45.991781    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-688e4a86-bpz9g" event={"ID":"a4095a2d-f338-4e7d-b7c1-7edf3fed00ff","Type":"ContainerStarted","Data":"bd5ebd12d9155f255d8001fbc1e18119180fb54da4e336a71618c1d0e4b133ca"}
Jul 28 18:08:45.992203 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:45.991867    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-688e4a86-bpz9g" event={"ID":"a4095a2d-f338-4e7d-b7c1-7edf3fed00ff","Type":"ContainerStarted","Data":"dbd706e0bdbffb6d1064c096b3a3b041fcb5a752b3283b09af48de75e61a0c0e"}
Jul 28 18:08:46.025920 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:46.025869    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-688e4a86-bpz9g" podStartSLOduration=1.025856221 podStartE2EDuration="1.025856221s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:08:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:08:46.02463383 +0000 UTC m=+2064.685790184" watchObservedRunningTime="2026-07-28 18:08:46.025856221 +0000 UTC m=+2064.687012573"
Jul 28 18:08:46.106082 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:46.106026    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:08:46.998192 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:46.998155    2563 generic.go:356] "Generic (PLEG): container finished" podID="a4095a2d-f338-4e7d-b7c1-7edf3fed00ff" containerID="bd5ebd12d9155f255d8001fbc1e18119180fb54da4e336a71618c1d0e4b133ca" exitCode=0
Jul 28 18:08:46.998603 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:46.998226    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-688e4a86-bpz9g" event={"ID":"a4095a2d-f338-4e7d-b7c1-7edf3fed00ff","Type":"ContainerDied","Data":"bd5ebd12d9155f255d8001fbc1e18119180fb54da4e336a71618c1d0e4b133ca"}
Jul 28 18:08:48.135682 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:48.135656    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-688e4a86-bpz9g"
Jul 28 18:08:48.295662 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:48.295582    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4095a2d-f338-4e7d-b7c1-7edf3fed00ff-maas-api-service-ca\") pod \"a4095a2d-f338-4e7d-b7c1-7edf3fed00ff\" (UID: \"a4095a2d-f338-4e7d-b7c1-7edf3fed00ff\") "
Jul 28 18:08:48.295920 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:48.295901    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4095a2d-f338-4e7d-b7c1-7edf3fed00ff-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a4095a2d-f338-4e7d-b7c1-7edf3fed00ff" (UID: "a4095a2d-f338-4e7d-b7c1-7edf3fed00ff"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:08:48.396488 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:48.396457    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4095a2d-f338-4e7d-b7c1-7edf3fed00ff-maas-api-service-ca\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:08:49.000710 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:49.000675    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-688e4a86-7dc7fcff79-jmbpj"]
Jul 28 18:08:49.000954 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:49.000921    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-688e4a86-7dc7fcff79-jmbpj" podUID="d9456476-ecd0-4313-8a2c-ee1c1b6cd1a4" containerName="maas-api" containerID="cri-o://5c27b959954ad3aa9779d83ed09a5d8d46260d810de98940ae687e358bd207a0" gracePeriod=30
Jul 28 18:08:49.012245 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:49.012223    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-688e4a86-bpz9g"
Jul 28 18:08:49.012377 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:49.012258    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-688e4a86-bpz9g" event={"ID":"a4095a2d-f338-4e7d-b7c1-7edf3fed00ff","Type":"ContainerDied","Data":"dbd706e0bdbffb6d1064c096b3a3b041fcb5a752b3283b09af48de75e61a0c0e"}
Jul 28 18:08:49.012377 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:49.012311    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dbd706e0bdbffb6d1064c096b3a3b041fcb5a752b3283b09af48de75e61a0c0e"
Jul 28 18:08:49.027997 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:49.027975    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:08:49.112733 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:49.112689    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:08:49.187645 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:49.187604    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:08:49.252624 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:49.252562    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-688e4a86-7dc7fcff79-jmbpj"
Jul 28 18:08:49.287012 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:49.286977    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:08:49.297538 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:49.297511    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-688e4a86-bpz9g"]
Jul 28 18:08:49.301291 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:49.301266    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-688e4a86-bpz9g"]
Jul 28 18:08:49.404637 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:49.404604    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9456476-ecd0-4313-8a2c-ee1c1b6cd1a4-maas-api-tls\") pod \"d9456476-ecd0-4313-8a2c-ee1c1b6cd1a4\" (UID: \"d9456476-ecd0-4313-8a2c-ee1c1b6cd1a4\") "
Jul 28 18:08:49.404786 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:49.404708    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xxx6b\" (UniqueName: \"kubernetes.io/projected/d9456476-ecd0-4313-8a2c-ee1c1b6cd1a4-kube-api-access-xxx6b\") pod \"d9456476-ecd0-4313-8a2c-ee1c1b6cd1a4\" (UID: \"d9456476-ecd0-4313-8a2c-ee1c1b6cd1a4\") "
Jul 28 18:08:49.404786 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:49.404776    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9456476-ecd0-4313-8a2c-ee1c1b6cd1a4-openshift-service-ca\") pod \"d9456476-ecd0-4313-8a2c-ee1c1b6cd1a4\" (UID: \"d9456476-ecd0-4313-8a2c-ee1c1b6cd1a4\") "
Jul 28 18:08:49.405171 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:49.405140    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9456476-ecd0-4313-8a2c-ee1c1b6cd1a4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d9456476-ecd0-4313-8a2c-ee1c1b6cd1a4" (UID: "d9456476-ecd0-4313-8a2c-ee1c1b6cd1a4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:08:49.406722 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:49.406691    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9456476-ecd0-4313-8a2c-ee1c1b6cd1a4-kube-api-access-xxx6b" (OuterVolumeSpecName: "kube-api-access-xxx6b") pod "d9456476-ecd0-4313-8a2c-ee1c1b6cd1a4" (UID: "d9456476-ecd0-4313-8a2c-ee1c1b6cd1a4"). InnerVolumeSpecName "kube-api-access-xxx6b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:08:49.406821 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:49.406795    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9456476-ecd0-4313-8a2c-ee1c1b6cd1a4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d9456476-ecd0-4313-8a2c-ee1c1b6cd1a4" (UID: "d9456476-ecd0-4313-8a2c-ee1c1b6cd1a4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:08:49.506296 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:49.506217    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9456476-ecd0-4313-8a2c-ee1c1b6cd1a4-openshift-service-ca\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:08:49.506296 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:49.506245    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9456476-ecd0-4313-8a2c-ee1c1b6cd1a4-maas-api-tls\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:08:49.506296 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:49.506257    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xxx6b\" (UniqueName: \"kubernetes.io/projected/d9456476-ecd0-4313-8a2c-ee1c1b6cd1a4-kube-api-access-xxx6b\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:08:49.681836 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:49.681801    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:08:49.852675 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:49.852643    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4095a2d-f338-4e7d-b7c1-7edf3fed00ff" path="/var/lib/kubelet/pods/a4095a2d-f338-4e7d-b7c1-7edf3fed00ff/volumes"
Jul 28 18:08:50.019474 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:50.019414    2563 generic.go:356] "Generic (PLEG): container finished" podID="d9456476-ecd0-4313-8a2c-ee1c1b6cd1a4" containerID="5c27b959954ad3aa9779d83ed09a5d8d46260d810de98940ae687e358bd207a0" exitCode=0
Jul 28 18:08:50.019474 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:50.019468    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-688e4a86-7dc7fcff79-jmbpj" event={"ID":"d9456476-ecd0-4313-8a2c-ee1c1b6cd1a4","Type":"ContainerDied","Data":"5c27b959954ad3aa9779d83ed09a5d8d46260d810de98940ae687e358bd207a0"}
Jul 28 18:08:50.019706 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:50.019520    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-688e4a86-7dc7fcff79-jmbpj" event={"ID":"d9456476-ecd0-4313-8a2c-ee1c1b6cd1a4","Type":"ContainerDied","Data":"e627be904ff1821f05cb77ac8a616558e37435c487bd9e4bc35cdabc03241403"}
Jul 28 18:08:50.019706 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:50.019474    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-688e4a86-7dc7fcff79-jmbpj"
Jul 28 18:08:50.019706 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:50.019542    2563 scope.go:117] "RemoveContainer" containerID="5c27b959954ad3aa9779d83ed09a5d8d46260d810de98940ae687e358bd207a0"
Jul 28 18:08:50.028078 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:50.028023    2563 scope.go:117] "RemoveContainer" containerID="5c27b959954ad3aa9779d83ed09a5d8d46260d810de98940ae687e358bd207a0"
Jul 28 18:08:50.028272 ip-10-0-136-126 kubenswrapper[2563]: E0728 18:08:50.028253    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c27b959954ad3aa9779d83ed09a5d8d46260d810de98940ae687e358bd207a0\": container with ID starting with 5c27b959954ad3aa9779d83ed09a5d8d46260d810de98940ae687e358bd207a0 not found: ID does not exist" containerID="5c27b959954ad3aa9779d83ed09a5d8d46260d810de98940ae687e358bd207a0"
Jul 28 18:08:50.028322 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:50.028278    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c27b959954ad3aa9779d83ed09a5d8d46260d810de98940ae687e358bd207a0"} err="failed to get container status \"5c27b959954ad3aa9779d83ed09a5d8d46260d810de98940ae687e358bd207a0\": rpc error: code = NotFound desc = could not find container \"5c27b959954ad3aa9779d83ed09a5d8d46260d810de98940ae687e358bd207a0\": container with ID starting with 5c27b959954ad3aa9779d83ed09a5d8d46260d810de98940ae687e358bd207a0 not found: ID does not exist"
Jul 28 18:08:50.040465 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:50.040135    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-688e4a86-7dc7fcff79-jmbpj"]
Jul 28 18:08:50.048457 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:50.048420    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-688e4a86-7dc7fcff79-jmbpj"]
Jul 28 18:08:51.856064 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:51.856031    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9456476-ecd0-4313-8a2c-ee1c1b6cd1a4" path="/var/lib/kubelet/pods/d9456476-ecd0-4313-8a2c-ee1c1b6cd1a4/volumes"
Jul 28 18:08:56.027016 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:56.026987    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:08:56.123481 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:56.123428    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:08:56.353843 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:56.353809    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:08:56.565123 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:56.565091    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:08:56.715840 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:56.715751    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:08:56.816254 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:56.816216    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:08:56.977386 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:08:56.977294    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:03.787098 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:09:03.787066    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:03.857271 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:09:03.857241    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:03.944861 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:09:03.944830    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:04.049250 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:09:04.049159    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:04.173034 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:09:04.173001    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:11.348181 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:09:11.348147    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:11.466362 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:09:11.466329    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:18.972860 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:09:18.972829    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:19.323738 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:09:19.323709    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:19.581553 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:09:19.581471    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:22.352368 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:09:22.352333    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:22.443929 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:09:22.443898    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:22.466306 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:09:22.466275    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a6b37785-5d5cd4944b-zmmc8"]
Jul 28 18:09:22.466655 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:09:22.466642    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9456476-ecd0-4313-8a2c-ee1c1b6cd1a4" containerName="maas-api"
Jul 28 18:09:22.466711 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:09:22.466656    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9456476-ecd0-4313-8a2c-ee1c1b6cd1a4" containerName="maas-api"
Jul 28 18:09:22.466711 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:09:22.466669    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4095a2d-f338-4e7d-b7c1-7edf3fed00ff" containerName="revoke-keys"
Jul 28 18:09:22.466711 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:09:22.466675    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4095a2d-f338-4e7d-b7c1-7edf3fed00ff" containerName="revoke-keys"
Jul 28 18:09:22.466818 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:09:22.466727    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9456476-ecd0-4313-8a2c-ee1c1b6cd1a4" containerName="maas-api"
Jul 28 18:09:22.466818 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:09:22.466735    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4095a2d-f338-4e7d-b7c1-7edf3fed00ff" containerName="revoke-keys"
Jul 28 18:09:22.470797 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:09:22.470778    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a6b37785-5d5cd4944b-zmmc8"
Jul 28 18:09:22.473325 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:09:22.473307    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-a6b37785-serving-cert\""
Jul 28 18:09:22.482733 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:09:22.482708    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a6b37785-5d5cd4944b-zmmc8"]
Jul 28 18:09:22.540984 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:09:22.540949    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:22.569084 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:09:22.569050    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mczk\" (UniqueName: \"kubernetes.io/projected/2446b345-88dc-4179-9fda-46966c12bda8-kube-api-access-8mczk\") pod \"maas-api-e2e-mt-a6b37785-5d5cd4944b-zmmc8\" (UID: \"2446b345-88dc-4179-9fda-46966c12bda8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a6b37785-5d5cd4944b-zmmc8"
Jul 28 18:09:22.569255 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:09:22.569113    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2446b345-88dc-4179-9fda-46966c12bda8-openshift-service-ca\") pod \"maas-api-e2e-mt-a6b37785-5d5cd4944b-zmmc8\" (UID: \"2446b345-88dc-4179-9fda-46966c12bda8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a6b37785-5d5cd4944b-zmmc8"
Jul 28 18:09:22.569255 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:09:22.569173    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2446b345-88dc-4179-9fda-46966c12bda8-maas-api-tls\") pod \"maas-api-e2e-mt-a6b37785-5d5cd4944b-zmmc8\" (UID: \"2446b345-88dc-4179-9fda-46966c12bda8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a6b37785-5d5cd4944b-zmmc8"
Jul 28 18:09:22.670031 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:09:22.669935    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2446b345-88dc-4179-9fda-46966c12bda8-maas-api-tls\") pod \"maas-api-e2e-mt-a6b37785-5d5cd4944b-zmmc8\" (UID: \"2446b345-88dc-4179-9fda-46966c12bda8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a6b37785-5d5cd4944b-zmmc8"
Jul 28 18:09:22.670031 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:09:22.669976    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mczk\" (UniqueName: \"kubernetes.io/projected/2446b345-88dc-4179-9fda-46966c12bda8-kube-api-access-8mczk\") pod \"maas-api-e2e-mt-a6b37785-5d5cd4944b-zmmc8\" (UID: \"2446b345-88dc-4179-9fda-46966c12bda8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a6b37785-5d5cd4944b-zmmc8"
Jul 28 18:09:22.670267 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:09:22.670044    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2446b345-88dc-4179-9fda-46966c12bda8-openshift-service-ca\") pod \"maas-api-e2e-mt-a6b37785-5d5cd4944b-zmmc8\" (UID: \"2446b345-88dc-4179-9fda-46966c12bda8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a6b37785-5d5cd4944b-zmmc8"
Jul 28 18:09:22.670847 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:09:22.670815    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2446b345-88dc-4179-9fda-46966c12bda8-openshift-service-ca\") pod \"maas-api-e2e-mt-a6b37785-5d5cd4944b-zmmc8\" (UID: \"2446b345-88dc-4179-9fda-46966c12bda8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a6b37785-5d5cd4944b-zmmc8"
Jul 28 18:09:22.672707 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:09:22.672687    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2446b345-88dc-4179-9fda-46966c12bda8-maas-api-tls\") pod \"maas-api-e2e-mt-a6b37785-5d5cd4944b-zmmc8\" (UID: \"2446b345-88dc-4179-9fda-46966c12bda8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a6b37785-5d5cd4944b-zmmc8"
Jul 28 18:09:22.680957 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:09:22.680938    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mczk\" (UniqueName: \"kubernetes.io/projected/2446b345-88dc-4179-9fda-46966c12bda8-kube-api-access-8mczk\") pod \"maas-api-e2e-mt-a6b37785-5d5cd4944b-zmmc8\" (UID: \"2446b345-88dc-4179-9fda-46966c12bda8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a6b37785-5d5cd4944b-zmmc8"
Jul 28 18:09:22.722971 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:09:22.722935    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:22.781290 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:09:22.781257    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a6b37785-5d5cd4944b-zmmc8"
Jul 28 18:09:22.927138 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:09:22.927102    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a6b37785-5d5cd4944b-zmmc8"]
Jul 28 18:09:23.198316 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:09:23.198237    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a6b37785-5d5cd4944b-zmmc8" event={"ID":"2446b345-88dc-4179-9fda-46966c12bda8","Type":"ContainerStarted","Data":"f76cd0f85eea1d9607eda6654892a23e10d96018e84c50bf961978b1f0406e31"}
Jul 28 18:09:23.212965 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:09:23.212932    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:23.329580 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:09:23.329542    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:24.204685 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:09:24.204646    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a6b37785-5d5cd4944b-zmmc8" event={"ID":"2446b345-88dc-4179-9fda-46966c12bda8","Type":"ContainerStarted","Data":"dac934fbebac3cc6bfd1eb9581ba3ec018b9c6b46f2ce31005a298c21a849676"}
Jul 28 18:09:24.205089 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:09:24.204744    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a6b37785-5d5cd4944b-zmmc8"
Jul 28 18:09:24.230780 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:09:24.230738    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a6b37785-5d5cd4944b-zmmc8" podStartSLOduration=1.764451977 podStartE2EDuration="2.230723931s" totalImagesPullingTime="466.271954ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:09:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:09:24.229797258 +0000 UTC m=+2102.890953613" watchObservedRunningTime="2026-07-28 18:09:24.230723931 +0000 UTC m=+2102.891880287"
Jul 28 18:09:30.222099 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:09:30.222063    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a6b37785-5d5cd4944b-zmmc8"
Jul 28 18:09:47.348535 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:09:47.348499    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:09:51.428577 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:09:51.428544    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:10:22.019564 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:10:22.019535    2563 scope.go:117] "RemoveContainer" containerID="a928d3de518aa8b78f8eaa12dd21189f05c085c3b574ba2d9b5365ac36a90e32"
Jul 28 18:10:22.263080 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:10:22.263045    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a6b37785-vfrbt"]
Jul 28 18:10:22.267809 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:10:22.267789    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a6b37785-vfrbt"
Jul 28 18:10:22.272455 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:10:22.272390    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9xcv2\""
Jul 28 18:10:22.279229 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:10:22.279208    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0dee2433-99e7-4433-9464-83f435ac36df-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-a6b37785-vfrbt\" (UID: \"0dee2433-99e7-4433-9464-83f435ac36df\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a6b37785-vfrbt"
Jul 28 18:10:22.285792 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:10:22.285770    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a6b37785-vfrbt"]
Jul 28 18:10:22.379849 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:10:22.379809    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0dee2433-99e7-4433-9464-83f435ac36df-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-a6b37785-vfrbt\" (UID: \"0dee2433-99e7-4433-9464-83f435ac36df\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a6b37785-vfrbt"
Jul 28 18:10:22.380416 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:10:22.380396    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0dee2433-99e7-4433-9464-83f435ac36df-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-a6b37785-vfrbt\" (UID: \"0dee2433-99e7-4433-9464-83f435ac36df\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a6b37785-vfrbt"
Jul 28 18:10:22.577521 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:10:22.577492    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a6b37785-vfrbt"
Jul 28 18:10:22.718093 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:10:22.718063    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a6b37785-vfrbt"]
Jul 28 18:10:22.719664 ip-10-0-136-126 kubenswrapper[2563]: W0728 18:10:22.719637    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0dee2433_99e7_4433_9464_83f435ac36df.slice/crio-c48882db867ce8b97fc8288b5916a3ce23920061e538fe6ed87d31cce9ccce33 WatchSource:0}: Error finding container c48882db867ce8b97fc8288b5916a3ce23920061e538fe6ed87d31cce9ccce33: Status 404 returned error can't find the container with id c48882db867ce8b97fc8288b5916a3ce23920061e538fe6ed87d31cce9ccce33
Jul 28 18:10:23.528335 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:10:23.528298    2563 generic.go:356] "Generic (PLEG): container finished" podID="0dee2433-99e7-4433-9464-83f435ac36df" containerID="801e6402c1f8c3e84e5c18aef4ec489f85e0661e65d76ff68cdcfc92e416bbdb" exitCode=0
Jul 28 18:10:23.528776 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:10:23.528382    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a6b37785-vfrbt" event={"ID":"0dee2433-99e7-4433-9464-83f435ac36df","Type":"ContainerDied","Data":"801e6402c1f8c3e84e5c18aef4ec489f85e0661e65d76ff68cdcfc92e416bbdb"}
Jul 28 18:10:23.528776 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:10:23.528429    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a6b37785-vfrbt" event={"ID":"0dee2433-99e7-4433-9464-83f435ac36df","Type":"ContainerStarted","Data":"c48882db867ce8b97fc8288b5916a3ce23920061e538fe6ed87d31cce9ccce33"}
Jul 28 18:10:24.664850 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:10:24.664823    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a6b37785-vfrbt"
Jul 28 18:10:24.700566 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:10:24.700541    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0dee2433-99e7-4433-9464-83f435ac36df-maas-api-service-ca\") pod \"0dee2433-99e7-4433-9464-83f435ac36df\" (UID: \"0dee2433-99e7-4433-9464-83f435ac36df\") "
Jul 28 18:10:24.700865 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:10:24.700844    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0dee2433-99e7-4433-9464-83f435ac36df-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0dee2433-99e7-4433-9464-83f435ac36df" (UID: "0dee2433-99e7-4433-9464-83f435ac36df"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:10:24.802119 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:10:24.802036    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0dee2433-99e7-4433-9464-83f435ac36df-maas-api-service-ca\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:10:25.541148 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:10:25.541115    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a6b37785-vfrbt"
Jul 28 18:10:25.541148 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:10:25.541138    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a6b37785-vfrbt" event={"ID":"0dee2433-99e7-4433-9464-83f435ac36df","Type":"ContainerDied","Data":"c48882db867ce8b97fc8288b5916a3ce23920061e538fe6ed87d31cce9ccce33"}
Jul 28 18:10:25.541357 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:10:25.541183    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c48882db867ce8b97fc8288b5916a3ce23920061e538fe6ed87d31cce9ccce33"
Jul 28 18:10:38.212902 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:10:38.212870    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a6b37785-5d5cd4944b-zmmc8"]
Jul 28 18:10:38.215338 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:10:38.213052    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a6b37785-5d5cd4944b-zmmc8" podUID="2446b345-88dc-4179-9fda-46966c12bda8" containerName="maas-api" containerID="cri-o://dac934fbebac3cc6bfd1eb9581ba3ec018b9c6b46f2ce31005a298c21a849676" gracePeriod=30
Jul 28 18:10:38.250753 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:10:38.250720    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:10:38.341429 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:10:38.341400    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:10:38.416875 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:10:38.416843    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:10:38.466174 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:10:38.466118    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a6b37785-5d5cd4944b-zmmc8"
Jul 28 18:10:38.478312 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:10:38.478286    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a6b37785-vfrbt"]
Jul 28 18:10:38.480679 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:10:38.480656    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a6b37785-vfrbt"]
Jul 28 18:10:38.485129 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:10:38.485101    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:10:38.518005 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:10:38.517977    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2446b345-88dc-4179-9fda-46966c12bda8-openshift-service-ca\") pod \"2446b345-88dc-4179-9fda-46966c12bda8\" (UID: \"2446b345-88dc-4179-9fda-46966c12bda8\") "
Jul 28 18:10:38.518005 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:10:38.518011    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8mczk\" (UniqueName: \"kubernetes.io/projected/2446b345-88dc-4179-9fda-46966c12bda8-kube-api-access-8mczk\") pod \"2446b345-88dc-4179-9fda-46966c12bda8\" (UID: \"2446b345-88dc-4179-9fda-46966c12bda8\") "
Jul 28 18:10:38.518221 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:10:38.518075    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2446b345-88dc-4179-9fda-46966c12bda8-maas-api-tls\") pod \"2446b345-88dc-4179-9fda-46966c12bda8\" (UID: \"2446b345-88dc-4179-9fda-46966c12bda8\") "
Jul 28 18:10:38.518351 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:10:38.518327    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2446b345-88dc-4179-9fda-46966c12bda8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2446b345-88dc-4179-9fda-46966c12bda8" (UID: "2446b345-88dc-4179-9fda-46966c12bda8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:10:38.520182 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:10:38.520152    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2446b345-88dc-4179-9fda-46966c12bda8-kube-api-access-8mczk" (OuterVolumeSpecName: "kube-api-access-8mczk") pod "2446b345-88dc-4179-9fda-46966c12bda8" (UID: "2446b345-88dc-4179-9fda-46966c12bda8"). InnerVolumeSpecName "kube-api-access-8mczk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:10:38.520275 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:10:38.520158    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2446b345-88dc-4179-9fda-46966c12bda8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2446b345-88dc-4179-9fda-46966c12bda8" (UID: "2446b345-88dc-4179-9fda-46966c12bda8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:10:38.610735 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:10:38.610704    2563 generic.go:356] "Generic (PLEG): container finished" podID="2446b345-88dc-4179-9fda-46966c12bda8" containerID="dac934fbebac3cc6bfd1eb9581ba3ec018b9c6b46f2ce31005a298c21a849676" exitCode=0
Jul 28 18:10:38.610899 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:10:38.610762    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a6b37785-5d5cd4944b-zmmc8"
Jul 28 18:10:38.610899 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:10:38.610794    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a6b37785-5d5cd4944b-zmmc8" event={"ID":"2446b345-88dc-4179-9fda-46966c12bda8","Type":"ContainerDied","Data":"dac934fbebac3cc6bfd1eb9581ba3ec018b9c6b46f2ce31005a298c21a849676"}
Jul 28 18:10:38.610899 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:10:38.610879    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a6b37785-5d5cd4944b-zmmc8" event={"ID":"2446b345-88dc-4179-9fda-46966c12bda8","Type":"ContainerDied","Data":"f76cd0f85eea1d9607eda6654892a23e10d96018e84c50bf961978b1f0406e31"}
Jul 28 18:10:38.611019 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:10:38.610903    2563 scope.go:117] "RemoveContainer" containerID="dac934fbebac3cc6bfd1eb9581ba3ec018b9c6b46f2ce31005a298c21a849676"
Jul 28 18:10:38.619008 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:10:38.618948    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2446b345-88dc-4179-9fda-46966c12bda8-maas-api-tls\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:10:38.619008 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:10:38.618966    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2446b345-88dc-4179-9fda-46966c12bda8-openshift-service-ca\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:10:38.619008 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:10:38.618976    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8mczk\" (UniqueName: \"kubernetes.io/projected/2446b345-88dc-4179-9fda-46966c12bda8-kube-api-access-8mczk\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:10:38.620774 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:10:38.620757    2563 scope.go:117] "RemoveContainer" containerID="dac934fbebac3cc6bfd1eb9581ba3ec018b9c6b46f2ce31005a298c21a849676"
Jul 28 18:10:38.621023 ip-10-0-136-126 kubenswrapper[2563]: E0728 18:10:38.621004    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dac934fbebac3cc6bfd1eb9581ba3ec018b9c6b46f2ce31005a298c21a849676\": container with ID starting with dac934fbebac3cc6bfd1eb9581ba3ec018b9c6b46f2ce31005a298c21a849676 not found: ID does not exist" containerID="dac934fbebac3cc6bfd1eb9581ba3ec018b9c6b46f2ce31005a298c21a849676"
Jul 28 18:10:38.621068 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:10:38.621032    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dac934fbebac3cc6bfd1eb9581ba3ec018b9c6b46f2ce31005a298c21a849676"} err="failed to get container status \"dac934fbebac3cc6bfd1eb9581ba3ec018b9c6b46f2ce31005a298c21a849676\": rpc error: code = NotFound desc = could not find container \"dac934fbebac3cc6bfd1eb9581ba3ec018b9c6b46f2ce31005a298c21a849676\": container with ID starting with dac934fbebac3cc6bfd1eb9581ba3ec018b9c6b46f2ce31005a298c21a849676 not found: ID does not exist"
Jul 28 18:10:38.638251 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:10:38.638227    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a6b37785-5d5cd4944b-zmmc8"]
Jul 28 18:10:38.641244 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:10:38.641223    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a6b37785-5d5cd4944b-zmmc8"]
Jul 28 18:10:39.852995 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:10:39.852965    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0dee2433-99e7-4433-9464-83f435ac36df" path="/var/lib/kubelet/pods/0dee2433-99e7-4433-9464-83f435ac36df/volumes"
Jul 28 18:10:39.853359 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:10:39.853329    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2446b345-88dc-4179-9fda-46966c12bda8" path="/var/lib/kubelet/pods/2446b345-88dc-4179-9fda-46966c12bda8/volumes"
Jul 28 18:10:42.106366 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:10:42.106335    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:10:47.272811 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:10:47.272779    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:11:22.041325 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:11:22.041292    2563 scope.go:117] "RemoveContainer" containerID="fbdaafaf81f306363c0778f50007d5ed48d0cdf23b74989fa91695a615ec3e69"
Jul 28 18:11:24.118999 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:11:24.118952    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:11:25.332753 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:11:25.332679    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:11:29.997388 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:11:29.997342    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:11:32.061709 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:11:32.061674    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:11:35.790722 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:11:35.790689    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:11:58.520486 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:11:58.520375    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:11:58.587022 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:11:58.586979    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:11:58.826877 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:11:58.826829    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:11:59.064430 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:11:59.064372    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:12:00.254481 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:12:00.254419    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:12:00.357309 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:12:00.357246    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:12:00.455397 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:12:00.455364    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:12:06.301790 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:12:06.301751    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:12:06.387225 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:12:06.387154    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:12:06.488651 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:12:06.488619    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:12:08.088420 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:12:08.088384    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:12:17.081755 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:12:17.081725    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:12:17.180557 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:12:17.180522    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:12:17.822745 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:12:17.822711    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:12:22.069228 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:12:22.069195    2563 scope.go:117] "RemoveContainer" containerID="4e7d3fe179322c4c1a32afd39801966677fad7bf4446131d7dfebd822e32886d"
Jul 28 18:12:22.078045 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:12:22.078020    2563 scope.go:117] "RemoveContainer" containerID="8710bba8e2537dbefe9b37b6f848977db9fffdb25aef0741fba0d2c04bca2a14"
Jul 28 18:12:25.817098 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:12:25.817066    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:12:26.611323 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:12:26.611291    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:12:26.711968 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:12:26.711934    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:12:28.777653 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:12:28.777614    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:12:29.649088 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:12:29.649050    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:12:36.604082 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:12:36.604045    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:12:37.007329 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:12:37.007247    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:12:58.019993 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:12:58.019962    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:13:06.071704 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:06.071675    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:13:06.144098 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:06.144071    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:13:06.489956 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:06.489871    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:13:06.952580 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:06.952546    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:13:07.136608 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:07.136573    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:13:10.424757 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:10.424724    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:13:10.513990 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:10.513955    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:13:10.670039 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:10.670006    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-1cce3e-5759d75769-645qh"]
Jul 28 18:13:10.670377 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:10.670360    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2446b345-88dc-4179-9fda-46966c12bda8" containerName="maas-api"
Jul 28 18:13:10.670430 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:10.670381    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2446b345-88dc-4179-9fda-46966c12bda8" containerName="maas-api"
Jul 28 18:13:10.670430 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:10.670395    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0dee2433-99e7-4433-9464-83f435ac36df" containerName="revoke-keys"
Jul 28 18:13:10.670430 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:10.670403    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0dee2433-99e7-4433-9464-83f435ac36df" containerName="revoke-keys"
Jul 28 18:13:10.670558 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:10.670498    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="0dee2433-99e7-4433-9464-83f435ac36df" containerName="revoke-keys"
Jul 28 18:13:10.670558 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:10.670508    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2446b345-88dc-4179-9fda-46966c12bda8" containerName="maas-api"
Jul 28 18:13:10.673590 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:10.673572    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1cce3e-5759d75769-645qh"
Jul 28 18:13:10.677746 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:10.677677    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-1cce3e-serving-cert\""
Jul 28 18:13:10.708405 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:10.708378    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-1cce3e-5759d75769-645qh"]
Jul 28 18:13:10.713012 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:10.712989    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:13:10.721395 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:10.721371    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df9e55db-0fc7-47cd-8e31-d2e0492c2fae-openshift-service-ca\") pod \"maas-api-e2e-inf-a-1cce3e-5759d75769-645qh\" (UID: \"df9e55db-0fc7-47cd-8e31-d2e0492c2fae\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1cce3e-5759d75769-645qh"
Jul 28 18:13:10.721527 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:10.721404    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mphhr\" (UniqueName: \"kubernetes.io/projected/df9e55db-0fc7-47cd-8e31-d2e0492c2fae-kube-api-access-mphhr\") pod \"maas-api-e2e-inf-a-1cce3e-5759d75769-645qh\" (UID: \"df9e55db-0fc7-47cd-8e31-d2e0492c2fae\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1cce3e-5759d75769-645qh"
Jul 28 18:13:10.721527 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:10.721477    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df9e55db-0fc7-47cd-8e31-d2e0492c2fae-maas-api-tls\") pod \"maas-api-e2e-inf-a-1cce3e-5759d75769-645qh\" (UID: \"df9e55db-0fc7-47cd-8e31-d2e0492c2fae\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1cce3e-5759d75769-645qh"
Jul 28 18:13:10.821920 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:10.821882    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mphhr\" (UniqueName: \"kubernetes.io/projected/df9e55db-0fc7-47cd-8e31-d2e0492c2fae-kube-api-access-mphhr\") pod \"maas-api-e2e-inf-a-1cce3e-5759d75769-645qh\" (UID: \"df9e55db-0fc7-47cd-8e31-d2e0492c2fae\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1cce3e-5759d75769-645qh"
Jul 28 18:13:10.822078 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:10.821934    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df9e55db-0fc7-47cd-8e31-d2e0492c2fae-maas-api-tls\") pod \"maas-api-e2e-inf-a-1cce3e-5759d75769-645qh\" (UID: \"df9e55db-0fc7-47cd-8e31-d2e0492c2fae\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1cce3e-5759d75769-645qh"
Jul 28 18:13:10.822078 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:10.821989    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df9e55db-0fc7-47cd-8e31-d2e0492c2fae-openshift-service-ca\") pod \"maas-api-e2e-inf-a-1cce3e-5759d75769-645qh\" (UID: \"df9e55db-0fc7-47cd-8e31-d2e0492c2fae\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1cce3e-5759d75769-645qh"
Jul 28 18:13:10.822157 ip-10-0-136-126 kubenswrapper[2563]: E0728 18:13:10.822091    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-1cce3e-serving-cert: secret "maas-api-e2e-inf-a-1cce3e-serving-cert" not found
Jul 28 18:13:10.822199 ip-10-0-136-126 kubenswrapper[2563]: E0728 18:13:10.822168    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df9e55db-0fc7-47cd-8e31-d2e0492c2fae-maas-api-tls podName:df9e55db-0fc7-47cd-8e31-d2e0492c2fae nodeName:}" failed. No retries permitted until 2026-07-28 18:13:11.322149832 +0000 UTC m=+2329.983306172 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/df9e55db-0fc7-47cd-8e31-d2e0492c2fae-maas-api-tls") pod "maas-api-e2e-inf-a-1cce3e-5759d75769-645qh" (UID: "df9e55db-0fc7-47cd-8e31-d2e0492c2fae") : secret "maas-api-e2e-inf-a-1cce3e-serving-cert" not found
Jul 28 18:13:10.822660 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:10.822641    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df9e55db-0fc7-47cd-8e31-d2e0492c2fae-openshift-service-ca\") pod \"maas-api-e2e-inf-a-1cce3e-5759d75769-645qh\" (UID: \"df9e55db-0fc7-47cd-8e31-d2e0492c2fae\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1cce3e-5759d75769-645qh"
Jul 28 18:13:10.838761 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:10.838732    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mphhr\" (UniqueName: \"kubernetes.io/projected/df9e55db-0fc7-47cd-8e31-d2e0492c2fae-kube-api-access-mphhr\") pod \"maas-api-e2e-inf-a-1cce3e-5759d75769-645qh\" (UID: \"df9e55db-0fc7-47cd-8e31-d2e0492c2fae\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1cce3e-5759d75769-645qh"
Jul 28 18:13:10.902775 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:10.902748    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:13:11.324871 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:11.324833    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df9e55db-0fc7-47cd-8e31-d2e0492c2fae-maas-api-tls\") pod \"maas-api-e2e-inf-a-1cce3e-5759d75769-645qh\" (UID: \"df9e55db-0fc7-47cd-8e31-d2e0492c2fae\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1cce3e-5759d75769-645qh"
Jul 28 18:13:11.327213 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:11.327190    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df9e55db-0fc7-47cd-8e31-d2e0492c2fae-maas-api-tls\") pod \"maas-api-e2e-inf-a-1cce3e-5759d75769-645qh\" (UID: \"df9e55db-0fc7-47cd-8e31-d2e0492c2fae\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1cce3e-5759d75769-645qh"
Jul 28 18:13:11.545221 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:11.545186    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:13:11.583373 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:11.583298    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1cce3e-5759d75769-645qh"
Jul 28 18:13:11.636452 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:11.636201    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:13:11.734826 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:11.734789    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-1cce3e-5759d75769-645qh"]
Jul 28 18:13:11.735636 ip-10-0-136-126 kubenswrapper[2563]: W0728 18:13:11.735607    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf9e55db_0fc7_47cd_8e31_d2e0492c2fae.slice/crio-26dc50066f05470dc2a7a833eb6ab1f207d2f167a087ca4e8a14bc77dcfbb121 WatchSource:0}: Error finding container 26dc50066f05470dc2a7a833eb6ab1f207d2f167a087ca4e8a14bc77dcfbb121: Status 404 returned error can't find the container with id 26dc50066f05470dc2a7a833eb6ab1f207d2f167a087ca4e8a14bc77dcfbb121
Jul 28 18:13:11.737389 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:11.737366    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 18:13:12.415504 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:12.415458    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1cce3e-5759d75769-645qh" event={"ID":"df9e55db-0fc7-47cd-8e31-d2e0492c2fae","Type":"ContainerStarted","Data":"60a1d3993ecdecf260fd8d8933508084502d28f95ef4130a51083ec6f3d64c8f"}
Jul 28 18:13:12.415707 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:12.415544    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1cce3e-5759d75769-645qh" event={"ID":"df9e55db-0fc7-47cd-8e31-d2e0492c2fae","Type":"ContainerStarted","Data":"26dc50066f05470dc2a7a833eb6ab1f207d2f167a087ca4e8a14bc77dcfbb121"}
Jul 28 18:13:12.415707 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:12.415587    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1cce3e-5759d75769-645qh"
Jul 28 18:13:12.449065 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:12.448970    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1cce3e-5759d75769-645qh" podStartSLOduration=2.0040145 podStartE2EDuration="2.448955586s" totalImagesPullingTime="444.941086ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:13:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:13:12.44740465 +0000 UTC m=+2331.108561006" watchObservedRunningTime="2026-07-28 18:13:12.448955586 +0000 UTC m=+2331.110111941"
Jul 28 18:13:18.427253 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:18.427176    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1cce3e-5759d75769-645qh"
Jul 28 18:13:35.518673 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:35.518643    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:13:35.898498 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:35.898468    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:13:36.380159 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:36.380128    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:13:36.578981 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:36.578948    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:13:39.600065 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:39.600027    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:13:39.709612 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:39.709581    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:13:39.865592 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:39.865502    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-95c506-55b8fcccb6-c4krk"]
Jul 28 18:13:39.868775 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:39.868754    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-95c506-55b8fcccb6-c4krk"
Jul 28 18:13:39.873098 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:39.872800    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-95c506-serving-cert\""
Jul 28 18:13:39.873668 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:39.873644    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-95c506-55b8fcccb6-c4krk"]
Jul 28 18:13:39.902538 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:39.902512    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:13:39.975684 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:39.975648    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a13e7776-0ae8-47fb-b3e6-597c29fea161-maas-api-tls\") pod \"maas-api-e2e-inf-b-95c506-55b8fcccb6-c4krk\" (UID: \"a13e7776-0ae8-47fb-b3e6-597c29fea161\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-95c506-55b8fcccb6-c4krk"
Jul 28 18:13:39.975855 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:39.975699    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a13e7776-0ae8-47fb-b3e6-597c29fea161-openshift-service-ca\") pod \"maas-api-e2e-inf-b-95c506-55b8fcccb6-c4krk\" (UID: \"a13e7776-0ae8-47fb-b3e6-597c29fea161\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-95c506-55b8fcccb6-c4krk"
Jul 28 18:13:39.975855 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:39.975779    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqx64\" (UniqueName: \"kubernetes.io/projected/a13e7776-0ae8-47fb-b3e6-597c29fea161-kube-api-access-hqx64\") pod \"maas-api-e2e-inf-b-95c506-55b8fcccb6-c4krk\" (UID: \"a13e7776-0ae8-47fb-b3e6-597c29fea161\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-95c506-55b8fcccb6-c4krk"
Jul 28 18:13:40.077188 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:40.077161    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a13e7776-0ae8-47fb-b3e6-597c29fea161-maas-api-tls\") pod \"maas-api-e2e-inf-b-95c506-55b8fcccb6-c4krk\" (UID: \"a13e7776-0ae8-47fb-b3e6-597c29fea161\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-95c506-55b8fcccb6-c4krk"
Jul 28 18:13:40.077350 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:40.077203    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a13e7776-0ae8-47fb-b3e6-597c29fea161-openshift-service-ca\") pod \"maas-api-e2e-inf-b-95c506-55b8fcccb6-c4krk\" (UID: \"a13e7776-0ae8-47fb-b3e6-597c29fea161\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-95c506-55b8fcccb6-c4krk"
Jul 28 18:13:40.077350 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:40.077230    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqx64\" (UniqueName: \"kubernetes.io/projected/a13e7776-0ae8-47fb-b3e6-597c29fea161-kube-api-access-hqx64\") pod \"maas-api-e2e-inf-b-95c506-55b8fcccb6-c4krk\" (UID: \"a13e7776-0ae8-47fb-b3e6-597c29fea161\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-95c506-55b8fcccb6-c4krk"
Jul 28 18:13:40.077350 ip-10-0-136-126 kubenswrapper[2563]: E0728 18:13:40.077311    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-95c506-serving-cert: secret "maas-api-e2e-inf-b-95c506-serving-cert" not found
Jul 28 18:13:40.077491 ip-10-0-136-126 kubenswrapper[2563]: E0728 18:13:40.077385    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a13e7776-0ae8-47fb-b3e6-597c29fea161-maas-api-tls podName:a13e7776-0ae8-47fb-b3e6-597c29fea161 nodeName:}" failed. No retries permitted until 2026-07-28 18:13:40.577368184 +0000 UTC m=+2359.238524517 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a13e7776-0ae8-47fb-b3e6-597c29fea161-maas-api-tls") pod "maas-api-e2e-inf-b-95c506-55b8fcccb6-c4krk" (UID: "a13e7776-0ae8-47fb-b3e6-597c29fea161") : secret "maas-api-e2e-inf-b-95c506-serving-cert" not found
Jul 28 18:13:40.077808 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:40.077789    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a13e7776-0ae8-47fb-b3e6-597c29fea161-openshift-service-ca\") pod \"maas-api-e2e-inf-b-95c506-55b8fcccb6-c4krk\" (UID: \"a13e7776-0ae8-47fb-b3e6-597c29fea161\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-95c506-55b8fcccb6-c4krk"
Jul 28 18:13:40.090707 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:40.090685    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqx64\" (UniqueName: \"kubernetes.io/projected/a13e7776-0ae8-47fb-b3e6-597c29fea161-kube-api-access-hqx64\") pod \"maas-api-e2e-inf-b-95c506-55b8fcccb6-c4krk\" (UID: \"a13e7776-0ae8-47fb-b3e6-597c29fea161\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-95c506-55b8fcccb6-c4krk"
Jul 28 18:13:40.148067 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:40.147995    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:13:40.579803 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:40.579770    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:13:40.581965 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:40.581943    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a13e7776-0ae8-47fb-b3e6-597c29fea161-maas-api-tls\") pod \"maas-api-e2e-inf-b-95c506-55b8fcccb6-c4krk\" (UID: \"a13e7776-0ae8-47fb-b3e6-597c29fea161\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-95c506-55b8fcccb6-c4krk"
Jul 28 18:13:40.584384 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:40.584360    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a13e7776-0ae8-47fb-b3e6-597c29fea161-maas-api-tls\") pod \"maas-api-e2e-inf-b-95c506-55b8fcccb6-c4krk\" (UID: \"a13e7776-0ae8-47fb-b3e6-597c29fea161\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-95c506-55b8fcccb6-c4krk"
Jul 28 18:13:40.679904 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:40.679872    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:13:40.725305 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:40.725277    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-95c506-f469f5844-scv75"]
Jul 28 18:13:40.728851 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:40.728830    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-95c506-f469f5844-scv75"
Jul 28 18:13:40.732856 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:40.732835    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-95c506\""
Jul 28 18:13:40.745326 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:40.745298    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-95c506-f469f5844-scv75"]
Jul 28 18:13:40.748523 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:40.748495    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-95c506-5b8ccbd94d-xmmh9"]
Jul 28 18:13:40.751662 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:40.751643    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-95c506-5b8ccbd94d-xmmh9"
Jul 28 18:13:40.754579 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:40.754562    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-95c506-dockercfg-w6jlf\""
Jul 28 18:13:40.765592 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:40.765572    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-95c506-5b8ccbd94d-xmmh9"]
Jul 28 18:13:40.780139 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:40.780111    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-95c506-55b8fcccb6-c4krk"
Jul 28 18:13:40.885652 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:40.885622    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9znpn\" (UniqueName: \"kubernetes.io/projected/c8d97283-963e-478a-b641-459c17ab71ca-kube-api-access-9znpn\") pod \"payload-processing-e2e-inf-b-95c506-5b8ccbd94d-xmmh9\" (UID: \"c8d97283-963e-478a-b641-459c17ab71ca\") " pod="openshift-ingress/payload-processing-e2e-inf-b-95c506-5b8ccbd94d-xmmh9"
Jul 28 18:13:40.885848 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:40.885662    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1933138a-91ef-4698-a911-eefc04421307-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-95c506-f469f5844-scv75\" (UID: \"1933138a-91ef-4698-a911-eefc04421307\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-95c506-f469f5844-scv75"
Jul 28 18:13:40.885848 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:40.885708    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6txnt\" (UniqueName: \"kubernetes.io/projected/1933138a-91ef-4698-a911-eefc04421307-kube-api-access-6txnt\") pod \"payload-pre-processing-e2e-inf-b-95c506-f469f5844-scv75\" (UID: \"1933138a-91ef-4698-a911-eefc04421307\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-95c506-f469f5844-scv75"
Jul 28 18:13:40.885848 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:40.885731    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8d97283-963e-478a-b641-459c17ab71ca-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-95c506-5b8ccbd94d-xmmh9\" (UID: \"c8d97283-963e-478a-b641-459c17ab71ca\") " pod="openshift-ingress/payload-processing-e2e-inf-b-95c506-5b8ccbd94d-xmmh9"
Jul 28 18:13:40.930229 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:40.930198    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-95c506-55b8fcccb6-c4krk"]
Jul 28 18:13:40.931623 ip-10-0-136-126 kubenswrapper[2563]: W0728 18:13:40.931598    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda13e7776_0ae8_47fb_b3e6_597c29fea161.slice/crio-e21a212d8150befa9b9b8dd33fa0e8a731585db77385bd9a8c1eb8e7da52a6ba WatchSource:0}: Error finding container e21a212d8150befa9b9b8dd33fa0e8a731585db77385bd9a8c1eb8e7da52a6ba: Status 404 returned error can't find the container with id e21a212d8150befa9b9b8dd33fa0e8a731585db77385bd9a8c1eb8e7da52a6ba
Jul 28 18:13:40.986835 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:40.986803    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9znpn\" (UniqueName: \"kubernetes.io/projected/c8d97283-963e-478a-b641-459c17ab71ca-kube-api-access-9znpn\") pod \"payload-processing-e2e-inf-b-95c506-5b8ccbd94d-xmmh9\" (UID: \"c8d97283-963e-478a-b641-459c17ab71ca\") " pod="openshift-ingress/payload-processing-e2e-inf-b-95c506-5b8ccbd94d-xmmh9"
Jul 28 18:13:40.986951 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:40.986850    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1933138a-91ef-4698-a911-eefc04421307-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-95c506-f469f5844-scv75\" (UID: \"1933138a-91ef-4698-a911-eefc04421307\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-95c506-f469f5844-scv75"
Jul 28 18:13:40.986951 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:40.986919    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6txnt\" (UniqueName: \"kubernetes.io/projected/1933138a-91ef-4698-a911-eefc04421307-kube-api-access-6txnt\") pod \"payload-pre-processing-e2e-inf-b-95c506-f469f5844-scv75\" (UID: \"1933138a-91ef-4698-a911-eefc04421307\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-95c506-f469f5844-scv75"
Jul 28 18:13:40.987066 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:40.986949    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8d97283-963e-478a-b641-459c17ab71ca-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-95c506-5b8ccbd94d-xmmh9\" (UID: \"c8d97283-963e-478a-b641-459c17ab71ca\") " pod="openshift-ingress/payload-processing-e2e-inf-b-95c506-5b8ccbd94d-xmmh9"
Jul 28 18:13:40.987581 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:40.987558    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1933138a-91ef-4698-a911-eefc04421307-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-95c506-f469f5844-scv75\" (UID: \"1933138a-91ef-4698-a911-eefc04421307\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-95c506-f469f5844-scv75"
Jul 28 18:13:40.987680 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:40.987648    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8d97283-963e-478a-b641-459c17ab71ca-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-95c506-5b8ccbd94d-xmmh9\" (UID: \"c8d97283-963e-478a-b641-459c17ab71ca\") " pod="openshift-ingress/payload-processing-e2e-inf-b-95c506-5b8ccbd94d-xmmh9"
Jul 28 18:13:40.997397 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:40.997368    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6txnt\" (UniqueName: \"kubernetes.io/projected/1933138a-91ef-4698-a911-eefc04421307-kube-api-access-6txnt\") pod \"payload-pre-processing-e2e-inf-b-95c506-f469f5844-scv75\" (UID: \"1933138a-91ef-4698-a911-eefc04421307\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-95c506-f469f5844-scv75"
Jul 28 18:13:40.997510 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:40.997459    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9znpn\" (UniqueName: \"kubernetes.io/projected/c8d97283-963e-478a-b641-459c17ab71ca-kube-api-access-9znpn\") pod \"payload-processing-e2e-inf-b-95c506-5b8ccbd94d-xmmh9\" (UID: \"c8d97283-963e-478a-b641-459c17ab71ca\") " pod="openshift-ingress/payload-processing-e2e-inf-b-95c506-5b8ccbd94d-xmmh9"
Jul 28 18:13:41.040317 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:41.040278    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-95c506-f469f5844-scv75"
Jul 28 18:13:41.062562 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:41.062535    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-95c506-5b8ccbd94d-xmmh9"
Jul 28 18:13:41.199795 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:41.199760    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-95c506-f469f5844-scv75"]
Jul 28 18:13:41.203125 ip-10-0-136-126 kubenswrapper[2563]: W0728 18:13:41.203091    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1933138a_91ef_4698_a911_eefc04421307.slice/crio-e297a482cc145eaeae0482d519ea0890bbf8df84022387031da7c75f5b5b7e5e WatchSource:0}: Error finding container e297a482cc145eaeae0482d519ea0890bbf8df84022387031da7c75f5b5b7e5e: Status 404 returned error can't find the container with id e297a482cc145eaeae0482d519ea0890bbf8df84022387031da7c75f5b5b7e5e
Jul 28 18:13:41.232917 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:41.232874    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-95c506-5b8ccbd94d-xmmh9"]
Jul 28 18:13:41.233778 ip-10-0-136-126 kubenswrapper[2563]: W0728 18:13:41.233748    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8d97283_963e_478a_b641_459c17ab71ca.slice/crio-fd077525d33aee9847dc2bca9a4adcc1ea2ecccf52d421d3203e38805e21fbe6 WatchSource:0}: Error finding container fd077525d33aee9847dc2bca9a4adcc1ea2ecccf52d421d3203e38805e21fbe6: Status 404 returned error can't find the container with id fd077525d33aee9847dc2bca9a4adcc1ea2ecccf52d421d3203e38805e21fbe6
Jul 28 18:13:41.562678 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:41.562598    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-95c506-55b8fcccb6-c4krk" event={"ID":"a13e7776-0ae8-47fb-b3e6-597c29fea161","Type":"ContainerStarted","Data":"e21a212d8150befa9b9b8dd33fa0e8a731585db77385bd9a8c1eb8e7da52a6ba"}
Jul 28 18:13:41.563746 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:41.563722    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-95c506-5b8ccbd94d-xmmh9" event={"ID":"c8d97283-963e-478a-b641-459c17ab71ca","Type":"ContainerStarted","Data":"fd077525d33aee9847dc2bca9a4adcc1ea2ecccf52d421d3203e38805e21fbe6"}
Jul 28 18:13:41.564767 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:41.564743    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-95c506-f469f5844-scv75" event={"ID":"1933138a-91ef-4698-a911-eefc04421307","Type":"ContainerStarted","Data":"e297a482cc145eaeae0482d519ea0890bbf8df84022387031da7c75f5b5b7e5e"}
Jul 28 18:13:42.519203 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:42.519167    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:13:42.572895 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:42.572854    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-95c506-55b8fcccb6-c4krk" event={"ID":"a13e7776-0ae8-47fb-b3e6-597c29fea161","Type":"ContainerStarted","Data":"f2cc031a9fc66ff9755bcfc97edd7768eb56b444a65a2631f3388a8d4355bcf4"}
Jul 28 18:13:42.573088 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:42.572978    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-95c506-55b8fcccb6-c4krk"
Jul 28 18:13:42.596939 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:42.596888    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-95c506-55b8fcccb6-c4krk" podStartSLOduration=2.496926174 podStartE2EDuration="3.596872762s" totalImagesPullingTime="1.099946588s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:13:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:13:42.593361039 +0000 UTC m=+2361.254517395" watchObservedRunningTime="2026-07-28 18:13:42.596872762 +0000 UTC m=+2361.258029116"
Jul 28 18:13:42.604140 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:42.604109    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:13:43.579316 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:43.579279    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-95c506-f469f5844-scv75" event={"ID":"1933138a-91ef-4698-a911-eefc04421307","Type":"ContainerStarted","Data":"032ee396c45cdc715c08d091a594adcf64039b38ef56bee8ee51b663762bf041"}
Jul 28 18:13:43.579764 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:43.579350    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-95c506-f469f5844-scv75"
Jul 28 18:13:43.580980 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:43.580957    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-95c506-5b8ccbd94d-xmmh9" event={"ID":"c8d97283-963e-478a-b641-459c17ab71ca","Type":"ContainerStarted","Data":"918bd837a6aabbf5c75d079cb6ee1ab31f1425fca3f8b253c1b5f8a475724965"}
Jul 28 18:13:43.601092 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:43.601050    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-95c506-f469f5844-scv75" podStartSLOduration=1.714240295 podStartE2EDuration="3.60103673s" totalImagesPullingTime="1.886796435s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:13:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:13:43.600846707 +0000 UTC m=+2362.262003061" watchObservedRunningTime="2026-07-28 18:13:43.60103673 +0000 UTC m=+2362.262193085"
Jul 28 18:13:44.586403 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:44.586372    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-95c506-5b8ccbd94d-xmmh9"
Jul 28 18:13:48.586825 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:48.586788    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-95c506-55b8fcccb6-c4krk"
Jul 28 18:13:48.616726 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:48.616674    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-95c506-5b8ccbd94d-xmmh9" podStartSLOduration=6.479642016 podStartE2EDuration="8.616656211s" totalImagesPullingTime="2.137014195s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:13:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:13:43.622207214 +0000 UTC m=+2362.283363571" watchObservedRunningTime="2026-07-28 18:13:48.616656211 +0000 UTC m=+2367.277812569"
Jul 28 18:13:54.589138 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:54.589101    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-95c506-f469f5844-scv75"
Jul 28 18:13:55.593625 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:13:55.593595    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-95c506-5b8ccbd94d-xmmh9"
Jul 28 18:14:04.725695 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:14:04.725626    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:14:06.430152 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:14:06.430119    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:14:16.687953 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:14:16.687917    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:14:17.316149 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:14:17.316118    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:14:56.539947 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:14:56.539870    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:15:00.138962 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:00.138921    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-1cce3e-29754375-f2ccd"]
Jul 28 18:15:00.142674 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:00.142650    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-95c506-29754375-q8wck"]
Jul 28 18:15:00.142801 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:00.142776    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-1cce3e-29754375-f2ccd"
Jul 28 18:15:00.145415 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:00.145397    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9xcv2\""
Jul 28 18:15:00.146001 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:00.145978    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754375-275ww"]
Jul 28 18:15:00.146088 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:00.145985    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-95c506-29754375-q8wck"
Jul 28 18:15:00.149704 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:00.149684    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754375-275ww"
Jul 28 18:15:00.150934 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:00.150766    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-1cce3e-29754375-f2ccd"]
Jul 28 18:15:00.153565 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:00.153532    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-95c506-29754375-q8wck"]
Jul 28 18:15:00.156633 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:00.156609    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754375-275ww"]
Jul 28 18:15:00.156739 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:00.156726    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-1cce3e-29754375-f2ccd"
Jul 28 18:15:00.166560 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:00.166539    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-95c506-29754375-q8wck"
Jul 28 18:15:00.201268 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:00.201095    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754375-275ww"
Jul 28 18:15:00.328030 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:00.327998    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-1cce3e-29754375-f2ccd"]
Jul 28 18:15:00.334107 ip-10-0-136-126 kubenswrapper[2563]: W0728 18:15:00.332948    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ca885cc_fc6a_4cda_bedc_673b2051be6b.slice/crio-c886a463d767126983eeee029c8726c0b764fd7a78db05a4ee24b56b88ab8b4a WatchSource:0}: Error finding container c886a463d767126983eeee029c8726c0b764fd7a78db05a4ee24b56b88ab8b4a: Status 404 returned error can't find the container with id c886a463d767126983eeee029c8726c0b764fd7a78db05a4ee24b56b88ab8b4a
Jul 28 18:15:00.386056 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:00.386027    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754375-275ww"]
Jul 28 18:15:00.387569 ip-10-0-136-126 kubenswrapper[2563]: W0728 18:15:00.387541    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0d32692_ba92_4679_bb2f_ddee85668e55.slice/crio-6371ac1f787a3f3607283d727337547a5139d9d916265f2709152af67946d16e WatchSource:0}: Error finding container 6371ac1f787a3f3607283d727337547a5139d9d916265f2709152af67946d16e: Status 404 returned error can't find the container with id 6371ac1f787a3f3607283d727337547a5139d9d916265f2709152af67946d16e
Jul 28 18:15:00.565951 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:00.565923    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-95c506-29754375-q8wck"]
Jul 28 18:15:00.566889 ip-10-0-136-126 kubenswrapper[2563]: W0728 18:15:00.566862    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca1c8216_ebe5_4c94_9cc7_a9dbfb068cbd.slice/crio-03e13ae527af42a54d122d7bb99965fc52b78e1fb89a1fc6c08fe2596a17fb4f WatchSource:0}: Error finding container 03e13ae527af42a54d122d7bb99965fc52b78e1fb89a1fc6c08fe2596a17fb4f: Status 404 returned error can't find the container with id 03e13ae527af42a54d122d7bb99965fc52b78e1fb89a1fc6c08fe2596a17fb4f
Jul 28 18:15:01.010053 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:01.009949    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754375-275ww" event={"ID":"b0d32692-ba92-4679-bb2f-ddee85668e55","Type":"ContainerStarted","Data":"f53317c45299c3dc0392cacbce646499d4fd44d9aeb523ea52cb5cfba4e288da"}
Jul 28 18:15:01.010053 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:01.010008    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754375-275ww" event={"ID":"b0d32692-ba92-4679-bb2f-ddee85668e55","Type":"ContainerStarted","Data":"6371ac1f787a3f3607283d727337547a5139d9d916265f2709152af67946d16e"}
Jul 28 18:15:01.011461 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:01.011424    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-1cce3e-29754375-f2ccd" event={"ID":"3ca885cc-fc6a-4cda-bedc-673b2051be6b","Type":"ContainerStarted","Data":"18fb68b71d037fbe6bb853ccc3783bb3d5bb5b91d0bec95699df246fc8a95b80"}
Jul 28 18:15:01.011578 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:01.011481    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-1cce3e-29754375-f2ccd" event={"ID":"3ca885cc-fc6a-4cda-bedc-673b2051be6b","Type":"ContainerStarted","Data":"c886a463d767126983eeee029c8726c0b764fd7a78db05a4ee24b56b88ab8b4a"}
Jul 28 18:15:01.012835 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:01.012814    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-95c506-29754375-q8wck" event={"ID":"ca1c8216-ebe5-4c94-9cc7-a9dbfb068cbd","Type":"ContainerStarted","Data":"1aa742324e05e69035a3c5cb5343b4b161b6ce9addf432e226a985743e5040ba"}
Jul 28 18:15:01.012934 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:01.012865    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-95c506-29754375-q8wck" event={"ID":"ca1c8216-ebe5-4c94-9cc7-a9dbfb068cbd","Type":"ContainerStarted","Data":"03e13ae527af42a54d122d7bb99965fc52b78e1fb89a1fc6c08fe2596a17fb4f"}
Jul 28 18:15:01.029827 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:01.029778    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754375-275ww" podStartSLOduration=1.029763728 podStartE2EDuration="1.029763728s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:15:01.027404901 +0000 UTC m=+2439.688561257" watchObservedRunningTime="2026-07-28 18:15:01.029763728 +0000 UTC m=+2439.690920074"
Jul 28 18:15:01.046655 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:01.046614    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-1cce3e-29754375-f2ccd" podStartSLOduration=1.046599993 podStartE2EDuration="1.046599993s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:15:01.043598774 +0000 UTC m=+2439.704755121" watchObservedRunningTime="2026-07-28 18:15:01.046599993 +0000 UTC m=+2439.707756344"
Jul 28 18:15:01.062520 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:01.062481    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-95c506-29754375-q8wck" podStartSLOduration=1.062468251 podStartE2EDuration="1.062468251s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:15:01.060744208 +0000 UTC m=+2439.721900561" watchObservedRunningTime="2026-07-28 18:15:01.062468251 +0000 UTC m=+2439.723624602"
Jul 28 18:15:02.019793 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:02.019759    2563 generic.go:356] "Generic (PLEG): container finished" podID="ca1c8216-ebe5-4c94-9cc7-a9dbfb068cbd" containerID="1aa742324e05e69035a3c5cb5343b4b161b6ce9addf432e226a985743e5040ba" exitCode=0
Jul 28 18:15:02.020195 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:02.019846    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-95c506-29754375-q8wck" event={"ID":"ca1c8216-ebe5-4c94-9cc7-a9dbfb068cbd","Type":"ContainerDied","Data":"1aa742324e05e69035a3c5cb5343b4b161b6ce9addf432e226a985743e5040ba"}
Jul 28 18:15:02.021256 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:02.021238    2563 generic.go:356] "Generic (PLEG): container finished" podID="b0d32692-ba92-4679-bb2f-ddee85668e55" containerID="f53317c45299c3dc0392cacbce646499d4fd44d9aeb523ea52cb5cfba4e288da" exitCode=0
Jul 28 18:15:02.021319 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:02.021264    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754375-275ww" event={"ID":"b0d32692-ba92-4679-bb2f-ddee85668e55","Type":"ContainerDied","Data":"f53317c45299c3dc0392cacbce646499d4fd44d9aeb523ea52cb5cfba4e288da"}
Jul 28 18:15:02.022782 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:02.022766    2563 generic.go:356] "Generic (PLEG): container finished" podID="3ca885cc-fc6a-4cda-bedc-673b2051be6b" containerID="18fb68b71d037fbe6bb853ccc3783bb3d5bb5b91d0bec95699df246fc8a95b80" exitCode=0
Jul 28 18:15:02.022856 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:02.022790    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-1cce3e-29754375-f2ccd" event={"ID":"3ca885cc-fc6a-4cda-bedc-673b2051be6b","Type":"ContainerDied","Data":"18fb68b71d037fbe6bb853ccc3783bb3d5bb5b91d0bec95699df246fc8a95b80"}
Jul 28 18:15:03.176611 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:03.176586    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-1cce3e-29754375-f2ccd"
Jul 28 18:15:03.210253 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:03.210232    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-95c506-29754375-q8wck"
Jul 28 18:15:03.244018 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:03.243994    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754375-275ww"
Jul 28 18:15:04.035260 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:04.035227    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-1cce3e-29754375-f2ccd"
Jul 28 18:15:04.035260 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:04.035250    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-1cce3e-29754375-f2ccd" event={"ID":"3ca885cc-fc6a-4cda-bedc-673b2051be6b","Type":"ContainerDied","Data":"c886a463d767126983eeee029c8726c0b764fd7a78db05a4ee24b56b88ab8b4a"}
Jul 28 18:15:04.035529 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:04.035306    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c886a463d767126983eeee029c8726c0b764fd7a78db05a4ee24b56b88ab8b4a"
Jul 28 18:15:04.036848 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:04.036832    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-95c506-29754375-q8wck"
Jul 28 18:15:04.036961 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:04.036887    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-95c506-29754375-q8wck" event={"ID":"ca1c8216-ebe5-4c94-9cc7-a9dbfb068cbd","Type":"ContainerDied","Data":"03e13ae527af42a54d122d7bb99965fc52b78e1fb89a1fc6c08fe2596a17fb4f"}
Jul 28 18:15:04.036961 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:04.036936    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="03e13ae527af42a54d122d7bb99965fc52b78e1fb89a1fc6c08fe2596a17fb4f"
Jul 28 18:15:04.038495 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:04.038473    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754375-275ww" event={"ID":"b0d32692-ba92-4679-bb2f-ddee85668e55","Type":"ContainerDied","Data":"6371ac1f787a3f3607283d727337547a5139d9d916265f2709152af67946d16e"}
Jul 28 18:15:04.038600 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:04.038499    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754375-275ww"
Jul 28 18:15:04.038600 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:04.038514    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6371ac1f787a3f3607283d727337547a5139d9d916265f2709152af67946d16e"
Jul 28 18:15:07.215557 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:07.215520    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:15:07.279863 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:07.279828    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:15:22.117772 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:22.117737    2563 scope.go:117] "RemoveContainer" containerID="bd5ebd12d9155f255d8001fbc1e18119180fb54da4e336a71618c1d0e4b133ca"
Jul 28 18:15:55.836151 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:55.836114    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:15:56.919000 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:56.918960    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:15:57.050647 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:57.050612    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1cce3e-xprpk"]
Jul 28 18:15:57.051024 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:57.051011    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0d32692-ba92-4679-bb2f-ddee85668e55" containerName="cleanup"
Jul 28 18:15:57.051077 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:57.051025    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0d32692-ba92-4679-bb2f-ddee85668e55" containerName="cleanup"
Jul 28 18:15:57.051077 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:57.051035    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca1c8216-ebe5-4c94-9cc7-a9dbfb068cbd" containerName="cleanup"
Jul 28 18:15:57.051077 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:57.051041    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca1c8216-ebe5-4c94-9cc7-a9dbfb068cbd" containerName="cleanup"
Jul 28 18:15:57.051077 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:57.051050    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ca885cc-fc6a-4cda-bedc-673b2051be6b" containerName="cleanup"
Jul 28 18:15:57.051077 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:57.051056    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ca885cc-fc6a-4cda-bedc-673b2051be6b" containerName="cleanup"
Jul 28 18:15:57.051229 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:57.051114    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca1c8216-ebe5-4c94-9cc7-a9dbfb068cbd" containerName="cleanup"
Jul 28 18:15:57.051229 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:57.051124    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ca885cc-fc6a-4cda-bedc-673b2051be6b" containerName="cleanup"
Jul 28 18:15:57.051229 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:57.051131    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0d32692-ba92-4679-bb2f-ddee85668e55" containerName="cleanup"
Jul 28 18:15:57.053822 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:57.053806    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1cce3e-xprpk"
Jul 28 18:15:57.056679 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:57.056657    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9xcv2\""
Jul 28 18:15:57.064397 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:57.064367    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1cce3e-xprpk"]
Jul 28 18:15:57.166064 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:57.166028    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd96b09e-9ea4-4e28-bfc1-5a1c9a9828e7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-1cce3e-xprpk\" (UID: \"cd96b09e-9ea4-4e28-bfc1-5a1c9a9828e7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1cce3e-xprpk"
Jul 28 18:15:57.266931 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:57.266841    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd96b09e-9ea4-4e28-bfc1-5a1c9a9828e7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-1cce3e-xprpk\" (UID: \"cd96b09e-9ea4-4e28-bfc1-5a1c9a9828e7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1cce3e-xprpk"
Jul 28 18:15:57.267517 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:57.267500    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd96b09e-9ea4-4e28-bfc1-5a1c9a9828e7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-1cce3e-xprpk\" (UID: \"cd96b09e-9ea4-4e28-bfc1-5a1c9a9828e7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1cce3e-xprpk"
Jul 28 18:15:57.365049 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:57.365013    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1cce3e-xprpk"
Jul 28 18:15:57.500374 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:57.500338    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1cce3e-xprpk"]
Jul 28 18:15:57.501788 ip-10-0-136-126 kubenswrapper[2563]: W0728 18:15:57.501761    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd96b09e_9ea4_4e28_bfc1_5a1c9a9828e7.slice/crio-f062985279af22d7af308df48edfc33be879dffe40b1fcba640dbb0b0a4b4d1d WatchSource:0}: Error finding container f062985279af22d7af308df48edfc33be879dffe40b1fcba640dbb0b0a4b4d1d: Status 404 returned error can't find the container with id f062985279af22d7af308df48edfc33be879dffe40b1fcba640dbb0b0a4b4d1d
Jul 28 18:15:58.338604 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:58.338568    2563 generic.go:356] "Generic (PLEG): container finished" podID="cd96b09e-9ea4-4e28-bfc1-5a1c9a9828e7" containerID="c1c511d36298c4cf3a35fbab9aeb60f502ae77afdb2256b35fbc8dfc417f86fe" exitCode=0
Jul 28 18:15:58.339064 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:58.338627    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1cce3e-xprpk" event={"ID":"cd96b09e-9ea4-4e28-bfc1-5a1c9a9828e7","Type":"ContainerDied","Data":"c1c511d36298c4cf3a35fbab9aeb60f502ae77afdb2256b35fbc8dfc417f86fe"}
Jul 28 18:15:58.339064 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:58.338696    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1cce3e-xprpk" event={"ID":"cd96b09e-9ea4-4e28-bfc1-5a1c9a9828e7","Type":"ContainerStarted","Data":"f062985279af22d7af308df48edfc33be879dffe40b1fcba640dbb0b0a4b4d1d"}
Jul 28 18:15:59.480763 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:59.480742    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1cce3e-xprpk"
Jul 28 18:15:59.585414 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:59.585383    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd96b09e-9ea4-4e28-bfc1-5a1c9a9828e7-maas-api-service-ca\") pod \"cd96b09e-9ea4-4e28-bfc1-5a1c9a9828e7\" (UID: \"cd96b09e-9ea4-4e28-bfc1-5a1c9a9828e7\") "
Jul 28 18:15:59.585746 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:59.585725    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd96b09e-9ea4-4e28-bfc1-5a1c9a9828e7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "cd96b09e-9ea4-4e28-bfc1-5a1c9a9828e7" (UID: "cd96b09e-9ea4-4e28-bfc1-5a1c9a9828e7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:15:59.686407 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:15:59.686322    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd96b09e-9ea4-4e28-bfc1-5a1c9a9828e7-maas-api-service-ca\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:16:00.351593 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:00.351565    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1cce3e-xprpk"
Jul 28 18:16:00.351593 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:00.351573    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1cce3e-xprpk" event={"ID":"cd96b09e-9ea4-4e28-bfc1-5a1c9a9828e7","Type":"ContainerDied","Data":"f062985279af22d7af308df48edfc33be879dffe40b1fcba640dbb0b0a4b4d1d"}
Jul 28 18:16:00.351796 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:00.351620    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f062985279af22d7af308df48edfc33be879dffe40b1fcba640dbb0b0a4b4d1d"
Jul 28 18:16:07.803789 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:07.803755    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-1cce3e-5759d75769-645qh"]
Jul 28 18:16:07.804177 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:07.803975    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1cce3e-5759d75769-645qh" podUID="df9e55db-0fc7-47cd-8e31-d2e0492c2fae" containerName="maas-api" containerID="cri-o://60a1d3993ecdecf260fd8d8933508084502d28f95ef4130a51083ec6f3d64c8f" gracePeriod=30
Jul 28 18:16:07.830368 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:07.830321    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:16:07.840973 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:07.840939    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-1cce3e-29754375-f2ccd"]
Jul 28 18:16:07.843684 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:07.843661    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-1cce3e-29754375-f2ccd"]
Jul 28 18:16:07.856121 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:07.856087    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ca885cc-fc6a-4cda-bedc-673b2051be6b" path="/var/lib/kubelet/pods/3ca885cc-fc6a-4cda-bedc-673b2051be6b/volumes"
Jul 28 18:16:07.913490 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:07.913459    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:16:07.980272 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:07.980240    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:16:08.051730 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:08.051703    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1cce3e-xprpk"]
Jul 28 18:16:08.055801 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:08.055733    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:16:08.056916 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:08.056895    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1cce3e-xprpk"]
Jul 28 18:16:08.059686 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:08.059668    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1cce3e-5759d75769-645qh"
Jul 28 18:16:08.163053 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:08.163022    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df9e55db-0fc7-47cd-8e31-d2e0492c2fae-maas-api-tls\") pod \"df9e55db-0fc7-47cd-8e31-d2e0492c2fae\" (UID: \"df9e55db-0fc7-47cd-8e31-d2e0492c2fae\") "
Jul 28 18:16:08.163222 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:08.163091    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mphhr\" (UniqueName: \"kubernetes.io/projected/df9e55db-0fc7-47cd-8e31-d2e0492c2fae-kube-api-access-mphhr\") pod \"df9e55db-0fc7-47cd-8e31-d2e0492c2fae\" (UID: \"df9e55db-0fc7-47cd-8e31-d2e0492c2fae\") "
Jul 28 18:16:08.163222 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:08.163171    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df9e55db-0fc7-47cd-8e31-d2e0492c2fae-openshift-service-ca\") pod \"df9e55db-0fc7-47cd-8e31-d2e0492c2fae\" (UID: \"df9e55db-0fc7-47cd-8e31-d2e0492c2fae\") "
Jul 28 18:16:08.163576 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:08.163546    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df9e55db-0fc7-47cd-8e31-d2e0492c2fae-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "df9e55db-0fc7-47cd-8e31-d2e0492c2fae" (UID: "df9e55db-0fc7-47cd-8e31-d2e0492c2fae"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:16:08.165243 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:08.165213    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df9e55db-0fc7-47cd-8e31-d2e0492c2fae-kube-api-access-mphhr" (OuterVolumeSpecName: "kube-api-access-mphhr") pod "df9e55db-0fc7-47cd-8e31-d2e0492c2fae" (UID: "df9e55db-0fc7-47cd-8e31-d2e0492c2fae"). InnerVolumeSpecName "kube-api-access-mphhr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:16:08.165354 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:08.165296    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df9e55db-0fc7-47cd-8e31-d2e0492c2fae-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "df9e55db-0fc7-47cd-8e31-d2e0492c2fae" (UID: "df9e55db-0fc7-47cd-8e31-d2e0492c2fae"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:16:08.264202 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:08.264162    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df9e55db-0fc7-47cd-8e31-d2e0492c2fae-openshift-service-ca\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:16:08.264202 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:08.264195    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df9e55db-0fc7-47cd-8e31-d2e0492c2fae-maas-api-tls\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:16:08.264202 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:08.264208    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mphhr\" (UniqueName: \"kubernetes.io/projected/df9e55db-0fc7-47cd-8e31-d2e0492c2fae-kube-api-access-mphhr\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:16:08.396020 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:08.395983    2563 generic.go:356] "Generic (PLEG): container finished" podID="df9e55db-0fc7-47cd-8e31-d2e0492c2fae" containerID="60a1d3993ecdecf260fd8d8933508084502d28f95ef4130a51083ec6f3d64c8f" exitCode=0
Jul 28 18:16:08.396224 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:08.396031    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1cce3e-5759d75769-645qh"
Jul 28 18:16:08.396224 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:08.396057    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1cce3e-5759d75769-645qh" event={"ID":"df9e55db-0fc7-47cd-8e31-d2e0492c2fae","Type":"ContainerDied","Data":"60a1d3993ecdecf260fd8d8933508084502d28f95ef4130a51083ec6f3d64c8f"}
Jul 28 18:16:08.396224 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:08.396101    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1cce3e-5759d75769-645qh" event={"ID":"df9e55db-0fc7-47cd-8e31-d2e0492c2fae","Type":"ContainerDied","Data":"26dc50066f05470dc2a7a833eb6ab1f207d2f167a087ca4e8a14bc77dcfbb121"}
Jul 28 18:16:08.396224 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:08.396122    2563 scope.go:117] "RemoveContainer" containerID="60a1d3993ecdecf260fd8d8933508084502d28f95ef4130a51083ec6f3d64c8f"
Jul 28 18:16:08.405509 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:08.405484    2563 scope.go:117] "RemoveContainer" containerID="60a1d3993ecdecf260fd8d8933508084502d28f95ef4130a51083ec6f3d64c8f"
Jul 28 18:16:08.405761 ip-10-0-136-126 kubenswrapper[2563]: E0728 18:16:08.405743    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60a1d3993ecdecf260fd8d8933508084502d28f95ef4130a51083ec6f3d64c8f\": container with ID starting with 60a1d3993ecdecf260fd8d8933508084502d28f95ef4130a51083ec6f3d64c8f not found: ID does not exist" containerID="60a1d3993ecdecf260fd8d8933508084502d28f95ef4130a51083ec6f3d64c8f"
Jul 28 18:16:08.405815 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:08.405769    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60a1d3993ecdecf260fd8d8933508084502d28f95ef4130a51083ec6f3d64c8f"} err="failed to get container status \"60a1d3993ecdecf260fd8d8933508084502d28f95ef4130a51083ec6f3d64c8f\": rpc error: code = NotFound desc = could not find container \"60a1d3993ecdecf260fd8d8933508084502d28f95ef4130a51083ec6f3d64c8f\": container with ID starting with 60a1d3993ecdecf260fd8d8933508084502d28f95ef4130a51083ec6f3d64c8f not found: ID does not exist"
Jul 28 18:16:08.419742 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:08.419718    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-1cce3e-5759d75769-645qh"]
Jul 28 18:16:08.424682 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:08.424661    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-1cce3e-5759d75769-645qh"]
Jul 28 18:16:08.492578 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:08.492543    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:16:09.852907 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:09.852876    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd96b09e-9ea4-4e28-bfc1-5a1c9a9828e7" path="/var/lib/kubelet/pods/cd96b09e-9ea4-4e28-bfc1-5a1c9a9828e7/volumes"
Jul 28 18:16:09.853504 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:09.853232    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df9e55db-0fc7-47cd-8e31-d2e0492c2fae" path="/var/lib/kubelet/pods/df9e55db-0fc7-47cd-8e31-d2e0492c2fae/volumes"
Jul 28 18:16:15.924951 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:15.924871    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:16:16.025530 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:16.025500    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:16:16.207946 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:16.207862    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:16:16.477066 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:16.476957    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:16:16.597449 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:16.597405    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-95c506-vrhph"]
Jul 28 18:16:16.597805 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:16.597792    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd96b09e-9ea4-4e28-bfc1-5a1c9a9828e7" containerName="revoke-keys"
Jul 28 18:16:16.597849 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:16.597807    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd96b09e-9ea4-4e28-bfc1-5a1c9a9828e7" containerName="revoke-keys"
Jul 28 18:16:16.597849 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:16.597825    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df9e55db-0fc7-47cd-8e31-d2e0492c2fae" containerName="maas-api"
Jul 28 18:16:16.597849 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:16.597830    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="df9e55db-0fc7-47cd-8e31-d2e0492c2fae" containerName="maas-api"
Jul 28 18:16:16.597947 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:16.597885    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="df9e55db-0fc7-47cd-8e31-d2e0492c2fae" containerName="maas-api"
Jul 28 18:16:16.597947 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:16.597895    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd96b09e-9ea4-4e28-bfc1-5a1c9a9828e7" containerName="revoke-keys"
Jul 28 18:16:16.602400 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:16.602383    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-95c506-vrhph"
Jul 28 18:16:16.605061 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:16.605041    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9xcv2\""
Jul 28 18:16:16.610788 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:16.610766    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-95c506-vrhph"]
Jul 28 18:16:16.738258 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:16.738171    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/852c1c18-b7ee-49b2-9b7e-e620b999b796-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-95c506-vrhph\" (UID: \"852c1c18-b7ee-49b2-9b7e-e620b999b796\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-95c506-vrhph"
Jul 28 18:16:16.839074 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:16.839033    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/852c1c18-b7ee-49b2-9b7e-e620b999b796-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-95c506-vrhph\" (UID: \"852c1c18-b7ee-49b2-9b7e-e620b999b796\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-95c506-vrhph"
Jul 28 18:16:16.839710 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:16.839692    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/852c1c18-b7ee-49b2-9b7e-e620b999b796-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-95c506-vrhph\" (UID: \"852c1c18-b7ee-49b2-9b7e-e620b999b796\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-95c506-vrhph"
Jul 28 18:16:16.914730 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:16.914693    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-95c506-vrhph"
Jul 28 18:16:17.055521 ip-10-0-136-126 kubenswrapper[2563]: W0728 18:16:17.055494    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod852c1c18_b7ee_49b2_9b7e_e620b999b796.slice/crio-1a83c619b4a22862089756439b152e703a6d2c64caa45789a6f49e5764aee1ca WatchSource:0}: Error finding container 1a83c619b4a22862089756439b152e703a6d2c64caa45789a6f49e5764aee1ca: Status 404 returned error can't find the container with id 1a83c619b4a22862089756439b152e703a6d2c64caa45789a6f49e5764aee1ca
Jul 28 18:16:17.055837 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:17.055727    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-95c506-vrhph"]
Jul 28 18:16:17.449647 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:17.449615    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-95c506-vrhph" event={"ID":"852c1c18-b7ee-49b2-9b7e-e620b999b796","Type":"ContainerStarted","Data":"4bb207244627a090f8db625ab1fec7c71530cd6c819571abcaeef3c04b448e33"}
Jul 28 18:16:17.449812 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:17.449667    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-95c506-vrhph" event={"ID":"852c1c18-b7ee-49b2-9b7e-e620b999b796","Type":"ContainerStarted","Data":"1a83c619b4a22862089756439b152e703a6d2c64caa45789a6f49e5764aee1ca"}
Jul 28 18:16:17.466668 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:17.466590    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-95c506-vrhph" podStartSLOduration=1.466573741 podStartE2EDuration="1.466573741s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:16:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:16:17.466425737 +0000 UTC m=+2516.127582092" watchObservedRunningTime="2026-07-28 18:16:17.466573741 +0000 UTC m=+2516.127730097"
Jul 28 18:16:18.456714 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:18.456674    2563 generic.go:356] "Generic (PLEG): container finished" podID="852c1c18-b7ee-49b2-9b7e-e620b999b796" containerID="4bb207244627a090f8db625ab1fec7c71530cd6c819571abcaeef3c04b448e33" exitCode=0
Jul 28 18:16:18.457079 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:18.456731    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-95c506-vrhph" event={"ID":"852c1c18-b7ee-49b2-9b7e-e620b999b796","Type":"ContainerDied","Data":"4bb207244627a090f8db625ab1fec7c71530cd6c819571abcaeef3c04b448e33"}
Jul 28 18:16:19.594847 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:19.594824    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-95c506-vrhph"
Jul 28 18:16:19.766783 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:19.766707    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/852c1c18-b7ee-49b2-9b7e-e620b999b796-maas-api-service-ca\") pod \"852c1c18-b7ee-49b2-9b7e-e620b999b796\" (UID: \"852c1c18-b7ee-49b2-9b7e-e620b999b796\") "
Jul 28 18:16:19.767095 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:19.767071    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/852c1c18-b7ee-49b2-9b7e-e620b999b796-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "852c1c18-b7ee-49b2-9b7e-e620b999b796" (UID: "852c1c18-b7ee-49b2-9b7e-e620b999b796"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:16:19.867451 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:19.867409    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/852c1c18-b7ee-49b2-9b7e-e620b999b796-maas-api-service-ca\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:16:20.469013 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:20.468974    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-95c506-vrhph" event={"ID":"852c1c18-b7ee-49b2-9b7e-e620b999b796","Type":"ContainerDied","Data":"1a83c619b4a22862089756439b152e703a6d2c64caa45789a6f49e5764aee1ca"}
Jul 28 18:16:20.469013 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:20.469004    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-95c506-vrhph"
Jul 28 18:16:20.469298 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:20.469036    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1a83c619b4a22862089756439b152e703a6d2c64caa45789a6f49e5764aee1ca"
Jul 28 18:16:21.808986 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:21.808947    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-95c506-55b8fcccb6-c4krk"]
Jul 28 18:16:21.809478 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:21.809202    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-95c506-55b8fcccb6-c4krk" podUID="a13e7776-0ae8-47fb-b3e6-597c29fea161" containerName="maas-api" containerID="cri-o://f2cc031a9fc66ff9755bcfc97edd7768eb56b444a65a2631f3388a8d4355bcf4" gracePeriod=30
Jul 28 18:16:21.834729 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:21.834698    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:16:21.843196 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:21.843163    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-95c506-29754375-q8wck"]
Jul 28 18:16:21.845524 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:21.845501    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-95c506-29754375-q8wck"]
Jul 28 18:16:21.855176 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:21.855145    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca1c8216-ebe5-4c94-9cc7-a9dbfb068cbd" path="/var/lib/kubelet/pods/ca1c8216-ebe5-4c94-9cc7-a9dbfb068cbd/volumes"
Jul 28 18:16:21.880455 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:21.880411    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-95c506-5b8ccbd94d-xmmh9"]
Jul 28 18:16:21.880675 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:21.880649    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-95c506-5b8ccbd94d-xmmh9" podUID="c8d97283-963e-478a-b641-459c17ab71ca" containerName="payload-processing" containerID="cri-o://918bd837a6aabbf5c75d079cb6ee1ab31f1425fca3f8b253c1b5f8a475724965" gracePeriod=30
Jul 28 18:16:21.890293 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:21.890264    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-95c506-f469f5844-scv75"]
Jul 28 18:16:21.890524 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:21.890497    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-95c506-f469f5844-scv75" podUID="1933138a-91ef-4698-a911-eefc04421307" containerName="payload-pre-processing" containerID="cri-o://032ee396c45cdc715c08d091a594adcf64039b38ef56bee8ee51b663762bf041" gracePeriod=30
Jul 28 18:16:21.926902 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:21.926871    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:16:21.997739 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:21.997612    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:16:22.073160 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:22.073101    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-95c506-vrhph"]
Jul 28 18:16:22.074279 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:22.074238    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:16:22.085801 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:22.085773    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-95c506-vrhph"]
Jul 28 18:16:22.183549 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:22.183516    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-95c506-5b8ccbd94d-xmmh9"
Jul 28 18:16:22.185606 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:22.185586    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-95c506-55b8fcccb6-c4krk"
Jul 28 18:16:22.200585 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:22.200566    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-95c506-f469f5844-scv75"
Jul 28 18:16:22.289107 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:22.289075    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9znpn\" (UniqueName: \"kubernetes.io/projected/c8d97283-963e-478a-b641-459c17ab71ca-kube-api-access-9znpn\") pod \"c8d97283-963e-478a-b641-459c17ab71ca\" (UID: \"c8d97283-963e-478a-b641-459c17ab71ca\") "
Jul 28 18:16:22.289274 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:22.289118    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1933138a-91ef-4698-a911-eefc04421307-plugins-config-volume\") pod \"1933138a-91ef-4698-a911-eefc04421307\" (UID: \"1933138a-91ef-4698-a911-eefc04421307\") "
Jul 28 18:16:22.289274 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:22.289159    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a13e7776-0ae8-47fb-b3e6-597c29fea161-maas-api-tls\") pod \"a13e7776-0ae8-47fb-b3e6-597c29fea161\" (UID: \"a13e7776-0ae8-47fb-b3e6-597c29fea161\") "
Jul 28 18:16:22.289274 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:22.289191    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hqx64\" (UniqueName: \"kubernetes.io/projected/a13e7776-0ae8-47fb-b3e6-597c29fea161-kube-api-access-hqx64\") pod \"a13e7776-0ae8-47fb-b3e6-597c29fea161\" (UID: \"a13e7776-0ae8-47fb-b3e6-597c29fea161\") "
Jul 28 18:16:22.289274 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:22.289238    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8d97283-963e-478a-b641-459c17ab71ca-plugins-config-volume\") pod \"c8d97283-963e-478a-b641-459c17ab71ca\" (UID: \"c8d97283-963e-478a-b641-459c17ab71ca\") "
Jul 28 18:16:22.289274 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:22.289255    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6txnt\" (UniqueName: \"kubernetes.io/projected/1933138a-91ef-4698-a911-eefc04421307-kube-api-access-6txnt\") pod \"1933138a-91ef-4698-a911-eefc04421307\" (UID: \"1933138a-91ef-4698-a911-eefc04421307\") "
Jul 28 18:16:22.289560 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:22.289282    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a13e7776-0ae8-47fb-b3e6-597c29fea161-openshift-service-ca\") pod \"a13e7776-0ae8-47fb-b3e6-597c29fea161\" (UID: \"a13e7776-0ae8-47fb-b3e6-597c29fea161\") "
Jul 28 18:16:22.289560 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:22.289502    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1933138a-91ef-4698-a911-eefc04421307-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1933138a-91ef-4698-a911-eefc04421307" (UID: "1933138a-91ef-4698-a911-eefc04421307"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:16:22.289815 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:22.289789    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8d97283-963e-478a-b641-459c17ab71ca-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c8d97283-963e-478a-b641-459c17ab71ca" (UID: "c8d97283-963e-478a-b641-459c17ab71ca"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:16:22.289815 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:22.289807    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a13e7776-0ae8-47fb-b3e6-597c29fea161-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a13e7776-0ae8-47fb-b3e6-597c29fea161" (UID: "a13e7776-0ae8-47fb-b3e6-597c29fea161"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:16:22.291861 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:22.291834    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a13e7776-0ae8-47fb-b3e6-597c29fea161-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a13e7776-0ae8-47fb-b3e6-597c29fea161" (UID: "a13e7776-0ae8-47fb-b3e6-597c29fea161"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:16:22.291978 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:22.291889    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a13e7776-0ae8-47fb-b3e6-597c29fea161-kube-api-access-hqx64" (OuterVolumeSpecName: "kube-api-access-hqx64") pod "a13e7776-0ae8-47fb-b3e6-597c29fea161" (UID: "a13e7776-0ae8-47fb-b3e6-597c29fea161"). InnerVolumeSpecName "kube-api-access-hqx64". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:16:22.291978 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:22.291906    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1933138a-91ef-4698-a911-eefc04421307-kube-api-access-6txnt" (OuterVolumeSpecName: "kube-api-access-6txnt") pod "1933138a-91ef-4698-a911-eefc04421307" (UID: "1933138a-91ef-4698-a911-eefc04421307"). InnerVolumeSpecName "kube-api-access-6txnt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:16:22.291978 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:22.291943    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8d97283-963e-478a-b641-459c17ab71ca-kube-api-access-9znpn" (OuterVolumeSpecName: "kube-api-access-9znpn") pod "c8d97283-963e-478a-b641-459c17ab71ca" (UID: "c8d97283-963e-478a-b641-459c17ab71ca"). InnerVolumeSpecName "kube-api-access-9znpn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:16:22.390619 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:22.390576    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hqx64\" (UniqueName: \"kubernetes.io/projected/a13e7776-0ae8-47fb-b3e6-597c29fea161-kube-api-access-hqx64\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:16:22.390619 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:22.390613    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8d97283-963e-478a-b641-459c17ab71ca-plugins-config-volume\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:16:22.390619 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:22.390627    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6txnt\" (UniqueName: \"kubernetes.io/projected/1933138a-91ef-4698-a911-eefc04421307-kube-api-access-6txnt\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:16:22.390885 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:22.390639    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a13e7776-0ae8-47fb-b3e6-597c29fea161-openshift-service-ca\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:16:22.390885 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:22.390651    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9znpn\" (UniqueName: \"kubernetes.io/projected/c8d97283-963e-478a-b641-459c17ab71ca-kube-api-access-9znpn\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:16:22.390885 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:22.390665    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1933138a-91ef-4698-a911-eefc04421307-plugins-config-volume\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:16:22.390885 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:22.390676    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a13e7776-0ae8-47fb-b3e6-597c29fea161-maas-api-tls\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:16:22.475048 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:22.475008    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:16:22.481896 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:22.481863    2563 generic.go:356] "Generic (PLEG): container finished" podID="1933138a-91ef-4698-a911-eefc04421307" containerID="032ee396c45cdc715c08d091a594adcf64039b38ef56bee8ee51b663762bf041" exitCode=0
Jul 28 18:16:22.482049 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:22.481905    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-95c506-f469f5844-scv75"
Jul 28 18:16:22.482049 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:22.481946    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-95c506-f469f5844-scv75" event={"ID":"1933138a-91ef-4698-a911-eefc04421307","Type":"ContainerDied","Data":"032ee396c45cdc715c08d091a594adcf64039b38ef56bee8ee51b663762bf041"}
Jul 28 18:16:22.482049 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:22.482023    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-95c506-f469f5844-scv75" event={"ID":"1933138a-91ef-4698-a911-eefc04421307","Type":"ContainerDied","Data":"e297a482cc145eaeae0482d519ea0890bbf8df84022387031da7c75f5b5b7e5e"}
Jul 28 18:16:22.482164 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:22.482056    2563 scope.go:117] "RemoveContainer" containerID="032ee396c45cdc715c08d091a594adcf64039b38ef56bee8ee51b663762bf041"
Jul 28 18:16:22.483169 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:22.483148    2563 generic.go:356] "Generic (PLEG): container finished" podID="c8d97283-963e-478a-b641-459c17ab71ca" containerID="918bd837a6aabbf5c75d079cb6ee1ab31f1425fca3f8b253c1b5f8a475724965" exitCode=0
Jul 28 18:16:22.483258 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:22.483198    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-95c506-5b8ccbd94d-xmmh9"
Jul 28 18:16:22.483258 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:22.483232    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-95c506-5b8ccbd94d-xmmh9" event={"ID":"c8d97283-963e-478a-b641-459c17ab71ca","Type":"ContainerDied","Data":"918bd837a6aabbf5c75d079cb6ee1ab31f1425fca3f8b253c1b5f8a475724965"}
Jul 28 18:16:22.483354 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:22.483286    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-95c506-5b8ccbd94d-xmmh9" event={"ID":"c8d97283-963e-478a-b641-459c17ab71ca","Type":"ContainerDied","Data":"fd077525d33aee9847dc2bca9a4adcc1ea2ecccf52d421d3203e38805e21fbe6"}
Jul 28 18:16:22.484714 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:22.484690    2563 generic.go:356] "Generic (PLEG): container finished" podID="a13e7776-0ae8-47fb-b3e6-597c29fea161" containerID="f2cc031a9fc66ff9755bcfc97edd7768eb56b444a65a2631f3388a8d4355bcf4" exitCode=0
Jul 28 18:16:22.484829 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:22.484736    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-95c506-55b8fcccb6-c4krk"
Jul 28 18:16:22.484829 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:22.484755    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-95c506-55b8fcccb6-c4krk" event={"ID":"a13e7776-0ae8-47fb-b3e6-597c29fea161","Type":"ContainerDied","Data":"f2cc031a9fc66ff9755bcfc97edd7768eb56b444a65a2631f3388a8d4355bcf4"}
Jul 28 18:16:22.484829 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:22.484792    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-95c506-55b8fcccb6-c4krk" event={"ID":"a13e7776-0ae8-47fb-b3e6-597c29fea161","Type":"ContainerDied","Data":"e21a212d8150befa9b9b8dd33fa0e8a731585db77385bd9a8c1eb8e7da52a6ba"}
Jul 28 18:16:22.493569 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:22.493066    2563 scope.go:117] "RemoveContainer" containerID="032ee396c45cdc715c08d091a594adcf64039b38ef56bee8ee51b663762bf041"
Jul 28 18:16:22.495977 ip-10-0-136-126 kubenswrapper[2563]: E0728 18:16:22.495894    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"032ee396c45cdc715c08d091a594adcf64039b38ef56bee8ee51b663762bf041\": container with ID starting with 032ee396c45cdc715c08d091a594adcf64039b38ef56bee8ee51b663762bf041 not found: ID does not exist" containerID="032ee396c45cdc715c08d091a594adcf64039b38ef56bee8ee51b663762bf041"
Jul 28 18:16:22.495977 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:22.495926    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"032ee396c45cdc715c08d091a594adcf64039b38ef56bee8ee51b663762bf041"} err="failed to get container status \"032ee396c45cdc715c08d091a594adcf64039b38ef56bee8ee51b663762bf041\": rpc error: code = NotFound desc = could not find container \"032ee396c45cdc715c08d091a594adcf64039b38ef56bee8ee51b663762bf041\": container with ID starting with 032ee396c45cdc715c08d091a594adcf64039b38ef56bee8ee51b663762bf041 not found: ID does not exist"
Jul 28 18:16:22.495977 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:22.495949    2563 scope.go:117] "RemoveContainer" containerID="918bd837a6aabbf5c75d079cb6ee1ab31f1425fca3f8b253c1b5f8a475724965"
Jul 28 18:16:22.504174 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:22.504157    2563 scope.go:117] "RemoveContainer" containerID="918bd837a6aabbf5c75d079cb6ee1ab31f1425fca3f8b253c1b5f8a475724965"
Jul 28 18:16:22.504418 ip-10-0-136-126 kubenswrapper[2563]: E0728 18:16:22.504400    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"918bd837a6aabbf5c75d079cb6ee1ab31f1425fca3f8b253c1b5f8a475724965\": container with ID starting with 918bd837a6aabbf5c75d079cb6ee1ab31f1425fca3f8b253c1b5f8a475724965 not found: ID does not exist" containerID="918bd837a6aabbf5c75d079cb6ee1ab31f1425fca3f8b253c1b5f8a475724965"
Jul 28 18:16:22.504539 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:22.504421    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"918bd837a6aabbf5c75d079cb6ee1ab31f1425fca3f8b253c1b5f8a475724965"} err="failed to get container status \"918bd837a6aabbf5c75d079cb6ee1ab31f1425fca3f8b253c1b5f8a475724965\": rpc error: code = NotFound desc = could not find container \"918bd837a6aabbf5c75d079cb6ee1ab31f1425fca3f8b253c1b5f8a475724965\": container with ID starting with 918bd837a6aabbf5c75d079cb6ee1ab31f1425fca3f8b253c1b5f8a475724965 not found: ID does not exist"
Jul 28 18:16:22.504539 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:22.504526    2563 scope.go:117] "RemoveContainer" containerID="f2cc031a9fc66ff9755bcfc97edd7768eb56b444a65a2631f3388a8d4355bcf4"
Jul 28 18:16:22.512174 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:22.512157    2563 scope.go:117] "RemoveContainer" containerID="f2cc031a9fc66ff9755bcfc97edd7768eb56b444a65a2631f3388a8d4355bcf4"
Jul 28 18:16:22.512418 ip-10-0-136-126 kubenswrapper[2563]: E0728 18:16:22.512402    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2cc031a9fc66ff9755bcfc97edd7768eb56b444a65a2631f3388a8d4355bcf4\": container with ID starting with f2cc031a9fc66ff9755bcfc97edd7768eb56b444a65a2631f3388a8d4355bcf4 not found: ID does not exist" containerID="f2cc031a9fc66ff9755bcfc97edd7768eb56b444a65a2631f3388a8d4355bcf4"
Jul 28 18:16:22.512535 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:22.512425    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2cc031a9fc66ff9755bcfc97edd7768eb56b444a65a2631f3388a8d4355bcf4"} err="failed to get container status \"f2cc031a9fc66ff9755bcfc97edd7768eb56b444a65a2631f3388a8d4355bcf4\": rpc error: code = NotFound desc = could not find container \"f2cc031a9fc66ff9755bcfc97edd7768eb56b444a65a2631f3388a8d4355bcf4\": container with ID starting with f2cc031a9fc66ff9755bcfc97edd7768eb56b444a65a2631f3388a8d4355bcf4 not found: ID does not exist"
Jul 28 18:16:22.514009 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:22.513991    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-95c506-f469f5844-scv75"]
Jul 28 18:16:22.522981 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:22.522954    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-95c506-f469f5844-scv75"]
Jul 28 18:16:22.535986 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:22.535952    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-95c506-5b8ccbd94d-xmmh9"]
Jul 28 18:16:22.544216 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:22.544193    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-95c506-5b8ccbd94d-xmmh9"]
Jul 28 18:16:22.556890 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:22.556862    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-95c506-55b8fcccb6-c4krk"]
Jul 28 18:16:22.564122 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:22.564097    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-95c506-55b8fcccb6-c4krk"]
Jul 28 18:16:23.853200 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:23.853168    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1933138a-91ef-4698-a911-eefc04421307" path="/var/lib/kubelet/pods/1933138a-91ef-4698-a911-eefc04421307/volumes"
Jul 28 18:16:23.853598 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:23.853530    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="852c1c18-b7ee-49b2-9b7e-e620b999b796" path="/var/lib/kubelet/pods/852c1c18-b7ee-49b2-9b7e-e620b999b796/volumes"
Jul 28 18:16:23.853866 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:23.853852    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a13e7776-0ae8-47fb-b3e6-597c29fea161" path="/var/lib/kubelet/pods/a13e7776-0ae8-47fb-b3e6-597c29fea161/volumes"
Jul 28 18:16:23.854152 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:23.854141    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8d97283-963e-478a-b641-459c17ab71ca" path="/var/lib/kubelet/pods/c8d97283-963e-478a-b641-459c17ab71ca/volumes"
Jul 28 18:16:29.309916 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:29.309884    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:16:29.414590 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:29.414555    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:16:29.643920 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:29.643887    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:16:29.867603 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:29.867573    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:16:30.220565 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:30.220536    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:16:31.581700 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:31.581671    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:16:31.665269 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:31.665242    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:16:33.405756 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:33.405713    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:16:33.502619 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:33.502587    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:16:33.536482 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:33.536454    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-7a2e6c-5884c54b58-nmk7n"]
Jul 28 18:16:33.536976 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:33.536956    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1933138a-91ef-4698-a911-eefc04421307" containerName="payload-pre-processing"
Jul 28 18:16:33.537037 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:33.536981    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1933138a-91ef-4698-a911-eefc04421307" containerName="payload-pre-processing"
Jul 28 18:16:33.537037 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:33.537007    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="852c1c18-b7ee-49b2-9b7e-e620b999b796" containerName="revoke-keys"
Jul 28 18:16:33.537037 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:33.537016    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="852c1c18-b7ee-49b2-9b7e-e620b999b796" containerName="revoke-keys"
Jul 28 18:16:33.537132 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:33.537041    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a13e7776-0ae8-47fb-b3e6-597c29fea161" containerName="maas-api"
Jul 28 18:16:33.537132 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:33.537050    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a13e7776-0ae8-47fb-b3e6-597c29fea161" containerName="maas-api"
Jul 28 18:16:33.537132 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:33.537060    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8d97283-963e-478a-b641-459c17ab71ca" containerName="payload-processing"
Jul 28 18:16:33.537132 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:33.537068    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8d97283-963e-478a-b641-459c17ab71ca" containerName="payload-processing"
Jul 28 18:16:33.537250 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:33.537164    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a13e7776-0ae8-47fb-b3e6-597c29fea161" containerName="maas-api"
Jul 28 18:16:33.537250 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:33.537174    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="852c1c18-b7ee-49b2-9b7e-e620b999b796" containerName="revoke-keys"
Jul 28 18:16:33.537250 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:33.537180    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8d97283-963e-478a-b641-459c17ab71ca" containerName="payload-processing"
Jul 28 18:16:33.537250 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:33.537193    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1933138a-91ef-4698-a911-eefc04421307" containerName="payload-pre-processing"
Jul 28 18:16:33.540535 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:33.540516    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7a2e6c-5884c54b58-nmk7n"
Jul 28 18:16:33.543002 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:33.542984    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-7a2e6c-serving-cert\""
Jul 28 18:16:33.550217 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:33.550191    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-7a2e6c-5884c54b58-nmk7n"]
Jul 28 18:16:33.574407 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:33.574382    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:16:33.584651 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:33.584629    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac5f3040-efb9-4ff9-ab2a-ae49a34dfb26-maas-api-tls\") pod \"maas-api-e2e-api-a-7a2e6c-5884c54b58-nmk7n\" (UID: \"ac5f3040-efb9-4ff9-ab2a-ae49a34dfb26\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7a2e6c-5884c54b58-nmk7n"
Jul 28 18:16:33.584757 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:33.584657    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8249v\" (UniqueName: \"kubernetes.io/projected/ac5f3040-efb9-4ff9-ab2a-ae49a34dfb26-kube-api-access-8249v\") pod \"maas-api-e2e-api-a-7a2e6c-5884c54b58-nmk7n\" (UID: \"ac5f3040-efb9-4ff9-ab2a-ae49a34dfb26\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7a2e6c-5884c54b58-nmk7n"
Jul 28 18:16:33.584757 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:33.584744    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac5f3040-efb9-4ff9-ab2a-ae49a34dfb26-openshift-service-ca\") pod \"maas-api-e2e-api-a-7a2e6c-5884c54b58-nmk7n\" (UID: \"ac5f3040-efb9-4ff9-ab2a-ae49a34dfb26\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7a2e6c-5884c54b58-nmk7n"
Jul 28 18:16:33.653855 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:33.653809    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:16:33.685175 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:33.685092    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac5f3040-efb9-4ff9-ab2a-ae49a34dfb26-openshift-service-ca\") pod \"maas-api-e2e-api-a-7a2e6c-5884c54b58-nmk7n\" (UID: \"ac5f3040-efb9-4ff9-ab2a-ae49a34dfb26\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7a2e6c-5884c54b58-nmk7n"
Jul 28 18:16:33.685175 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:33.685148    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac5f3040-efb9-4ff9-ab2a-ae49a34dfb26-maas-api-tls\") pod \"maas-api-e2e-api-a-7a2e6c-5884c54b58-nmk7n\" (UID: \"ac5f3040-efb9-4ff9-ab2a-ae49a34dfb26\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7a2e6c-5884c54b58-nmk7n"
Jul 28 18:16:33.685336 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:33.685187    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8249v\" (UniqueName: \"kubernetes.io/projected/ac5f3040-efb9-4ff9-ab2a-ae49a34dfb26-kube-api-access-8249v\") pod \"maas-api-e2e-api-a-7a2e6c-5884c54b58-nmk7n\" (UID: \"ac5f3040-efb9-4ff9-ab2a-ae49a34dfb26\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7a2e6c-5884c54b58-nmk7n"
Jul 28 18:16:33.685769 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:33.685746    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac5f3040-efb9-4ff9-ab2a-ae49a34dfb26-openshift-service-ca\") pod \"maas-api-e2e-api-a-7a2e6c-5884c54b58-nmk7n\" (UID: \"ac5f3040-efb9-4ff9-ab2a-ae49a34dfb26\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7a2e6c-5884c54b58-nmk7n"
Jul 28 18:16:33.687609 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:33.687590    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac5f3040-efb9-4ff9-ab2a-ae49a34dfb26-maas-api-tls\") pod \"maas-api-e2e-api-a-7a2e6c-5884c54b58-nmk7n\" (UID: \"ac5f3040-efb9-4ff9-ab2a-ae49a34dfb26\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7a2e6c-5884c54b58-nmk7n"
Jul 28 18:16:33.694925 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:33.694894    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8249v\" (UniqueName: \"kubernetes.io/projected/ac5f3040-efb9-4ff9-ab2a-ae49a34dfb26-kube-api-access-8249v\") pod \"maas-api-e2e-api-a-7a2e6c-5884c54b58-nmk7n\" (UID: \"ac5f3040-efb9-4ff9-ab2a-ae49a34dfb26\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7a2e6c-5884c54b58-nmk7n"
Jul 28 18:16:33.851357 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:33.851332    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7a2e6c-5884c54b58-nmk7n"
Jul 28 18:16:33.989542 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:33.989509    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-7a2e6c-5884c54b58-nmk7n"]
Jul 28 18:16:33.990943 ip-10-0-136-126 kubenswrapper[2563]: W0728 18:16:33.990916    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac5f3040_efb9_4ff9_ab2a_ae49a34dfb26.slice/crio-a92d91cdbe7ca4300d113fdc963088631e52acaf32b4dd7f3989fa092ea4cb43 WatchSource:0}: Error finding container a92d91cdbe7ca4300d113fdc963088631e52acaf32b4dd7f3989fa092ea4cb43: Status 404 returned error can't find the container with id a92d91cdbe7ca4300d113fdc963088631e52acaf32b4dd7f3989fa092ea4cb43
Jul 28 18:16:34.129740 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:34.129707    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:16:34.233385 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:34.233296    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:16:34.243670 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:34.243634    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-7a2e6c-5dd7c8c868-hwbxv"]
Jul 28 18:16:34.248302 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:34.248277    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7a2e6c-5dd7c8c868-hwbxv"
Jul 28 18:16:34.250822 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:34.250789    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-7a2e6c\""
Jul 28 18:16:34.258947 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:34.258922    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-7a2e6c-5dd7c8c868-hwbxv"]
Jul 28 18:16:34.261672 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:34.261651    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-7a2e6c-54c69f7cb8-7gqrx"]
Jul 28 18:16:34.265181 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:34.265165    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-7a2e6c-54c69f7cb8-7gqrx"
Jul 28 18:16:34.267722 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:34.267701    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-7a2e6c-dockercfg-zffd2\""
Jul 28 18:16:34.276862 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:34.276842    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-7a2e6c-54c69f7cb8-7gqrx"]
Jul 28 18:16:34.290397 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:34.290370    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhq8m\" (UniqueName: \"kubernetes.io/projected/349c455e-fca7-46b0-bc92-336150fc0c9e-kube-api-access-mhq8m\") pod \"payload-pre-processing-e2e-api-a-7a2e6c-5dd7c8c868-hwbxv\" (UID: \"349c455e-fca7-46b0-bc92-336150fc0c9e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-7a2e6c-5dd7c8c868-hwbxv"
Jul 28 18:16:34.290505 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:34.290422    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a70ca06-3827-4004-8e4e-69a292cf98bf-plugins-config-volume\") pod \"payload-processing-e2e-api-a-7a2e6c-54c69f7cb8-7gqrx\" (UID: \"8a70ca06-3827-4004-8e4e-69a292cf98bf\") " pod="openshift-ingress/payload-processing-e2e-api-a-7a2e6c-54c69f7cb8-7gqrx"
Jul 28 18:16:34.290582 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:34.290520    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5nkjj\" (UniqueName: \"kubernetes.io/projected/8a70ca06-3827-4004-8e4e-69a292cf98bf-kube-api-access-5nkjj\") pod \"payload-processing-e2e-api-a-7a2e6c-54c69f7cb8-7gqrx\" (UID: \"8a70ca06-3827-4004-8e4e-69a292cf98bf\") " pod="openshift-ingress/payload-processing-e2e-api-a-7a2e6c-54c69f7cb8-7gqrx"
Jul 28 18:16:34.290630 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:34.290579    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/349c455e-fca7-46b0-bc92-336150fc0c9e-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-7a2e6c-5dd7c8c868-hwbxv\" (UID: \"349c455e-fca7-46b0-bc92-336150fc0c9e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-7a2e6c-5dd7c8c868-hwbxv"
Jul 28 18:16:34.391507 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:34.391476    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhq8m\" (UniqueName: \"kubernetes.io/projected/349c455e-fca7-46b0-bc92-336150fc0c9e-kube-api-access-mhq8m\") pod \"payload-pre-processing-e2e-api-a-7a2e6c-5dd7c8c868-hwbxv\" (UID: \"349c455e-fca7-46b0-bc92-336150fc0c9e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-7a2e6c-5dd7c8c868-hwbxv"
Jul 28 18:16:34.391670 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:34.391522    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a70ca06-3827-4004-8e4e-69a292cf98bf-plugins-config-volume\") pod \"payload-processing-e2e-api-a-7a2e6c-54c69f7cb8-7gqrx\" (UID: \"8a70ca06-3827-4004-8e4e-69a292cf98bf\") " pod="openshift-ingress/payload-processing-e2e-api-a-7a2e6c-54c69f7cb8-7gqrx"
Jul 28 18:16:34.391670 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:34.391559    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5nkjj\" (UniqueName: \"kubernetes.io/projected/8a70ca06-3827-4004-8e4e-69a292cf98bf-kube-api-access-5nkjj\") pod \"payload-processing-e2e-api-a-7a2e6c-54c69f7cb8-7gqrx\" (UID: \"8a70ca06-3827-4004-8e4e-69a292cf98bf\") " pod="openshift-ingress/payload-processing-e2e-api-a-7a2e6c-54c69f7cb8-7gqrx"
Jul 28 18:16:34.391670 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:34.391600    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/349c455e-fca7-46b0-bc92-336150fc0c9e-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-7a2e6c-5dd7c8c868-hwbxv\" (UID: \"349c455e-fca7-46b0-bc92-336150fc0c9e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-7a2e6c-5dd7c8c868-hwbxv"
Jul 28 18:16:34.392343 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:34.392321    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/349c455e-fca7-46b0-bc92-336150fc0c9e-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-7a2e6c-5dd7c8c868-hwbxv\" (UID: \"349c455e-fca7-46b0-bc92-336150fc0c9e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-7a2e6c-5dd7c8c868-hwbxv"
Jul 28 18:16:34.392396 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:34.392323    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a70ca06-3827-4004-8e4e-69a292cf98bf-plugins-config-volume\") pod \"payload-processing-e2e-api-a-7a2e6c-54c69f7cb8-7gqrx\" (UID: \"8a70ca06-3827-4004-8e4e-69a292cf98bf\") " pod="openshift-ingress/payload-processing-e2e-api-a-7a2e6c-54c69f7cb8-7gqrx"
Jul 28 18:16:34.400855 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:34.400830    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhq8m\" (UniqueName: \"kubernetes.io/projected/349c455e-fca7-46b0-bc92-336150fc0c9e-kube-api-access-mhq8m\") pod \"payload-pre-processing-e2e-api-a-7a2e6c-5dd7c8c868-hwbxv\" (UID: \"349c455e-fca7-46b0-bc92-336150fc0c9e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-7a2e6c-5dd7c8c868-hwbxv"
Jul 28 18:16:34.400855 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:34.400854    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nkjj\" (UniqueName: \"kubernetes.io/projected/8a70ca06-3827-4004-8e4e-69a292cf98bf-kube-api-access-5nkjj\") pod \"payload-processing-e2e-api-a-7a2e6c-54c69f7cb8-7gqrx\" (UID: \"8a70ca06-3827-4004-8e4e-69a292cf98bf\") " pod="openshift-ingress/payload-processing-e2e-api-a-7a2e6c-54c69f7cb8-7gqrx"
Jul 28 18:16:34.560723 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:34.560696    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7a2e6c-5dd7c8c868-hwbxv"
Jul 28 18:16:34.562219 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:34.562183    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7a2e6c-5884c54b58-nmk7n" event={"ID":"ac5f3040-efb9-4ff9-ab2a-ae49a34dfb26","Type":"ContainerStarted","Data":"d0f83adf6d666f8f7d4c3d543745a282055e069e6afa3606ba510b24e73321dd"}
Jul 28 18:16:34.562318 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:34.562240    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7a2e6c-5884c54b58-nmk7n" event={"ID":"ac5f3040-efb9-4ff9-ab2a-ae49a34dfb26","Type":"ContainerStarted","Data":"a92d91cdbe7ca4300d113fdc963088631e52acaf32b4dd7f3989fa092ea4cb43"}
Jul 28 18:16:34.562318 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:34.562298    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7a2e6c-5884c54b58-nmk7n"
Jul 28 18:16:34.576047 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:34.576027    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-7a2e6c-54c69f7cb8-7gqrx"
Jul 28 18:16:34.584868 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:34.584820    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7a2e6c-5884c54b58-nmk7n" podStartSLOduration=1.102099739 podStartE2EDuration="1.584802519s" totalImagesPullingTime="482.70278ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:16:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:16:34.581134547 +0000 UTC m=+2533.242290910" watchObservedRunningTime="2026-07-28 18:16:34.584802519 +0000 UTC m=+2533.245958875"
Jul 28 18:16:34.739866 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:34.739834    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-7a2e6c-54c69f7cb8-7gqrx"]
Jul 28 18:16:34.742707 ip-10-0-136-126 kubenswrapper[2563]: W0728 18:16:34.742678    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a70ca06_3827_4004_8e4e_69a292cf98bf.slice/crio-28b91d36655262ec242d04c74136e9f5edb5badb543149d8d2b26320003a3406 WatchSource:0}: Error finding container 28b91d36655262ec242d04c74136e9f5edb5badb543149d8d2b26320003a3406: Status 404 returned error can't find the container with id 28b91d36655262ec242d04c74136e9f5edb5badb543149d8d2b26320003a3406
Jul 28 18:16:34.913924 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:34.913892    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-7a2e6c-5dd7c8c868-hwbxv"]
Jul 28 18:16:34.914782 ip-10-0-136-126 kubenswrapper[2563]: W0728 18:16:34.914740    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod349c455e_fca7_46b0_bc92_336150fc0c9e.slice/crio-c8e8fdf48cebec65761b83f8210449fba431f092695f600bd7fbc363b93e7437 WatchSource:0}: Error finding container c8e8fdf48cebec65761b83f8210449fba431f092695f600bd7fbc363b93e7437: Status 404 returned error can't find the container with id c8e8fdf48cebec65761b83f8210449fba431f092695f600bd7fbc363b93e7437
Jul 28 18:16:35.569040 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:35.569004    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7a2e6c-5dd7c8c868-hwbxv" event={"ID":"349c455e-fca7-46b0-bc92-336150fc0c9e","Type":"ContainerStarted","Data":"4287ba5219c78860b1c418aa3147aedc0a00ab4dd2ce6e7bab80af056e7a957d"}
Jul 28 18:16:35.569487 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:35.569065    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7a2e6c-5dd7c8c868-hwbxv" event={"ID":"349c455e-fca7-46b0-bc92-336150fc0c9e","Type":"ContainerStarted","Data":"c8e8fdf48cebec65761b83f8210449fba431f092695f600bd7fbc363b93e7437"}
Jul 28 18:16:35.569487 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:35.569094    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7a2e6c-5dd7c8c868-hwbxv"
Jul 28 18:16:35.570627 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:35.570604    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-7a2e6c-54c69f7cb8-7gqrx" event={"ID":"8a70ca06-3827-4004-8e4e-69a292cf98bf","Type":"ContainerStarted","Data":"51df95a3fa6184dcff27b2c45f96c1383c0e381527989e5d934eb6165428b8b1"}
Jul 28 18:16:35.570748 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:35.570648    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-7a2e6c-54c69f7cb8-7gqrx" event={"ID":"8a70ca06-3827-4004-8e4e-69a292cf98bf","Type":"ContainerStarted","Data":"28b91d36655262ec242d04c74136e9f5edb5badb543149d8d2b26320003a3406"}
Jul 28 18:16:35.570748 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:35.570694    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-7a2e6c-54c69f7cb8-7gqrx"
Jul 28 18:16:35.593700 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:35.593655    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7a2e6c-5dd7c8c868-hwbxv" podStartSLOduration=1.593642675 podStartE2EDuration="1.593642675s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:16:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:16:35.59313343 +0000 UTC m=+2534.254289784" watchObservedRunningTime="2026-07-28 18:16:35.593642675 +0000 UTC m=+2534.254799029"
Jul 28 18:16:35.611090 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:35.611051    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-7a2e6c-54c69f7cb8-7gqrx" podStartSLOduration=1.6110367490000002 podStartE2EDuration="1.611036749s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:16:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:16:35.610352703 +0000 UTC m=+2534.271509059" watchObservedRunningTime="2026-07-28 18:16:35.611036749 +0000 UTC m=+2534.272193125"
Jul 28 18:16:40.576356 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:40.576321    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7a2e6c-5884c54b58-nmk7n"
Jul 28 18:16:46.578975 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:46.578944    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-7a2e6c-54c69f7cb8-7gqrx"
Jul 28 18:16:46.579359 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:46.579018    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7a2e6c-5dd7c8c868-hwbxv"
Jul 28 18:16:58.479834 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:58.479799    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:16:58.995414 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:58.995377    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:16:59.340025 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:59.339994    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:16:59.796347 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:16:59.796265    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:17:02.487185 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:02.487153    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:17:02.587170 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:02.587133    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:17:02.613208 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:02.613180    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-1ff6f1-5d96c4f9df-fpftf"]
Jul 28 18:17:02.616608 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:02.616591    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1ff6f1-5d96c4f9df-fpftf"
Jul 28 18:17:02.620840 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:02.620823    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-1ff6f1-serving-cert\""
Jul 28 18:17:02.639778 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:02.639745    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-1ff6f1-5d96c4f9df-fpftf"]
Jul 28 18:17:02.692469 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:02.692424    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:17:02.746177 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:02.746082    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/45858501-6443-4cdd-9536-8f6e14f12742-openshift-service-ca\") pod \"maas-api-e2e-api-b-1ff6f1-5d96c4f9df-fpftf\" (UID: \"45858501-6443-4cdd-9536-8f6e14f12742\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1ff6f1-5d96c4f9df-fpftf"
Jul 28 18:17:02.746177 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:02.746151    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45858501-6443-4cdd-9536-8f6e14f12742-maas-api-tls\") pod \"maas-api-e2e-api-b-1ff6f1-5d96c4f9df-fpftf\" (UID: \"45858501-6443-4cdd-9536-8f6e14f12742\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1ff6f1-5d96c4f9df-fpftf"
Jul 28 18:17:02.746383 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:02.746181    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgnp8\" (UniqueName: \"kubernetes.io/projected/45858501-6443-4cdd-9536-8f6e14f12742-kube-api-access-xgnp8\") pod \"maas-api-e2e-api-b-1ff6f1-5d96c4f9df-fpftf\" (UID: \"45858501-6443-4cdd-9536-8f6e14f12742\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1ff6f1-5d96c4f9df-fpftf"
Jul 28 18:17:02.847260 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:02.847222    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45858501-6443-4cdd-9536-8f6e14f12742-maas-api-tls\") pod \"maas-api-e2e-api-b-1ff6f1-5d96c4f9df-fpftf\" (UID: \"45858501-6443-4cdd-9536-8f6e14f12742\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1ff6f1-5d96c4f9df-fpftf"
Jul 28 18:17:02.847260 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:02.847271    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgnp8\" (UniqueName: \"kubernetes.io/projected/45858501-6443-4cdd-9536-8f6e14f12742-kube-api-access-xgnp8\") pod \"maas-api-e2e-api-b-1ff6f1-5d96c4f9df-fpftf\" (UID: \"45858501-6443-4cdd-9536-8f6e14f12742\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1ff6f1-5d96c4f9df-fpftf"
Jul 28 18:17:02.847576 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:02.847376    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/45858501-6443-4cdd-9536-8f6e14f12742-openshift-service-ca\") pod \"maas-api-e2e-api-b-1ff6f1-5d96c4f9df-fpftf\" (UID: \"45858501-6443-4cdd-9536-8f6e14f12742\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1ff6f1-5d96c4f9df-fpftf"
Jul 28 18:17:02.847576 ip-10-0-136-126 kubenswrapper[2563]: E0728 18:17:02.847402    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-1ff6f1-serving-cert: secret "maas-api-e2e-api-b-1ff6f1-serving-cert" not found
Jul 28 18:17:02.847576 ip-10-0-136-126 kubenswrapper[2563]: E0728 18:17:02.847511    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45858501-6443-4cdd-9536-8f6e14f12742-maas-api-tls podName:45858501-6443-4cdd-9536-8f6e14f12742 nodeName:}" failed. No retries permitted until 2026-07-28 18:17:03.347488251 +0000 UTC m=+2562.008644602 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/45858501-6443-4cdd-9536-8f6e14f12742-maas-api-tls") pod "maas-api-e2e-api-b-1ff6f1-5d96c4f9df-fpftf" (UID: "45858501-6443-4cdd-9536-8f6e14f12742") : secret "maas-api-e2e-api-b-1ff6f1-serving-cert" not found
Jul 28 18:17:02.848074 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:02.848053    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/45858501-6443-4cdd-9536-8f6e14f12742-openshift-service-ca\") pod \"maas-api-e2e-api-b-1ff6f1-5d96c4f9df-fpftf\" (UID: \"45858501-6443-4cdd-9536-8f6e14f12742\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1ff6f1-5d96c4f9df-fpftf"
Jul 28 18:17:02.862998 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:02.862965    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgnp8\" (UniqueName: \"kubernetes.io/projected/45858501-6443-4cdd-9536-8f6e14f12742-kube-api-access-xgnp8\") pod \"maas-api-e2e-api-b-1ff6f1-5d96c4f9df-fpftf\" (UID: \"45858501-6443-4cdd-9536-8f6e14f12742\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1ff6f1-5d96c4f9df-fpftf"
Jul 28 18:17:03.085313 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:03.085274    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:17:03.351456 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:03.351370    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45858501-6443-4cdd-9536-8f6e14f12742-maas-api-tls\") pod \"maas-api-e2e-api-b-1ff6f1-5d96c4f9df-fpftf\" (UID: \"45858501-6443-4cdd-9536-8f6e14f12742\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1ff6f1-5d96c4f9df-fpftf"
Jul 28 18:17:03.353664 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:03.353642    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45858501-6443-4cdd-9536-8f6e14f12742-maas-api-tls\") pod \"maas-api-e2e-api-b-1ff6f1-5d96c4f9df-fpftf\" (UID: \"45858501-6443-4cdd-9536-8f6e14f12742\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1ff6f1-5d96c4f9df-fpftf"
Jul 28 18:17:03.365291 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:03.365266    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:17:03.463815 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:03.463780    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:17:03.483055 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:03.483025    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-1ff6f1-9cc645bb4-hq8qr"]
Jul 28 18:17:03.486806 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:03.486780    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1ff6f1-9cc645bb4-hq8qr"
Jul 28 18:17:03.491239 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:03.491217    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-1ff6f1\""
Jul 28 18:17:03.499820 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:03.499799    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-1ff6f1-9cc645bb4-hq8qr"]
Jul 28 18:17:03.539268 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:03.539235    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1ff6f1-5d96c4f9df-fpftf"
Jul 28 18:17:03.553785 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:03.553668    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79667739-961c-4d5f-80a9-a14a5869e7ca-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-1ff6f1-9cc645bb4-hq8qr\" (UID: \"79667739-961c-4d5f-80a9-a14a5869e7ca\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-1ff6f1-9cc645bb4-hq8qr"
Jul 28 18:17:03.553950 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:03.553791    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9scw2\" (UniqueName: \"kubernetes.io/projected/79667739-961c-4d5f-80a9-a14a5869e7ca-kube-api-access-9scw2\") pod \"payload-pre-processing-e2e-api-b-1ff6f1-9cc645bb4-hq8qr\" (UID: \"79667739-961c-4d5f-80a9-a14a5869e7ca\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-1ff6f1-9cc645bb4-hq8qr"
Jul 28 18:17:03.654988 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:03.654956    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79667739-961c-4d5f-80a9-a14a5869e7ca-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-1ff6f1-9cc645bb4-hq8qr\" (UID: \"79667739-961c-4d5f-80a9-a14a5869e7ca\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-1ff6f1-9cc645bb4-hq8qr"
Jul 28 18:17:03.655153 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:03.655017    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9scw2\" (UniqueName: \"kubernetes.io/projected/79667739-961c-4d5f-80a9-a14a5869e7ca-kube-api-access-9scw2\") pod \"payload-pre-processing-e2e-api-b-1ff6f1-9cc645bb4-hq8qr\" (UID: \"79667739-961c-4d5f-80a9-a14a5869e7ca\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-1ff6f1-9cc645bb4-hq8qr"
Jul 28 18:17:03.655934 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:03.655903    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79667739-961c-4d5f-80a9-a14a5869e7ca-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-1ff6f1-9cc645bb4-hq8qr\" (UID: \"79667739-961c-4d5f-80a9-a14a5869e7ca\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-1ff6f1-9cc645bb4-hq8qr"
Jul 28 18:17:03.665109 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:03.665082    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9scw2\" (UniqueName: \"kubernetes.io/projected/79667739-961c-4d5f-80a9-a14a5869e7ca-kube-api-access-9scw2\") pod \"payload-pre-processing-e2e-api-b-1ff6f1-9cc645bb4-hq8qr\" (UID: \"79667739-961c-4d5f-80a9-a14a5869e7ca\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-1ff6f1-9cc645bb4-hq8qr"
Jul 28 18:17:03.690097 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:03.690075    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-1ff6f1-5d96c4f9df-fpftf"]
Jul 28 18:17:03.691573 ip-10-0-136-126 kubenswrapper[2563]: W0728 18:17:03.691548    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45858501_6443_4cdd_9536_8f6e14f12742.slice/crio-e2a8b4f98fe8b25fad135486123f85152f76c8730fd21bf65ed08de831a4ef8d WatchSource:0}: Error finding container e2a8b4f98fe8b25fad135486123f85152f76c8730fd21bf65ed08de831a4ef8d: Status 404 returned error can't find the container with id e2a8b4f98fe8b25fad135486123f85152f76c8730fd21bf65ed08de831a4ef8d
Jul 28 18:17:03.732503 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:03.732471    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1ff6f1-5d96c4f9df-fpftf" event={"ID":"45858501-6443-4cdd-9536-8f6e14f12742","Type":"ContainerStarted","Data":"e2a8b4f98fe8b25fad135486123f85152f76c8730fd21bf65ed08de831a4ef8d"}
Jul 28 18:17:03.797985 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:03.797962    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1ff6f1-9cc645bb4-hq8qr"
Jul 28 18:17:04.141850 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:04.141811    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-1ff6f1-9cc645bb4-hq8qr"]
Jul 28 18:17:04.143540 ip-10-0-136-126 kubenswrapper[2563]: W0728 18:17:04.143509    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79667739_961c_4d5f_80a9_a14a5869e7ca.slice/crio-c84210c6b6905b86e7d5a0ce21b0fab22eccd96038b0650ade67ebddfac00521 WatchSource:0}: Error finding container c84210c6b6905b86e7d5a0ce21b0fab22eccd96038b0650ade67ebddfac00521: Status 404 returned error can't find the container with id c84210c6b6905b86e7d5a0ce21b0fab22eccd96038b0650ade67ebddfac00521
Jul 28 18:17:04.741047 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:04.741011    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1ff6f1-5d96c4f9df-fpftf" event={"ID":"45858501-6443-4cdd-9536-8f6e14f12742","Type":"ContainerStarted","Data":"ad85228f3fc122322fcaa92024c9569a8a25e1a17b43ff93b086beb4c9a8e3e7"}
Jul 28 18:17:04.741530 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:04.741073    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1ff6f1-5d96c4f9df-fpftf"
Jul 28 18:17:04.742679 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:04.742648    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1ff6f1-9cc645bb4-hq8qr" event={"ID":"79667739-961c-4d5f-80a9-a14a5869e7ca","Type":"ContainerStarted","Data":"ec24773bbf34a916c1dd5319b8b2f14326437566319e137deae2fd753bf29e65"}
Jul 28 18:17:04.742802 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:04.742739    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1ff6f1-9cc645bb4-hq8qr"
Jul 28 18:17:04.742802 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:04.742765    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1ff6f1-9cc645bb4-hq8qr" event={"ID":"79667739-961c-4d5f-80a9-a14a5869e7ca","Type":"ContainerStarted","Data":"c84210c6b6905b86e7d5a0ce21b0fab22eccd96038b0650ade67ebddfac00521"}
Jul 28 18:17:04.763048 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:04.762995    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1ff6f1-5d96c4f9df-fpftf" podStartSLOduration=2.453665926 podStartE2EDuration="2.762979746s" totalImagesPullingTime="309.31382ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:17:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:17:04.760572651 +0000 UTC m=+2563.421729007" watchObservedRunningTime="2026-07-28 18:17:04.762979746 +0000 UTC m=+2563.424136100"
Jul 28 18:17:04.782686 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:04.782644    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1ff6f1-9cc645bb4-hq8qr" podStartSLOduration=1.782629893 podStartE2EDuration="1.782629893s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:17:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:17:04.781083883 +0000 UTC m=+2563.442240239" watchObservedRunningTime="2026-07-28 18:17:04.782629893 +0000 UTC m=+2563.443786617"
Jul 28 18:17:10.754695 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:10.754666    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1ff6f1-5d96c4f9df-fpftf"
Jul 28 18:17:15.751219 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:15.751190    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1ff6f1-9cc645bb4-hq8qr"
Jul 28 18:17:22.159972 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:22.159941    2563 scope.go:117] "RemoveContainer" containerID="801e6402c1f8c3e84e5c18aef4ec489f85e0661e65d76ff68cdcfc92e416bbdb"
Jul 28 18:17:32.722923 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:32.722878    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:17:35.854148 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:35.854120    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7a2e6c-cmckm"]
Jul 28 18:17:35.857667 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:35.857650    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7a2e6c-cmckm"
Jul 28 18:17:35.860421 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:35.860402    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9xcv2\""
Jul 28 18:17:35.866297 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:35.866273    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7a2e6c-cmckm"]
Jul 28 18:17:35.937170 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:35.937138    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f84a8b8-46c2-4340-b2f8-bbeb1b67df94-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-7a2e6c-cmckm\" (UID: \"8f84a8b8-46c2-4340-b2f8-bbeb1b67df94\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7a2e6c-cmckm"
Jul 28 18:17:36.038493 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:36.038460    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f84a8b8-46c2-4340-b2f8-bbeb1b67df94-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-7a2e6c-cmckm\" (UID: \"8f84a8b8-46c2-4340-b2f8-bbeb1b67df94\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7a2e6c-cmckm"
Jul 28 18:17:36.039077 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:36.039052    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f84a8b8-46c2-4340-b2f8-bbeb1b67df94-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-7a2e6c-cmckm\" (UID: \"8f84a8b8-46c2-4340-b2f8-bbeb1b67df94\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7a2e6c-cmckm"
Jul 28 18:17:36.174876 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:36.174767    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7a2e6c-cmckm"
Jul 28 18:17:36.313469 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:36.313422    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7a2e6c-cmckm"]
Jul 28 18:17:36.318722 ip-10-0-136-126 kubenswrapper[2563]: W0728 18:17:36.318682    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f84a8b8_46c2_4340_b2f8_bbeb1b67df94.slice/crio-2ccba5b383520926bbd800c5608f9a973babda92763c3a2f4b320e96892241c0 WatchSource:0}: Error finding container 2ccba5b383520926bbd800c5608f9a973babda92763c3a2f4b320e96892241c0: Status 404 returned error can't find the container with id 2ccba5b383520926bbd800c5608f9a973babda92763c3a2f4b320e96892241c0
Jul 28 18:17:36.930114 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:36.930071    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7a2e6c-cmckm" event={"ID":"8f84a8b8-46c2-4340-b2f8-bbeb1b67df94","Type":"ContainerStarted","Data":"f1ca33f6ec39d26aa3b3f09d091684c87205b5e5b6b44ee24b9aaeaed4bc409e"}
Jul 28 18:17:36.930591 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:36.930129    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7a2e6c-cmckm" event={"ID":"8f84a8b8-46c2-4340-b2f8-bbeb1b67df94","Type":"ContainerStarted","Data":"2ccba5b383520926bbd800c5608f9a973babda92763c3a2f4b320e96892241c0"}
Jul 28 18:17:36.951522 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:36.951475    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7a2e6c-cmckm" podStartSLOduration=1.951454765 podStartE2EDuration="1.951454765s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:17:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:17:36.950807419 +0000 UTC m=+2595.611963765" watchObservedRunningTime="2026-07-28 18:17:36.951454765 +0000 UTC m=+2595.612611116"
Jul 28 18:17:37.936798 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:37.936763    2563 generic.go:356] "Generic (PLEG): container finished" podID="8f84a8b8-46c2-4340-b2f8-bbeb1b67df94" containerID="f1ca33f6ec39d26aa3b3f09d091684c87205b5e5b6b44ee24b9aaeaed4bc409e" exitCode=0
Jul 28 18:17:37.937211 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:37.936855    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7a2e6c-cmckm" event={"ID":"8f84a8b8-46c2-4340-b2f8-bbeb1b67df94","Type":"ContainerDied","Data":"f1ca33f6ec39d26aa3b3f09d091684c87205b5e5b6b44ee24b9aaeaed4bc409e"}
Jul 28 18:17:39.075048 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:39.075027    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7a2e6c-cmckm"
Jul 28 18:17:39.167826 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:39.167798    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f84a8b8-46c2-4340-b2f8-bbeb1b67df94-maas-api-service-ca\") pod \"8f84a8b8-46c2-4340-b2f8-bbeb1b67df94\" (UID: \"8f84a8b8-46c2-4340-b2f8-bbeb1b67df94\") "
Jul 28 18:17:39.168135 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:39.168112    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f84a8b8-46c2-4340-b2f8-bbeb1b67df94-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8f84a8b8-46c2-4340-b2f8-bbeb1b67df94" (UID: "8f84a8b8-46c2-4340-b2f8-bbeb1b67df94"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:17:39.268701 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:39.268628    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f84a8b8-46c2-4340-b2f8-bbeb1b67df94-maas-api-service-ca\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:17:39.948541 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:39.948508    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7a2e6c-cmckm" event={"ID":"8f84a8b8-46c2-4340-b2f8-bbeb1b67df94","Type":"ContainerDied","Data":"2ccba5b383520926bbd800c5608f9a973babda92763c3a2f4b320e96892241c0"}
Jul 28 18:17:39.948541 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:39.948536    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7a2e6c-cmckm"
Jul 28 18:17:39.948742 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:39.948558    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2ccba5b383520926bbd800c5608f9a973babda92763c3a2f4b320e96892241c0"
Jul 28 18:17:51.509237 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:51.509159    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-7a2e6c-5884c54b58-nmk7n"]
Jul 28 18:17:51.509657 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:51.509348    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7a2e6c-5884c54b58-nmk7n" podUID="ac5f3040-efb9-4ff9-ab2a-ae49a34dfb26" containerName="maas-api" containerID="cri-o://d0f83adf6d666f8f7d4c3d543745a282055e069e6afa3606ba510b24e73321dd" gracePeriod=30
Jul 28 18:17:51.552469 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:51.552426    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:17:51.616208 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:51.616180    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-7a2e6c-54c69f7cb8-7gqrx"]
Jul 28 18:17:51.616462 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:51.616414    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-7a2e6c-54c69f7cb8-7gqrx" podUID="8a70ca06-3827-4004-8e4e-69a292cf98bf" containerName="payload-processing" containerID="cri-o://51df95a3fa6184dcff27b2c45f96c1383c0e381527989e5d934eb6165428b8b1" gracePeriod=30
Jul 28 18:17:51.625857 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:51.625830    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-7a2e6c-5dd7c8c868-hwbxv"]
Jul 28 18:17:51.626059 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:51.626035    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7a2e6c-5dd7c8c868-hwbxv" podUID="349c455e-fca7-46b0-bc92-336150fc0c9e" containerName="payload-pre-processing" containerID="cri-o://4287ba5219c78860b1c418aa3147aedc0a00ab4dd2ce6e7bab80af056e7a957d" gracePeriod=30
Jul 28 18:17:51.668101 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:51.668072    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:17:51.776744 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:51.776668    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:17:51.779045 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:51.778984    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7a2e6c-5884c54b58-nmk7n"
Jul 28 18:17:51.878035 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:51.877741    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8249v\" (UniqueName: \"kubernetes.io/projected/ac5f3040-efb9-4ff9-ab2a-ae49a34dfb26-kube-api-access-8249v\") pod \"ac5f3040-efb9-4ff9-ab2a-ae49a34dfb26\" (UID: \"ac5f3040-efb9-4ff9-ab2a-ae49a34dfb26\") "
Jul 28 18:17:51.878035 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:51.877905    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac5f3040-efb9-4ff9-ab2a-ae49a34dfb26-maas-api-tls\") pod \"ac5f3040-efb9-4ff9-ab2a-ae49a34dfb26\" (UID: \"ac5f3040-efb9-4ff9-ab2a-ae49a34dfb26\") "
Jul 28 18:17:51.878035 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:51.877968    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac5f3040-efb9-4ff9-ab2a-ae49a34dfb26-openshift-service-ca\") pod \"ac5f3040-efb9-4ff9-ab2a-ae49a34dfb26\" (UID: \"ac5f3040-efb9-4ff9-ab2a-ae49a34dfb26\") "
Jul 28 18:17:51.879337 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:51.879227    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac5f3040-efb9-4ff9-ab2a-ae49a34dfb26-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ac5f3040-efb9-4ff9-ab2a-ae49a34dfb26" (UID: "ac5f3040-efb9-4ff9-ab2a-ae49a34dfb26"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:17:51.882995 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:51.882972    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac5f3040-efb9-4ff9-ab2a-ae49a34dfb26-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ac5f3040-efb9-4ff9-ab2a-ae49a34dfb26" (UID: "ac5f3040-efb9-4ff9-ab2a-ae49a34dfb26"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:17:51.884048 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:51.884023    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac5f3040-efb9-4ff9-ab2a-ae49a34dfb26-kube-api-access-8249v" (OuterVolumeSpecName: "kube-api-access-8249v") pod "ac5f3040-efb9-4ff9-ab2a-ae49a34dfb26" (UID: "ac5f3040-efb9-4ff9-ab2a-ae49a34dfb26"). InnerVolumeSpecName "kube-api-access-8249v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:17:51.907663 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:51.907643    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-7a2e6c-54c69f7cb8-7gqrx"
Jul 28 18:17:51.910900 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:51.910881    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7a2e6c-5dd7c8c868-hwbxv"
Jul 28 18:17:51.916134 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:51.916112    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:17:51.964638 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:51.964598    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7a2e6c-cmckm"]
Jul 28 18:17:51.967622 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:51.967597    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7a2e6c-cmckm"]
Jul 28 18:17:51.979305 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:51.979271    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/349c455e-fca7-46b0-bc92-336150fc0c9e-plugins-config-volume\") pod \"349c455e-fca7-46b0-bc92-336150fc0c9e\" (UID: \"349c455e-fca7-46b0-bc92-336150fc0c9e\") "
Jul 28 18:17:51.979461 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:51.979321    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mhq8m\" (UniqueName: \"kubernetes.io/projected/349c455e-fca7-46b0-bc92-336150fc0c9e-kube-api-access-mhq8m\") pod \"349c455e-fca7-46b0-bc92-336150fc0c9e\" (UID: \"349c455e-fca7-46b0-bc92-336150fc0c9e\") "
Jul 28 18:17:51.979461 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:51.979342    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a70ca06-3827-4004-8e4e-69a292cf98bf-plugins-config-volume\") pod \"8a70ca06-3827-4004-8e4e-69a292cf98bf\" (UID: \"8a70ca06-3827-4004-8e4e-69a292cf98bf\") "
Jul 28 18:17:51.979461 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:51.979367    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5nkjj\" (UniqueName: \"kubernetes.io/projected/8a70ca06-3827-4004-8e4e-69a292cf98bf-kube-api-access-5nkjj\") pod \"8a70ca06-3827-4004-8e4e-69a292cf98bf\" (UID: \"8a70ca06-3827-4004-8e4e-69a292cf98bf\") "
Jul 28 18:17:51.979738 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:51.979704    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/349c455e-fca7-46b0-bc92-336150fc0c9e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "349c455e-fca7-46b0-bc92-336150fc0c9e" (UID: "349c455e-fca7-46b0-bc92-336150fc0c9e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:17:51.980230 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:51.980099    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac5f3040-efb9-4ff9-ab2a-ae49a34dfb26-maas-api-tls\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:17:51.980230 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:51.980125    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac5f3040-efb9-4ff9-ab2a-ae49a34dfb26-openshift-service-ca\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:17:51.980230 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:51.980140    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/349c455e-fca7-46b0-bc92-336150fc0c9e-plugins-config-volume\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:17:51.980230 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:51.980155    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8249v\" (UniqueName: \"kubernetes.io/projected/ac5f3040-efb9-4ff9-ab2a-ae49a34dfb26-kube-api-access-8249v\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:17:51.980422 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:51.980275    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a70ca06-3827-4004-8e4e-69a292cf98bf-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8a70ca06-3827-4004-8e4e-69a292cf98bf" (UID: "8a70ca06-3827-4004-8e4e-69a292cf98bf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:17:51.982008 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:51.981984    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/349c455e-fca7-46b0-bc92-336150fc0c9e-kube-api-access-mhq8m" (OuterVolumeSpecName: "kube-api-access-mhq8m") pod "349c455e-fca7-46b0-bc92-336150fc0c9e" (UID: "349c455e-fca7-46b0-bc92-336150fc0c9e"). InnerVolumeSpecName "kube-api-access-mhq8m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:17:51.982089 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:51.982072    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a70ca06-3827-4004-8e4e-69a292cf98bf-kube-api-access-5nkjj" (OuterVolumeSpecName: "kube-api-access-5nkjj") pod "8a70ca06-3827-4004-8e4e-69a292cf98bf" (UID: "8a70ca06-3827-4004-8e4e-69a292cf98bf"). InnerVolumeSpecName "kube-api-access-5nkjj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:17:52.013329 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:52.013255    2563 generic.go:356] "Generic (PLEG): container finished" podID="349c455e-fca7-46b0-bc92-336150fc0c9e" containerID="4287ba5219c78860b1c418aa3147aedc0a00ab4dd2ce6e7bab80af056e7a957d" exitCode=0
Jul 28 18:17:52.013329 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:52.013304    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7a2e6c-5dd7c8c868-hwbxv"
Jul 28 18:17:52.013546 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:52.013356    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7a2e6c-5dd7c8c868-hwbxv" event={"ID":"349c455e-fca7-46b0-bc92-336150fc0c9e","Type":"ContainerDied","Data":"4287ba5219c78860b1c418aa3147aedc0a00ab4dd2ce6e7bab80af056e7a957d"}
Jul 28 18:17:52.013546 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:52.013426    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7a2e6c-5dd7c8c868-hwbxv" event={"ID":"349c455e-fca7-46b0-bc92-336150fc0c9e","Type":"ContainerDied","Data":"c8e8fdf48cebec65761b83f8210449fba431f092695f600bd7fbc363b93e7437"}
Jul 28 18:17:52.013546 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:52.013484    2563 scope.go:117] "RemoveContainer" containerID="4287ba5219c78860b1c418aa3147aedc0a00ab4dd2ce6e7bab80af056e7a957d"
Jul 28 18:17:52.014817 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:52.014791    2563 generic.go:356] "Generic (PLEG): container finished" podID="8a70ca06-3827-4004-8e4e-69a292cf98bf" containerID="51df95a3fa6184dcff27b2c45f96c1383c0e381527989e5d934eb6165428b8b1" exitCode=0
Jul 28 18:17:52.014941 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:52.014822    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-7a2e6c-54c69f7cb8-7gqrx" event={"ID":"8a70ca06-3827-4004-8e4e-69a292cf98bf","Type":"ContainerDied","Data":"51df95a3fa6184dcff27b2c45f96c1383c0e381527989e5d934eb6165428b8b1"}
Jul 28 18:17:52.014941 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:52.014880    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-7a2e6c-54c69f7cb8-7gqrx" event={"ID":"8a70ca06-3827-4004-8e4e-69a292cf98bf","Type":"ContainerDied","Data":"28b91d36655262ec242d04c74136e9f5edb5badb543149d8d2b26320003a3406"}
Jul 28 18:17:52.014941 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:52.014833    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-7a2e6c-54c69f7cb8-7gqrx"
Jul 28 18:17:52.016633 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:52.016577    2563 generic.go:356] "Generic (PLEG): container finished" podID="ac5f3040-efb9-4ff9-ab2a-ae49a34dfb26" containerID="d0f83adf6d666f8f7d4c3d543745a282055e069e6afa3606ba510b24e73321dd" exitCode=0
Jul 28 18:17:52.016633 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:52.016629    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7a2e6c-5884c54b58-nmk7n"
Jul 28 18:17:52.016836 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:52.016625    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7a2e6c-5884c54b58-nmk7n" event={"ID":"ac5f3040-efb9-4ff9-ab2a-ae49a34dfb26","Type":"ContainerDied","Data":"d0f83adf6d666f8f7d4c3d543745a282055e069e6afa3606ba510b24e73321dd"}
Jul 28 18:17:52.017049 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:52.017020    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7a2e6c-5884c54b58-nmk7n" event={"ID":"ac5f3040-efb9-4ff9-ab2a-ae49a34dfb26","Type":"ContainerDied","Data":"a92d91cdbe7ca4300d113fdc963088631e52acaf32b4dd7f3989fa092ea4cb43"}
Jul 28 18:17:52.023532 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:52.023376    2563 scope.go:117] "RemoveContainer" containerID="4287ba5219c78860b1c418aa3147aedc0a00ab4dd2ce6e7bab80af056e7a957d"
Jul 28 18:17:52.023946 ip-10-0-136-126 kubenswrapper[2563]: E0728 18:17:52.023790    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4287ba5219c78860b1c418aa3147aedc0a00ab4dd2ce6e7bab80af056e7a957d\": container with ID starting with 4287ba5219c78860b1c418aa3147aedc0a00ab4dd2ce6e7bab80af056e7a957d not found: ID does not exist" containerID="4287ba5219c78860b1c418aa3147aedc0a00ab4dd2ce6e7bab80af056e7a957d"
Jul 28 18:17:52.023946 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:52.023825    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4287ba5219c78860b1c418aa3147aedc0a00ab4dd2ce6e7bab80af056e7a957d"} err="failed to get container status \"4287ba5219c78860b1c418aa3147aedc0a00ab4dd2ce6e7bab80af056e7a957d\": rpc error: code = NotFound desc = could not find container \"4287ba5219c78860b1c418aa3147aedc0a00ab4dd2ce6e7bab80af056e7a957d\": container with ID starting with 4287ba5219c78860b1c418aa3147aedc0a00ab4dd2ce6e7bab80af056e7a957d not found: ID does not exist"
Jul 28 18:17:52.023946 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:52.023862    2563 scope.go:117] "RemoveContainer" containerID="51df95a3fa6184dcff27b2c45f96c1383c0e381527989e5d934eb6165428b8b1"
Jul 28 18:17:52.033767 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:52.033744    2563 scope.go:117] "RemoveContainer" containerID="51df95a3fa6184dcff27b2c45f96c1383c0e381527989e5d934eb6165428b8b1"
Jul 28 18:17:52.034084 ip-10-0-136-126 kubenswrapper[2563]: E0728 18:17:52.034059    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51df95a3fa6184dcff27b2c45f96c1383c0e381527989e5d934eb6165428b8b1\": container with ID starting with 51df95a3fa6184dcff27b2c45f96c1383c0e381527989e5d934eb6165428b8b1 not found: ID does not exist" containerID="51df95a3fa6184dcff27b2c45f96c1383c0e381527989e5d934eb6165428b8b1"
Jul 28 18:17:52.034184 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:52.034102    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51df95a3fa6184dcff27b2c45f96c1383c0e381527989e5d934eb6165428b8b1"} err="failed to get container status \"51df95a3fa6184dcff27b2c45f96c1383c0e381527989e5d934eb6165428b8b1\": rpc error: code = NotFound desc = could not find container \"51df95a3fa6184dcff27b2c45f96c1383c0e381527989e5d934eb6165428b8b1\": container with ID starting with 51df95a3fa6184dcff27b2c45f96c1383c0e381527989e5d934eb6165428b8b1 not found: ID does not exist"
Jul 28 18:17:52.034184 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:52.034123    2563 scope.go:117] "RemoveContainer" containerID="d0f83adf6d666f8f7d4c3d543745a282055e069e6afa3606ba510b24e73321dd"
Jul 28 18:17:52.042591 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:52.042570    2563 scope.go:117] "RemoveContainer" containerID="d0f83adf6d666f8f7d4c3d543745a282055e069e6afa3606ba510b24e73321dd"
Jul 28 18:17:52.042877 ip-10-0-136-126 kubenswrapper[2563]: E0728 18:17:52.042854    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0f83adf6d666f8f7d4c3d543745a282055e069e6afa3606ba510b24e73321dd\": container with ID starting with d0f83adf6d666f8f7d4c3d543745a282055e069e6afa3606ba510b24e73321dd not found: ID does not exist" containerID="d0f83adf6d666f8f7d4c3d543745a282055e069e6afa3606ba510b24e73321dd"
Jul 28 18:17:52.042955 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:52.042888    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0f83adf6d666f8f7d4c3d543745a282055e069e6afa3606ba510b24e73321dd"} err="failed to get container status \"d0f83adf6d666f8f7d4c3d543745a282055e069e6afa3606ba510b24e73321dd\": rpc error: code = NotFound desc = could not find container \"d0f83adf6d666f8f7d4c3d543745a282055e069e6afa3606ba510b24e73321dd\": container with ID starting with d0f83adf6d666f8f7d4c3d543745a282055e069e6afa3606ba510b24e73321dd not found: ID does not exist"
Jul 28 18:17:52.061454 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:52.060879    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-7a2e6c-5dd7c8c868-hwbxv"]
Jul 28 18:17:52.061454 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:52.060931    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-7a2e6c-5dd7c8c868-hwbxv"]
Jul 28 18:17:52.076908 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:52.076872    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-7a2e6c-54c69f7cb8-7gqrx"]
Jul 28 18:17:52.081057 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:52.081029    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mhq8m\" (UniqueName: \"kubernetes.io/projected/349c455e-fca7-46b0-bc92-336150fc0c9e-kube-api-access-mhq8m\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:17:52.081164 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:52.081061    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a70ca06-3827-4004-8e4e-69a292cf98bf-plugins-config-volume\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:17:52.081164 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:52.081078    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5nkjj\" (UniqueName: \"kubernetes.io/projected/8a70ca06-3827-4004-8e4e-69a292cf98bf-kube-api-access-5nkjj\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:17:52.081911 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:52.081894    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-7a2e6c-54c69f7cb8-7gqrx"]
Jul 28 18:17:52.124812 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:52.124786    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-7a2e6c-5884c54b58-nmk7n"]
Jul 28 18:17:52.126911 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:52.126889    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-7a2e6c-5884c54b58-nmk7n"]
Jul 28 18:17:52.436153 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:52.436107    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:17:53.853543 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:53.853511    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="349c455e-fca7-46b0-bc92-336150fc0c9e" path="/var/lib/kubelet/pods/349c455e-fca7-46b0-bc92-336150fc0c9e/volumes"
Jul 28 18:17:53.853915 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:53.853865    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a70ca06-3827-4004-8e4e-69a292cf98bf" path="/var/lib/kubelet/pods/8a70ca06-3827-4004-8e4e-69a292cf98bf/volumes"
Jul 28 18:17:53.854138 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:53.854125    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f84a8b8-46c2-4340-b2f8-bbeb1b67df94" path="/var/lib/kubelet/pods/8f84a8b8-46c2-4340-b2f8-bbeb1b67df94/volumes"
Jul 28 18:17:53.854483 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:53.854470    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac5f3040-efb9-4ff9-ab2a-ae49a34dfb26" path="/var/lib/kubelet/pods/ac5f3040-efb9-4ff9-ab2a-ae49a34dfb26/volumes"
Jul 28 18:17:59.860282 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:59.860251    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:17:59.964506 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:17:59.964472    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:18:00.556501 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:00.556462    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1ff6f1-zmd2n"]
Jul 28 18:18:00.556888 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:00.556870    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="349c455e-fca7-46b0-bc92-336150fc0c9e" containerName="payload-pre-processing"
Jul 28 18:18:00.556979 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:00.556890    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="349c455e-fca7-46b0-bc92-336150fc0c9e" containerName="payload-pre-processing"
Jul 28 18:18:00.556979 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:00.556925    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac5f3040-efb9-4ff9-ab2a-ae49a34dfb26" containerName="maas-api"
Jul 28 18:18:00.556979 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:00.556935    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac5f3040-efb9-4ff9-ab2a-ae49a34dfb26" containerName="maas-api"
Jul 28 18:18:00.556979 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:00.556950    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a70ca06-3827-4004-8e4e-69a292cf98bf" containerName="payload-processing"
Jul 28 18:18:00.556979 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:00.556958    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a70ca06-3827-4004-8e4e-69a292cf98bf" containerName="payload-processing"
Jul 28 18:18:00.556979 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:00.556969    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f84a8b8-46c2-4340-b2f8-bbeb1b67df94" containerName="revoke-keys"
Jul 28 18:18:00.556979 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:00.556977    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f84a8b8-46c2-4340-b2f8-bbeb1b67df94" containerName="revoke-keys"
Jul 28 18:18:00.557309 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:00.557071    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac5f3040-efb9-4ff9-ab2a-ae49a34dfb26" containerName="maas-api"
Jul 28 18:18:00.557309 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:00.557086    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="349c455e-fca7-46b0-bc92-336150fc0c9e" containerName="payload-pre-processing"
Jul 28 18:18:00.557309 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:00.557096    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a70ca06-3827-4004-8e4e-69a292cf98bf" containerName="payload-processing"
Jul 28 18:18:00.557309 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:00.557105    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f84a8b8-46c2-4340-b2f8-bbeb1b67df94" containerName="revoke-keys"
Jul 28 18:18:00.560173 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:00.560152    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1ff6f1-zmd2n"
Jul 28 18:18:00.562865 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:00.562844    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9xcv2\""
Jul 28 18:18:00.573840 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:00.573805    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1ff6f1-zmd2n"]
Jul 28 18:18:00.654696 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:00.654662    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73ca8130-8da3-4745-9f12-a329867e9c7f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-1ff6f1-zmd2n\" (UID: \"73ca8130-8da3-4745-9f12-a329867e9c7f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1ff6f1-zmd2n"
Jul 28 18:18:00.755372 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:00.755339    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73ca8130-8da3-4745-9f12-a329867e9c7f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-1ff6f1-zmd2n\" (UID: \"73ca8130-8da3-4745-9f12-a329867e9c7f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1ff6f1-zmd2n"
Jul 28 18:18:00.755937 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:00.755917    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73ca8130-8da3-4745-9f12-a329867e9c7f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-1ff6f1-zmd2n\" (UID: \"73ca8130-8da3-4745-9f12-a329867e9c7f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1ff6f1-zmd2n"
Jul 28 18:18:00.869836 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:00.869805    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1ff6f1-zmd2n"
Jul 28 18:18:00.968154 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:00.968122    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:18:01.008097 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:01.008074    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1ff6f1-zmd2n"]
Jul 28 18:18:01.010192 ip-10-0-136-126 kubenswrapper[2563]: W0728 18:18:01.010162    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73ca8130_8da3_4745_9f12_a329867e9c7f.slice/crio-8c4fbb3108da4e5f305fc8bd0907248ef83cec47aae2d5b88b0db1e1df8427c6 WatchSource:0}: Error finding container 8c4fbb3108da4e5f305fc8bd0907248ef83cec47aae2d5b88b0db1e1df8427c6: Status 404 returned error can't find the container with id 8c4fbb3108da4e5f305fc8bd0907248ef83cec47aae2d5b88b0db1e1df8427c6
Jul 28 18:18:01.075273 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:01.075239    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1ff6f1-zmd2n" event={"ID":"73ca8130-8da3-4745-9f12-a329867e9c7f","Type":"ContainerStarted","Data":"8c4fbb3108da4e5f305fc8bd0907248ef83cec47aae2d5b88b0db1e1df8427c6"}
Jul 28 18:18:01.119064 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:01.119036    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:18:02.082861 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:02.082757    2563 generic.go:356] "Generic (PLEG): container finished" podID="73ca8130-8da3-4745-9f12-a329867e9c7f" containerID="e39581aa97ad46e29bb3df4c66b20def811d50c1112b0148d0dbd1489c997cf7" exitCode=0
Jul 28 18:18:02.083231 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:02.082848    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1ff6f1-zmd2n" event={"ID":"73ca8130-8da3-4745-9f12-a329867e9c7f","Type":"ContainerDied","Data":"e39581aa97ad46e29bb3df4c66b20def811d50c1112b0148d0dbd1489c997cf7"}
Jul 28 18:18:03.228264 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:03.228236    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1ff6f1-zmd2n"
Jul 28 18:18:03.379593 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:03.379511    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73ca8130-8da3-4745-9f12-a329867e9c7f-maas-api-service-ca\") pod \"73ca8130-8da3-4745-9f12-a329867e9c7f\" (UID: \"73ca8130-8da3-4745-9f12-a329867e9c7f\") "
Jul 28 18:18:03.379897 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:03.379874    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73ca8130-8da3-4745-9f12-a329867e9c7f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "73ca8130-8da3-4745-9f12-a329867e9c7f" (UID: "73ca8130-8da3-4745-9f12-a329867e9c7f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:18:03.480421 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:03.480393    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73ca8130-8da3-4745-9f12-a329867e9c7f-maas-api-service-ca\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:18:04.095266 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:04.095234    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1ff6f1-zmd2n" event={"ID":"73ca8130-8da3-4745-9f12-a329867e9c7f","Type":"ContainerDied","Data":"8c4fbb3108da4e5f305fc8bd0907248ef83cec47aae2d5b88b0db1e1df8427c6"}
Jul 28 18:18:04.095266 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:04.095267    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1ff6f1-zmd2n"
Jul 28 18:18:04.095517 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:04.095303    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8c4fbb3108da4e5f305fc8bd0907248ef83cec47aae2d5b88b0db1e1df8427c6"
Jul 28 18:18:11.200825 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:11.200780    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-1ff6f1-5d96c4f9df-fpftf"]
Jul 28 18:18:11.201193 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:11.201006    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1ff6f1-5d96c4f9df-fpftf" podUID="45858501-6443-4cdd-9536-8f6e14f12742" containerName="maas-api" containerID="cri-o://ad85228f3fc122322fcaa92024c9569a8a25e1a17b43ff93b086beb4c9a8e3e7" gracePeriod=30
Jul 28 18:18:11.232026 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:11.231984    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:18:11.324973 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:11.324937    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-1ff6f1-9cc645bb4-hq8qr"]
Jul 28 18:18:11.325203 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:11.325175    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1ff6f1-9cc645bb4-hq8qr" podUID="79667739-961c-4d5f-80a9-a14a5869e7ca" containerName="payload-pre-processing" containerID="cri-o://ec24773bbf34a916c1dd5319b8b2f14326437566319e137deae2fd753bf29e65" gracePeriod=30
Jul 28 18:18:11.372157 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:11.372129    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:18:11.460711 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:11.460677    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:18:11.472385 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:11.472364    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1ff6f1-5d96c4f9df-fpftf"
Jul 28 18:18:11.573031 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:11.572996    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:18:11.603211 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:11.603188    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1ff6f1-9cc645bb4-hq8qr"
Jul 28 18:18:11.604228 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:11.604204    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1ff6f1-zmd2n"]
Jul 28 18:18:11.612459 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:11.612424    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1ff6f1-zmd2n"]
Jul 28 18:18:11.655651 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:11.655624    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45858501-6443-4cdd-9536-8f6e14f12742-maas-api-tls\") pod \"45858501-6443-4cdd-9536-8f6e14f12742\" (UID: \"45858501-6443-4cdd-9536-8f6e14f12742\") "
Jul 28 18:18:11.655651 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:11.655656    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xgnp8\" (UniqueName: \"kubernetes.io/projected/45858501-6443-4cdd-9536-8f6e14f12742-kube-api-access-xgnp8\") pod \"45858501-6443-4cdd-9536-8f6e14f12742\" (UID: \"45858501-6443-4cdd-9536-8f6e14f12742\") "
Jul 28 18:18:11.655858 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:11.655715    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/45858501-6443-4cdd-9536-8f6e14f12742-openshift-service-ca\") pod \"45858501-6443-4cdd-9536-8f6e14f12742\" (UID: \"45858501-6443-4cdd-9536-8f6e14f12742\") "
Jul 28 18:18:11.656110 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:11.656088    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45858501-6443-4cdd-9536-8f6e14f12742-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "45858501-6443-4cdd-9536-8f6e14f12742" (UID: "45858501-6443-4cdd-9536-8f6e14f12742"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:18:11.657737 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:11.657709    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45858501-6443-4cdd-9536-8f6e14f12742-kube-api-access-xgnp8" (OuterVolumeSpecName: "kube-api-access-xgnp8") pod "45858501-6443-4cdd-9536-8f6e14f12742" (UID: "45858501-6443-4cdd-9536-8f6e14f12742"). InnerVolumeSpecName "kube-api-access-xgnp8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:18:11.657737 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:11.657720    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/45858501-6443-4cdd-9536-8f6e14f12742-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "45858501-6443-4cdd-9536-8f6e14f12742" (UID: "45858501-6443-4cdd-9536-8f6e14f12742"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:18:11.756298 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:11.756214    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79667739-961c-4d5f-80a9-a14a5869e7ca-plugins-config-volume\") pod \"79667739-961c-4d5f-80a9-a14a5869e7ca\" (UID: \"79667739-961c-4d5f-80a9-a14a5869e7ca\") "
Jul 28 18:18:11.756458 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:11.756317    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9scw2\" (UniqueName: \"kubernetes.io/projected/79667739-961c-4d5f-80a9-a14a5869e7ca-kube-api-access-9scw2\") pod \"79667739-961c-4d5f-80a9-a14a5869e7ca\" (UID: \"79667739-961c-4d5f-80a9-a14a5869e7ca\") "
Jul 28 18:18:11.756519 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:11.756479    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45858501-6443-4cdd-9536-8f6e14f12742-maas-api-tls\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:18:11.756519 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:11.756490    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xgnp8\" (UniqueName: \"kubernetes.io/projected/45858501-6443-4cdd-9536-8f6e14f12742-kube-api-access-xgnp8\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:18:11.756519 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:11.756500    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/45858501-6443-4cdd-9536-8f6e14f12742-openshift-service-ca\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:18:11.756626 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:11.756592    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79667739-961c-4d5f-80a9-a14a5869e7ca-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "79667739-961c-4d5f-80a9-a14a5869e7ca" (UID: "79667739-961c-4d5f-80a9-a14a5869e7ca"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:18:11.758378 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:11.758355    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79667739-961c-4d5f-80a9-a14a5869e7ca-kube-api-access-9scw2" (OuterVolumeSpecName: "kube-api-access-9scw2") pod "79667739-961c-4d5f-80a9-a14a5869e7ca" (UID: "79667739-961c-4d5f-80a9-a14a5869e7ca"). InnerVolumeSpecName "kube-api-access-9scw2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:18:11.853829 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:11.853795    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73ca8130-8da3-4745-9f12-a329867e9c7f" path="/var/lib/kubelet/pods/73ca8130-8da3-4745-9f12-a329867e9c7f/volumes"
Jul 28 18:18:11.857059 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:11.857030    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79667739-961c-4d5f-80a9-a14a5869e7ca-plugins-config-volume\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:18:11.857059 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:11.857055    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9scw2\" (UniqueName: \"kubernetes.io/projected/79667739-961c-4d5f-80a9-a14a5869e7ca-kube-api-access-9scw2\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:18:12.008651 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:12.008559    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:18:12.139997 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:12.139967    2563 generic.go:356] "Generic (PLEG): container finished" podID="45858501-6443-4cdd-9536-8f6e14f12742" containerID="ad85228f3fc122322fcaa92024c9569a8a25e1a17b43ff93b086beb4c9a8e3e7" exitCode=0
Jul 28 18:18:12.140173 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:12.140020    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1ff6f1-5d96c4f9df-fpftf"
Jul 28 18:18:12.140173 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:12.140052    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1ff6f1-5d96c4f9df-fpftf" event={"ID":"45858501-6443-4cdd-9536-8f6e14f12742","Type":"ContainerDied","Data":"ad85228f3fc122322fcaa92024c9569a8a25e1a17b43ff93b086beb4c9a8e3e7"}
Jul 28 18:18:12.140173 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:12.140108    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1ff6f1-5d96c4f9df-fpftf" event={"ID":"45858501-6443-4cdd-9536-8f6e14f12742","Type":"ContainerDied","Data":"e2a8b4f98fe8b25fad135486123f85152f76c8730fd21bf65ed08de831a4ef8d"}
Jul 28 18:18:12.140173 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:12.140143    2563 scope.go:117] "RemoveContainer" containerID="ad85228f3fc122322fcaa92024c9569a8a25e1a17b43ff93b086beb4c9a8e3e7"
Jul 28 18:18:12.141558 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:12.141531    2563 generic.go:356] "Generic (PLEG): container finished" podID="79667739-961c-4d5f-80a9-a14a5869e7ca" containerID="ec24773bbf34a916c1dd5319b8b2f14326437566319e137deae2fd753bf29e65" exitCode=0
Jul 28 18:18:12.141686 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:12.141565    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1ff6f1-9cc645bb4-hq8qr" event={"ID":"79667739-961c-4d5f-80a9-a14a5869e7ca","Type":"ContainerDied","Data":"ec24773bbf34a916c1dd5319b8b2f14326437566319e137deae2fd753bf29e65"}
Jul 28 18:18:12.141686 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:12.141585    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1ff6f1-9cc645bb4-hq8qr"
Jul 28 18:18:12.141686 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:12.141612    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1ff6f1-9cc645bb4-hq8qr" event={"ID":"79667739-961c-4d5f-80a9-a14a5869e7ca","Type":"ContainerDied","Data":"c84210c6b6905b86e7d5a0ce21b0fab22eccd96038b0650ade67ebddfac00521"}
Jul 28 18:18:12.149648 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:12.149632    2563 scope.go:117] "RemoveContainer" containerID="ad85228f3fc122322fcaa92024c9569a8a25e1a17b43ff93b086beb4c9a8e3e7"
Jul 28 18:18:12.149887 ip-10-0-136-126 kubenswrapper[2563]: E0728 18:18:12.149871    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad85228f3fc122322fcaa92024c9569a8a25e1a17b43ff93b086beb4c9a8e3e7\": container with ID starting with ad85228f3fc122322fcaa92024c9569a8a25e1a17b43ff93b086beb4c9a8e3e7 not found: ID does not exist" containerID="ad85228f3fc122322fcaa92024c9569a8a25e1a17b43ff93b086beb4c9a8e3e7"
Jul 28 18:18:12.149948 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:12.149895    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad85228f3fc122322fcaa92024c9569a8a25e1a17b43ff93b086beb4c9a8e3e7"} err="failed to get container status \"ad85228f3fc122322fcaa92024c9569a8a25e1a17b43ff93b086beb4c9a8e3e7\": rpc error: code = NotFound desc = could not find container \"ad85228f3fc122322fcaa92024c9569a8a25e1a17b43ff93b086beb4c9a8e3e7\": container with ID starting with ad85228f3fc122322fcaa92024c9569a8a25e1a17b43ff93b086beb4c9a8e3e7 not found: ID does not exist"
Jul 28 18:18:12.149948 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:12.149911    2563 scope.go:117] "RemoveContainer" containerID="ec24773bbf34a916c1dd5319b8b2f14326437566319e137deae2fd753bf29e65"
Jul 28 18:18:12.157594 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:12.157579    2563 scope.go:117] "RemoveContainer" containerID="ec24773bbf34a916c1dd5319b8b2f14326437566319e137deae2fd753bf29e65"
Jul 28 18:18:12.157854 ip-10-0-136-126 kubenswrapper[2563]: E0728 18:18:12.157835    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec24773bbf34a916c1dd5319b8b2f14326437566319e137deae2fd753bf29e65\": container with ID starting with ec24773bbf34a916c1dd5319b8b2f14326437566319e137deae2fd753bf29e65 not found: ID does not exist" containerID="ec24773bbf34a916c1dd5319b8b2f14326437566319e137deae2fd753bf29e65"
Jul 28 18:18:12.157911 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:12.157862    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec24773bbf34a916c1dd5319b8b2f14326437566319e137deae2fd753bf29e65"} err="failed to get container status \"ec24773bbf34a916c1dd5319b8b2f14326437566319e137deae2fd753bf29e65\": rpc error: code = NotFound desc = could not find container \"ec24773bbf34a916c1dd5319b8b2f14326437566319e137deae2fd753bf29e65\": container with ID starting with ec24773bbf34a916c1dd5319b8b2f14326437566319e137deae2fd753bf29e65 not found: ID does not exist"
Jul 28 18:18:12.163446 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:12.163407    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-1ff6f1-9cc645bb4-hq8qr"]
Jul 28 18:18:12.165647 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:12.165626    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-1ff6f1-9cc645bb4-hq8qr"]
Jul 28 18:18:12.180443 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:12.180415    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-1ff6f1-5d96c4f9df-fpftf"]
Jul 28 18:18:12.186742 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:12.186721    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-1ff6f1-5d96c4f9df-fpftf"]
Jul 28 18:18:13.853623 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:13.853586    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45858501-6443-4cdd-9536-8f6e14f12742" path="/var/lib/kubelet/pods/45858501-6443-4cdd-9536-8f6e14f12742/volumes"
Jul 28 18:18:13.854090 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:13.854060    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79667739-961c-4d5f-80a9-a14a5869e7ca" path="/var/lib/kubelet/pods/79667739-961c-4d5f-80a9-a14a5869e7ca/volumes"
Jul 28 18:18:19.746471 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:19.746419    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:18:19.853447 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:19.853401    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:18:20.075251 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:20.075223    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:18:20.434999 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:20.434908    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:18:20.791418 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:20.791330    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:18:21.156745 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:21.156713    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:18:21.243726 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:21.243698    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:18:24.195475 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:24.195396    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:18:24.292382 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:24.292352    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:18:24.440218 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:24.440185    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-9b5b66-84d4dc76bf-tbcmk"]
Jul 28 18:18:24.440611 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:24.440593    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73ca8130-8da3-4745-9f12-a329867e9c7f" containerName="revoke-keys"
Jul 28 18:18:24.440711 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:24.440613    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="73ca8130-8da3-4745-9f12-a329867e9c7f" containerName="revoke-keys"
Jul 28 18:18:24.440711 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:24.440649    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79667739-961c-4d5f-80a9-a14a5869e7ca" containerName="payload-pre-processing"
Jul 28 18:18:24.440711 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:24.440658    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="79667739-961c-4d5f-80a9-a14a5869e7ca" containerName="payload-pre-processing"
Jul 28 18:18:24.440711 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:24.440674    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45858501-6443-4cdd-9536-8f6e14f12742" containerName="maas-api"
Jul 28 18:18:24.440711 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:24.440683    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="45858501-6443-4cdd-9536-8f6e14f12742" containerName="maas-api"
Jul 28 18:18:24.440963 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:24.440766    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="73ca8130-8da3-4745-9f12-a329867e9c7f" containerName="revoke-keys"
Jul 28 18:18:24.440963 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:24.440777    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="79667739-961c-4d5f-80a9-a14a5869e7ca" containerName="payload-pre-processing"
Jul 28 18:18:24.440963 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:24.440791    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="45858501-6443-4cdd-9536-8f6e14f12742" containerName="maas-api"
Jul 28 18:18:24.445673 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:24.445614    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b5b66-84d4dc76bf-tbcmk"
Jul 28 18:18:24.450217 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:24.450200    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-9b5b66-serving-cert\""
Jul 28 18:18:24.465293 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:24.465267    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-9b5b66-84d4dc76bf-tbcmk"]
Jul 28 18:18:24.495849 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:24.495824    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:18:24.573514 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:24.573479    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdlrm\" (UniqueName: \"kubernetes.io/projected/46cf1643-a035-46a6-82ac-f73568c5936b-kube-api-access-gdlrm\") pod \"maas-api-e2e-shared-a-9b5b66-84d4dc76bf-tbcmk\" (UID: \"46cf1643-a035-46a6-82ac-f73568c5936b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b5b66-84d4dc76bf-tbcmk"
Jul 28 18:18:24.573514 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:24.573522    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46cf1643-a035-46a6-82ac-f73568c5936b-openshift-service-ca\") pod \"maas-api-e2e-shared-a-9b5b66-84d4dc76bf-tbcmk\" (UID: \"46cf1643-a035-46a6-82ac-f73568c5936b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b5b66-84d4dc76bf-tbcmk"
Jul 28 18:18:24.573765 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:24.573620    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46cf1643-a035-46a6-82ac-f73568c5936b-maas-api-tls\") pod \"maas-api-e2e-shared-a-9b5b66-84d4dc76bf-tbcmk\" (UID: \"46cf1643-a035-46a6-82ac-f73568c5936b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b5b66-84d4dc76bf-tbcmk"
Jul 28 18:18:24.674909 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:24.674874    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdlrm\" (UniqueName: \"kubernetes.io/projected/46cf1643-a035-46a6-82ac-f73568c5936b-kube-api-access-gdlrm\") pod \"maas-api-e2e-shared-a-9b5b66-84d4dc76bf-tbcmk\" (UID: \"46cf1643-a035-46a6-82ac-f73568c5936b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b5b66-84d4dc76bf-tbcmk"
Jul 28 18:18:24.674909 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:24.674913    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46cf1643-a035-46a6-82ac-f73568c5936b-openshift-service-ca\") pod \"maas-api-e2e-shared-a-9b5b66-84d4dc76bf-tbcmk\" (UID: \"46cf1643-a035-46a6-82ac-f73568c5936b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b5b66-84d4dc76bf-tbcmk"
Jul 28 18:18:24.675120 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:24.674947    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46cf1643-a035-46a6-82ac-f73568c5936b-maas-api-tls\") pod \"maas-api-e2e-shared-a-9b5b66-84d4dc76bf-tbcmk\" (UID: \"46cf1643-a035-46a6-82ac-f73568c5936b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b5b66-84d4dc76bf-tbcmk"
Jul 28 18:18:24.675120 ip-10-0-136-126 kubenswrapper[2563]: E0728 18:18:24.675076    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-9b5b66-serving-cert: secret "maas-api-e2e-shared-a-9b5b66-serving-cert" not found
Jul 28 18:18:24.675199 ip-10-0-136-126 kubenswrapper[2563]: E0728 18:18:24.675138    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46cf1643-a035-46a6-82ac-f73568c5936b-maas-api-tls podName:46cf1643-a035-46a6-82ac-f73568c5936b nodeName:}" failed. No retries permitted until 2026-07-28 18:18:25.175120246 +0000 UTC m=+2643.836276593 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/46cf1643-a035-46a6-82ac-f73568c5936b-maas-api-tls") pod "maas-api-e2e-shared-a-9b5b66-84d4dc76bf-tbcmk" (UID: "46cf1643-a035-46a6-82ac-f73568c5936b") : secret "maas-api-e2e-shared-a-9b5b66-serving-cert" not found
Jul 28 18:18:24.675697 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:24.675675    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46cf1643-a035-46a6-82ac-f73568c5936b-openshift-service-ca\") pod \"maas-api-e2e-shared-a-9b5b66-84d4dc76bf-tbcmk\" (UID: \"46cf1643-a035-46a6-82ac-f73568c5936b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b5b66-84d4dc76bf-tbcmk"
Jul 28 18:18:24.686867 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:24.686843    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdlrm\" (UniqueName: \"kubernetes.io/projected/46cf1643-a035-46a6-82ac-f73568c5936b-kube-api-access-gdlrm\") pod \"maas-api-e2e-shared-a-9b5b66-84d4dc76bf-tbcmk\" (UID: \"46cf1643-a035-46a6-82ac-f73568c5936b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b5b66-84d4dc76bf-tbcmk"
Jul 28 18:18:24.789795 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:24.789705    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:18:25.179393 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:25.179357    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46cf1643-a035-46a6-82ac-f73568c5936b-maas-api-tls\") pod \"maas-api-e2e-shared-a-9b5b66-84d4dc76bf-tbcmk\" (UID: \"46cf1643-a035-46a6-82ac-f73568c5936b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b5b66-84d4dc76bf-tbcmk"
Jul 28 18:18:25.181732 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:25.181713    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46cf1643-a035-46a6-82ac-f73568c5936b-maas-api-tls\") pod \"maas-api-e2e-shared-a-9b5b66-84d4dc76bf-tbcmk\" (UID: \"46cf1643-a035-46a6-82ac-f73568c5936b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b5b66-84d4dc76bf-tbcmk"
Jul 28 18:18:25.355369 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:25.355336    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b5b66-84d4dc76bf-tbcmk"
Jul 28 18:18:25.387501 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:25.387473    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:18:25.504382 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:25.504347    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:18:25.522286 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:25.522263    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-9b5b66-84d4dc76bf-tbcmk"]
Jul 28 18:18:25.524727 ip-10-0-136-126 kubenswrapper[2563]: W0728 18:18:25.524696    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46cf1643_a035_46a6_82ac_f73568c5936b.slice/crio-f712c3fc7976376e94087ba7193c4cbfc77ad438bb2ee7ea7e1a4b4684639a33 WatchSource:0}: Error finding container f712c3fc7976376e94087ba7193c4cbfc77ad438bb2ee7ea7e1a4b4684639a33: Status 404 returned error can't find the container with id f712c3fc7976376e94087ba7193c4cbfc77ad438bb2ee7ea7e1a4b4684639a33
Jul 28 18:18:25.526056 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:25.526035    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 18:18:26.229822 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:26.229786    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b5b66-84d4dc76bf-tbcmk" event={"ID":"46cf1643-a035-46a6-82ac-f73568c5936b","Type":"ContainerStarted","Data":"c6474a00c89febf3ab08a4e59b14c155a7ccd54b13e1a2ce97afc545ffc5a20a"}
Jul 28 18:18:26.229990 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:26.229856    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b5b66-84d4dc76bf-tbcmk"
Jul 28 18:18:26.229990 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:26.229882    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b5b66-84d4dc76bf-tbcmk" event={"ID":"46cf1643-a035-46a6-82ac-f73568c5936b","Type":"ContainerStarted","Data":"f712c3fc7976376e94087ba7193c4cbfc77ad438bb2ee7ea7e1a4b4684639a33"}
Jul 28 18:18:26.265413 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:26.265364    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b5b66-84d4dc76bf-tbcmk" podStartSLOduration=1.7267507869999998 podStartE2EDuration="2.265349309s" totalImagesPullingTime="538.598522ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:18:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:18:26.26312983 +0000 UTC m=+2644.924286209" watchObservedRunningTime="2026-07-28 18:18:26.265349309 +0000 UTC m=+2644.926505664"
Jul 28 18:18:32.242281 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:32.242253    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b5b66-84d4dc76bf-tbcmk"
Jul 28 18:18:33.824235 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:33.824202    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:18:34.162612 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:34.162522    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:18:34.533308 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:34.533216    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:18:35.284161 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:35.284126    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:18:37.991528 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:37.991492    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:18:38.093895 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:38.093863    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:18:38.136934 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:38.136906    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-c73c1e-6ff87597b8-hdlcx"]
Jul 28 18:18:38.140778 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:38.140758    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c73c1e-6ff87597b8-hdlcx"
Jul 28 18:18:38.143827 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:38.143809    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-c73c1e-serving-cert\""
Jul 28 18:18:38.156248 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:38.156214    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-c73c1e-6ff87597b8-hdlcx"]
Jul 28 18:18:38.235929 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:38.235886    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:18:38.292221 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:38.292144    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdxth\" (UniqueName: \"kubernetes.io/projected/03c0eef1-73e0-4400-a6f7-1936634155a5-kube-api-access-hdxth\") pod \"maas-api-e2e-shared-b-c73c1e-6ff87597b8-hdlcx\" (UID: \"03c0eef1-73e0-4400-a6f7-1936634155a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c73c1e-6ff87597b8-hdlcx"
Jul 28 18:18:38.292221 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:38.292189    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03c0eef1-73e0-4400-a6f7-1936634155a5-maas-api-tls\") pod \"maas-api-e2e-shared-b-c73c1e-6ff87597b8-hdlcx\" (UID: \"03c0eef1-73e0-4400-a6f7-1936634155a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c73c1e-6ff87597b8-hdlcx"
Jul 28 18:18:38.292397 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:38.292277    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03c0eef1-73e0-4400-a6f7-1936634155a5-openshift-service-ca\") pod \"maas-api-e2e-shared-b-c73c1e-6ff87597b8-hdlcx\" (UID: \"03c0eef1-73e0-4400-a6f7-1936634155a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c73c1e-6ff87597b8-hdlcx"
Jul 28 18:18:38.392863 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:38.392824    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdxth\" (UniqueName: \"kubernetes.io/projected/03c0eef1-73e0-4400-a6f7-1936634155a5-kube-api-access-hdxth\") pod \"maas-api-e2e-shared-b-c73c1e-6ff87597b8-hdlcx\" (UID: \"03c0eef1-73e0-4400-a6f7-1936634155a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c73c1e-6ff87597b8-hdlcx"
Jul 28 18:18:38.392863 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:38.392869    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03c0eef1-73e0-4400-a6f7-1936634155a5-maas-api-tls\") pod \"maas-api-e2e-shared-b-c73c1e-6ff87597b8-hdlcx\" (UID: \"03c0eef1-73e0-4400-a6f7-1936634155a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c73c1e-6ff87597b8-hdlcx"
Jul 28 18:18:38.393081 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:38.392922    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03c0eef1-73e0-4400-a6f7-1936634155a5-openshift-service-ca\") pod \"maas-api-e2e-shared-b-c73c1e-6ff87597b8-hdlcx\" (UID: \"03c0eef1-73e0-4400-a6f7-1936634155a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c73c1e-6ff87597b8-hdlcx"
Jul 28 18:18:38.393081 ip-10-0-136-126 kubenswrapper[2563]: E0728 18:18:38.393041    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-c73c1e-serving-cert: secret "maas-api-e2e-shared-b-c73c1e-serving-cert" not found
Jul 28 18:18:38.393146 ip-10-0-136-126 kubenswrapper[2563]: E0728 18:18:38.393115    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03c0eef1-73e0-4400-a6f7-1936634155a5-maas-api-tls podName:03c0eef1-73e0-4400-a6f7-1936634155a5 nodeName:}" failed. No retries permitted until 2026-07-28 18:18:38.893096562 +0000 UTC m=+2657.554252915 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/03c0eef1-73e0-4400-a6f7-1936634155a5-maas-api-tls") pod "maas-api-e2e-shared-b-c73c1e-6ff87597b8-hdlcx" (UID: "03c0eef1-73e0-4400-a6f7-1936634155a5") : secret "maas-api-e2e-shared-b-c73c1e-serving-cert" not found
Jul 28 18:18:38.393550 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:38.393534    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03c0eef1-73e0-4400-a6f7-1936634155a5-openshift-service-ca\") pod \"maas-api-e2e-shared-b-c73c1e-6ff87597b8-hdlcx\" (UID: \"03c0eef1-73e0-4400-a6f7-1936634155a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c73c1e-6ff87597b8-hdlcx"
Jul 28 18:18:38.402051 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:38.402030    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdxth\" (UniqueName: \"kubernetes.io/projected/03c0eef1-73e0-4400-a6f7-1936634155a5-kube-api-access-hdxth\") pod \"maas-api-e2e-shared-b-c73c1e-6ff87597b8-hdlcx\" (UID: \"03c0eef1-73e0-4400-a6f7-1936634155a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c73c1e-6ff87597b8-hdlcx"
Jul 28 18:18:38.858416 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:38.858380    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:18:38.898909 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:38.898874    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03c0eef1-73e0-4400-a6f7-1936634155a5-maas-api-tls\") pod \"maas-api-e2e-shared-b-c73c1e-6ff87597b8-hdlcx\" (UID: \"03c0eef1-73e0-4400-a6f7-1936634155a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c73c1e-6ff87597b8-hdlcx"
Jul 28 18:18:38.901543 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:38.901515    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03c0eef1-73e0-4400-a6f7-1936634155a5-maas-api-tls\") pod \"maas-api-e2e-shared-b-c73c1e-6ff87597b8-hdlcx\" (UID: \"03c0eef1-73e0-4400-a6f7-1936634155a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c73c1e-6ff87597b8-hdlcx"
Jul 28 18:18:38.965120 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:38.965065    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:18:39.044936 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:39.044904    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:18:39.050846 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:39.050825    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c73c1e-6ff87597b8-hdlcx"
Jul 28 18:18:39.075953 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:39.075920    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-c73c1e-668b4b8ff8-w79d6"]
Jul 28 18:18:39.079860 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:39.079839    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-c73c1e-668b4b8ff8-w79d6"
Jul 28 18:18:39.082590 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:39.082570    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-c73c1e-dockercfg-kwmmd\""
Jul 28 18:18:39.082732 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:39.082573    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-c73c1e\""
Jul 28 18:18:39.089732 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:39.088919    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-c73c1e-668b4b8ff8-w79d6"]
Jul 28 18:18:39.145512 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:39.145423    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-c73c1e-569f987c9b-jptl4"]
Jul 28 18:18:39.149453 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:39.149410    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-c73c1e-569f987c9b-jptl4"
Jul 28 18:18:39.159186 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:39.159127    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-c73c1e-569f987c9b-jptl4"]
Jul 28 18:18:39.201515 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:39.201484    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b10f1515-8d9d-4a52-ba48-50c787a5e9b5-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-c73c1e-668b4b8ff8-w79d6\" (UID: \"b10f1515-8d9d-4a52-ba48-50c787a5e9b5\") " pod="openshift-ingress/payload-processing-e2e-shared-b-c73c1e-668b4b8ff8-w79d6"
Jul 28 18:18:39.201685 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:39.201528    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gl6ng\" (UniqueName: \"kubernetes.io/projected/b10f1515-8d9d-4a52-ba48-50c787a5e9b5-kube-api-access-gl6ng\") pod \"payload-processing-e2e-shared-b-c73c1e-668b4b8ff8-w79d6\" (UID: \"b10f1515-8d9d-4a52-ba48-50c787a5e9b5\") " pod="openshift-ingress/payload-processing-e2e-shared-b-c73c1e-668b4b8ff8-w79d6"
Jul 28 18:18:39.221361 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:39.221336    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-c73c1e-6ff87597b8-hdlcx"]
Jul 28 18:18:39.222842 ip-10-0-136-126 kubenswrapper[2563]: W0728 18:18:39.222809    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03c0eef1_73e0_4400_a6f7_1936634155a5.slice/crio-70618a9a84931e77ab54cd4a67b195300176acea4efce1a99f765f5d3152b7b9 WatchSource:0}: Error finding container 70618a9a84931e77ab54cd4a67b195300176acea4efce1a99f765f5d3152b7b9: Status 404 returned error can't find the container with id 70618a9a84931e77ab54cd4a67b195300176acea4efce1a99f765f5d3152b7b9
Jul 28 18:18:39.302063 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:39.302019    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzlft\" (UniqueName: \"kubernetes.io/projected/fe167dee-9478-42ec-97a6-b291dbb3cf60-kube-api-access-nzlft\") pod \"payload-pre-processing-e2e-shared-b-c73c1e-569f987c9b-jptl4\" (UID: \"fe167dee-9478-42ec-97a6-b291dbb3cf60\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-c73c1e-569f987c9b-jptl4"
Jul 28 18:18:39.302259 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:39.302088    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe167dee-9478-42ec-97a6-b291dbb3cf60-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-c73c1e-569f987c9b-jptl4\" (UID: \"fe167dee-9478-42ec-97a6-b291dbb3cf60\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-c73c1e-569f987c9b-jptl4"
Jul 28 18:18:39.302259 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:39.302214    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b10f1515-8d9d-4a52-ba48-50c787a5e9b5-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-c73c1e-668b4b8ff8-w79d6\" (UID: \"b10f1515-8d9d-4a52-ba48-50c787a5e9b5\") " pod="openshift-ingress/payload-processing-e2e-shared-b-c73c1e-668b4b8ff8-w79d6"
Jul 28 18:18:39.302259 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:39.302253    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gl6ng\" (UniqueName: \"kubernetes.io/projected/b10f1515-8d9d-4a52-ba48-50c787a5e9b5-kube-api-access-gl6ng\") pod \"payload-processing-e2e-shared-b-c73c1e-668b4b8ff8-w79d6\" (UID: \"b10f1515-8d9d-4a52-ba48-50c787a5e9b5\") " pod="openshift-ingress/payload-processing-e2e-shared-b-c73c1e-668b4b8ff8-w79d6"
Jul 28 18:18:39.303016 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:39.302992    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b10f1515-8d9d-4a52-ba48-50c787a5e9b5-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-c73c1e-668b4b8ff8-w79d6\" (UID: \"b10f1515-8d9d-4a52-ba48-50c787a5e9b5\") " pod="openshift-ingress/payload-processing-e2e-shared-b-c73c1e-668b4b8ff8-w79d6"
Jul 28 18:18:39.303926 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:39.303904    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c73c1e-6ff87597b8-hdlcx" event={"ID":"03c0eef1-73e0-4400-a6f7-1936634155a5","Type":"ContainerStarted","Data":"70618a9a84931e77ab54cd4a67b195300176acea4efce1a99f765f5d3152b7b9"}
Jul 28 18:18:39.312804 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:39.312783    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gl6ng\" (UniqueName: \"kubernetes.io/projected/b10f1515-8d9d-4a52-ba48-50c787a5e9b5-kube-api-access-gl6ng\") pod \"payload-processing-e2e-shared-b-c73c1e-668b4b8ff8-w79d6\" (UID: \"b10f1515-8d9d-4a52-ba48-50c787a5e9b5\") " pod="openshift-ingress/payload-processing-e2e-shared-b-c73c1e-668b4b8ff8-w79d6"
Jul 28 18:18:39.392835 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:39.392761    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-c73c1e-668b4b8ff8-w79d6"
Jul 28 18:18:39.403005 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:39.402982    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzlft\" (UniqueName: \"kubernetes.io/projected/fe167dee-9478-42ec-97a6-b291dbb3cf60-kube-api-access-nzlft\") pod \"payload-pre-processing-e2e-shared-b-c73c1e-569f987c9b-jptl4\" (UID: \"fe167dee-9478-42ec-97a6-b291dbb3cf60\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-c73c1e-569f987c9b-jptl4"
Jul 28 18:18:39.403142 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:39.403030    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe167dee-9478-42ec-97a6-b291dbb3cf60-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-c73c1e-569f987c9b-jptl4\" (UID: \"fe167dee-9478-42ec-97a6-b291dbb3cf60\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-c73c1e-569f987c9b-jptl4"
Jul 28 18:18:39.403663 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:39.403643    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe167dee-9478-42ec-97a6-b291dbb3cf60-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-c73c1e-569f987c9b-jptl4\" (UID: \"fe167dee-9478-42ec-97a6-b291dbb3cf60\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-c73c1e-569f987c9b-jptl4"
Jul 28 18:18:39.413831 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:39.413808    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzlft\" (UniqueName: \"kubernetes.io/projected/fe167dee-9478-42ec-97a6-b291dbb3cf60-kube-api-access-nzlft\") pod \"payload-pre-processing-e2e-shared-b-c73c1e-569f987c9b-jptl4\" (UID: \"fe167dee-9478-42ec-97a6-b291dbb3cf60\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-c73c1e-569f987c9b-jptl4"
Jul 28 18:18:39.462756 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:39.462720    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-c73c1e-569f987c9b-jptl4"
Jul 28 18:18:39.541419 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:39.541393    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-c73c1e-668b4b8ff8-w79d6"]
Jul 28 18:18:39.543812 ip-10-0-136-126 kubenswrapper[2563]: W0728 18:18:39.543784    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb10f1515_8d9d_4a52_ba48_50c787a5e9b5.slice/crio-6a9c234f39977e3fbcd38f1c7562e9ec35f56c3ca07ba82c0403bf2f7741c682 WatchSource:0}: Error finding container 6a9c234f39977e3fbcd38f1c7562e9ec35f56c3ca07ba82c0403bf2f7741c682: Status 404 returned error can't find the container with id 6a9c234f39977e3fbcd38f1c7562e9ec35f56c3ca07ba82c0403bf2f7741c682
Jul 28 18:18:39.616094 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:39.616070    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-c73c1e-569f987c9b-jptl4"]
Jul 28 18:18:39.617064 ip-10-0-136-126 kubenswrapper[2563]: W0728 18:18:39.617029    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe167dee_9478_42ec_97a6_b291dbb3cf60.slice/crio-dd172b232f6b98d39dbfba7be117c1f96224a847c2f63416697b97327673a8c2 WatchSource:0}: Error finding container dd172b232f6b98d39dbfba7be117c1f96224a847c2f63416697b97327673a8c2: Status 404 returned error can't find the container with id dd172b232f6b98d39dbfba7be117c1f96224a847c2f63416697b97327673a8c2
Jul 28 18:18:40.311254 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:40.311218    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-c73c1e-668b4b8ff8-w79d6" event={"ID":"b10f1515-8d9d-4a52-ba48-50c787a5e9b5","Type":"ContainerStarted","Data":"19257d4ac1aaab6fc037f6c44ef7bc4f728b5f8a00e71daddee306de53221730"}
Jul 28 18:18:40.311723 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:40.311278    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-c73c1e-668b4b8ff8-w79d6" event={"ID":"b10f1515-8d9d-4a52-ba48-50c787a5e9b5","Type":"ContainerStarted","Data":"6a9c234f39977e3fbcd38f1c7562e9ec35f56c3ca07ba82c0403bf2f7741c682"}
Jul 28 18:18:40.311723 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:40.311396    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-c73c1e-668b4b8ff8-w79d6"
Jul 28 18:18:40.312886 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:40.312852    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-c73c1e-569f987c9b-jptl4" event={"ID":"fe167dee-9478-42ec-97a6-b291dbb3cf60","Type":"ContainerStarted","Data":"201e9440c1e8cd22bc09fdd7780872769aceaec4360953fcdf07a4ba673cd52a"}
Jul 28 18:18:40.313033 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:40.312905    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-c73c1e-569f987c9b-jptl4" event={"ID":"fe167dee-9478-42ec-97a6-b291dbb3cf60","Type":"ContainerStarted","Data":"dd172b232f6b98d39dbfba7be117c1f96224a847c2f63416697b97327673a8c2"}
Jul 28 18:18:40.313033 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:40.312934    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-c73c1e-569f987c9b-jptl4"
Jul 28 18:18:40.314401 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:40.314379    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c73c1e-6ff87597b8-hdlcx" event={"ID":"03c0eef1-73e0-4400-a6f7-1936634155a5","Type":"ContainerStarted","Data":"53e694a4701d67eda16ef3c91c4cf481470bda4c0837dbdb037feec2b0edb1db"}
Jul 28 18:18:40.314537 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:40.314417    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c73c1e-6ff87597b8-hdlcx"
Jul 28 18:18:40.330333 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:40.330295    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-c73c1e-668b4b8ff8-w79d6" podStartSLOduration=1.3302822349999999 podStartE2EDuration="1.330282235s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:18:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:18:40.329912643 +0000 UTC m=+2658.991069001" watchObservedRunningTime="2026-07-28 18:18:40.330282235 +0000 UTC m=+2658.991438591"
Jul 28 18:18:40.354780 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:40.354741    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-c73c1e-569f987c9b-jptl4" podStartSLOduration=1.354729045 podStartE2EDuration="1.354729045s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:18:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:18:40.354368404 +0000 UTC m=+2659.015524757" watchObservedRunningTime="2026-07-28 18:18:40.354729045 +0000 UTC m=+2659.015885400"
Jul 28 18:18:40.378424 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:40.378378    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c73c1e-6ff87597b8-hdlcx" podStartSLOduration=1.959838682 podStartE2EDuration="2.378365515s" totalImagesPullingTime="418.526833ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:18:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:18:40.376574574 +0000 UTC m=+2659.037730928" watchObservedRunningTime="2026-07-28 18:18:40.378365515 +0000 UTC m=+2659.039521904"
Jul 28 18:18:46.326040 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:46.326007    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c73c1e-6ff87597b8-hdlcx"
Jul 28 18:18:51.322538 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:51.322497    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-c73c1e-668b4b8ff8-w79d6"
Jul 28 18:18:51.323108 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:18:51.322606    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-c73c1e-569f987c9b-jptl4"
Jul 28 18:19:03.063791 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:19:03.063745    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:19:09.602478 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:19:09.602410    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:19:38.330587 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:19:38.330548    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:20:34.557923 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:20:34.557889    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:21:22.241811 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:21:22.241783    2563 scope.go:117] "RemoveContainer" containerID="1aa742324e05e69035a3c5cb5343b4b161b6ce9addf432e226a985743e5040ba"
Jul 28 18:21:22.251819 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:21:22.251799    2563 scope.go:117] "RemoveContainer" containerID="18fb68b71d037fbe6bb853ccc3783bb3d5bb5b91d0bec95699df246fc8a95b80"
Jul 28 18:21:52.118991 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:21:52.118945    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:22:22.276422 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:22:22.276333    2563 scope.go:117] "RemoveContainer" containerID="c1c511d36298c4cf3a35fbab9aeb60f502ae77afdb2256b35fbc8dfc417f86fe"
Jul 28 18:22:22.285315 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:22:22.285294    2563 scope.go:117] "RemoveContainer" containerID="4bb207244627a090f8db625ab1fec7c71530cd6c819571abcaeef3c04b448e33"
Jul 28 18:22:42.698463 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:22:42.698414    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:23:36.630636 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:23:36.630601    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:24:22.317220 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:24:22.317185    2563 scope.go:117] "RemoveContainer" containerID="f1ca33f6ec39d26aa3b3f09d091684c87205b5e5b6b44ee24b9aaeaed4bc409e"
Jul 28 18:24:22.325916 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:24:22.325893    2563 scope.go:117] "RemoveContainer" containerID="e39581aa97ad46e29bb3df4c66b20def811d50c1112b0148d0dbd1489c997cf7"
Jul 28 18:24:22.423305 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:24:22.423274    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:24:23.789101 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:24:23.789055    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:24:31.850770 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:24:31.850739    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:24:33.653444 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:24:33.653398    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:24:59.403860 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:24:59.403828    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:25:43.002611 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:25:43.002577    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:25:58.924970 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:25:58.924931    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:25:59.927712 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:25:59.927682    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:26:04.126764 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:26:04.126719    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:26:27.324692 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:26:27.324656    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:26:41.051919 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:26:41.051886    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:26:58.088259 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:26:58.088179    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-96ffg"]
Jul 28 18:26:58.091735 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:26:58.091714    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-96ffg"
Jul 28 18:26:58.094842 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:26:58.094823    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-lwx2w\""
Jul 28 18:26:58.101876 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:26:58.101854    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-96ffg"]
Jul 28 18:26:58.107515 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:26:58.107492    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qkkj\" (UniqueName: \"kubernetes.io/projected/63e0bf18-8468-4efb-8684-31ee5f7e5543-kube-api-access-9qkkj\") pod \"kuadrant-operator-catalog-96ffg\" (UID: \"63e0bf18-8468-4efb-8684-31ee5f7e5543\") " pod="kuadrant-system/kuadrant-operator-catalog-96ffg"
Jul 28 18:26:58.208066 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:26:58.208037    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qkkj\" (UniqueName: \"kubernetes.io/projected/63e0bf18-8468-4efb-8684-31ee5f7e5543-kube-api-access-9qkkj\") pod \"kuadrant-operator-catalog-96ffg\" (UID: \"63e0bf18-8468-4efb-8684-31ee5f7e5543\") " pod="kuadrant-system/kuadrant-operator-catalog-96ffg"
Jul 28 18:26:58.218264 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:26:58.218242    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qkkj\" (UniqueName: \"kubernetes.io/projected/63e0bf18-8468-4efb-8684-31ee5f7e5543-kube-api-access-9qkkj\") pod \"kuadrant-operator-catalog-96ffg\" (UID: \"63e0bf18-8468-4efb-8684-31ee5f7e5543\") " pod="kuadrant-system/kuadrant-operator-catalog-96ffg"
Jul 28 18:26:58.401958 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:26:58.401858    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-96ffg"
Jul 28 18:26:58.538402 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:26:58.538369    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-96ffg"]
Jul 28 18:26:58.539746 ip-10-0-136-126 kubenswrapper[2563]: W0728 18:26:58.539718    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63e0bf18_8468_4efb_8684_31ee5f7e5543.slice/crio-97277ec01b248f138b7f3846a123c2f7350120032bd768c03744bb8c6ed77d1e WatchSource:0}: Error finding container 97277ec01b248f138b7f3846a123c2f7350120032bd768c03744bb8c6ed77d1e: Status 404 returned error can't find the container with id 97277ec01b248f138b7f3846a123c2f7350120032bd768c03744bb8c6ed77d1e
Jul 28 18:26:58.541154 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:26:58.541134    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 18:26:59.039580 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:26:59.039542    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-96ffg" event={"ID":"63e0bf18-8468-4efb-8684-31ee5f7e5543","Type":"ContainerStarted","Data":"97277ec01b248f138b7f3846a123c2f7350120032bd768c03744bb8c6ed77d1e"}
Jul 28 18:27:01.053808 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:01.053768    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-96ffg" event={"ID":"63e0bf18-8468-4efb-8684-31ee5f7e5543","Type":"ContainerStarted","Data":"9bc432645a66425729ccf926c7d0b571b96809937f545e8adea5809b05c1dfff"}
Jul 28 18:27:01.073136 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:01.073093    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-96ffg" podStartSLOduration=1.388879708 podStartE2EDuration="3.073078288s" totalImagesPullingTime="1.68419858s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:26:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:27:01.071736117 +0000 UTC m=+3159.732892473" watchObservedRunningTime="2026-07-28 18:27:01.073078288 +0000 UTC m=+3159.734234643"
Jul 28 18:27:08.402826 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:08.402788    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-96ffg"
Jul 28 18:27:08.403321 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:08.402850    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-96ffg"
Jul 28 18:27:08.425594 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:08.425567    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-96ffg"
Jul 28 18:27:09.117202 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:09.117166    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-96ffg"
Jul 28 18:27:09.163817 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:09.163784    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-96ffg"]
Jul 28 18:27:11.106921 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:11.106883    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-96ffg" podUID="63e0bf18-8468-4efb-8684-31ee5f7e5543" containerName="registry-server" containerID="cri-o://9bc432645a66425729ccf926c7d0b571b96809937f545e8adea5809b05c1dfff" gracePeriod=2
Jul 28 18:27:11.367383 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:11.367325    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-96ffg"
Jul 28 18:27:11.428195 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:11.428159    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9qkkj\" (UniqueName: \"kubernetes.io/projected/63e0bf18-8468-4efb-8684-31ee5f7e5543-kube-api-access-9qkkj\") pod \"63e0bf18-8468-4efb-8684-31ee5f7e5543\" (UID: \"63e0bf18-8468-4efb-8684-31ee5f7e5543\") "
Jul 28 18:27:11.430242 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:11.430215    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63e0bf18-8468-4efb-8684-31ee5f7e5543-kube-api-access-9qkkj" (OuterVolumeSpecName: "kube-api-access-9qkkj") pod "63e0bf18-8468-4efb-8684-31ee5f7e5543" (UID: "63e0bf18-8468-4efb-8684-31ee5f7e5543"). InnerVolumeSpecName "kube-api-access-9qkkj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:27:11.529570 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:11.529536    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9qkkj\" (UniqueName: \"kubernetes.io/projected/63e0bf18-8468-4efb-8684-31ee5f7e5543-kube-api-access-9qkkj\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:27:12.113523 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:12.113414    2563 generic.go:356] "Generic (PLEG): container finished" podID="63e0bf18-8468-4efb-8684-31ee5f7e5543" containerID="9bc432645a66425729ccf926c7d0b571b96809937f545e8adea5809b05c1dfff" exitCode=0
Jul 28 18:27:12.113523 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:12.113486    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-96ffg"
Jul 28 18:27:12.113523 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:12.113496    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-96ffg" event={"ID":"63e0bf18-8468-4efb-8684-31ee5f7e5543","Type":"ContainerDied","Data":"9bc432645a66425729ccf926c7d0b571b96809937f545e8adea5809b05c1dfff"}
Jul 28 18:27:12.114048 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:12.113555    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-96ffg" event={"ID":"63e0bf18-8468-4efb-8684-31ee5f7e5543","Type":"ContainerDied","Data":"97277ec01b248f138b7f3846a123c2f7350120032bd768c03744bb8c6ed77d1e"}
Jul 28 18:27:12.114048 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:12.113579    2563 scope.go:117] "RemoveContainer" containerID="9bc432645a66425729ccf926c7d0b571b96809937f545e8adea5809b05c1dfff"
Jul 28 18:27:12.123244 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:12.123224    2563 scope.go:117] "RemoveContainer" containerID="9bc432645a66425729ccf926c7d0b571b96809937f545e8adea5809b05c1dfff"
Jul 28 18:27:12.123522 ip-10-0-136-126 kubenswrapper[2563]: E0728 18:27:12.123502    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9bc432645a66425729ccf926c7d0b571b96809937f545e8adea5809b05c1dfff\": container with ID starting with 9bc432645a66425729ccf926c7d0b571b96809937f545e8adea5809b05c1dfff not found: ID does not exist" containerID="9bc432645a66425729ccf926c7d0b571b96809937f545e8adea5809b05c1dfff"
Jul 28 18:27:12.123602 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:12.123535    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bc432645a66425729ccf926c7d0b571b96809937f545e8adea5809b05c1dfff"} err="failed to get container status \"9bc432645a66425729ccf926c7d0b571b96809937f545e8adea5809b05c1dfff\": rpc error: code = NotFound desc = could not find container \"9bc432645a66425729ccf926c7d0b571b96809937f545e8adea5809b05c1dfff\": container with ID starting with 9bc432645a66425729ccf926c7d0b571b96809937f545e8adea5809b05c1dfff not found: ID does not exist"
Jul 28 18:27:12.135535 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:12.135509    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-96ffg"]
Jul 28 18:27:12.140666 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:12.140645    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-96ffg"]
Jul 28 18:27:13.853695 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:13.853663    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63e0bf18-8468-4efb-8684-31ee5f7e5543" path="/var/lib/kubelet/pods/63e0bf18-8468-4efb-8684-31ee5f7e5543/volumes"
Jul 28 18:27:49.019121 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:49.019087    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:27:49.916276 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:49.916241    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:27:50.902987 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:50.902954    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7734-139928809554480"]
Jul 28 18:27:50.903354 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:50.903311    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63e0bf18-8468-4efb-8684-31ee5f7e5543" containerName="registry-server"
Jul 28 18:27:50.903354 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:50.903322    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="63e0bf18-8468-4efb-8684-31ee5f7e5543" containerName="registry-server"
Jul 28 18:27:50.903457 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:50.903391    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="63e0bf18-8468-4efb-8684-31ee5f7e5543" containerName="registry-server"
Jul 28 18:27:50.906555 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:50.906538    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7734-139928809554480"
Jul 28 18:27:50.909136 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:50.909115    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-87q2r\""
Jul 28 18:27:50.912785 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:50.912762    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7734-139928809554480"]
Jul 28 18:27:50.980161 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:50.980132    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzw6r\" (UniqueName: \"kubernetes.io/projected/8bb9bba3-a2d1-42aa-a08b-956a6be45e21-kube-api-access-mzw6r\") pod \"test-curl-7734-139928809554480\" (UID: \"8bb9bba3-a2d1-42aa-a08b-956a6be45e21\") " pod="opendatahub/test-curl-7734-139928809554480"
Jul 28 18:27:51.081562 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:51.081530    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzw6r\" (UniqueName: \"kubernetes.io/projected/8bb9bba3-a2d1-42aa-a08b-956a6be45e21-kube-api-access-mzw6r\") pod \"test-curl-7734-139928809554480\" (UID: \"8bb9bba3-a2d1-42aa-a08b-956a6be45e21\") " pod="opendatahub/test-curl-7734-139928809554480"
Jul 28 18:27:51.091486 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:51.091456    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzw6r\" (UniqueName: \"kubernetes.io/projected/8bb9bba3-a2d1-42aa-a08b-956a6be45e21-kube-api-access-mzw6r\") pod \"test-curl-7734-139928809554480\" (UID: \"8bb9bba3-a2d1-42aa-a08b-956a6be45e21\") " pod="opendatahub/test-curl-7734-139928809554480"
Jul 28 18:27:51.217532 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:51.217454    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7734-139928809554480"
Jul 28 18:27:51.356410 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:51.356385    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7734-139928809554480"]
Jul 28 18:27:51.357904 ip-10-0-136-126 kubenswrapper[2563]: W0728 18:27:51.357878    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8bb9bba3_a2d1_42aa_a08b_956a6be45e21.slice/crio-33cb72d6558a7b71a420d2151e032a7ccd1ae105b0b162ef3ca2c08cad424455 WatchSource:0}: Error finding container 33cb72d6558a7b71a420d2151e032a7ccd1ae105b0b162ef3ca2c08cad424455: Status 404 returned error can't find the container with id 33cb72d6558a7b71a420d2151e032a7ccd1ae105b0b162ef3ca2c08cad424455
Jul 28 18:27:52.335638 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:52.335605    2563 generic.go:356] "Generic (PLEG): container finished" podID="8bb9bba3-a2d1-42aa-a08b-956a6be45e21" containerID="c0664c45551254503657069fa124b3bbe3c14bb068892bf8b30e9b9a7a549722" exitCode=0
Jul 28 18:27:52.336011 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:52.335698    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7734-139928809554480" event={"ID":"8bb9bba3-a2d1-42aa-a08b-956a6be45e21","Type":"ContainerDied","Data":"c0664c45551254503657069fa124b3bbe3c14bb068892bf8b30e9b9a7a549722"}
Jul 28 18:27:52.336011 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:52.335741    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7734-139928809554480" event={"ID":"8bb9bba3-a2d1-42aa-a08b-956a6be45e21","Type":"ContainerStarted","Data":"33cb72d6558a7b71a420d2151e032a7ccd1ae105b0b162ef3ca2c08cad424455"}
Jul 28 18:27:53.474910 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:53.474886    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7734-139928809554480"
Jul 28 18:27:53.497812 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:53.497785    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7734-139928809554480_8bb9bba3-a2d1-42aa-a08b-956a6be45e21/test-curl-7734-139928809554480/0.log"
Jul 28 18:27:53.505940 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:53.505917    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mzw6r\" (UniqueName: \"kubernetes.io/projected/8bb9bba3-a2d1-42aa-a08b-956a6be45e21-kube-api-access-mzw6r\") pod \"8bb9bba3-a2d1-42aa-a08b-956a6be45e21\" (UID: \"8bb9bba3-a2d1-42aa-a08b-956a6be45e21\") "
Jul 28 18:27:53.508125 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:53.508098    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bb9bba3-a2d1-42aa-a08b-956a6be45e21-kube-api-access-mzw6r" (OuterVolumeSpecName: "kube-api-access-mzw6r") pod "8bb9bba3-a2d1-42aa-a08b-956a6be45e21" (UID: "8bb9bba3-a2d1-42aa-a08b-956a6be45e21"). InnerVolumeSpecName "kube-api-access-mzw6r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:27:53.528351 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:53.528322    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7734-139928809554480"]
Jul 28 18:27:53.535059 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:53.535035    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7734-139928809554480"]
Jul 28 18:27:53.606592 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:53.606511    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mzw6r\" (UniqueName: \"kubernetes.io/projected/8bb9bba3-a2d1-42aa-a08b-956a6be45e21-kube-api-access-mzw6r\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:27:53.663631 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:53.663592    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7734-139928809587120"]
Jul 28 18:27:53.663983 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:53.663970    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bb9bba3-a2d1-42aa-a08b-956a6be45e21" containerName="test-curl-7734-139928809554480"
Jul 28 18:27:53.664027 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:53.663985    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bb9bba3-a2d1-42aa-a08b-956a6be45e21" containerName="test-curl-7734-139928809554480"
Jul 28 18:27:53.664077 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:53.664067    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bb9bba3-a2d1-42aa-a08b-956a6be45e21" containerName="test-curl-7734-139928809554480"
Jul 28 18:27:53.667078 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:53.667057    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7734-139928809587120"
Jul 28 18:27:53.671897 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:53.671872    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7734-139928809587120"]
Jul 28 18:27:53.707707 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:53.707672    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgzjg\" (UniqueName: \"kubernetes.io/projected/ab08bc0c-cd60-400d-8f2b-e48e8ba5ed86-kube-api-access-jgzjg\") pod \"test-curl-7734-139928809587120\" (UID: \"ab08bc0c-cd60-400d-8f2b-e48e8ba5ed86\") " pod="opendatahub/test-curl-7734-139928809587120"
Jul 28 18:27:53.808780 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:53.808739    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgzjg\" (UniqueName: \"kubernetes.io/projected/ab08bc0c-cd60-400d-8f2b-e48e8ba5ed86-kube-api-access-jgzjg\") pod \"test-curl-7734-139928809587120\" (UID: \"ab08bc0c-cd60-400d-8f2b-e48e8ba5ed86\") " pod="opendatahub/test-curl-7734-139928809587120"
Jul 28 18:27:53.818223 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:53.818196    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgzjg\" (UniqueName: \"kubernetes.io/projected/ab08bc0c-cd60-400d-8f2b-e48e8ba5ed86-kube-api-access-jgzjg\") pod \"test-curl-7734-139928809587120\" (UID: \"ab08bc0c-cd60-400d-8f2b-e48e8ba5ed86\") " pod="opendatahub/test-curl-7734-139928809587120"
Jul 28 18:27:53.854888 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:53.854858    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8bb9bba3-a2d1-42aa-a08b-956a6be45e21" path="/var/lib/kubelet/pods/8bb9bba3-a2d1-42aa-a08b-956a6be45e21/volumes"
Jul 28 18:27:53.978499 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:53.978462    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7734-139928809587120"
Jul 28 18:27:54.109032 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:54.109000    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7734-139928809587120"]
Jul 28 18:27:54.111461 ip-10-0-136-126 kubenswrapper[2563]: W0728 18:27:54.111408    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podab08bc0c_cd60_400d_8f2b_e48e8ba5ed86.slice/crio-df6b3663e49f428ae91f3f67b7fcacd135007dcae06f5cee304e31c61852aeef WatchSource:0}: Error finding container df6b3663e49f428ae91f3f67b7fcacd135007dcae06f5cee304e31c61852aeef: Status 404 returned error can't find the container with id df6b3663e49f428ae91f3f67b7fcacd135007dcae06f5cee304e31c61852aeef
Jul 28 18:27:54.347459 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:54.347413    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7734-139928809587120" event={"ID":"ab08bc0c-cd60-400d-8f2b-e48e8ba5ed86","Type":"ContainerStarted","Data":"df6b3663e49f428ae91f3f67b7fcacd135007dcae06f5cee304e31c61852aeef"}
Jul 28 18:27:54.348606 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:54.348572    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7734-139928809554480"
Jul 28 18:27:54.348606 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:54.348597    2563 scope.go:117] "RemoveContainer" containerID="c0664c45551254503657069fa124b3bbe3c14bb068892bf8b30e9b9a7a549722"
Jul 28 18:27:55.355422 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:55.355388    2563 generic.go:356] "Generic (PLEG): container finished" podID="ab08bc0c-cd60-400d-8f2b-e48e8ba5ed86" containerID="840b2d7713ebed09da275e425eb87aa72cf43827f95f956b284f1eb2f159cd8d" exitCode=0
Jul 28 18:27:55.355915 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:55.355475    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7734-139928809587120" event={"ID":"ab08bc0c-cd60-400d-8f2b-e48e8ba5ed86","Type":"ContainerDied","Data":"840b2d7713ebed09da275e425eb87aa72cf43827f95f956b284f1eb2f159cd8d"}
Jul 28 18:27:56.501860 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:56.501835    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7734-139928809587120"
Jul 28 18:27:56.525344 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:56.525317    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7734-139928809587120_ab08bc0c-cd60-400d-8f2b-e48e8ba5ed86/test-curl-7734-139928809587120/0.log"
Jul 28 18:27:56.531624 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:56.531594    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jgzjg\" (UniqueName: \"kubernetes.io/projected/ab08bc0c-cd60-400d-8f2b-e48e8ba5ed86-kube-api-access-jgzjg\") pod \"ab08bc0c-cd60-400d-8f2b-e48e8ba5ed86\" (UID: \"ab08bc0c-cd60-400d-8f2b-e48e8ba5ed86\") "
Jul 28 18:27:56.533766 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:56.533733    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab08bc0c-cd60-400d-8f2b-e48e8ba5ed86-kube-api-access-jgzjg" (OuterVolumeSpecName: "kube-api-access-jgzjg") pod "ab08bc0c-cd60-400d-8f2b-e48e8ba5ed86" (UID: "ab08bc0c-cd60-400d-8f2b-e48e8ba5ed86"). InnerVolumeSpecName "kube-api-access-jgzjg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:27:56.554800 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:56.554765    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7734-139928809587120"]
Jul 28 18:27:56.558960 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:56.558937    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7734-139928809587120"]
Jul 28 18:27:56.632737 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:56.632637    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jgzjg\" (UniqueName: \"kubernetes.io/projected/ab08bc0c-cd60-400d-8f2b-e48e8ba5ed86-kube-api-access-jgzjg\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:27:56.739555 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:56.739518    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7734-139928809629712"]
Jul 28 18:27:56.739922 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:56.739908    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab08bc0c-cd60-400d-8f2b-e48e8ba5ed86" containerName="test-curl-7734-139928809587120"
Jul 28 18:27:56.739969 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:56.739924    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab08bc0c-cd60-400d-8f2b-e48e8ba5ed86" containerName="test-curl-7734-139928809587120"
Jul 28 18:27:56.740013 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:56.740002    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab08bc0c-cd60-400d-8f2b-e48e8ba5ed86" containerName="test-curl-7734-139928809587120"
Jul 28 18:27:56.743081 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:56.743063    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7734-139928809629712"
Jul 28 18:27:56.750577 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:56.750545    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7734-139928809629712"]
Jul 28 18:27:56.835321 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:56.835276    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wgcp\" (UniqueName: \"kubernetes.io/projected/3e939354-b515-4507-a2e8-39827f2c2a55-kube-api-access-4wgcp\") pod \"test-curl-7734-139928809629712\" (UID: \"3e939354-b515-4507-a2e8-39827f2c2a55\") " pod="opendatahub/test-curl-7734-139928809629712"
Jul 28 18:27:56.936470 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:56.936341    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wgcp\" (UniqueName: \"kubernetes.io/projected/3e939354-b515-4507-a2e8-39827f2c2a55-kube-api-access-4wgcp\") pod \"test-curl-7734-139928809629712\" (UID: \"3e939354-b515-4507-a2e8-39827f2c2a55\") " pod="opendatahub/test-curl-7734-139928809629712"
Jul 28 18:27:56.951045 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:56.951011    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wgcp\" (UniqueName: \"kubernetes.io/projected/3e939354-b515-4507-a2e8-39827f2c2a55-kube-api-access-4wgcp\") pod \"test-curl-7734-139928809629712\" (UID: \"3e939354-b515-4507-a2e8-39827f2c2a55\") " pod="opendatahub/test-curl-7734-139928809629712"
Jul 28 18:27:57.055092 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:57.055048    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7734-139928809629712"
Jul 28 18:27:57.202034 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:57.202004    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7734-139928809629712"]
Jul 28 18:27:57.203925 ip-10-0-136-126 kubenswrapper[2563]: W0728 18:27:57.203892    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3e939354_b515_4507_a2e8_39827f2c2a55.slice/crio-581b8a4532e7e9fdf22639059f3b486c5b58fe1722c64aaf1ee26cc7b92c39fe WatchSource:0}: Error finding container 581b8a4532e7e9fdf22639059f3b486c5b58fe1722c64aaf1ee26cc7b92c39fe: Status 404 returned error can't find the container with id 581b8a4532e7e9fdf22639059f3b486c5b58fe1722c64aaf1ee26cc7b92c39fe
Jul 28 18:27:57.374123 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:57.374093    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7734-139928809587120"
Jul 28 18:27:57.374317 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:57.374127    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="df6b3663e49f428ae91f3f67b7fcacd135007dcae06f5cee304e31c61852aeef"
Jul 28 18:27:57.375330 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:57.375302    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7734-139928809629712" event={"ID":"3e939354-b515-4507-a2e8-39827f2c2a55","Type":"ContainerStarted","Data":"581b8a4532e7e9fdf22639059f3b486c5b58fe1722c64aaf1ee26cc7b92c39fe"}
Jul 28 18:27:57.385543 ip-10-0-136-126 kubenswrapper[2563]: E0728 18:27:57.385518    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7734-139928809587120\" is forbidden: User \"system:node:ip-10-0-136-126.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-126.ec2.internal' and this object" podUID="ab08bc0c-cd60-400d-8f2b-e48e8ba5ed86" pod="opendatahub/test-curl-7734-139928809587120"
Jul 28 18:27:57.854294 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:57.854252    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab08bc0c-cd60-400d-8f2b-e48e8ba5ed86" path="/var/lib/kubelet/pods/ab08bc0c-cd60-400d-8f2b-e48e8ba5ed86/volumes"
Jul 28 18:27:58.382090 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:58.382052    2563 generic.go:356] "Generic (PLEG): container finished" podID="3e939354-b515-4507-a2e8-39827f2c2a55" containerID="1cd913c740be27de7c2ee85138b2d13523a3013b6b292f00ec96e13cd89c06f1" exitCode=0
Jul 28 18:27:58.382276 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:58.382137    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7734-139928809629712" event={"ID":"3e939354-b515-4507-a2e8-39827f2c2a55","Type":"ContainerDied","Data":"1cd913c740be27de7c2ee85138b2d13523a3013b6b292f00ec96e13cd89c06f1"}
Jul 28 18:27:59.526864 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:59.526835    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7734-139928809629712"
Jul 28 18:27:59.550303 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:59.550277    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7734-139928809629712_3e939354-b515-4507-a2e8-39827f2c2a55/test-curl-7734-139928809629712/0.log"
Jul 28 18:27:59.559401 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:59.559366    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4wgcp\" (UniqueName: \"kubernetes.io/projected/3e939354-b515-4507-a2e8-39827f2c2a55-kube-api-access-4wgcp\") pod \"3e939354-b515-4507-a2e8-39827f2c2a55\" (UID: \"3e939354-b515-4507-a2e8-39827f2c2a55\") "
Jul 28 18:27:59.561484 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:59.561456    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e939354-b515-4507-a2e8-39827f2c2a55-kube-api-access-4wgcp" (OuterVolumeSpecName: "kube-api-access-4wgcp") pod "3e939354-b515-4507-a2e8-39827f2c2a55" (UID: "3e939354-b515-4507-a2e8-39827f2c2a55"). InnerVolumeSpecName "kube-api-access-4wgcp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:27:59.578644 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:59.578617    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7734-139928809629712"]
Jul 28 18:27:59.581603 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:59.581579    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7734-139928809629712"]
Jul 28 18:27:59.660616 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:59.660520    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4wgcp\" (UniqueName: \"kubernetes.io/projected/3e939354-b515-4507-a2e8-39827f2c2a55-kube-api-access-4wgcp\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:27:59.705557 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:59.705524    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7734-139928809629712"]
Jul 28 18:27:59.705912 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:59.705900    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e939354-b515-4507-a2e8-39827f2c2a55" containerName="test-curl-7734-139928809629712"
Jul 28 18:27:59.705969 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:59.705914    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e939354-b515-4507-a2e8-39827f2c2a55" containerName="test-curl-7734-139928809629712"
Jul 28 18:27:59.706017 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:59.705993    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e939354-b515-4507-a2e8-39827f2c2a55" containerName="test-curl-7734-139928809629712"
Jul 28 18:27:59.708738 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:59.708723    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7734-139928809629712"
Jul 28 18:27:59.716654 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:59.716629    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7734-139928809629712"]
Jul 28 18:27:59.761992 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:59.761964    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfgq8\" (UniqueName: \"kubernetes.io/projected/4100460e-f605-4128-8d67-962fc4a65d37-kube-api-access-bfgq8\") pod \"test-curl-7734-139928809629712\" (UID: \"4100460e-f605-4128-8d67-962fc4a65d37\") " pod="opendatahub/test-curl-7734-139928809629712"
Jul 28 18:27:59.854283 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:59.854248    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e939354-b515-4507-a2e8-39827f2c2a55" path="/var/lib/kubelet/pods/3e939354-b515-4507-a2e8-39827f2c2a55/volumes"
Jul 28 18:27:59.863284 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:59.863257    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfgq8\" (UniqueName: \"kubernetes.io/projected/4100460e-f605-4128-8d67-962fc4a65d37-kube-api-access-bfgq8\") pod \"test-curl-7734-139928809629712\" (UID: \"4100460e-f605-4128-8d67-962fc4a65d37\") " pod="opendatahub/test-curl-7734-139928809629712"
Jul 28 18:27:59.872732 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:27:59.872713    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfgq8\" (UniqueName: \"kubernetes.io/projected/4100460e-f605-4128-8d67-962fc4a65d37-kube-api-access-bfgq8\") pod \"test-curl-7734-139928809629712\" (UID: \"4100460e-f605-4128-8d67-962fc4a65d37\") " pod="opendatahub/test-curl-7734-139928809629712"
Jul 28 18:28:00.019689 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:00.019609    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7734-139928809629712"
Jul 28 18:28:00.154224 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:00.154199    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7734-139928809629712"]
Jul 28 18:28:00.155758 ip-10-0-136-126 kubenswrapper[2563]: W0728 18:28:00.155732    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4100460e_f605_4128_8d67_962fc4a65d37.slice/crio-d52f1b0414f13e60997fc8e2898275b54134a9584793202a42d45212e467a117 WatchSource:0}: Error finding container d52f1b0414f13e60997fc8e2898275b54134a9584793202a42d45212e467a117: Status 404 returned error can't find the container with id d52f1b0414f13e60997fc8e2898275b54134a9584793202a42d45212e467a117
Jul 28 18:28:00.394263 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:00.394228    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7734-139928809629712" event={"ID":"4100460e-f605-4128-8d67-962fc4a65d37","Type":"ContainerStarted","Data":"d52f1b0414f13e60997fc8e2898275b54134a9584793202a42d45212e467a117"}
Jul 28 18:28:00.395576 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:00.395560    2563 scope.go:117] "RemoveContainer" containerID="1cd913c740be27de7c2ee85138b2d13523a3013b6b292f00ec96e13cd89c06f1"
Jul 28 18:28:00.395669 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:00.395581    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7734-139928809629712"
Jul 28 18:28:01.403819 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:01.403785    2563 generic.go:356] "Generic (PLEG): container finished" podID="4100460e-f605-4128-8d67-962fc4a65d37" containerID="4d3d0c43eb9d1c29d613fbd688db3ec1b8a25d2066b58eac31181e33ee52a9c0" exitCode=0
Jul 28 18:28:01.404194 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:01.403865    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7734-139928809629712" event={"ID":"4100460e-f605-4128-8d67-962fc4a65d37","Type":"ContainerDied","Data":"4d3d0c43eb9d1c29d613fbd688db3ec1b8a25d2066b58eac31181e33ee52a9c0"}
Jul 28 18:28:02.558825 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:02.558799    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7734-139928809629712"
Jul 28 18:28:02.583866 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:02.583837    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7734-139928809629712_4100460e-f605-4128-8d67-962fc4a65d37/test-curl-7734-139928809629712/0.log"
Jul 28 18:28:02.587059 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:02.587040    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bfgq8\" (UniqueName: \"kubernetes.io/projected/4100460e-f605-4128-8d67-962fc4a65d37-kube-api-access-bfgq8\") pod \"4100460e-f605-4128-8d67-962fc4a65d37\" (UID: \"4100460e-f605-4128-8d67-962fc4a65d37\") "
Jul 28 18:28:02.589076 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:02.589053    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4100460e-f605-4128-8d67-962fc4a65d37-kube-api-access-bfgq8" (OuterVolumeSpecName: "kube-api-access-bfgq8") pod "4100460e-f605-4128-8d67-962fc4a65d37" (UID: "4100460e-f605-4128-8d67-962fc4a65d37"). InnerVolumeSpecName "kube-api-access-bfgq8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:28:02.612286 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:02.612256    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7734-139928809629712"]
Jul 28 18:28:02.617379 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:02.617352    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7734-139928809629712"]
Jul 28 18:28:02.687835 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:02.687748    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bfgq8\" (UniqueName: \"kubernetes.io/projected/4100460e-f605-4128-8d67-962fc4a65d37-kube-api-access-bfgq8\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:28:02.747319 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:02.747283    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7734-139928809630720"]
Jul 28 18:28:02.747710 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:02.747696    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4100460e-f605-4128-8d67-962fc4a65d37" containerName="test-curl-7734-139928809629712"
Jul 28 18:28:02.747765 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:02.747712    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4100460e-f605-4128-8d67-962fc4a65d37" containerName="test-curl-7734-139928809629712"
Jul 28 18:28:02.747799 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:02.747792    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4100460e-f605-4128-8d67-962fc4a65d37" containerName="test-curl-7734-139928809629712"
Jul 28 18:28:02.750803 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:02.750781    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7734-139928809630720"
Jul 28 18:28:02.758144 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:02.758115    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7734-139928809630720"]
Jul 28 18:28:02.788567 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:02.788525    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c84jc\" (UniqueName: \"kubernetes.io/projected/2c918e2a-a91e-46fc-bdf5-accb0dce3071-kube-api-access-c84jc\") pod \"test-curl-7734-139928809630720\" (UID: \"2c918e2a-a91e-46fc-bdf5-accb0dce3071\") " pod="opendatahub/test-curl-7734-139928809630720"
Jul 28 18:28:02.889387 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:02.889349    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c84jc\" (UniqueName: \"kubernetes.io/projected/2c918e2a-a91e-46fc-bdf5-accb0dce3071-kube-api-access-c84jc\") pod \"test-curl-7734-139928809630720\" (UID: \"2c918e2a-a91e-46fc-bdf5-accb0dce3071\") " pod="opendatahub/test-curl-7734-139928809630720"
Jul 28 18:28:02.899851 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:02.899823    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c84jc\" (UniqueName: \"kubernetes.io/projected/2c918e2a-a91e-46fc-bdf5-accb0dce3071-kube-api-access-c84jc\") pod \"test-curl-7734-139928809630720\" (UID: \"2c918e2a-a91e-46fc-bdf5-accb0dce3071\") " pod="opendatahub/test-curl-7734-139928809630720"
Jul 28 18:28:03.062881 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:03.062774    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7734-139928809630720"
Jul 28 18:28:03.196782 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:03.196754    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7734-139928809630720"]
Jul 28 18:28:03.197846 ip-10-0-136-126 kubenswrapper[2563]: W0728 18:28:03.197823    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2c918e2a_a91e_46fc_bdf5_accb0dce3071.slice/crio-765a5a012e7ef84a2b884e4a393ab394d3e347e7f27d06e2373e323d8a7f662d WatchSource:0}: Error finding container 765a5a012e7ef84a2b884e4a393ab394d3e347e7f27d06e2373e323d8a7f662d: Status 404 returned error can't find the container with id 765a5a012e7ef84a2b884e4a393ab394d3e347e7f27d06e2373e323d8a7f662d
Jul 28 18:28:03.415761 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:03.415726    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7734-139928809630720" event={"ID":"2c918e2a-a91e-46fc-bdf5-accb0dce3071","Type":"ContainerStarted","Data":"765a5a012e7ef84a2b884e4a393ab394d3e347e7f27d06e2373e323d8a7f662d"}
Jul 28 18:28:03.416969 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:03.416946    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d52f1b0414f13e60997fc8e2898275b54134a9584793202a42d45212e467a117"
Jul 28 18:28:03.416969 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:03.416949    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7734-139928809629712"
Jul 28 18:28:03.427269 ip-10-0-136-126 kubenswrapper[2563]: E0728 18:28:03.427243    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7734-139928809629712\" is forbidden: User \"system:node:ip-10-0-136-126.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-126.ec2.internal' and this object" podUID="4100460e-f605-4128-8d67-962fc4a65d37" pod="opendatahub/test-curl-7734-139928809629712"
Jul 28 18:28:03.854430 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:03.854399    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4100460e-f605-4128-8d67-962fc4a65d37" path="/var/lib/kubelet/pods/4100460e-f605-4128-8d67-962fc4a65d37/volumes"
Jul 28 18:28:04.424153 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:04.424115    2563 generic.go:356] "Generic (PLEG): container finished" podID="2c918e2a-a91e-46fc-bdf5-accb0dce3071" containerID="2119da980280f3a5bc4fd6dbd5b3d642189fb1aefcda2da1e6089a4bb5d94f29" exitCode=0
Jul 28 18:28:04.424345 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:04.424160    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7734-139928809630720" event={"ID":"2c918e2a-a91e-46fc-bdf5-accb0dce3071","Type":"ContainerDied","Data":"2119da980280f3a5bc4fd6dbd5b3d642189fb1aefcda2da1e6089a4bb5d94f29"}
Jul 28 18:28:05.566524 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:05.566497    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7734-139928809630720"
Jul 28 18:28:05.587708 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:05.587687    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7734-139928809630720_2c918e2a-a91e-46fc-bdf5-accb0dce3071/test-curl-7734-139928809630720/0.log"
Jul 28 18:28:05.615735 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:05.615709    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c84jc\" (UniqueName: \"kubernetes.io/projected/2c918e2a-a91e-46fc-bdf5-accb0dce3071-kube-api-access-c84jc\") pod \"2c918e2a-a91e-46fc-bdf5-accb0dce3071\" (UID: \"2c918e2a-a91e-46fc-bdf5-accb0dce3071\") "
Jul 28 18:28:05.616651 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:05.616629    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7734-139928809630720"]
Jul 28 18:28:05.617723 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:05.617691    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c918e2a-a91e-46fc-bdf5-accb0dce3071-kube-api-access-c84jc" (OuterVolumeSpecName: "kube-api-access-c84jc") pod "2c918e2a-a91e-46fc-bdf5-accb0dce3071" (UID: "2c918e2a-a91e-46fc-bdf5-accb0dce3071"). InnerVolumeSpecName "kube-api-access-c84jc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:28:05.622398 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:05.622374    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7734-139928809630720"]
Jul 28 18:28:05.716784 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:05.716700    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c84jc\" (UniqueName: \"kubernetes.io/projected/2c918e2a-a91e-46fc-bdf5-accb0dce3071-kube-api-access-c84jc\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:28:05.853964 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:05.853928    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c918e2a-a91e-46fc-bdf5-accb0dce3071" path="/var/lib/kubelet/pods/2c918e2a-a91e-46fc-bdf5-accb0dce3071/volumes"
Jul 28 18:28:06.436417 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:06.436382    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7734-139928809630720"
Jul 28 18:28:06.436624 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:06.436414    2563 scope.go:117] "RemoveContainer" containerID="2119da980280f3a5bc4fd6dbd5b3d642189fb1aefcda2da1e6089a4bb5d94f29"
Jul 28 18:28:09.024308 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:09.024276    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:28:09.347538 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:09.347502    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:28:09.477846 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:09.477818    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:28:12.863008 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:12.862967    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:28:12.940647 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:12.940617    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:28:12.945056 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:12.945024    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-faea92-6b4c94ccc-s4ld7"]
Jul 28 18:28:12.945481 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:12.945464    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c918e2a-a91e-46fc-bdf5-accb0dce3071" containerName="test-curl-7734-139928809630720"
Jul 28 18:28:12.945564 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:12.945482    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c918e2a-a91e-46fc-bdf5-accb0dce3071" containerName="test-curl-7734-139928809630720"
Jul 28 18:28:12.945618 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:12.945605    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c918e2a-a91e-46fc-bdf5-accb0dce3071" containerName="test-curl-7734-139928809630720"
Jul 28 18:28:12.949844 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:12.949823    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-faea92-6b4c94ccc-s4ld7"
Jul 28 18:28:12.952341 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:12.952320    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-faea92-serving-cert\""
Jul 28 18:28:12.960008 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:12.959983    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-faea92-6b4c94ccc-s4ld7"]
Jul 28 18:28:13.005687 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:13.005645    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:28:13.087597 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:13.087523    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f12d0d22-4131-455a-9541-1fed5c26c08e-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-faea92-6b4c94ccc-s4ld7\" (UID: \"f12d0d22-4131-455a-9541-1fed5c26c08e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-faea92-6b4c94ccc-s4ld7"
Jul 28 18:28:13.087748 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:13.087635    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcmnv\" (UniqueName: \"kubernetes.io/projected/f12d0d22-4131-455a-9541-1fed5c26c08e-kube-api-access-vcmnv\") pod \"maas-api-e2e-ipp-a-faea92-6b4c94ccc-s4ld7\" (UID: \"f12d0d22-4131-455a-9541-1fed5c26c08e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-faea92-6b4c94ccc-s4ld7"
Jul 28 18:28:13.087748 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:13.087678    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f12d0d22-4131-455a-9541-1fed5c26c08e-maas-api-tls\") pod \"maas-api-e2e-ipp-a-faea92-6b4c94ccc-s4ld7\" (UID: \"f12d0d22-4131-455a-9541-1fed5c26c08e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-faea92-6b4c94ccc-s4ld7"
Jul 28 18:28:13.188650 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:13.188569    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f12d0d22-4131-455a-9541-1fed5c26c08e-maas-api-tls\") pod \"maas-api-e2e-ipp-a-faea92-6b4c94ccc-s4ld7\" (UID: \"f12d0d22-4131-455a-9541-1fed5c26c08e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-faea92-6b4c94ccc-s4ld7"
Jul 28 18:28:13.188650 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:13.188629    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f12d0d22-4131-455a-9541-1fed5c26c08e-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-faea92-6b4c94ccc-s4ld7\" (UID: \"f12d0d22-4131-455a-9541-1fed5c26c08e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-faea92-6b4c94ccc-s4ld7"
Jul 28 18:28:13.188824 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:13.188682    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcmnv\" (UniqueName: \"kubernetes.io/projected/f12d0d22-4131-455a-9541-1fed5c26c08e-kube-api-access-vcmnv\") pod \"maas-api-e2e-ipp-a-faea92-6b4c94ccc-s4ld7\" (UID: \"f12d0d22-4131-455a-9541-1fed5c26c08e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-faea92-6b4c94ccc-s4ld7"
Jul 28 18:28:13.188824 ip-10-0-136-126 kubenswrapper[2563]: E0728 18:28:13.188739    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-faea92-serving-cert: secret "maas-api-e2e-ipp-a-faea92-serving-cert" not found
Jul 28 18:28:13.188824 ip-10-0-136-126 kubenswrapper[2563]: E0728 18:28:13.188817    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f12d0d22-4131-455a-9541-1fed5c26c08e-maas-api-tls podName:f12d0d22-4131-455a-9541-1fed5c26c08e nodeName:}" failed. No retries permitted until 2026-07-28 18:28:13.688796332 +0000 UTC m=+3232.349952682 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f12d0d22-4131-455a-9541-1fed5c26c08e-maas-api-tls") pod "maas-api-e2e-ipp-a-faea92-6b4c94ccc-s4ld7" (UID: "f12d0d22-4131-455a-9541-1fed5c26c08e") : secret "maas-api-e2e-ipp-a-faea92-serving-cert" not found
Jul 28 18:28:13.189370 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:13.189351    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f12d0d22-4131-455a-9541-1fed5c26c08e-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-faea92-6b4c94ccc-s4ld7\" (UID: \"f12d0d22-4131-455a-9541-1fed5c26c08e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-faea92-6b4c94ccc-s4ld7"
Jul 28 18:28:13.199424 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:13.199396    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcmnv\" (UniqueName: \"kubernetes.io/projected/f12d0d22-4131-455a-9541-1fed5c26c08e-kube-api-access-vcmnv\") pod \"maas-api-e2e-ipp-a-faea92-6b4c94ccc-s4ld7\" (UID: \"f12d0d22-4131-455a-9541-1fed5c26c08e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-faea92-6b4c94ccc-s4ld7"
Jul 28 18:28:13.437334 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:13.437294    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:28:13.562155 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:13.562078    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:28:13.665565 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:13.665529    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-faea92-688ff66c9b-mpgxs"]
Jul 28 18:28:13.667008 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:13.666977    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:28:13.669528 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:13.669511    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-faea92-688ff66c9b-mpgxs"
Jul 28 18:28:13.671875 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:13.671854    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-faea92-dockercfg-csvlf\""
Jul 28 18:28:13.672008 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:13.671854    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-faea92\""
Jul 28 18:28:13.679672 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:13.679650    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-faea92-688ff66c9b-mpgxs"]
Jul 28 18:28:13.693710 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:13.693666    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f12d0d22-4131-455a-9541-1fed5c26c08e-maas-api-tls\") pod \"maas-api-e2e-ipp-a-faea92-6b4c94ccc-s4ld7\" (UID: \"f12d0d22-4131-455a-9541-1fed5c26c08e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-faea92-6b4c94ccc-s4ld7"
Jul 28 18:28:13.696418 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:13.696397    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f12d0d22-4131-455a-9541-1fed5c26c08e-maas-api-tls\") pod \"maas-api-e2e-ipp-a-faea92-6b4c94ccc-s4ld7\" (UID: \"f12d0d22-4131-455a-9541-1fed5c26c08e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-faea92-6b4c94ccc-s4ld7"
Jul 28 18:28:13.794539 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:13.794500    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jqtg\" (UniqueName: \"kubernetes.io/projected/53008497-534a-4934-84ed-d6698ba729fa-kube-api-access-4jqtg\") pod \"payload-pre-processing-e2e-ipp-a-faea92-688ff66c9b-mpgxs\" (UID: \"53008497-534a-4934-84ed-d6698ba729fa\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-faea92-688ff66c9b-mpgxs"
Jul 28 18:28:13.794539 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:13.794546    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53008497-534a-4934-84ed-d6698ba729fa-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-faea92-688ff66c9b-mpgxs\" (UID: \"53008497-534a-4934-84ed-d6698ba729fa\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-faea92-688ff66c9b-mpgxs"
Jul 28 18:28:13.861577 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:13.861549    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-faea92-6b4c94ccc-s4ld7"
Jul 28 18:28:13.895766 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:13.895733    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jqtg\" (UniqueName: \"kubernetes.io/projected/53008497-534a-4934-84ed-d6698ba729fa-kube-api-access-4jqtg\") pod \"payload-pre-processing-e2e-ipp-a-faea92-688ff66c9b-mpgxs\" (UID: \"53008497-534a-4934-84ed-d6698ba729fa\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-faea92-688ff66c9b-mpgxs"
Jul 28 18:28:13.896184 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:13.895781    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53008497-534a-4934-84ed-d6698ba729fa-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-faea92-688ff66c9b-mpgxs\" (UID: \"53008497-534a-4934-84ed-d6698ba729fa\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-faea92-688ff66c9b-mpgxs"
Jul 28 18:28:13.896570 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:13.896547    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53008497-534a-4934-84ed-d6698ba729fa-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-faea92-688ff66c9b-mpgxs\" (UID: \"53008497-534a-4934-84ed-d6698ba729fa\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-faea92-688ff66c9b-mpgxs"
Jul 28 18:28:13.904923 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:13.904900    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jqtg\" (UniqueName: \"kubernetes.io/projected/53008497-534a-4934-84ed-d6698ba729fa-kube-api-access-4jqtg\") pod \"payload-pre-processing-e2e-ipp-a-faea92-688ff66c9b-mpgxs\" (UID: \"53008497-534a-4934-84ed-d6698ba729fa\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-faea92-688ff66c9b-mpgxs"
Jul 28 18:28:13.981017 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:13.980987    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-faea92-688ff66c9b-mpgxs"
Jul 28 18:28:14.002389 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:14.002350    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-faea92-6b4c94ccc-s4ld7"]
Jul 28 18:28:14.002519 ip-10-0-136-126 kubenswrapper[2563]: W0728 18:28:14.002381    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf12d0d22_4131_455a_9541_1fed5c26c08e.slice/crio-4177732444f90b004f5f7da3c3743f00522b1e4e6fc640a07fb3f2ea3d389737 WatchSource:0}: Error finding container 4177732444f90b004f5f7da3c3743f00522b1e4e6fc640a07fb3f2ea3d389737: Status 404 returned error can't find the container with id 4177732444f90b004f5f7da3c3743f00522b1e4e6fc640a07fb3f2ea3d389737
Jul 28 18:28:14.117346 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:14.117319    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-faea92-688ff66c9b-mpgxs"]
Jul 28 18:28:14.119326 ip-10-0-136-126 kubenswrapper[2563]: W0728 18:28:14.119293    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53008497_534a_4934_84ed_d6698ba729fa.slice/crio-b64b046c01ad258dd88198ae4094205cedfb2f4bfccc3049ed7384fbfa4bdc85 WatchSource:0}: Error finding container b64b046c01ad258dd88198ae4094205cedfb2f4bfccc3049ed7384fbfa4bdc85: Status 404 returned error can't find the container with id b64b046c01ad258dd88198ae4094205cedfb2f4bfccc3049ed7384fbfa4bdc85
Jul 28 18:28:14.486189 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:14.486153    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-faea92-688ff66c9b-mpgxs" event={"ID":"53008497-534a-4934-84ed-d6698ba729fa","Type":"ContainerStarted","Data":"58ae62535622d36cc03487bfd08131ae1800e83cdd72331819978bbd9362d2f1"}
Jul 28 18:28:14.486386 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:14.486221    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-faea92-688ff66c9b-mpgxs" event={"ID":"53008497-534a-4934-84ed-d6698ba729fa","Type":"ContainerStarted","Data":"b64b046c01ad258dd88198ae4094205cedfb2f4bfccc3049ed7384fbfa4bdc85"}
Jul 28 18:28:14.486386 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:14.486251    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-faea92-688ff66c9b-mpgxs"
Jul 28 18:28:14.487486 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:14.487461    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-faea92-6b4c94ccc-s4ld7" event={"ID":"f12d0d22-4131-455a-9541-1fed5c26c08e","Type":"ContainerStarted","Data":"4177732444f90b004f5f7da3c3743f00522b1e4e6fc640a07fb3f2ea3d389737"}
Jul 28 18:28:14.503669 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:14.503629    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-faea92-688ff66c9b-mpgxs" podStartSLOduration=1.503614841 podStartE2EDuration="1.503614841s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:28:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:28:14.501806392 +0000 UTC m=+3233.162962748" watchObservedRunningTime="2026-07-28 18:28:14.503614841 +0000 UTC m=+3233.164771174"
Jul 28 18:28:15.494193 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:15.494153    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-faea92-6b4c94ccc-s4ld7" event={"ID":"f12d0d22-4131-455a-9541-1fed5c26c08e","Type":"ContainerStarted","Data":"0210b6eea764b2d5826af85500cef76f7a573ecc982e85ad075006aca98c20f8"}
Jul 28 18:28:15.494682 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:15.494257    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-faea92-6b4c94ccc-s4ld7"
Jul 28 18:28:15.513777 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:15.513733    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-faea92-6b4c94ccc-s4ld7" podStartSLOduration=3.070402149 podStartE2EDuration="3.513718959s" totalImagesPullingTime="443.31681ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:28:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:28:15.51171305 +0000 UTC m=+3234.172869404" watchObservedRunningTime="2026-07-28 18:28:15.513718959 +0000 UTC m=+3234.174875314"
Jul 28 18:28:21.505925 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:21.505897    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-faea92-6b4c94ccc-s4ld7"
Jul 28 18:28:25.496083 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:25.496048    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-faea92-688ff66c9b-mpgxs"
Jul 28 18:28:37.907866 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:37.907818    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:28:38.598967 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:38.598932    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:28:38.937603 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:38.937500    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:28:39.154201 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:39.154168    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:28:42.034715 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:42.034682    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:28:42.119661 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:42.119598    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:28:42.192258 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:42.192197    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-d53588-c88b8c8c7-5glvj"]
Jul 28 18:28:42.196267 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:42.196244    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d53588-c88b8c8c7-5glvj"
Jul 28 18:28:42.199004 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:42.198978    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-d53588-serving-cert\""
Jul 28 18:28:42.210810 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:42.210772    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-d53588-c88b8c8c7-5glvj"]
Jul 28 18:28:42.237800 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:42.237767    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:28:42.243604 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:42.243578    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd7df201-b522-4f85-9cb6-9d4453e3024b-maas-api-tls\") pod \"maas-api-e2e-ipp-b-d53588-c88b8c8c7-5glvj\" (UID: \"cd7df201-b522-4f85-9cb6-9d4453e3024b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d53588-c88b8c8c7-5glvj"
Jul 28 18:28:42.243735 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:42.243608    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fxj5\" (UniqueName: \"kubernetes.io/projected/cd7df201-b522-4f85-9cb6-9d4453e3024b-kube-api-access-2fxj5\") pod \"maas-api-e2e-ipp-b-d53588-c88b8c8c7-5glvj\" (UID: \"cd7df201-b522-4f85-9cb6-9d4453e3024b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d53588-c88b8c8c7-5glvj"
Jul 28 18:28:42.243735 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:42.243694    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd7df201-b522-4f85-9cb6-9d4453e3024b-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-d53588-c88b8c8c7-5glvj\" (UID: \"cd7df201-b522-4f85-9cb6-9d4453e3024b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d53588-c88b8c8c7-5glvj"
Jul 28 18:28:42.344430 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:42.344397    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd7df201-b522-4f85-9cb6-9d4453e3024b-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-d53588-c88b8c8c7-5glvj\" (UID: \"cd7df201-b522-4f85-9cb6-9d4453e3024b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d53588-c88b8c8c7-5glvj"
Jul 28 18:28:42.344635 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:42.344554    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd7df201-b522-4f85-9cb6-9d4453e3024b-maas-api-tls\") pod \"maas-api-e2e-ipp-b-d53588-c88b8c8c7-5glvj\" (UID: \"cd7df201-b522-4f85-9cb6-9d4453e3024b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d53588-c88b8c8c7-5glvj"
Jul 28 18:28:42.344635 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:42.344576    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fxj5\" (UniqueName: \"kubernetes.io/projected/cd7df201-b522-4f85-9cb6-9d4453e3024b-kube-api-access-2fxj5\") pod \"maas-api-e2e-ipp-b-d53588-c88b8c8c7-5glvj\" (UID: \"cd7df201-b522-4f85-9cb6-9d4453e3024b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d53588-c88b8c8c7-5glvj"
Jul 28 18:28:42.344763 ip-10-0-136-126 kubenswrapper[2563]: E0728 18:28:42.344660    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-d53588-serving-cert: secret "maas-api-e2e-ipp-b-d53588-serving-cert" not found
Jul 28 18:28:42.344763 ip-10-0-136-126 kubenswrapper[2563]: E0728 18:28:42.344754    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd7df201-b522-4f85-9cb6-9d4453e3024b-maas-api-tls podName:cd7df201-b522-4f85-9cb6-9d4453e3024b nodeName:}" failed. No retries permitted until 2026-07-28 18:28:42.844730949 +0000 UTC m=+3261.505887303 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cd7df201-b522-4f85-9cb6-9d4453e3024b-maas-api-tls") pod "maas-api-e2e-ipp-b-d53588-c88b8c8c7-5glvj" (UID: "cd7df201-b522-4f85-9cb6-9d4453e3024b") : secret "maas-api-e2e-ipp-b-d53588-serving-cert" not found
Jul 28 18:28:42.345019 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:42.344999    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd7df201-b522-4f85-9cb6-9d4453e3024b-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-d53588-c88b8c8c7-5glvj\" (UID: \"cd7df201-b522-4f85-9cb6-9d4453e3024b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d53588-c88b8c8c7-5glvj"
Jul 28 18:28:42.354390 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:42.354368    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fxj5\" (UniqueName: \"kubernetes.io/projected/cd7df201-b522-4f85-9cb6-9d4453e3024b-kube-api-access-2fxj5\") pod \"maas-api-e2e-ipp-b-d53588-c88b8c8c7-5glvj\" (UID: \"cd7df201-b522-4f85-9cb6-9d4453e3024b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d53588-c88b8c8c7-5glvj"
Jul 28 18:28:42.437525 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:42.437467    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:28:42.749825 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:42.749726    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:28:42.848845 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:42.848805    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd7df201-b522-4f85-9cb6-9d4453e3024b-maas-api-tls\") pod \"maas-api-e2e-ipp-b-d53588-c88b8c8c7-5glvj\" (UID: \"cd7df201-b522-4f85-9cb6-9d4453e3024b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d53588-c88b8c8c7-5glvj"
Jul 28 18:28:42.849563 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:42.849532    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:28:42.851545 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:42.851522    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd7df201-b522-4f85-9cb6-9d4453e3024b-maas-api-tls\") pod \"maas-api-e2e-ipp-b-d53588-c88b8c8c7-5glvj\" (UID: \"cd7df201-b522-4f85-9cb6-9d4453e3024b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d53588-c88b8c8c7-5glvj"
Jul 28 18:28:42.883114 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:42.883083    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d53588-56b7656f98-jtxfv"]
Jul 28 18:28:42.886827 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:42.886802    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-d53588-56b7656f98-jtxfv"
Jul 28 18:28:42.889224 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:42.889206    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-d53588\""
Jul 28 18:28:42.889380 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:42.889363    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-d53588-dockercfg-wwbsw\""
Jul 28 18:28:42.897546 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:42.897527    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d53588-56b7656f98-jtxfv"]
Jul 28 18:28:42.930509 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:42.930482    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-d53588-658f7f56b4-sh2tp"]
Jul 28 18:28:42.934347 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:42.934317    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d53588-658f7f56b4-sh2tp"
Jul 28 18:28:42.943613 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:42.943563    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-d53588-658f7f56b4-sh2tp"]
Jul 28 18:28:42.949699 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:42.949677    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61673ff4-6ce9-4257-b54c-fd3f9912d924-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-d53588-56b7656f98-jtxfv\" (UID: \"61673ff4-6ce9-4257-b54c-fd3f9912d924\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d53588-56b7656f98-jtxfv"
Jul 28 18:28:42.949809 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:42.949715    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glf69\" (UniqueName: \"kubernetes.io/projected/61673ff4-6ce9-4257-b54c-fd3f9912d924-kube-api-access-glf69\") pod \"payload-processing-e2e-ipp-b-d53588-56b7656f98-jtxfv\" (UID: \"61673ff4-6ce9-4257-b54c-fd3f9912d924\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d53588-56b7656f98-jtxfv"
Jul 28 18:28:43.050773 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:43.050690    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61673ff4-6ce9-4257-b54c-fd3f9912d924-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-d53588-56b7656f98-jtxfv\" (UID: \"61673ff4-6ce9-4257-b54c-fd3f9912d924\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d53588-56b7656f98-jtxfv"
Jul 28 18:28:43.050773 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:43.050740    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glf69\" (UniqueName: \"kubernetes.io/projected/61673ff4-6ce9-4257-b54c-fd3f9912d924-kube-api-access-glf69\") pod \"payload-processing-e2e-ipp-b-d53588-56b7656f98-jtxfv\" (UID: \"61673ff4-6ce9-4257-b54c-fd3f9912d924\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d53588-56b7656f98-jtxfv"
Jul 28 18:28:43.051191 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:43.050791    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvmpx\" (UniqueName: \"kubernetes.io/projected/31553ab6-42e2-42f0-9f06-ba203456e212-kube-api-access-vvmpx\") pod \"payload-pre-processing-e2e-ipp-b-d53588-658f7f56b4-sh2tp\" (UID: \"31553ab6-42e2-42f0-9f06-ba203456e212\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d53588-658f7f56b4-sh2tp"
Jul 28 18:28:43.051191 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:43.050861    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31553ab6-42e2-42f0-9f06-ba203456e212-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-d53588-658f7f56b4-sh2tp\" (UID: \"31553ab6-42e2-42f0-9f06-ba203456e212\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d53588-658f7f56b4-sh2tp"
Jul 28 18:28:43.051404 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:43.051385    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61673ff4-6ce9-4257-b54c-fd3f9912d924-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-d53588-56b7656f98-jtxfv\" (UID: \"61673ff4-6ce9-4257-b54c-fd3f9912d924\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d53588-56b7656f98-jtxfv"
Jul 28 18:28:43.060284 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:43.060264    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-glf69\" (UniqueName: \"kubernetes.io/projected/61673ff4-6ce9-4257-b54c-fd3f9912d924-kube-api-access-glf69\") pod \"payload-processing-e2e-ipp-b-d53588-56b7656f98-jtxfv\" (UID: \"61673ff4-6ce9-4257-b54c-fd3f9912d924\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d53588-56b7656f98-jtxfv"
Jul 28 18:28:43.111688 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:43.111654    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d53588-c88b8c8c7-5glvj"
Jul 28 18:28:43.151375 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:43.151342    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31553ab6-42e2-42f0-9f06-ba203456e212-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-d53588-658f7f56b4-sh2tp\" (UID: \"31553ab6-42e2-42f0-9f06-ba203456e212\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d53588-658f7f56b4-sh2tp"
Jul 28 18:28:43.151538 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:43.151409    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvmpx\" (UniqueName: \"kubernetes.io/projected/31553ab6-42e2-42f0-9f06-ba203456e212-kube-api-access-vvmpx\") pod \"payload-pre-processing-e2e-ipp-b-d53588-658f7f56b4-sh2tp\" (UID: \"31553ab6-42e2-42f0-9f06-ba203456e212\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d53588-658f7f56b4-sh2tp"
Jul 28 18:28:43.152312 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:43.152272    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31553ab6-42e2-42f0-9f06-ba203456e212-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-d53588-658f7f56b4-sh2tp\" (UID: \"31553ab6-42e2-42f0-9f06-ba203456e212\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d53588-658f7f56b4-sh2tp"
Jul 28 18:28:43.160825 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:43.160800    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvmpx\" (UniqueName: \"kubernetes.io/projected/31553ab6-42e2-42f0-9f06-ba203456e212-kube-api-access-vvmpx\") pod \"payload-pre-processing-e2e-ipp-b-d53588-658f7f56b4-sh2tp\" (UID: \"31553ab6-42e2-42f0-9f06-ba203456e212\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d53588-658f7f56b4-sh2tp"
Jul 28 18:28:43.199026 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:43.198762    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-d53588-56b7656f98-jtxfv"
Jul 28 18:28:43.247847 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:43.247812    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d53588-658f7f56b4-sh2tp"
Jul 28 18:28:43.252121 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:43.251976    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-d53588-c88b8c8c7-5glvj"]
Jul 28 18:28:43.253609 ip-10-0-136-126 kubenswrapper[2563]: W0728 18:28:43.253581    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd7df201_b522_4f85_9cb6_9d4453e3024b.slice/crio-4a9bb060c4373b98da04a6f30b14c591fff082b7b42cd18cf52a169528292f84 WatchSource:0}: Error finding container 4a9bb060c4373b98da04a6f30b14c591fff082b7b42cd18cf52a169528292f84: Status 404 returned error can't find the container with id 4a9bb060c4373b98da04a6f30b14c591fff082b7b42cd18cf52a169528292f84
Jul 28 18:28:43.357742 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:43.357260    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d53588-56b7656f98-jtxfv"]
Jul 28 18:28:43.360460 ip-10-0-136-126 kubenswrapper[2563]: W0728 18:28:43.360390    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61673ff4_6ce9_4257_b54c_fd3f9912d924.slice/crio-8bd58152cc1136a35626d48f0b4216f3483e68dd65c7fa5cad94fa73fb448b43 WatchSource:0}: Error finding container 8bd58152cc1136a35626d48f0b4216f3483e68dd65c7fa5cad94fa73fb448b43: Status 404 returned error can't find the container with id 8bd58152cc1136a35626d48f0b4216f3483e68dd65c7fa5cad94fa73fb448b43
Jul 28 18:28:43.621734 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:43.621709    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-d53588-658f7f56b4-sh2tp"]
Jul 28 18:28:43.622563 ip-10-0-136-126 kubenswrapper[2563]: W0728 18:28:43.622540    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31553ab6_42e2_42f0_9f06_ba203456e212.slice/crio-32d739757f5f3265bda867da97c64ff154608b6cf772e599641d57f7b3054d43 WatchSource:0}: Error finding container 32d739757f5f3265bda867da97c64ff154608b6cf772e599641d57f7b3054d43: Status 404 returned error can't find the container with id 32d739757f5f3265bda867da97c64ff154608b6cf772e599641d57f7b3054d43
Jul 28 18:28:43.648371 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:43.648335    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d53588-c88b8c8c7-5glvj" event={"ID":"cd7df201-b522-4f85-9cb6-9d4453e3024b","Type":"ContainerStarted","Data":"bbbb1c6d5adef54ab2511210b0a7c18c7d7b7155a0f1b9e6e64483a4efd1c05a"}
Jul 28 18:28:43.648511 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:43.648405    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d53588-c88b8c8c7-5glvj"
Jul 28 18:28:43.648511 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:43.648427    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d53588-c88b8c8c7-5glvj" event={"ID":"cd7df201-b522-4f85-9cb6-9d4453e3024b","Type":"ContainerStarted","Data":"4a9bb060c4373b98da04a6f30b14c591fff082b7b42cd18cf52a169528292f84"}
Jul 28 18:28:43.650008 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:43.649982    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-d53588-56b7656f98-jtxfv" event={"ID":"61673ff4-6ce9-4257-b54c-fd3f9912d924","Type":"ContainerStarted","Data":"094d29b91bb85458bba8d35bf05892a290b4637382cc6e8f20f0802fc6a5dcc9"}
Jul 28 18:28:43.650131 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:43.650038    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-d53588-56b7656f98-jtxfv"
Jul 28 18:28:43.650131 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:43.650063    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-d53588-56b7656f98-jtxfv" event={"ID":"61673ff4-6ce9-4257-b54c-fd3f9912d924","Type":"ContainerStarted","Data":"8bd58152cc1136a35626d48f0b4216f3483e68dd65c7fa5cad94fa73fb448b43"}
Jul 28 18:28:43.651226 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:43.651202    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d53588-658f7f56b4-sh2tp" event={"ID":"31553ab6-42e2-42f0-9f06-ba203456e212","Type":"ContainerStarted","Data":"32d739757f5f3265bda867da97c64ff154608b6cf772e599641d57f7b3054d43"}
Jul 28 18:28:43.670718 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:43.670672    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d53588-c88b8c8c7-5glvj" podStartSLOduration=1.359458241 podStartE2EDuration="1.670659405s" totalImagesPullingTime="311.201164ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:28:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:28:43.668710701 +0000 UTC m=+3262.329867068" watchObservedRunningTime="2026-07-28 18:28:43.670659405 +0000 UTC m=+3262.331815759"
Jul 28 18:28:43.686401 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:43.686354    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-d53588-56b7656f98-jtxfv" podStartSLOduration=1.686341 podStartE2EDuration="1.686341s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:28:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:28:43.685120448 +0000 UTC m=+3262.346276805" watchObservedRunningTime="2026-07-28 18:28:43.686341 +0000 UTC m=+3262.347497354"
Jul 28 18:28:44.658607 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:44.658569    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d53588-658f7f56b4-sh2tp" event={"ID":"31553ab6-42e2-42f0-9f06-ba203456e212","Type":"ContainerStarted","Data":"4e77cc0902e1dc8eab546396b75ff3bfa5797ca995d2a02c4d04badd90cefe04"}
Jul 28 18:28:44.659078 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:44.658734    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d53588-658f7f56b4-sh2tp"
Jul 28 18:28:44.677858 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:44.677804    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d53588-658f7f56b4-sh2tp" podStartSLOduration=2.677786403 podStartE2EDuration="2.677786403s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:28:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:28:44.677211609 +0000 UTC m=+3263.338367990" watchObservedRunningTime="2026-07-28 18:28:44.677786403 +0000 UTC m=+3263.338942760"
Jul 28 18:28:49.664844 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:49.664813    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d53588-c88b8c8c7-5glvj"
Jul 28 18:28:53.210721 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:53.210692    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:28:53.309966 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:53.309925    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:28:54.660890 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:54.660860    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-d53588-56b7656f98-jtxfv"
Jul 28 18:28:55.667973 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:28:55.667941    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d53588-658f7f56b4-sh2tp"
Jul 28 18:29:07.056487 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:29:07.056427    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:29:14.336741 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:29:14.336700    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-d53588-56b7656f98-jtxfv_61673ff4-6ce9-4257-b54c-fd3f9912d924/payload-processing/0.log"
Jul 28 18:29:19.584451 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:29:19.584409    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:29:55.997176 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:29:55.997087    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:29:59.548343 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:29:59.548314    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-d53588-56b7656f98-jtxfv_61673ff4-6ce9-4257-b54c-fd3f9912d924/payload-processing/0.log"
Jul 28 18:29:59.869719 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:29:59.869686    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:30:00.150928 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:00.150847    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-c73c1e-29754390-wr8qw"]
Jul 28 18:30:00.154769 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:00.154752    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-faea92-29754390-pbfr8"]
Jul 28 18:30:00.154891 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:00.154869    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-c73c1e-29754390-wr8qw"
Jul 28 18:30:00.157664 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:00.157634    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9xcv2\""
Jul 28 18:30:00.158333 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:00.158318    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-faea92-29754390-pbfr8"
Jul 28 18:30:00.169008 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:00.168834    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-c73c1e-29754390-wr8qw"
Jul 28 18:30:00.169901 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:00.169559    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-c73c1e-29754390-wr8qw"]
Jul 28 18:30:00.173582 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:00.173544    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-faea92-29754390-pbfr8"]
Jul 28 18:30:00.182789 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:00.182712    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-faea92-29754390-pbfr8"
Jul 28 18:30:00.224420 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:00.224383    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:30:00.349623 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:00.349588    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-c73c1e-29754390-wr8qw"]
Jul 28 18:30:00.353198 ip-10-0-136-126 kubenswrapper[2563]: W0728 18:30:00.353165    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod400c8f56_826e_4ee3_8b03_3339bf38f057.slice/crio-34a2433e2a99714c50fe2246d0df890a95a5e4c9ba8e36d9e49dd7813f68a729 WatchSource:0}: Error finding container 34a2433e2a99714c50fe2246d0df890a95a5e4c9ba8e36d9e49dd7813f68a729: Status 404 returned error can't find the container with id 34a2433e2a99714c50fe2246d0df890a95a5e4c9ba8e36d9e49dd7813f68a729
Jul 28 18:30:00.379947 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:00.379923    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-faea92-29754390-pbfr8"]
Jul 28 18:30:00.380849 ip-10-0-136-126 kubenswrapper[2563]: W0728 18:30:00.380817    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9a71981_6aca_433e_84ed_dd98a4a693a5.slice/crio-2430d19447f85826bcab03181ba0c0503f50e31c49addb4abbfc1f0e81b1b3e8 WatchSource:0}: Error finding container 2430d19447f85826bcab03181ba0c0503f50e31c49addb4abbfc1f0e81b1b3e8: Status 404 returned error can't find the container with id 2430d19447f85826bcab03181ba0c0503f50e31c49addb4abbfc1f0e81b1b3e8
Jul 28 18:30:00.755902 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:00.755815    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:30:01.102697 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:01.102664    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-faea92-29754390-pbfr8" event={"ID":"b9a71981-6aca-433e-84ed-dd98a4a693a5","Type":"ContainerStarted","Data":"ee6199683d028fbb822cbfc9a83b7ec54a1c16d72acd176a59ba55a4b5772752"}
Jul 28 18:30:01.102894 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:01.102717    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-faea92-29754390-pbfr8" event={"ID":"b9a71981-6aca-433e-84ed-dd98a4a693a5","Type":"ContainerStarted","Data":"2430d19447f85826bcab03181ba0c0503f50e31c49addb4abbfc1f0e81b1b3e8"}
Jul 28 18:30:01.104160 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:01.104135    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-c73c1e-29754390-wr8qw" event={"ID":"400c8f56-826e-4ee3-8b03-3339bf38f057","Type":"ContainerStarted","Data":"d9228a0ea90ed25734f64b9d936a96c67f078c1d16517ea10ff645cbe74d61e1"}
Jul 28 18:30:01.104261 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:01.104176    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-c73c1e-29754390-wr8qw" event={"ID":"400c8f56-826e-4ee3-8b03-3339bf38f057","Type":"ContainerStarted","Data":"34a2433e2a99714c50fe2246d0df890a95a5e4c9ba8e36d9e49dd7813f68a729"}
Jul 28 18:30:01.121643 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:01.121600    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-faea92-29754390-pbfr8" podStartSLOduration=1.121587076 podStartE2EDuration="1.121587076s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:30:01.119848862 +0000 UTC m=+3339.781005216" watchObservedRunningTime="2026-07-28 18:30:01.121587076 +0000 UTC m=+3339.782743430"
Jul 28 18:30:01.141804 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:01.141759    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-c73c1e-29754390-wr8qw" podStartSLOduration=1.141742744 podStartE2EDuration="1.141742744s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:30:01.138206002 +0000 UTC m=+3339.799362358" watchObservedRunningTime="2026-07-28 18:30:01.141742744 +0000 UTC m=+3339.802899096"
Jul 28 18:30:02.115747 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:02.115712    2563 generic.go:356] "Generic (PLEG): container finished" podID="400c8f56-826e-4ee3-8b03-3339bf38f057" containerID="d9228a0ea90ed25734f64b9d936a96c67f078c1d16517ea10ff645cbe74d61e1" exitCode=0
Jul 28 18:30:02.116308 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:02.115941    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-c73c1e-29754390-wr8qw" event={"ID":"400c8f56-826e-4ee3-8b03-3339bf38f057","Type":"ContainerDied","Data":"d9228a0ea90ed25734f64b9d936a96c67f078c1d16517ea10ff645cbe74d61e1"}
Jul 28 18:30:02.118202 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:02.118176    2563 generic.go:356] "Generic (PLEG): container finished" podID="b9a71981-6aca-433e-84ed-dd98a4a693a5" containerID="ee6199683d028fbb822cbfc9a83b7ec54a1c16d72acd176a59ba55a4b5772752" exitCode=0
Jul 28 18:30:02.118297 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:02.118228    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-faea92-29754390-pbfr8" event={"ID":"b9a71981-6aca-433e-84ed-dd98a4a693a5","Type":"ContainerDied","Data":"ee6199683d028fbb822cbfc9a83b7ec54a1c16d72acd176a59ba55a4b5772752"}
Jul 28 18:30:03.285041 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:03.285015    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-c73c1e-29754390-wr8qw"
Jul 28 18:30:03.287965 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:03.287948    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-faea92-29754390-pbfr8"
Jul 28 18:30:03.533188 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:03.533097    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:30:03.623120 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:03.623055    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:30:04.129967 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:04.129871    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-c73c1e-29754390-wr8qw"
Jul 28 18:30:04.129967 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:04.129901    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-c73c1e-29754390-wr8qw" event={"ID":"400c8f56-826e-4ee3-8b03-3339bf38f057","Type":"ContainerDied","Data":"34a2433e2a99714c50fe2246d0df890a95a5e4c9ba8e36d9e49dd7813f68a729"}
Jul 28 18:30:04.129967 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:04.129956    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="34a2433e2a99714c50fe2246d0df890a95a5e4c9ba8e36d9e49dd7813f68a729"
Jul 28 18:30:04.131238 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:04.131215    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-faea92-29754390-pbfr8" event={"ID":"b9a71981-6aca-433e-84ed-dd98a4a693a5","Type":"ContainerDied","Data":"2430d19447f85826bcab03181ba0c0503f50e31c49addb4abbfc1f0e81b1b3e8"}
Jul 28 18:30:04.131352 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:04.131247    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-faea92-29754390-pbfr8"
Jul 28 18:30:04.131352 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:04.131262    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2430d19447f85826bcab03181ba0c0503f50e31c49addb4abbfc1f0e81b1b3e8"
Jul 28 18:30:04.265548 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:04.265515    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2ad79-f664b84d5-mzzxz"]
Jul 28 18:30:04.266169 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:04.266149    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="400c8f56-826e-4ee3-8b03-3339bf38f057" containerName="cleanup"
Jul 28 18:30:04.266218 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:04.266175    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="400c8f56-826e-4ee3-8b03-3339bf38f057" containerName="cleanup"
Jul 28 18:30:04.266218 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:04.266201    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9a71981-6aca-433e-84ed-dd98a4a693a5" containerName="cleanup"
Jul 28 18:30:04.266218 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:04.266209    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9a71981-6aca-433e-84ed-dd98a4a693a5" containerName="cleanup"
Jul 28 18:30:04.266356 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:04.266343    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9a71981-6aca-433e-84ed-dd98a4a693a5" containerName="cleanup"
Jul 28 18:30:04.266398 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:04.266362    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="400c8f56-826e-4ee3-8b03-3339bf38f057" containerName="cleanup"
Jul 28 18:30:04.269861 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:04.269840    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2ad79-f664b84d5-mzzxz"
Jul 28 18:30:04.272212 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:04.272193    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-b2ad79-serving-cert\""
Jul 28 18:30:04.278544 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:04.278522    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2ad79-f664b84d5-mzzxz"]
Jul 28 18:30:04.309206 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:04.309176    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:30:04.372693 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:04.372422    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8s4n\" (UniqueName: \"kubernetes.io/projected/7bef84b4-2ca4-4eb3-a79c-34a96b5efb70-kube-api-access-h8s4n\") pod \"maas-api-e2e-ipp-cleanup-b2ad79-f664b84d5-mzzxz\" (UID: \"7bef84b4-2ca4-4eb3-a79c-34a96b5efb70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2ad79-f664b84d5-mzzxz"
Jul 28 18:30:04.372693 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:04.372494    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7bef84b4-2ca4-4eb3-a79c-34a96b5efb70-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-b2ad79-f664b84d5-mzzxz\" (UID: \"7bef84b4-2ca4-4eb3-a79c-34a96b5efb70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2ad79-f664b84d5-mzzxz"
Jul 28 18:30:04.372693 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:04.372519    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7bef84b4-2ca4-4eb3-a79c-34a96b5efb70-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-b2ad79-f664b84d5-mzzxz\" (UID: \"7bef84b4-2ca4-4eb3-a79c-34a96b5efb70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2ad79-f664b84d5-mzzxz"
Jul 28 18:30:04.473370 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:04.473271    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8s4n\" (UniqueName: \"kubernetes.io/projected/7bef84b4-2ca4-4eb3-a79c-34a96b5efb70-kube-api-access-h8s4n\") pod \"maas-api-e2e-ipp-cleanup-b2ad79-f664b84d5-mzzxz\" (UID: \"7bef84b4-2ca4-4eb3-a79c-34a96b5efb70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2ad79-f664b84d5-mzzxz"
Jul 28 18:30:04.473370 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:04.473320    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7bef84b4-2ca4-4eb3-a79c-34a96b5efb70-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-b2ad79-f664b84d5-mzzxz\" (UID: \"7bef84b4-2ca4-4eb3-a79c-34a96b5efb70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2ad79-f664b84d5-mzzxz"
Jul 28 18:30:04.473370 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:04.473338    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7bef84b4-2ca4-4eb3-a79c-34a96b5efb70-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-b2ad79-f664b84d5-mzzxz\" (UID: \"7bef84b4-2ca4-4eb3-a79c-34a96b5efb70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2ad79-f664b84d5-mzzxz"
Jul 28 18:30:04.473657 ip-10-0-136-126 kubenswrapper[2563]: E0728 18:30:04.473466    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2ad79-serving-cert: secret "maas-api-e2e-ipp-cleanup-b2ad79-serving-cert" not found
Jul 28 18:30:04.473657 ip-10-0-136-126 kubenswrapper[2563]: E0728 18:30:04.473536    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bef84b4-2ca4-4eb3-a79c-34a96b5efb70-maas-api-tls podName:7bef84b4-2ca4-4eb3-a79c-34a96b5efb70 nodeName:}" failed. No retries permitted until 2026-07-28 18:30:04.973519101 +0000 UTC m=+3343.634675437 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7bef84b4-2ca4-4eb3-a79c-34a96b5efb70-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-b2ad79-f664b84d5-mzzxz" (UID: "7bef84b4-2ca4-4eb3-a79c-34a96b5efb70") : secret "maas-api-e2e-ipp-cleanup-b2ad79-serving-cert" not found
Jul 28 18:30:04.473992 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:04.473974    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7bef84b4-2ca4-4eb3-a79c-34a96b5efb70-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-b2ad79-f664b84d5-mzzxz\" (UID: \"7bef84b4-2ca4-4eb3-a79c-34a96b5efb70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2ad79-f664b84d5-mzzxz"
Jul 28 18:30:04.484776 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:04.484749    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8s4n\" (UniqueName: \"kubernetes.io/projected/7bef84b4-2ca4-4eb3-a79c-34a96b5efb70-kube-api-access-h8s4n\") pod \"maas-api-e2e-ipp-cleanup-b2ad79-f664b84d5-mzzxz\" (UID: \"7bef84b4-2ca4-4eb3-a79c-34a96b5efb70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2ad79-f664b84d5-mzzxz"
Jul 28 18:30:04.941182 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:04.941144    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:30:04.977107 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:04.977072    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7bef84b4-2ca4-4eb3-a79c-34a96b5efb70-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-b2ad79-f664b84d5-mzzxz\" (UID: \"7bef84b4-2ca4-4eb3-a79c-34a96b5efb70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2ad79-f664b84d5-mzzxz"
Jul 28 18:30:04.979941 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:04.979915    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7bef84b4-2ca4-4eb3-a79c-34a96b5efb70-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-b2ad79-f664b84d5-mzzxz\" (UID: \"7bef84b4-2ca4-4eb3-a79c-34a96b5efb70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2ad79-f664b84d5-mzzxz"
Jul 28 18:30:05.041991 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:05.041952    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:30:05.090766 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:05.090727    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-b2ad79-7ff5bfff66-b84z4"]
Jul 28 18:30:05.094616 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:05.094591    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-b2ad79-7ff5bfff66-b84z4"
Jul 28 18:30:05.096990 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:05.096972    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-b2ad79-dockercfg-x5kkz\""
Jul 28 18:30:05.097111 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:05.096995    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-b2ad79\""
Jul 28 18:30:05.106342 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:05.106322    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-b2ad79-7ff5bfff66-b84z4"]
Jul 28 18:30:05.114729 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:05.114705    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:30:05.154802 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:05.154771    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b2ad79-5c9bd7878d-6fkv2"]
Jul 28 18:30:05.158418 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:05.158403    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b2ad79-5c9bd7878d-6fkv2"
Jul 28 18:30:05.168361 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:05.168327    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b2ad79-5c9bd7878d-6fkv2"]
Jul 28 18:30:05.178121 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:05.178100    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkmdl\" (UniqueName: \"kubernetes.io/projected/4a4333a5-791b-42d7-a9cf-5dc3c3a95165-kube-api-access-zkmdl\") pod \"payload-processing-e2e-ipp-cleanup-b2ad79-7ff5bfff66-b84z4\" (UID: \"4a4333a5-791b-42d7-a9cf-5dc3c3a95165\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-b2ad79-7ff5bfff66-b84z4"
Jul 28 18:30:05.178222 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:05.178139    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a4333a5-791b-42d7-a9cf-5dc3c3a95165-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-b2ad79-7ff5bfff66-b84z4\" (UID: \"4a4333a5-791b-42d7-a9cf-5dc3c3a95165\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-b2ad79-7ff5bfff66-b84z4"
Jul 28 18:30:05.183013 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:05.182990    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2ad79-f664b84d5-mzzxz"
Jul 28 18:30:05.279086 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:05.278930    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8982z\" (UniqueName: \"kubernetes.io/projected/a5013d74-382f-4636-a97e-3126bc23be29-kube-api-access-8982z\") pod \"payload-pre-processing-e2e-ipp-cleanup-b2ad79-5c9bd7878d-6fkv2\" (UID: \"a5013d74-382f-4636-a97e-3126bc23be29\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b2ad79-5c9bd7878d-6fkv2"
Jul 28 18:30:05.279086 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:05.279051    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5013d74-382f-4636-a97e-3126bc23be29-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-b2ad79-5c9bd7878d-6fkv2\" (UID: \"a5013d74-382f-4636-a97e-3126bc23be29\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b2ad79-5c9bd7878d-6fkv2"
Jul 28 18:30:05.279192 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:05.279106    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkmdl\" (UniqueName: \"kubernetes.io/projected/4a4333a5-791b-42d7-a9cf-5dc3c3a95165-kube-api-access-zkmdl\") pod \"payload-processing-e2e-ipp-cleanup-b2ad79-7ff5bfff66-b84z4\" (UID: \"4a4333a5-791b-42d7-a9cf-5dc3c3a95165\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-b2ad79-7ff5bfff66-b84z4"
Jul 28 18:30:05.279253 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:05.279199    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a4333a5-791b-42d7-a9cf-5dc3c3a95165-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-b2ad79-7ff5bfff66-b84z4\" (UID: \"4a4333a5-791b-42d7-a9cf-5dc3c3a95165\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-b2ad79-7ff5bfff66-b84z4"
Jul 28 18:30:05.280072 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:05.280046    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a4333a5-791b-42d7-a9cf-5dc3c3a95165-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-b2ad79-7ff5bfff66-b84z4\" (UID: \"4a4333a5-791b-42d7-a9cf-5dc3c3a95165\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-b2ad79-7ff5bfff66-b84z4"
Jul 28 18:30:05.289111 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:05.289084    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkmdl\" (UniqueName: \"kubernetes.io/projected/4a4333a5-791b-42d7-a9cf-5dc3c3a95165-kube-api-access-zkmdl\") pod \"payload-processing-e2e-ipp-cleanup-b2ad79-7ff5bfff66-b84z4\" (UID: \"4a4333a5-791b-42d7-a9cf-5dc3c3a95165\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-b2ad79-7ff5bfff66-b84z4"
Jul 28 18:30:05.333661 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:05.333626    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2ad79-f664b84d5-mzzxz"]
Jul 28 18:30:05.334670 ip-10-0-136-126 kubenswrapper[2563]: W0728 18:30:05.334643    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bef84b4_2ca4_4eb3_a79c_34a96b5efb70.slice/crio-683300fb90792549be09ff9b1598cec9ca8d9ce75fbd3fd0ccd0d9a6b2df6f1e WatchSource:0}: Error finding container 683300fb90792549be09ff9b1598cec9ca8d9ce75fbd3fd0ccd0d9a6b2df6f1e: Status 404 returned error can't find the container with id 683300fb90792549be09ff9b1598cec9ca8d9ce75fbd3fd0ccd0d9a6b2df6f1e
Jul 28 18:30:05.379976 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:05.379949    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8982z\" (UniqueName: \"kubernetes.io/projected/a5013d74-382f-4636-a97e-3126bc23be29-kube-api-access-8982z\") pod \"payload-pre-processing-e2e-ipp-cleanup-b2ad79-5c9bd7878d-6fkv2\" (UID: \"a5013d74-382f-4636-a97e-3126bc23be29\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b2ad79-5c9bd7878d-6fkv2"
Jul 28 18:30:05.380109 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:05.380012    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5013d74-382f-4636-a97e-3126bc23be29-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-b2ad79-5c9bd7878d-6fkv2\" (UID: \"a5013d74-382f-4636-a97e-3126bc23be29\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b2ad79-5c9bd7878d-6fkv2"
Jul 28 18:30:05.380666 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:05.380646    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5013d74-382f-4636-a97e-3126bc23be29-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-b2ad79-5c9bd7878d-6fkv2\" (UID: \"a5013d74-382f-4636-a97e-3126bc23be29\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b2ad79-5c9bd7878d-6fkv2"
Jul 28 18:30:05.388897 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:05.388879    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8982z\" (UniqueName: \"kubernetes.io/projected/a5013d74-382f-4636-a97e-3126bc23be29-kube-api-access-8982z\") pod \"payload-pre-processing-e2e-ipp-cleanup-b2ad79-5c9bd7878d-6fkv2\" (UID: \"a5013d74-382f-4636-a97e-3126bc23be29\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b2ad79-5c9bd7878d-6fkv2"
Jul 28 18:30:05.407796 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:05.407772    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-b2ad79-7ff5bfff66-b84z4"
Jul 28 18:30:05.470946 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:05.470915    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b2ad79-5c9bd7878d-6fkv2"
Jul 28 18:30:05.558581 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:05.557024    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-b2ad79-7ff5bfff66-b84z4"]
Jul 28 18:30:05.561264 ip-10-0-136-126 kubenswrapper[2563]: W0728 18:30:05.561236    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a4333a5_791b_42d7_a9cf_5dc3c3a95165.slice/crio-d42da16035575e1a065a8f79e58344e4cf7c5d05eac4328ab44fdc56315bb063 WatchSource:0}: Error finding container d42da16035575e1a065a8f79e58344e4cf7c5d05eac4328ab44fdc56315bb063: Status 404 returned error can't find the container with id d42da16035575e1a065a8f79e58344e4cf7c5d05eac4328ab44fdc56315bb063
Jul 28 18:30:05.849696 ip-10-0-136-126 kubenswrapper[2563]: W0728 18:30:05.849668    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5013d74_382f_4636_a97e_3126bc23be29.slice/crio-b634bf2dde1b85393a21be197bc73b71aa6d85b0f59e1739672f6a12ea4cf6a8 WatchSource:0}: Error finding container b634bf2dde1b85393a21be197bc73b71aa6d85b0f59e1739672f6a12ea4cf6a8: Status 404 returned error can't find the container with id b634bf2dde1b85393a21be197bc73b71aa6d85b0f59e1739672f6a12ea4cf6a8
Jul 28 18:30:05.857090 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:05.857061    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b2ad79-5c9bd7878d-6fkv2"]
Jul 28 18:30:06.148214 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:06.148171    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b2ad79-5c9bd7878d-6fkv2" event={"ID":"a5013d74-382f-4636-a97e-3126bc23be29","Type":"ContainerStarted","Data":"6e8178a5b017372252d19d486a1818db732c5f2336cc2af59cbed72926b9619a"}
Jul 28 18:30:06.148395 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:06.148282    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b2ad79-5c9bd7878d-6fkv2" event={"ID":"a5013d74-382f-4636-a97e-3126bc23be29","Type":"ContainerStarted","Data":"b634bf2dde1b85393a21be197bc73b71aa6d85b0f59e1739672f6a12ea4cf6a8"}
Jul 28 18:30:06.148395 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:06.148314    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b2ad79-5c9bd7878d-6fkv2"
Jul 28 18:30:06.150014 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:06.149991    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2ad79-f664b84d5-mzzxz" event={"ID":"7bef84b4-2ca4-4eb3-a79c-34a96b5efb70","Type":"ContainerStarted","Data":"8ef6c5d9128da67a0f9827ba1485b9a77cb9bbabe1fe0316a5eb67ba8f80c6aa"}
Jul 28 18:30:06.150144 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:06.150041    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2ad79-f664b84d5-mzzxz" event={"ID":"7bef84b4-2ca4-4eb3-a79c-34a96b5efb70","Type":"ContainerStarted","Data":"683300fb90792549be09ff9b1598cec9ca8d9ce75fbd3fd0ccd0d9a6b2df6f1e"}
Jul 28 18:30:06.150144 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:06.150072    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2ad79-f664b84d5-mzzxz"
Jul 28 18:30:06.151462 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:06.151426    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-b2ad79-7ff5bfff66-b84z4" event={"ID":"4a4333a5-791b-42d7-a9cf-5dc3c3a95165","Type":"ContainerStarted","Data":"9e3d775a41c1d5842f0083baba4d01ea6ca33e6d90a1c484ee34439c529fe7f4"}
Jul 28 18:30:06.151560 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:06.151480    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-b2ad79-7ff5bfff66-b84z4" event={"ID":"4a4333a5-791b-42d7-a9cf-5dc3c3a95165","Type":"ContainerStarted","Data":"d42da16035575e1a065a8f79e58344e4cf7c5d05eac4328ab44fdc56315bb063"}
Jul 28 18:30:06.151560 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:06.151556    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-b2ad79-7ff5bfff66-b84z4"
Jul 28 18:30:06.168022 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:06.167977    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b2ad79-5c9bd7878d-6fkv2" podStartSLOduration=1.167963761 podStartE2EDuration="1.167963761s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:30:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:30:06.166400247 +0000 UTC m=+3344.827556604" watchObservedRunningTime="2026-07-28 18:30:06.167963761 +0000 UTC m=+3344.829120188"
Jul 28 18:30:06.184531 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:06.184488    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2ad79-f664b84d5-mzzxz" podStartSLOduration=1.724899049 podStartE2EDuration="2.184475684s" totalImagesPullingTime="459.576635ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:30:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:30:06.183908732 +0000 UTC m=+3344.845065080" watchObservedRunningTime="2026-07-28 18:30:06.184475684 +0000 UTC m=+3344.845632040"
Jul 28 18:30:06.200415 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:06.200375    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-b2ad79-7ff5bfff66-b84z4" podStartSLOduration=1.200364615 podStartE2EDuration="1.200364615s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:30:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:30:06.200209383 +0000 UTC m=+3344.861365739" watchObservedRunningTime="2026-07-28 18:30:06.200364615 +0000 UTC m=+3344.861520970"
Jul 28 18:30:12.164228 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:12.164196    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2ad79-f664b84d5-mzzxz"
Jul 28 18:30:17.161160 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:17.161117    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b2ad79-5c9bd7878d-6fkv2"
Jul 28 18:30:17.161568 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:17.161192    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-b2ad79-7ff5bfff66-b84z4"
Jul 28 18:30:18.242135 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:18.242099    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:30:25.844997 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:25.844962    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2ad79-f664b84d5-mzzxz"]
Jul 28 18:30:25.845487 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:25.845225    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2ad79-f664b84d5-mzzxz" podUID="7bef84b4-2ca4-4eb3-a79c-34a96b5efb70" containerName="maas-api" containerID="cri-o://8ef6c5d9128da67a0f9827ba1485b9a77cb9bbabe1fe0316a5eb67ba8f80c6aa" gracePeriod=30
Jul 28 18:30:25.869834 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:25.869810    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:30:25.910126 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:25.910092    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-b2ad79-7ff5bfff66-b84z4"]
Jul 28 18:30:25.910527 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:25.910328    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-b2ad79-7ff5bfff66-b84z4" podUID="4a4333a5-791b-42d7-a9cf-5dc3c3a95165" containerName="payload-processing" containerID="cri-o://9e3d775a41c1d5842f0083baba4d01ea6ca33e6d90a1c484ee34439c529fe7f4" gracePeriod=30
Jul 28 18:30:25.916314 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:25.916172    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b2ad79-5c9bd7878d-6fkv2"]
Jul 28 18:30:25.916461 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:25.916394    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b2ad79-5c9bd7878d-6fkv2" podUID="a5013d74-382f-4636-a97e-3126bc23be29" containerName="payload-pre-processing" containerID="cri-o://6e8178a5b017372252d19d486a1818db732c5f2336cc2af59cbed72926b9619a" gracePeriod=30
Jul 28 18:30:25.959099 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:25.959072    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:30:26.188321 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:26.188298    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-b2ad79-7ff5bfff66-b84z4"
Jul 28 18:30:26.192820 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:26.192768    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:30:26.217751 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:26.217730    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2ad79-f664b84d5-mzzxz"
Jul 28 18:30:26.252222 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:26.252200    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b2ad79-5c9bd7878d-6fkv2"
Jul 28 18:30:26.257742 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:26.257718    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:30:26.278354 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:26.278328    2563 generic.go:356] "Generic (PLEG): container finished" podID="4a4333a5-791b-42d7-a9cf-5dc3c3a95165" containerID="9e3d775a41c1d5842f0083baba4d01ea6ca33e6d90a1c484ee34439c529fe7f4" exitCode=0
Jul 28 18:30:26.278506 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:26.278377    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-b2ad79-7ff5bfff66-b84z4"
Jul 28 18:30:26.278506 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:26.278415    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-b2ad79-7ff5bfff66-b84z4" event={"ID":"4a4333a5-791b-42d7-a9cf-5dc3c3a95165","Type":"ContainerDied","Data":"9e3d775a41c1d5842f0083baba4d01ea6ca33e6d90a1c484ee34439c529fe7f4"}
Jul 28 18:30:26.278506 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:26.278483    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-b2ad79-7ff5bfff66-b84z4" event={"ID":"4a4333a5-791b-42d7-a9cf-5dc3c3a95165","Type":"ContainerDied","Data":"d42da16035575e1a065a8f79e58344e4cf7c5d05eac4328ab44fdc56315bb063"}
Jul 28 18:30:26.278662 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:26.278510    2563 scope.go:117] "RemoveContainer" containerID="9e3d775a41c1d5842f0083baba4d01ea6ca33e6d90a1c484ee34439c529fe7f4"
Jul 28 18:30:26.279553 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:26.279532    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zkmdl\" (UniqueName: \"kubernetes.io/projected/4a4333a5-791b-42d7-a9cf-5dc3c3a95165-kube-api-access-zkmdl\") pod \"4a4333a5-791b-42d7-a9cf-5dc3c3a95165\" (UID: \"4a4333a5-791b-42d7-a9cf-5dc3c3a95165\") "
Jul 28 18:30:26.279634 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:26.279600    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a4333a5-791b-42d7-a9cf-5dc3c3a95165-plugins-config-volume\") pod \"4a4333a5-791b-42d7-a9cf-5dc3c3a95165\" (UID: \"4a4333a5-791b-42d7-a9cf-5dc3c3a95165\") "
Jul 28 18:30:26.280076 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:26.280052    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a4333a5-791b-42d7-a9cf-5dc3c3a95165-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4a4333a5-791b-42d7-a9cf-5dc3c3a95165" (UID: "4a4333a5-791b-42d7-a9cf-5dc3c3a95165"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:30:26.280160 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:26.280074    2563 generic.go:356] "Generic (PLEG): container finished" podID="a5013d74-382f-4636-a97e-3126bc23be29" containerID="6e8178a5b017372252d19d486a1818db732c5f2336cc2af59cbed72926b9619a" exitCode=0
Jul 28 18:30:26.280160 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:26.280143    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b2ad79-5c9bd7878d-6fkv2"
Jul 28 18:30:26.280273 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:26.280228    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b2ad79-5c9bd7878d-6fkv2" event={"ID":"a5013d74-382f-4636-a97e-3126bc23be29","Type":"ContainerDied","Data":"6e8178a5b017372252d19d486a1818db732c5f2336cc2af59cbed72926b9619a"}
Jul 28 18:30:26.280328 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:26.280269    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b2ad79-5c9bd7878d-6fkv2" event={"ID":"a5013d74-382f-4636-a97e-3126bc23be29","Type":"ContainerDied","Data":"b634bf2dde1b85393a21be197bc73b71aa6d85b0f59e1739672f6a12ea4cf6a8"}
Jul 28 18:30:26.281826 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:26.281752    2563 generic.go:356] "Generic (PLEG): container finished" podID="7bef84b4-2ca4-4eb3-a79c-34a96b5efb70" containerID="8ef6c5d9128da67a0f9827ba1485b9a77cb9bbabe1fe0316a5eb67ba8f80c6aa" exitCode=0
Jul 28 18:30:26.281826 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:26.281792    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2ad79-f664b84d5-mzzxz" event={"ID":"7bef84b4-2ca4-4eb3-a79c-34a96b5efb70","Type":"ContainerDied","Data":"8ef6c5d9128da67a0f9827ba1485b9a77cb9bbabe1fe0316a5eb67ba8f80c6aa"}
Jul 28 18:30:26.281977 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:26.281826    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2ad79-f664b84d5-mzzxz"
Jul 28 18:30:26.281977 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:26.281840    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2ad79-f664b84d5-mzzxz" event={"ID":"7bef84b4-2ca4-4eb3-a79c-34a96b5efb70","Type":"ContainerDied","Data":"683300fb90792549be09ff9b1598cec9ca8d9ce75fbd3fd0ccd0d9a6b2df6f1e"}
Jul 28 18:30:26.281977 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:26.281850    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a4333a5-791b-42d7-a9cf-5dc3c3a95165-kube-api-access-zkmdl" (OuterVolumeSpecName: "kube-api-access-zkmdl") pod "4a4333a5-791b-42d7-a9cf-5dc3c3a95165" (UID: "4a4333a5-791b-42d7-a9cf-5dc3c3a95165"). InnerVolumeSpecName "kube-api-access-zkmdl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:30:26.288924 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:26.288905    2563 scope.go:117] "RemoveContainer" containerID="9e3d775a41c1d5842f0083baba4d01ea6ca33e6d90a1c484ee34439c529fe7f4"
Jul 28 18:30:26.289211 ip-10-0-136-126 kubenswrapper[2563]: E0728 18:30:26.289191    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e3d775a41c1d5842f0083baba4d01ea6ca33e6d90a1c484ee34439c529fe7f4\": container with ID starting with 9e3d775a41c1d5842f0083baba4d01ea6ca33e6d90a1c484ee34439c529fe7f4 not found: ID does not exist" containerID="9e3d775a41c1d5842f0083baba4d01ea6ca33e6d90a1c484ee34439c529fe7f4"
Jul 28 18:30:26.289281 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:26.289220    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e3d775a41c1d5842f0083baba4d01ea6ca33e6d90a1c484ee34439c529fe7f4"} err="failed to get container status \"9e3d775a41c1d5842f0083baba4d01ea6ca33e6d90a1c484ee34439c529fe7f4\": rpc error: code = NotFound desc = could not find container \"9e3d775a41c1d5842f0083baba4d01ea6ca33e6d90a1c484ee34439c529fe7f4\": container with ID starting with 9e3d775a41c1d5842f0083baba4d01ea6ca33e6d90a1c484ee34439c529fe7f4 not found: ID does not exist"
Jul 28 18:30:26.289281 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:26.289237    2563 scope.go:117] "RemoveContainer" containerID="6e8178a5b017372252d19d486a1818db732c5f2336cc2af59cbed72926b9619a"
Jul 28 18:30:26.297169 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:26.297148    2563 scope.go:117] "RemoveContainer" containerID="6e8178a5b017372252d19d486a1818db732c5f2336cc2af59cbed72926b9619a"
Jul 28 18:30:26.297449 ip-10-0-136-126 kubenswrapper[2563]: E0728 18:30:26.297419    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e8178a5b017372252d19d486a1818db732c5f2336cc2af59cbed72926b9619a\": container with ID starting with 6e8178a5b017372252d19d486a1818db732c5f2336cc2af59cbed72926b9619a not found: ID does not exist" containerID="6e8178a5b017372252d19d486a1818db732c5f2336cc2af59cbed72926b9619a"
Jul 28 18:30:26.297520 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:26.297455    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e8178a5b017372252d19d486a1818db732c5f2336cc2af59cbed72926b9619a"} err="failed to get container status \"6e8178a5b017372252d19d486a1818db732c5f2336cc2af59cbed72926b9619a\": rpc error: code = NotFound desc = could not find container \"6e8178a5b017372252d19d486a1818db732c5f2336cc2af59cbed72926b9619a\": container with ID starting with 6e8178a5b017372252d19d486a1818db732c5f2336cc2af59cbed72926b9619a not found: ID does not exist"
Jul 28 18:30:26.297520 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:26.297471    2563 scope.go:117] "RemoveContainer" containerID="8ef6c5d9128da67a0f9827ba1485b9a77cb9bbabe1fe0316a5eb67ba8f80c6aa"
Jul 28 18:30:26.305807 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:26.305786    2563 scope.go:117] "RemoveContainer" containerID="8ef6c5d9128da67a0f9827ba1485b9a77cb9bbabe1fe0316a5eb67ba8f80c6aa"
Jul 28 18:30:26.306054 ip-10-0-136-126 kubenswrapper[2563]: E0728 18:30:26.306028    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ef6c5d9128da67a0f9827ba1485b9a77cb9bbabe1fe0316a5eb67ba8f80c6aa\": container with ID starting with 8ef6c5d9128da67a0f9827ba1485b9a77cb9bbabe1fe0316a5eb67ba8f80c6aa not found: ID does not exist" containerID="8ef6c5d9128da67a0f9827ba1485b9a77cb9bbabe1fe0316a5eb67ba8f80c6aa"
Jul 28 18:30:26.306120 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:26.306054    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ef6c5d9128da67a0f9827ba1485b9a77cb9bbabe1fe0316a5eb67ba8f80c6aa"} err="failed to get container status \"8ef6c5d9128da67a0f9827ba1485b9a77cb9bbabe1fe0316a5eb67ba8f80c6aa\": rpc error: code = NotFound desc = could not find container \"8ef6c5d9128da67a0f9827ba1485b9a77cb9bbabe1fe0316a5eb67ba8f80c6aa\": container with ID starting with 8ef6c5d9128da67a0f9827ba1485b9a77cb9bbabe1fe0316a5eb67ba8f80c6aa not found: ID does not exist"
Jul 28 18:30:26.380217 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:26.380146    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7bef84b4-2ca4-4eb3-a79c-34a96b5efb70-maas-api-tls\") pod \"7bef84b4-2ca4-4eb3-a79c-34a96b5efb70\" (UID: \"7bef84b4-2ca4-4eb3-a79c-34a96b5efb70\") "
Jul 28 18:30:26.380217 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:26.380205    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7bef84b4-2ca4-4eb3-a79c-34a96b5efb70-openshift-service-ca\") pod \"7bef84b4-2ca4-4eb3-a79c-34a96b5efb70\" (UID: \"7bef84b4-2ca4-4eb3-a79c-34a96b5efb70\") "
Jul 28 18:30:26.380376 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:26.380317    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5013d74-382f-4636-a97e-3126bc23be29-plugins-config-volume\") pod \"a5013d74-382f-4636-a97e-3126bc23be29\" (UID: \"a5013d74-382f-4636-a97e-3126bc23be29\") "
Jul 28 18:30:26.380413 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:26.380390    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8982z\" (UniqueName: \"kubernetes.io/projected/a5013d74-382f-4636-a97e-3126bc23be29-kube-api-access-8982z\") pod \"a5013d74-382f-4636-a97e-3126bc23be29\" (UID: \"a5013d74-382f-4636-a97e-3126bc23be29\") "
Jul 28 18:30:26.380518 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:26.380501    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h8s4n\" (UniqueName: \"kubernetes.io/projected/7bef84b4-2ca4-4eb3-a79c-34a96b5efb70-kube-api-access-h8s4n\") pod \"7bef84b4-2ca4-4eb3-a79c-34a96b5efb70\" (UID: \"7bef84b4-2ca4-4eb3-a79c-34a96b5efb70\") "
Jul 28 18:30:26.380638 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:26.380615    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7bef84b4-2ca4-4eb3-a79c-34a96b5efb70-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7bef84b4-2ca4-4eb3-a79c-34a96b5efb70" (UID: "7bef84b4-2ca4-4eb3-a79c-34a96b5efb70"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:30:26.380703 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:26.380670    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5013d74-382f-4636-a97e-3126bc23be29-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a5013d74-382f-4636-a97e-3126bc23be29" (UID: "a5013d74-382f-4636-a97e-3126bc23be29"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:30:26.380855 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:26.380838    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7bef84b4-2ca4-4eb3-a79c-34a96b5efb70-openshift-service-ca\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:30:26.380916 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:26.380860    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zkmdl\" (UniqueName: \"kubernetes.io/projected/4a4333a5-791b-42d7-a9cf-5dc3c3a95165-kube-api-access-zkmdl\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:30:26.380916 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:26.380876    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a4333a5-791b-42d7-a9cf-5dc3c3a95165-plugins-config-volume\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:30:26.380916 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:26.380892    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5013d74-382f-4636-a97e-3126bc23be29-plugins-config-volume\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:30:26.382328 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:26.382301    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7bef84b4-2ca4-4eb3-a79c-34a96b5efb70-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7bef84b4-2ca4-4eb3-a79c-34a96b5efb70" (UID: "7bef84b4-2ca4-4eb3-a79c-34a96b5efb70"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:30:26.382462 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:26.382342    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5013d74-382f-4636-a97e-3126bc23be29-kube-api-access-8982z" (OuterVolumeSpecName: "kube-api-access-8982z") pod "a5013d74-382f-4636-a97e-3126bc23be29" (UID: "a5013d74-382f-4636-a97e-3126bc23be29"). InnerVolumeSpecName "kube-api-access-8982z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:30:26.382462 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:26.382392    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bef84b4-2ca4-4eb3-a79c-34a96b5efb70-kube-api-access-h8s4n" (OuterVolumeSpecName: "kube-api-access-h8s4n") pod "7bef84b4-2ca4-4eb3-a79c-34a96b5efb70" (UID: "7bef84b4-2ca4-4eb3-a79c-34a96b5efb70"). InnerVolumeSpecName "kube-api-access-h8s4n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:30:26.478031 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:26.478000    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:30:26.482189 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:26.482167    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8982z\" (UniqueName: \"kubernetes.io/projected/a5013d74-382f-4636-a97e-3126bc23be29-kube-api-access-8982z\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:30:26.482189 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:26.482190    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h8s4n\" (UniqueName: \"kubernetes.io/projected/7bef84b4-2ca4-4eb3-a79c-34a96b5efb70-kube-api-access-h8s4n\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:30:26.482315 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:26.482201    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7bef84b4-2ca4-4eb3-a79c-34a96b5efb70-maas-api-tls\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:30:26.602663 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:26.602634    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-b2ad79-7ff5bfff66-b84z4"]
Jul 28 18:30:26.607088 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:26.607067    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-b2ad79-7ff5bfff66-b84z4"]
Jul 28 18:30:26.617555 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:26.617532    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b2ad79-5c9bd7878d-6fkv2"]
Jul 28 18:30:26.621992 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:26.621973    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b2ad79-5c9bd7878d-6fkv2"]
Jul 28 18:30:26.633757 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:26.633682    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2ad79-f664b84d5-mzzxz"]
Jul 28 18:30:26.639847 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:26.639812    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2ad79-f664b84d5-mzzxz"]
Jul 28 18:30:27.853776 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:27.853744    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a4333a5-791b-42d7-a9cf-5dc3c3a95165" path="/var/lib/kubelet/pods/4a4333a5-791b-42d7-a9cf-5dc3c3a95165/volumes"
Jul 28 18:30:27.854120 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:27.854111    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7bef84b4-2ca4-4eb3-a79c-34a96b5efb70" path="/var/lib/kubelet/pods/7bef84b4-2ca4-4eb3-a79c-34a96b5efb70/volumes"
Jul 28 18:30:27.854414 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:27.854401    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5013d74-382f-4636-a97e-3126bc23be29" path="/var/lib/kubelet/pods/a5013d74-382f-4636-a97e-3126bc23be29/volumes"
Jul 28 18:30:33.476680 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:33.476644    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:30:33.575734 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:33.575694    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:30:33.769630 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:33.769543    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:30:34.008504 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:34.008465    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:30:52.905818 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:52.905781    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-faea92-6b4c94ccc-s4ld7"]
Jul 28 18:30:52.906313 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:52.906048    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-faea92-6b4c94ccc-s4ld7" podUID="f12d0d22-4131-455a-9541-1fed5c26c08e" containerName="maas-api" containerID="cri-o://0210b6eea764b2d5826af85500cef76f7a573ecc982e85ad075006aca98c20f8" gracePeriod=30
Jul 28 18:30:52.926321 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:52.926290    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:30:52.932423 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:52.932397    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-faea92-29754390-pbfr8"]
Jul 28 18:30:52.937320 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:52.937297    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-faea92-29754390-pbfr8"]
Jul 28 18:30:52.975727 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:52.975696    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-faea92-688ff66c9b-mpgxs"]
Jul 28 18:30:52.975982 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:52.975940    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-faea92-688ff66c9b-mpgxs" podUID="53008497-534a-4934-84ed-d6698ba729fa" containerName="payload-pre-processing" containerID="cri-o://58ae62535622d36cc03487bfd08131ae1800e83cdd72331819978bbd9362d2f1" gracePeriod=30
Jul 28 18:30:52.997222 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:52.997198    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:30:53.052996 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:53.052967    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:30:53.153746 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:53.153661    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:30:53.239133 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:53.239108    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-faea92-6b4c94ccc-s4ld7"
Jul 28 18:30:53.268604 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:53.268580    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-faea92-688ff66c9b-mpgxs"
Jul 28 18:30:53.321022 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:53.320987    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vcmnv\" (UniqueName: \"kubernetes.io/projected/f12d0d22-4131-455a-9541-1fed5c26c08e-kube-api-access-vcmnv\") pod \"f12d0d22-4131-455a-9541-1fed5c26c08e\" (UID: \"f12d0d22-4131-455a-9541-1fed5c26c08e\") "
Jul 28 18:30:53.321194 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:53.321044    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f12d0d22-4131-455a-9541-1fed5c26c08e-openshift-service-ca\") pod \"f12d0d22-4131-455a-9541-1fed5c26c08e\" (UID: \"f12d0d22-4131-455a-9541-1fed5c26c08e\") "
Jul 28 18:30:53.321266 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:53.321232    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53008497-534a-4934-84ed-d6698ba729fa-plugins-config-volume\") pod \"53008497-534a-4934-84ed-d6698ba729fa\" (UID: \"53008497-534a-4934-84ed-d6698ba729fa\") "
Jul 28 18:30:53.321364 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:53.321349    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f12d0d22-4131-455a-9541-1fed5c26c08e-maas-api-tls\") pod \"f12d0d22-4131-455a-9541-1fed5c26c08e\" (UID: \"f12d0d22-4131-455a-9541-1fed5c26c08e\") "
Jul 28 18:30:53.321450 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:53.321382    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4jqtg\" (UniqueName: \"kubernetes.io/projected/53008497-534a-4934-84ed-d6698ba729fa-kube-api-access-4jqtg\") pod \"53008497-534a-4934-84ed-d6698ba729fa\" (UID: \"53008497-534a-4934-84ed-d6698ba729fa\") "
Jul 28 18:30:53.321450 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:53.321348    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f12d0d22-4131-455a-9541-1fed5c26c08e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f12d0d22-4131-455a-9541-1fed5c26c08e" (UID: "f12d0d22-4131-455a-9541-1fed5c26c08e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:30:53.321585 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:53.321566    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53008497-534a-4934-84ed-d6698ba729fa-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "53008497-534a-4934-84ed-d6698ba729fa" (UID: "53008497-534a-4934-84ed-d6698ba729fa"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:30:53.321710 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:53.321691    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f12d0d22-4131-455a-9541-1fed5c26c08e-openshift-service-ca\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:30:53.321710 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:53.321711    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53008497-534a-4934-84ed-d6698ba729fa-plugins-config-volume\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:30:53.323234 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:53.323207    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f12d0d22-4131-455a-9541-1fed5c26c08e-kube-api-access-vcmnv" (OuterVolumeSpecName: "kube-api-access-vcmnv") pod "f12d0d22-4131-455a-9541-1fed5c26c08e" (UID: "f12d0d22-4131-455a-9541-1fed5c26c08e"). InnerVolumeSpecName "kube-api-access-vcmnv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:30:53.323319 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:53.323258    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f12d0d22-4131-455a-9541-1fed5c26c08e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f12d0d22-4131-455a-9541-1fed5c26c08e" (UID: "f12d0d22-4131-455a-9541-1fed5c26c08e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:30:53.323319 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:53.323296    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53008497-534a-4934-84ed-d6698ba729fa-kube-api-access-4jqtg" (OuterVolumeSpecName: "kube-api-access-4jqtg") pod "53008497-534a-4934-84ed-d6698ba729fa" (UID: "53008497-534a-4934-84ed-d6698ba729fa"). InnerVolumeSpecName "kube-api-access-4jqtg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:30:53.422486 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:53.422429    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f12d0d22-4131-455a-9541-1fed5c26c08e-maas-api-tls\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:30:53.422486 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:53.422482    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4jqtg\" (UniqueName: \"kubernetes.io/projected/53008497-534a-4934-84ed-d6698ba729fa-kube-api-access-4jqtg\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:30:53.422486 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:53.422493    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vcmnv\" (UniqueName: \"kubernetes.io/projected/f12d0d22-4131-455a-9541-1fed5c26c08e-kube-api-access-vcmnv\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:30:53.449695 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:53.449608    2563 generic.go:356] "Generic (PLEG): container finished" podID="53008497-534a-4934-84ed-d6698ba729fa" containerID="58ae62535622d36cc03487bfd08131ae1800e83cdd72331819978bbd9362d2f1" exitCode=0
Jul 28 18:30:53.449695 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:53.449652    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-faea92-688ff66c9b-mpgxs"
Jul 28 18:30:53.449903 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:53.449701    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-faea92-688ff66c9b-mpgxs" event={"ID":"53008497-534a-4934-84ed-d6698ba729fa","Type":"ContainerDied","Data":"58ae62535622d36cc03487bfd08131ae1800e83cdd72331819978bbd9362d2f1"}
Jul 28 18:30:53.449903 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:53.449766    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-faea92-688ff66c9b-mpgxs" event={"ID":"53008497-534a-4934-84ed-d6698ba729fa","Type":"ContainerDied","Data":"b64b046c01ad258dd88198ae4094205cedfb2f4bfccc3049ed7384fbfa4bdc85"}
Jul 28 18:30:53.449903 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:53.449805    2563 scope.go:117] "RemoveContainer" containerID="58ae62535622d36cc03487bfd08131ae1800e83cdd72331819978bbd9362d2f1"
Jul 28 18:30:53.451416 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:53.451393    2563 generic.go:356] "Generic (PLEG): container finished" podID="f12d0d22-4131-455a-9541-1fed5c26c08e" containerID="0210b6eea764b2d5826af85500cef76f7a573ecc982e85ad075006aca98c20f8" exitCode=0
Jul 28 18:30:53.451538 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:53.451444    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-faea92-6b4c94ccc-s4ld7"
Jul 28 18:30:53.451538 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:53.451464    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-faea92-6b4c94ccc-s4ld7" event={"ID":"f12d0d22-4131-455a-9541-1fed5c26c08e","Type":"ContainerDied","Data":"0210b6eea764b2d5826af85500cef76f7a573ecc982e85ad075006aca98c20f8"}
Jul 28 18:30:53.451538 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:53.451506    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-faea92-6b4c94ccc-s4ld7" event={"ID":"f12d0d22-4131-455a-9541-1fed5c26c08e","Type":"ContainerDied","Data":"4177732444f90b004f5f7da3c3743f00522b1e4e6fc640a07fb3f2ea3d389737"}
Jul 28 18:30:53.459340 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:53.459317    2563 scope.go:117] "RemoveContainer" containerID="58ae62535622d36cc03487bfd08131ae1800e83cdd72331819978bbd9362d2f1"
Jul 28 18:30:53.459612 ip-10-0-136-126 kubenswrapper[2563]: E0728 18:30:53.459595    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58ae62535622d36cc03487bfd08131ae1800e83cdd72331819978bbd9362d2f1\": container with ID starting with 58ae62535622d36cc03487bfd08131ae1800e83cdd72331819978bbd9362d2f1 not found: ID does not exist" containerID="58ae62535622d36cc03487bfd08131ae1800e83cdd72331819978bbd9362d2f1"
Jul 28 18:30:53.459664 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:53.459619    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58ae62535622d36cc03487bfd08131ae1800e83cdd72331819978bbd9362d2f1"} err="failed to get container status \"58ae62535622d36cc03487bfd08131ae1800e83cdd72331819978bbd9362d2f1\": rpc error: code = NotFound desc = could not find container \"58ae62535622d36cc03487bfd08131ae1800e83cdd72331819978bbd9362d2f1\": container with ID starting with 58ae62535622d36cc03487bfd08131ae1800e83cdd72331819978bbd9362d2f1 not found: ID does not exist"
Jul 28 18:30:53.459664 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:53.459642    2563 scope.go:117] "RemoveContainer" containerID="0210b6eea764b2d5826af85500cef76f7a573ecc982e85ad075006aca98c20f8"
Jul 28 18:30:53.467717 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:53.467703    2563 scope.go:117] "RemoveContainer" containerID="0210b6eea764b2d5826af85500cef76f7a573ecc982e85ad075006aca98c20f8"
Jul 28 18:30:53.467966 ip-10-0-136-126 kubenswrapper[2563]: E0728 18:30:53.467949    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0210b6eea764b2d5826af85500cef76f7a573ecc982e85ad075006aca98c20f8\": container with ID starting with 0210b6eea764b2d5826af85500cef76f7a573ecc982e85ad075006aca98c20f8 not found: ID does not exist" containerID="0210b6eea764b2d5826af85500cef76f7a573ecc982e85ad075006aca98c20f8"
Jul 28 18:30:53.468016 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:53.467974    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0210b6eea764b2d5826af85500cef76f7a573ecc982e85ad075006aca98c20f8"} err="failed to get container status \"0210b6eea764b2d5826af85500cef76f7a573ecc982e85ad075006aca98c20f8\": rpc error: code = NotFound desc = could not find container \"0210b6eea764b2d5826af85500cef76f7a573ecc982e85ad075006aca98c20f8\": container with ID starting with 0210b6eea764b2d5826af85500cef76f7a573ecc982e85ad075006aca98c20f8 not found: ID does not exist"
Jul 28 18:30:53.474278 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:53.474253    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-faea92-688ff66c9b-mpgxs"]
Jul 28 18:30:53.480874 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:53.480853    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-faea92-688ff66c9b-mpgxs"]
Jul 28 18:30:53.492003 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:53.491983    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-faea92-6b4c94ccc-s4ld7"]
Jul 28 18:30:53.496162 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:53.496143    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-faea92-6b4c94ccc-s4ld7"]
Jul 28 18:30:53.574239 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:53.574207    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:30:53.853815 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:53.853783    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53008497-534a-4934-84ed-d6698ba729fa" path="/var/lib/kubelet/pods/53008497-534a-4934-84ed-d6698ba729fa/volumes"
Jul 28 18:30:53.854071 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:53.854058    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9a71981-6aca-433e-84ed-dd98a4a693a5" path="/var/lib/kubelet/pods/b9a71981-6aca-433e-84ed-dd98a4a693a5/volumes"
Jul 28 18:30:53.854359 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:53.854347    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f12d0d22-4131-455a-9541-1fed5c26c08e" path="/var/lib/kubelet/pods/f12d0d22-4131-455a-9541-1fed5c26c08e/volumes"
Jul 28 18:30:58.657234 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:30:58.657196    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:31:05.118117 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:05.118082    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:31:05.218957 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:05.218921    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:31:35.310280 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:35.310242    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:31:36.551287 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:36.551250    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:31:43.294351 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:43.294314    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:31:43.393059 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:43.393034    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:31:43.591687 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:43.591657    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:31:43.854361 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:43.854286    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:31:51.467189 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:51.467158    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-d53588-c88b8c8c7-5glvj"]
Jul 28 18:31:51.467643 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:51.467365    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d53588-c88b8c8c7-5glvj" podUID="cd7df201-b522-4f85-9cb6-9d4453e3024b" containerName="maas-api" containerID="cri-o://bbbb1c6d5adef54ab2511210b0a7c18c7d7b7155a0f1b9e6e64483a4efd1c05a" gracePeriod=30
Jul 28 18:31:51.491374 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:51.491349    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:31:51.529861 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:51.529828    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d53588-56b7656f98-jtxfv"]
Jul 28 18:31:51.530090 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:51.530064    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-d53588-56b7656f98-jtxfv" podUID="61673ff4-6ce9-4257-b54c-fd3f9912d924" containerName="payload-processing" containerID="cri-o://094d29b91bb85458bba8d35bf05892a290b4637382cc6e8f20f0802fc6a5dcc9" gracePeriod=30
Jul 28 18:31:51.539759 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:51.539725    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-d53588-658f7f56b4-sh2tp"]
Jul 28 18:31:51.540026 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:51.539966    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d53588-658f7f56b4-sh2tp" podUID="31553ab6-42e2-42f0-9f06-ba203456e212" containerName="payload-pre-processing" containerID="cri-o://4e77cc0902e1dc8eab546396b75ff3bfa5797ca995d2a02c4d04badd90cefe04" gracePeriod=30
Jul 28 18:31:51.561000 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:51.560972    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:31:51.661744 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:51.661712    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:31:51.761905 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:51.761732    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:31:51.781617 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:51.781579    2563 generic.go:356] "Generic (PLEG): container finished" podID="61673ff4-6ce9-4257-b54c-fd3f9912d924" containerID="094d29b91bb85458bba8d35bf05892a290b4637382cc6e8f20f0802fc6a5dcc9" exitCode=0
Jul 28 18:31:51.781734 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:51.781651    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-d53588-56b7656f98-jtxfv" event={"ID":"61673ff4-6ce9-4257-b54c-fd3f9912d924","Type":"ContainerDied","Data":"094d29b91bb85458bba8d35bf05892a290b4637382cc6e8f20f0802fc6a5dcc9"}
Jul 28 18:31:51.783924 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:51.783903    2563 generic.go:356] "Generic (PLEG): container finished" podID="31553ab6-42e2-42f0-9f06-ba203456e212" containerID="4e77cc0902e1dc8eab546396b75ff3bfa5797ca995d2a02c4d04badd90cefe04" exitCode=0
Jul 28 18:31:51.784127 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:51.784071    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d53588-658f7f56b4-sh2tp" event={"ID":"31553ab6-42e2-42f0-9f06-ba203456e212","Type":"ContainerDied","Data":"4e77cc0902e1dc8eab546396b75ff3bfa5797ca995d2a02c4d04badd90cefe04"}
Jul 28 18:31:51.786070 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:51.786048    2563 generic.go:356] "Generic (PLEG): container finished" podID="cd7df201-b522-4f85-9cb6-9d4453e3024b" containerID="bbbb1c6d5adef54ab2511210b0a7c18c7d7b7155a0f1b9e6e64483a4efd1c05a" exitCode=0
Jul 28 18:31:51.786180 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:51.786093    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d53588-c88b8c8c7-5glvj" event={"ID":"cd7df201-b522-4f85-9cb6-9d4453e3024b","Type":"ContainerDied","Data":"bbbb1c6d5adef54ab2511210b0a7c18c7d7b7155a0f1b9e6e64483a4efd1c05a"}
Jul 28 18:31:51.814618 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:51.814590    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d53588-c88b8c8c7-5glvj"
Jul 28 18:31:51.851393 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:51.851366    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-d53588-56b7656f98-jtxfv"
Jul 28 18:31:51.858769 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:51.858745    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d53588-658f7f56b4-sh2tp"
Jul 28 18:31:51.950022 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:51.949985    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2fxj5\" (UniqueName: \"kubernetes.io/projected/cd7df201-b522-4f85-9cb6-9d4453e3024b-kube-api-access-2fxj5\") pod \"cd7df201-b522-4f85-9cb6-9d4453e3024b\" (UID: \"cd7df201-b522-4f85-9cb6-9d4453e3024b\") "
Jul 28 18:31:51.950204 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:51.950069    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd7df201-b522-4f85-9cb6-9d4453e3024b-maas-api-tls\") pod \"cd7df201-b522-4f85-9cb6-9d4453e3024b\" (UID: \"cd7df201-b522-4f85-9cb6-9d4453e3024b\") "
Jul 28 18:31:51.950204 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:51.950095    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd7df201-b522-4f85-9cb6-9d4453e3024b-openshift-service-ca\") pod \"cd7df201-b522-4f85-9cb6-9d4453e3024b\" (UID: \"cd7df201-b522-4f85-9cb6-9d4453e3024b\") "
Jul 28 18:31:51.950204 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:51.950135    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-glf69\" (UniqueName: \"kubernetes.io/projected/61673ff4-6ce9-4257-b54c-fd3f9912d924-kube-api-access-glf69\") pod \"61673ff4-6ce9-4257-b54c-fd3f9912d924\" (UID: \"61673ff4-6ce9-4257-b54c-fd3f9912d924\") "
Jul 28 18:31:51.950365 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:51.950207    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61673ff4-6ce9-4257-b54c-fd3f9912d924-plugins-config-volume\") pod \"61673ff4-6ce9-4257-b54c-fd3f9912d924\" (UID: \"61673ff4-6ce9-4257-b54c-fd3f9912d924\") "
Jul 28 18:31:51.950365 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:51.950240    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vvmpx\" (UniqueName: \"kubernetes.io/projected/31553ab6-42e2-42f0-9f06-ba203456e212-kube-api-access-vvmpx\") pod \"31553ab6-42e2-42f0-9f06-ba203456e212\" (UID: \"31553ab6-42e2-42f0-9f06-ba203456e212\") "
Jul 28 18:31:51.950365 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:51.950273    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31553ab6-42e2-42f0-9f06-ba203456e212-plugins-config-volume\") pod \"31553ab6-42e2-42f0-9f06-ba203456e212\" (UID: \"31553ab6-42e2-42f0-9f06-ba203456e212\") "
Jul 28 18:31:51.950572 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:51.950525    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd7df201-b522-4f85-9cb6-9d4453e3024b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cd7df201-b522-4f85-9cb6-9d4453e3024b" (UID: "cd7df201-b522-4f85-9cb6-9d4453e3024b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:31:51.950626 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:51.950597    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61673ff4-6ce9-4257-b54c-fd3f9912d924-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "61673ff4-6ce9-4257-b54c-fd3f9912d924" (UID: "61673ff4-6ce9-4257-b54c-fd3f9912d924"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:31:51.951474 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:51.951123    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd7df201-b522-4f85-9cb6-9d4453e3024b-openshift-service-ca\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:31:51.951474 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:51.951145    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61673ff4-6ce9-4257-b54c-fd3f9912d924-plugins-config-volume\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:31:51.951474 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:51.951361    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31553ab6-42e2-42f0-9f06-ba203456e212-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "31553ab6-42e2-42f0-9f06-ba203456e212" (UID: "31553ab6-42e2-42f0-9f06-ba203456e212"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:31:51.952578 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:51.952557    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61673ff4-6ce9-4257-b54c-fd3f9912d924-kube-api-access-glf69" (OuterVolumeSpecName: "kube-api-access-glf69") pod "61673ff4-6ce9-4257-b54c-fd3f9912d924" (UID: "61673ff4-6ce9-4257-b54c-fd3f9912d924"). InnerVolumeSpecName "kube-api-access-glf69". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:31:51.952696 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:51.952678    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd7df201-b522-4f85-9cb6-9d4453e3024b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cd7df201-b522-4f85-9cb6-9d4453e3024b" (UID: "cd7df201-b522-4f85-9cb6-9d4453e3024b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:31:51.952788 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:51.952773    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd7df201-b522-4f85-9cb6-9d4453e3024b-kube-api-access-2fxj5" (OuterVolumeSpecName: "kube-api-access-2fxj5") pod "cd7df201-b522-4f85-9cb6-9d4453e3024b" (UID: "cd7df201-b522-4f85-9cb6-9d4453e3024b"). InnerVolumeSpecName "kube-api-access-2fxj5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:31:51.953223 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:51.953202    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31553ab6-42e2-42f0-9f06-ba203456e212-kube-api-access-vvmpx" (OuterVolumeSpecName: "kube-api-access-vvmpx") pod "31553ab6-42e2-42f0-9f06-ba203456e212" (UID: "31553ab6-42e2-42f0-9f06-ba203456e212"). InnerVolumeSpecName "kube-api-access-vvmpx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:31:52.052403 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:52.052309    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd7df201-b522-4f85-9cb6-9d4453e3024b-maas-api-tls\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:31:52.052403 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:52.052344    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-glf69\" (UniqueName: \"kubernetes.io/projected/61673ff4-6ce9-4257-b54c-fd3f9912d924-kube-api-access-glf69\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:31:52.052403 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:52.052357    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vvmpx\" (UniqueName: \"kubernetes.io/projected/31553ab6-42e2-42f0-9f06-ba203456e212-kube-api-access-vvmpx\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:31:52.052403 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:52.052369    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31553ab6-42e2-42f0-9f06-ba203456e212-plugins-config-volume\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:31:52.052403 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:52.052380    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2fxj5\" (UniqueName: \"kubernetes.io/projected/cd7df201-b522-4f85-9cb6-9d4453e3024b-kube-api-access-2fxj5\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:31:52.114649 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:52.114616    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:31:52.793561 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:52.793467    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d53588-c88b8c8c7-5glvj" event={"ID":"cd7df201-b522-4f85-9cb6-9d4453e3024b","Type":"ContainerDied","Data":"4a9bb060c4373b98da04a6f30b14c591fff082b7b42cd18cf52a169528292f84"}
Jul 28 18:31:52.793561 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:52.793484    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d53588-c88b8c8c7-5glvj"
Jul 28 18:31:52.793561 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:52.793536    2563 scope.go:117] "RemoveContainer" containerID="bbbb1c6d5adef54ab2511210b0a7c18c7d7b7155a0f1b9e6e64483a4efd1c05a"
Jul 28 18:31:52.795208 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:52.795187    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-d53588-56b7656f98-jtxfv"
Jul 28 18:31:52.795311 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:52.795219    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-d53588-56b7656f98-jtxfv" event={"ID":"61673ff4-6ce9-4257-b54c-fd3f9912d924","Type":"ContainerDied","Data":"8bd58152cc1136a35626d48f0b4216f3483e68dd65c7fa5cad94fa73fb448b43"}
Jul 28 18:31:52.796634 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:52.796599    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d53588-658f7f56b4-sh2tp"
Jul 28 18:31:52.796729 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:52.796606    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d53588-658f7f56b4-sh2tp" event={"ID":"31553ab6-42e2-42f0-9f06-ba203456e212","Type":"ContainerDied","Data":"32d739757f5f3265bda867da97c64ff154608b6cf772e599641d57f7b3054d43"}
Jul 28 18:31:52.803675 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:52.803657    2563 scope.go:117] "RemoveContainer" containerID="094d29b91bb85458bba8d35bf05892a290b4637382cc6e8f20f0802fc6a5dcc9"
Jul 28 18:31:52.813564 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:52.813546    2563 scope.go:117] "RemoveContainer" containerID="4e77cc0902e1dc8eab546396b75ff3bfa5797ca995d2a02c4d04badd90cefe04"
Jul 28 18:31:52.824619 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:52.824595    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-d53588-658f7f56b4-sh2tp"]
Jul 28 18:31:52.829090 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:52.829056    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-d53588-658f7f56b4-sh2tp"]
Jul 28 18:31:52.840947 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:52.840923    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d53588-56b7656f98-jtxfv"]
Jul 28 18:31:52.845902 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:52.845867    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d53588-56b7656f98-jtxfv"]
Jul 28 18:31:52.858271 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:52.858241    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-d53588-c88b8c8c7-5glvj"]
Jul 28 18:31:52.862485 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:52.862459    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-d53588-c88b8c8c7-5glvj"]
Jul 28 18:31:53.853595 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:53.853561    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31553ab6-42e2-42f0-9f06-ba203456e212" path="/var/lib/kubelet/pods/31553ab6-42e2-42f0-9f06-ba203456e212/volumes"
Jul 28 18:31:53.853977 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:53.853919    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61673ff4-6ce9-4257-b54c-fd3f9912d924" path="/var/lib/kubelet/pods/61673ff4-6ce9-4257-b54c-fd3f9912d924/volumes"
Jul 28 18:31:53.854249 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:53.854237    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd7df201-b522-4f85-9cb6-9d4453e3024b" path="/var/lib/kubelet/pods/cd7df201-b522-4f85-9cb6-9d4453e3024b/volumes"
Jul 28 18:31:58.897529 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:58.897492    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:31:58.958837 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:58.958798    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:31:59.210124 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:59.210049    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:31:59.455641 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:31:59.455609    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:32:07.797369 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:07.797333    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-9b5b66-84d4dc76bf-tbcmk"]
Jul 28 18:32:07.797929 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:07.797588    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b5b66-84d4dc76bf-tbcmk" podUID="46cf1643-a035-46a6-82ac-f73568c5936b" containerName="maas-api" containerID="cri-o://c6474a00c89febf3ab08a4e59b14c155a7ccd54b13e1a2ce97afc545ffc5a20a" gracePeriod=30
Jul 28 18:32:07.827164 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:07.827139    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:32:08.012807 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:08.012765    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:32:08.074366 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:08.074341    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b5b66-84d4dc76bf-tbcmk"
Jul 28 18:32:08.098370 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:08.098344    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:32:08.155136 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:08.155101    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:32:08.201230 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:08.201172    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46cf1643-a035-46a6-82ac-f73568c5936b-maas-api-tls\") pod \"46cf1643-a035-46a6-82ac-f73568c5936b\" (UID: \"46cf1643-a035-46a6-82ac-f73568c5936b\") "
Jul 28 18:32:08.201478 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:08.201302    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gdlrm\" (UniqueName: \"kubernetes.io/projected/46cf1643-a035-46a6-82ac-f73568c5936b-kube-api-access-gdlrm\") pod \"46cf1643-a035-46a6-82ac-f73568c5936b\" (UID: \"46cf1643-a035-46a6-82ac-f73568c5936b\") "
Jul 28 18:32:08.201478 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:08.201337    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46cf1643-a035-46a6-82ac-f73568c5936b-openshift-service-ca\") pod \"46cf1643-a035-46a6-82ac-f73568c5936b\" (UID: \"46cf1643-a035-46a6-82ac-f73568c5936b\") "
Jul 28 18:32:08.202079 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:08.202038    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46cf1643-a035-46a6-82ac-f73568c5936b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "46cf1643-a035-46a6-82ac-f73568c5936b" (UID: "46cf1643-a035-46a6-82ac-f73568c5936b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:32:08.203578 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:08.203554    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46cf1643-a035-46a6-82ac-f73568c5936b-kube-api-access-gdlrm" (OuterVolumeSpecName: "kube-api-access-gdlrm") pod "46cf1643-a035-46a6-82ac-f73568c5936b" (UID: "46cf1643-a035-46a6-82ac-f73568c5936b"). InnerVolumeSpecName "kube-api-access-gdlrm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:32:08.203681 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:08.203577    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46cf1643-a035-46a6-82ac-f73568c5936b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "46cf1643-a035-46a6-82ac-f73568c5936b" (UID: "46cf1643-a035-46a6-82ac-f73568c5936b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:32:08.302385 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:08.302343    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46cf1643-a035-46a6-82ac-f73568c5936b-maas-api-tls\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:32:08.302385 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:08.302373    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gdlrm\" (UniqueName: \"kubernetes.io/projected/46cf1643-a035-46a6-82ac-f73568c5936b-kube-api-access-gdlrm\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:32:08.302385 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:08.302383    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46cf1643-a035-46a6-82ac-f73568c5936b-openshift-service-ca\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:32:08.571876 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:08.571846    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:32:08.905331 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:08.905241    2563 generic.go:356] "Generic (PLEG): container finished" podID="46cf1643-a035-46a6-82ac-f73568c5936b" containerID="c6474a00c89febf3ab08a4e59b14c155a7ccd54b13e1a2ce97afc545ffc5a20a" exitCode=0
Jul 28 18:32:08.905331 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:08.905288    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b5b66-84d4dc76bf-tbcmk"
Jul 28 18:32:08.905795 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:08.905326    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b5b66-84d4dc76bf-tbcmk" event={"ID":"46cf1643-a035-46a6-82ac-f73568c5936b","Type":"ContainerDied","Data":"c6474a00c89febf3ab08a4e59b14c155a7ccd54b13e1a2ce97afc545ffc5a20a"}
Jul 28 18:32:08.905795 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:08.905376    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b5b66-84d4dc76bf-tbcmk" event={"ID":"46cf1643-a035-46a6-82ac-f73568c5936b","Type":"ContainerDied","Data":"f712c3fc7976376e94087ba7193c4cbfc77ad438bb2ee7ea7e1a4b4684639a33"}
Jul 28 18:32:08.905795 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:08.905398    2563 scope.go:117] "RemoveContainer" containerID="c6474a00c89febf3ab08a4e59b14c155a7ccd54b13e1a2ce97afc545ffc5a20a"
Jul 28 18:32:08.915445 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:08.915416    2563 scope.go:117] "RemoveContainer" containerID="c6474a00c89febf3ab08a4e59b14c155a7ccd54b13e1a2ce97afc545ffc5a20a"
Jul 28 18:32:08.915712 ip-10-0-136-126 kubenswrapper[2563]: E0728 18:32:08.915693    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6474a00c89febf3ab08a4e59b14c155a7ccd54b13e1a2ce97afc545ffc5a20a\": container with ID starting with c6474a00c89febf3ab08a4e59b14c155a7ccd54b13e1a2ce97afc545ffc5a20a not found: ID does not exist" containerID="c6474a00c89febf3ab08a4e59b14c155a7ccd54b13e1a2ce97afc545ffc5a20a"
Jul 28 18:32:08.915761 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:08.915729    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6474a00c89febf3ab08a4e59b14c155a7ccd54b13e1a2ce97afc545ffc5a20a"} err="failed to get container status \"c6474a00c89febf3ab08a4e59b14c155a7ccd54b13e1a2ce97afc545ffc5a20a\": rpc error: code = NotFound desc = could not find container \"c6474a00c89febf3ab08a4e59b14c155a7ccd54b13e1a2ce97afc545ffc5a20a\": container with ID starting with c6474a00c89febf3ab08a4e59b14c155a7ccd54b13e1a2ce97afc545ffc5a20a not found: ID does not exist"
Jul 28 18:32:08.928188 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:08.928170    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-9b5b66-84d4dc76bf-tbcmk"]
Jul 28 18:32:08.933187 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:08.933168    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-9b5b66-84d4dc76bf-tbcmk"]
Jul 28 18:32:09.854792 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:09.854754    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46cf1643-a035-46a6-82ac-f73568c5936b" path="/var/lib/kubelet/pods/46cf1643-a035-46a6-82ac-f73568c5936b/volumes"
Jul 28 18:32:15.871941 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:15.871911    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:32:15.972363 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:15.972325    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:32:16.204237 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:16.204111    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:32:16.402169 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:16.402137    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:32:16.522726 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:16.522646    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c73c1e-hmcsw"]
Jul 28 18:32:16.523029 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:16.523014    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd7df201-b522-4f85-9cb6-9d4453e3024b" containerName="maas-api"
Jul 28 18:32:16.523075 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:16.523031    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd7df201-b522-4f85-9cb6-9d4453e3024b" containerName="maas-api"
Jul 28 18:32:16.523075 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:16.523040    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a4333a5-791b-42d7-a9cf-5dc3c3a95165" containerName="payload-processing"
Jul 28 18:32:16.523075 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:16.523046    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a4333a5-791b-42d7-a9cf-5dc3c3a95165" containerName="payload-processing"
Jul 28 18:32:16.523075 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:16.523055    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5013d74-382f-4636-a97e-3126bc23be29" containerName="payload-pre-processing"
Jul 28 18:32:16.523075 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:16.523060    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5013d74-382f-4636-a97e-3126bc23be29" containerName="payload-pre-processing"
Jul 28 18:32:16.523075 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:16.523067    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31553ab6-42e2-42f0-9f06-ba203456e212" containerName="payload-pre-processing"
Jul 28 18:32:16.523075 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:16.523073    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="31553ab6-42e2-42f0-9f06-ba203456e212" containerName="payload-pre-processing"
Jul 28 18:32:16.523284 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:16.523085    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f12d0d22-4131-455a-9541-1fed5c26c08e" containerName="maas-api"
Jul 28 18:32:16.523284 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:16.523091    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f12d0d22-4131-455a-9541-1fed5c26c08e" containerName="maas-api"
Jul 28 18:32:16.523284 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:16.523096    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bef84b4-2ca4-4eb3-a79c-34a96b5efb70" containerName="maas-api"
Jul 28 18:32:16.523284 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:16.523102    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bef84b4-2ca4-4eb3-a79c-34a96b5efb70" containerName="maas-api"
Jul 28 18:32:16.523284 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:16.523109    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61673ff4-6ce9-4257-b54c-fd3f9912d924" containerName="payload-processing"
Jul 28 18:32:16.523284 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:16.523115    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="61673ff4-6ce9-4257-b54c-fd3f9912d924" containerName="payload-processing"
Jul 28 18:32:16.523284 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:16.523123    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46cf1643-a035-46a6-82ac-f73568c5936b" containerName="maas-api"
Jul 28 18:32:16.523284 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:16.523129    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="46cf1643-a035-46a6-82ac-f73568c5936b" containerName="maas-api"
Jul 28 18:32:16.523284 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:16.523136    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53008497-534a-4934-84ed-d6698ba729fa" containerName="payload-pre-processing"
Jul 28 18:32:16.523284 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:16.523141    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="53008497-534a-4934-84ed-d6698ba729fa" containerName="payload-pre-processing"
Jul 28 18:32:16.523284 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:16.523198    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="31553ab6-42e2-42f0-9f06-ba203456e212" containerName="payload-pre-processing"
Jul 28 18:32:16.523284 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:16.523206    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7bef84b4-2ca4-4eb3-a79c-34a96b5efb70" containerName="maas-api"
Jul 28 18:32:16.523284 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:16.523213    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f12d0d22-4131-455a-9541-1fed5c26c08e" containerName="maas-api"
Jul 28 18:32:16.523284 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:16.523218    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd7df201-b522-4f85-9cb6-9d4453e3024b" containerName="maas-api"
Jul 28 18:32:16.523284 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:16.523225    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="61673ff4-6ce9-4257-b54c-fd3f9912d924" containerName="payload-processing"
Jul 28 18:32:16.523284 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:16.523230    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5013d74-382f-4636-a97e-3126bc23be29" containerName="payload-pre-processing"
Jul 28 18:32:16.523284 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:16.523235    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="46cf1643-a035-46a6-82ac-f73568c5936b" containerName="maas-api"
Jul 28 18:32:16.523284 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:16.523240    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a4333a5-791b-42d7-a9cf-5dc3c3a95165" containerName="payload-processing"
Jul 28 18:32:16.523284 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:16.523245    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="53008497-534a-4934-84ed-d6698ba729fa" containerName="payload-pre-processing"
Jul 28 18:32:16.527281 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:16.527261    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c73c1e-hmcsw"
Jul 28 18:32:16.529816 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:16.529796    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9xcv2\""
Jul 28 18:32:16.534905 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:16.534717    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c73c1e-hmcsw"]
Jul 28 18:32:16.673097 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:16.673057    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41fc738b-ec7c-464b-8f0e-0678e1dc0fa3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-c73c1e-hmcsw\" (UID: \"41fc738b-ec7c-464b-8f0e-0678e1dc0fa3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c73c1e-hmcsw"
Jul 28 18:32:16.774505 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:16.774397    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41fc738b-ec7c-464b-8f0e-0678e1dc0fa3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-c73c1e-hmcsw\" (UID: \"41fc738b-ec7c-464b-8f0e-0678e1dc0fa3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c73c1e-hmcsw"
Jul 28 18:32:16.775031 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:16.775009    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41fc738b-ec7c-464b-8f0e-0678e1dc0fa3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-c73c1e-hmcsw\" (UID: \"41fc738b-ec7c-464b-8f0e-0678e1dc0fa3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c73c1e-hmcsw"
Jul 28 18:32:16.839041 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:16.839007    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c73c1e-hmcsw"
Jul 28 18:32:16.978067 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:16.978036    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c73c1e-hmcsw"]
Jul 28 18:32:16.979486 ip-10-0-136-126 kubenswrapper[2563]: W0728 18:32:16.979463    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41fc738b_ec7c_464b_8f0e_0678e1dc0fa3.slice/crio-3eb7ae91314dc69c681d098921225f17baf45c1909b01f5b149cbfaf4599fbce WatchSource:0}: Error finding container 3eb7ae91314dc69c681d098921225f17baf45c1909b01f5b149cbfaf4599fbce: Status 404 returned error can't find the container with id 3eb7ae91314dc69c681d098921225f17baf45c1909b01f5b149cbfaf4599fbce
Jul 28 18:32:17.960915 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:17.960817    2563 generic.go:356] "Generic (PLEG): container finished" podID="41fc738b-ec7c-464b-8f0e-0678e1dc0fa3" containerID="aafe88c8c725f026a057a70193d908b41df6b834289efcdcef4109ca1ef3f2ce" exitCode=0
Jul 28 18:32:17.960915 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:17.960896    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c73c1e-hmcsw" event={"ID":"41fc738b-ec7c-464b-8f0e-0678e1dc0fa3","Type":"ContainerDied","Data":"aafe88c8c725f026a057a70193d908b41df6b834289efcdcef4109ca1ef3f2ce"}
Jul 28 18:32:17.961133 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:17.960968    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c73c1e-hmcsw" event={"ID":"41fc738b-ec7c-464b-8f0e-0678e1dc0fa3","Type":"ContainerStarted","Data":"3eb7ae91314dc69c681d098921225f17baf45c1909b01f5b149cbfaf4599fbce"}
Jul 28 18:32:19.094948 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:19.094921    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c73c1e-hmcsw"
Jul 28 18:32:19.195055 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:19.195022    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41fc738b-ec7c-464b-8f0e-0678e1dc0fa3-maas-api-service-ca\") pod \"41fc738b-ec7c-464b-8f0e-0678e1dc0fa3\" (UID: \"41fc738b-ec7c-464b-8f0e-0678e1dc0fa3\") "
Jul 28 18:32:19.195367 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:19.195345    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41fc738b-ec7c-464b-8f0e-0678e1dc0fa3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "41fc738b-ec7c-464b-8f0e-0678e1dc0fa3" (UID: "41fc738b-ec7c-464b-8f0e-0678e1dc0fa3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:32:19.295617 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:19.295541    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41fc738b-ec7c-464b-8f0e-0678e1dc0fa3-maas-api-service-ca\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:32:19.973887 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:19.973851    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c73c1e-hmcsw" event={"ID":"41fc738b-ec7c-464b-8f0e-0678e1dc0fa3","Type":"ContainerDied","Data":"3eb7ae91314dc69c681d098921225f17baf45c1909b01f5b149cbfaf4599fbce"}
Jul 28 18:32:19.974059 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:19.973907    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3eb7ae91314dc69c681d098921225f17baf45c1909b01f5b149cbfaf4599fbce"
Jul 28 18:32:19.974059 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:19.973868    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c73c1e-hmcsw"
Jul 28 18:32:21.660039 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:21.660003    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-c73c1e-6ff87597b8-hdlcx"]
Jul 28 18:32:21.660803 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:21.660765    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c73c1e-6ff87597b8-hdlcx" podUID="03c0eef1-73e0-4400-a6f7-1936634155a5" containerName="maas-api" containerID="cri-o://53e694a4701d67eda16ef3c91c4cf481470bda4c0837dbdb037feec2b0edb1db" gracePeriod=30
Jul 28 18:32:21.691547 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:21.691507    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:32:21.692082 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:21.692060    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-c73c1e-29754390-wr8qw"]
Jul 28 18:32:21.695114 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:21.695091    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-c73c1e-29754390-wr8qw"]
Jul 28 18:32:21.729995 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:21.729961    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-c73c1e-668b4b8ff8-w79d6"]
Jul 28 18:32:21.730236 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:21.730208    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-c73c1e-668b4b8ff8-w79d6" podUID="b10f1515-8d9d-4a52-ba48-50c787a5e9b5" containerName="payload-processing" containerID="cri-o://19257d4ac1aaab6fc037f6c44ef7bc4f728b5f8a00e71daddee306de53221730" gracePeriod=30
Jul 28 18:32:21.737137 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:21.736145    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-c73c1e-569f987c9b-jptl4"]
Jul 28 18:32:21.737137 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:21.736375    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-c73c1e-569f987c9b-jptl4" podUID="fe167dee-9478-42ec-97a6-b291dbb3cf60" containerName="payload-pre-processing" containerID="cri-o://201e9440c1e8cd22bc09fdd7780872769aceaec4360953fcdf07a4ba673cd52a" gracePeriod=30
Jul 28 18:32:21.765398 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:21.765366    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:32:21.855514 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:21.855485    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="400c8f56-826e-4ee3-8b03-3339bf38f057" path="/var/lib/kubelet/pods/400c8f56-826e-4ee3-8b03-3339bf38f057/volumes"
Jul 28 18:32:21.864376 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:21.864321    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:32:21.880266 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:21.880235    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c73c1e-hmcsw"]
Jul 28 18:32:21.883452 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:21.883397    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c73c1e-hmcsw"]
Jul 28 18:32:21.929263 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:21.929232    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:32:21.989144 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:21.989110    2563 generic.go:356] "Generic (PLEG): container finished" podID="b10f1515-8d9d-4a52-ba48-50c787a5e9b5" containerID="19257d4ac1aaab6fc037f6c44ef7bc4f728b5f8a00e71daddee306de53221730" exitCode=0
Jul 28 18:32:21.989303 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:21.989165    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-c73c1e-668b4b8ff8-w79d6" event={"ID":"b10f1515-8d9d-4a52-ba48-50c787a5e9b5","Type":"ContainerDied","Data":"19257d4ac1aaab6fc037f6c44ef7bc4f728b5f8a00e71daddee306de53221730"}
Jul 28 18:32:21.991679 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:21.991579    2563 generic.go:356] "Generic (PLEG): container finished" podID="fe167dee-9478-42ec-97a6-b291dbb3cf60" containerID="201e9440c1e8cd22bc09fdd7780872769aceaec4360953fcdf07a4ba673cd52a" exitCode=0
Jul 28 18:32:21.991679 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:21.991613    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-c73c1e-569f987c9b-jptl4" event={"ID":"fe167dee-9478-42ec-97a6-b291dbb3cf60","Type":"ContainerDied","Data":"201e9440c1e8cd22bc09fdd7780872769aceaec4360953fcdf07a4ba673cd52a"}
Jul 28 18:32:21.993407 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:21.993379    2563 generic.go:356] "Generic (PLEG): container finished" podID="03c0eef1-73e0-4400-a6f7-1936634155a5" containerID="53e694a4701d67eda16ef3c91c4cf481470bda4c0837dbdb037feec2b0edb1db" exitCode=0
Jul 28 18:32:21.994077 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:21.993644    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c73c1e-6ff87597b8-hdlcx" event={"ID":"03c0eef1-73e0-4400-a6f7-1936634155a5","Type":"ContainerDied","Data":"53e694a4701d67eda16ef3c91c4cf481470bda4c0837dbdb037feec2b0edb1db"}
Jul 28 18:32:21.994077 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:21.993705    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c73c1e-6ff87597b8-hdlcx" event={"ID":"03c0eef1-73e0-4400-a6f7-1936634155a5","Type":"ContainerDied","Data":"70618a9a84931e77ab54cd4a67b195300176acea4efce1a99f765f5d3152b7b9"}
Jul 28 18:32:21.994077 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:21.993732    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="70618a9a84931e77ab54cd4a67b195300176acea4efce1a99f765f5d3152b7b9"
Jul 28 18:32:22.008699 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:22.008673    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c73c1e-6ff87597b8-hdlcx"
Jul 28 18:32:22.014949 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:22.014923    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03c0eef1-73e0-4400-a6f7-1936634155a5-maas-api-tls\") pod \"03c0eef1-73e0-4400-a6f7-1936634155a5\" (UID: \"03c0eef1-73e0-4400-a6f7-1936634155a5\") "
Jul 28 18:32:22.015088 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:22.015006    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03c0eef1-73e0-4400-a6f7-1936634155a5-openshift-service-ca\") pod \"03c0eef1-73e0-4400-a6f7-1936634155a5\" (UID: \"03c0eef1-73e0-4400-a6f7-1936634155a5\") "
Jul 28 18:32:22.015088 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:22.015049    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hdxth\" (UniqueName: \"kubernetes.io/projected/03c0eef1-73e0-4400-a6f7-1936634155a5-kube-api-access-hdxth\") pod \"03c0eef1-73e0-4400-a6f7-1936634155a5\" (UID: \"03c0eef1-73e0-4400-a6f7-1936634155a5\") "
Jul 28 18:32:22.015412 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:22.015375    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03c0eef1-73e0-4400-a6f7-1936634155a5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "03c0eef1-73e0-4400-a6f7-1936634155a5" (UID: "03c0eef1-73e0-4400-a6f7-1936634155a5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:32:22.017645 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:22.017621    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03c0eef1-73e0-4400-a6f7-1936634155a5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "03c0eef1-73e0-4400-a6f7-1936634155a5" (UID: "03c0eef1-73e0-4400-a6f7-1936634155a5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:32:22.018960 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:22.018936    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03c0eef1-73e0-4400-a6f7-1936634155a5-kube-api-access-hdxth" (OuterVolumeSpecName: "kube-api-access-hdxth") pod "03c0eef1-73e0-4400-a6f7-1936634155a5" (UID: "03c0eef1-73e0-4400-a6f7-1936634155a5"). InnerVolumeSpecName "kube-api-access-hdxth". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:32:22.050865 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:22.050811    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-c73c1e-668b4b8ff8-w79d6"
Jul 28 18:32:22.053825 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:22.053803    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-c73c1e-569f987c9b-jptl4"
Jul 28 18:32:22.116370 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:22.116336    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b10f1515-8d9d-4a52-ba48-50c787a5e9b5-plugins-config-volume\") pod \"b10f1515-8d9d-4a52-ba48-50c787a5e9b5\" (UID: \"b10f1515-8d9d-4a52-ba48-50c787a5e9b5\") "
Jul 28 18:32:22.116370 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:22.116374    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe167dee-9478-42ec-97a6-b291dbb3cf60-plugins-config-volume\") pod \"fe167dee-9478-42ec-97a6-b291dbb3cf60\" (UID: \"fe167dee-9478-42ec-97a6-b291dbb3cf60\") "
Jul 28 18:32:22.116624 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:22.116399    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nzlft\" (UniqueName: \"kubernetes.io/projected/fe167dee-9478-42ec-97a6-b291dbb3cf60-kube-api-access-nzlft\") pod \"fe167dee-9478-42ec-97a6-b291dbb3cf60\" (UID: \"fe167dee-9478-42ec-97a6-b291dbb3cf60\") "
Jul 28 18:32:22.116624 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:22.116537    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gl6ng\" (UniqueName: \"kubernetes.io/projected/b10f1515-8d9d-4a52-ba48-50c787a5e9b5-kube-api-access-gl6ng\") pod \"b10f1515-8d9d-4a52-ba48-50c787a5e9b5\" (UID: \"b10f1515-8d9d-4a52-ba48-50c787a5e9b5\") "
Jul 28 18:32:22.116789 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:22.116770    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03c0eef1-73e0-4400-a6f7-1936634155a5-openshift-service-ca\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:32:22.116866 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:22.116795    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hdxth\" (UniqueName: \"kubernetes.io/projected/03c0eef1-73e0-4400-a6f7-1936634155a5-kube-api-access-hdxth\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:32:22.116866 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:22.116802    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe167dee-9478-42ec-97a6-b291dbb3cf60-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fe167dee-9478-42ec-97a6-b291dbb3cf60" (UID: "fe167dee-9478-42ec-97a6-b291dbb3cf60"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:32:22.116866 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:22.116808    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b10f1515-8d9d-4a52-ba48-50c787a5e9b5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b10f1515-8d9d-4a52-ba48-50c787a5e9b5" (UID: "b10f1515-8d9d-4a52-ba48-50c787a5e9b5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:32:22.116866 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:22.116809    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03c0eef1-73e0-4400-a6f7-1936634155a5-maas-api-tls\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:32:22.118640 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:22.118615    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe167dee-9478-42ec-97a6-b291dbb3cf60-kube-api-access-nzlft" (OuterVolumeSpecName: "kube-api-access-nzlft") pod "fe167dee-9478-42ec-97a6-b291dbb3cf60" (UID: "fe167dee-9478-42ec-97a6-b291dbb3cf60"). InnerVolumeSpecName "kube-api-access-nzlft". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:32:22.118755 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:22.118671    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b10f1515-8d9d-4a52-ba48-50c787a5e9b5-kube-api-access-gl6ng" (OuterVolumeSpecName: "kube-api-access-gl6ng") pod "b10f1515-8d9d-4a52-ba48-50c787a5e9b5" (UID: "b10f1515-8d9d-4a52-ba48-50c787a5e9b5"). InnerVolumeSpecName "kube-api-access-gl6ng". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:32:22.217618 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:22.217525    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b10f1515-8d9d-4a52-ba48-50c787a5e9b5-plugins-config-volume\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:32:22.217618 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:22.217556    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe167dee-9478-42ec-97a6-b291dbb3cf60-plugins-config-volume\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:32:22.217618 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:22.217569    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nzlft\" (UniqueName: \"kubernetes.io/projected/fe167dee-9478-42ec-97a6-b291dbb3cf60-kube-api-access-nzlft\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:32:22.217618 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:22.217580    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gl6ng\" (UniqueName: \"kubernetes.io/projected/b10f1515-8d9d-4a52-ba48-50c787a5e9b5-kube-api-access-gl6ng\") on node \"ip-10-0-136-126.ec2.internal\" DevicePath \"\""
Jul 28 18:32:22.282090 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:22.282058    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:32:22.455199 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:22.455166    2563 scope.go:117] "RemoveContainer" containerID="53e694a4701d67eda16ef3c91c4cf481470bda4c0837dbdb037feec2b0edb1db"
Jul 28 18:32:22.463967 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:22.463951    2563 scope.go:117] "RemoveContainer" containerID="19257d4ac1aaab6fc037f6c44ef7bc4f728b5f8a00e71daddee306de53221730"
Jul 28 18:32:22.471575 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:22.471548    2563 scope.go:117] "RemoveContainer" containerID="201e9440c1e8cd22bc09fdd7780872769aceaec4360953fcdf07a4ba673cd52a"
Jul 28 18:32:22.998763 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:22.998727    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-c73c1e-569f987c9b-jptl4"
Jul 28 18:32:22.999257 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:22.998781    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-c73c1e-668b4b8ff8-w79d6" event={"ID":"b10f1515-8d9d-4a52-ba48-50c787a5e9b5","Type":"ContainerDied","Data":"6a9c234f39977e3fbcd38f1c7562e9ec35f56c3ca07ba82c0403bf2f7741c682"}
Jul 28 18:32:22.999257 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:22.998844    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-c73c1e-569f987c9b-jptl4" event={"ID":"fe167dee-9478-42ec-97a6-b291dbb3cf60","Type":"ContainerDied","Data":"dd172b232f6b98d39dbfba7be117c1f96224a847c2f63416697b97327673a8c2"}
Jul 28 18:32:22.999257 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:22.998888    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c73c1e-6ff87597b8-hdlcx"
Jul 28 18:32:22.999257 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:22.999001    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-c73c1e-668b4b8ff8-w79d6"
Jul 28 18:32:23.047109 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:23.047077    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-c73c1e-6ff87597b8-hdlcx"]
Jul 28 18:32:23.053652 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:23.053630    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-c73c1e-6ff87597b8-hdlcx"]
Jul 28 18:32:23.083138 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:23.083109    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-c73c1e-668b4b8ff8-w79d6"]
Jul 28 18:32:23.087955 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:23.087930    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-c73c1e-668b4b8ff8-w79d6"]
Jul 28 18:32:23.099611 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:23.099587    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-c73c1e-569f987c9b-jptl4"]
Jul 28 18:32:23.104213 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:23.104190    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-c73c1e-569f987c9b-jptl4"]
Jul 28 18:32:23.853504 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:23.853472    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03c0eef1-73e0-4400-a6f7-1936634155a5" path="/var/lib/kubelet/pods/03c0eef1-73e0-4400-a6f7-1936634155a5/volumes"
Jul 28 18:32:23.853801 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:23.853789    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41fc738b-ec7c-464b-8f0e-0678e1dc0fa3" path="/var/lib/kubelet/pods/41fc738b-ec7c-464b-8f0e-0678e1dc0fa3/volumes"
Jul 28 18:32:23.854090 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:23.854079    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b10f1515-8d9d-4a52-ba48-50c787a5e9b5" path="/var/lib/kubelet/pods/b10f1515-8d9d-4a52-ba48-50c787a5e9b5/volumes"
Jul 28 18:32:23.854376 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:23.854365    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe167dee-9478-42ec-97a6-b291dbb3cf60" path="/var/lib/kubelet/pods/fe167dee-9478-42ec-97a6-b291dbb3cf60/volumes"
Jul 28 18:32:28.645394 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:28.645353    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:32:28.745910 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:28.745877    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:32:28.924089 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:28.924007    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:32:29.172065 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:29.172032    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:32:30.248578 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:30.248545    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:32:30.346082 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:30.346053    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:32:35.270709 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:35.270679    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-5bfrd_d1058246-3d40-4ead-83f6-bed96bf7777c/manager/0.log"
Jul 28 18:32:35.399786 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:35.399734    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-zghk7_76cb19a8-73c6-4277-a484-608429e16222/manager/0.log"
Jul 28 18:32:35.657776 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:35.657749    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-ng7x4_0ae086ab-fa26-4ea1-94a2-b35920c32d3b/server/0.log"
Jul 28 18:32:35.779117 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:35.779084    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-fn4nb_7b824351-3974-48c2-a51d-279feba50bb2/manager/0.log"
Jul 28 18:32:35.896990 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:35.896964    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c7c5665b5-9g8wm_091c1bab-94c3-43f8-92c9-6c4937a172ce/copy-manifests/0.log"
Jul 28 18:32:35.910564 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:35.910499    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c7c5665b5-9g8wm_091c1bab-94c3-43f8-92c9-6c4937a172ce/manager/0.log"
Jul 28 18:32:36.514024 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:36.513997    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-7d4cd97cdc-zdk6z_46fc5fc3-5d95-4878-b4ab-a3e73285a72b/maas-api/0.log"
Jul 28 18:32:36.633866 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:36.633830    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29754360-n2m5s_ab4c837e-4522-4662-b6f9-60c6cb3566b5/cleanup/0.log"
Jul 28 18:32:36.752985 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:36.752957    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29754375-275ww_b0d32692-ba92-4679-bb2f-ddee85668e55/cleanup/0.log"
Jul 28 18:32:37.848053 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:37.848028    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pnv6x_fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa/pull/0.log"
Jul 28 18:32:37.855597 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:37.855563    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pnv6x_fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa/extract/0.log"
Jul 28 18:32:37.863029 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:37.863013    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pnv6x_fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa/util/0.log"
Jul 28 18:32:37.978084 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:37.978052    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nvggl_cd6773b9-1881-431a-bd08-757ec5e03c00/util/0.log"
Jul 28 18:32:37.985945 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:37.985920    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nvggl_cd6773b9-1881-431a-bd08-757ec5e03c00/pull/0.log"
Jul 28 18:32:37.994696 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:37.994678    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nvggl_cd6773b9-1881-431a-bd08-757ec5e03c00/extract/0.log"
Jul 28 18:32:38.115092 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:38.115025    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlgc2_010eb983-47d5-4c20-a9d4-f8f27c32d136/util/0.log"
Jul 28 18:32:38.122215 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:38.122192    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlgc2_010eb983-47d5-4c20-a9d4-f8f27c32d136/pull/0.log"
Jul 28 18:32:38.129199 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:38.129179    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlgc2_010eb983-47d5-4c20-a9d4-f8f27c32d136/extract/0.log"
Jul 28 18:32:38.248608 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:38.248585    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1scs6r_3f67ec8b-7142-4b63-9df8-ed079ca8a9f7/util/0.log"
Jul 28 18:32:38.255671 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:38.255652    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1scs6r_3f67ec8b-7142-4b63-9df8-ed079ca8a9f7/pull/0.log"
Jul 28 18:32:38.262886 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:38.262869    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1scs6r_3f67ec8b-7142-4b63-9df8-ed079ca8a9f7/extract/0.log"
Jul 28 18:32:38.691329 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:38.691299    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-w487q_f6365e9a-e607-4e3a-b0f9-d7a904d120d6/manager/0.log"
Jul 28 18:32:39.068071 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:39.068032    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-6gbws_f61cb9c1-bef2-44df-b009-93d12dee3429/manager/0.log"
Jul 28 18:32:39.310100 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:39.310070    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-j56s6_0bb917a1-5d9d-4d4b-ae0e-3718aa32d030/manager/0.log"
Jul 28 18:32:39.852369 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:39.852341    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-8pm6v_05df90ce-13cd-478f-864f-103c7cfa9689/discovery/0.log"
Jul 28 18:32:39.971308 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:39.971279    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6b8f9cc6c-4mggp_a977444b-6bdf-4448-9859-266756ff657f/kube-auth-proxy/0.log"
Jul 28 18:32:40.640858 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:40.640832    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7c67d86669-hftv7_0641104e-1805-4a32-97fd-e9ee7c550152/router/0.log"
Jul 28 18:32:41.252825 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:41.252798    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-g67gp_9efa933f-0829-4477-9d6b-b7ee749f3ef6/main/0.log"
Jul 28 18:32:47.666378 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:47.666280    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:32:47.733252 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:47.733220    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log" limit=1024
Jul 28 18:32:50.702376 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:50.702342    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-z75m5_8a8edb3a-34c9-4a66-81aa-49474fdfdf37/global-pull-secret-syncer/0.log"
Jul 28 18:32:50.794253 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:50.794228    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-2qvs8_7e2a258e-828b-4bbd-96b8-56cb5a7dd36f/konnectivity-agent/0.log"
Jul 28 18:32:50.895491 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:50.895456    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-126.ec2.internal_bf90ceb74d2562be9fba76df0fbf255c/haproxy/0.log"
Jul 28 18:32:55.597867 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:55.597840    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pnv6x_fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa/extract/0.log"
Jul 28 18:32:55.621956 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:55.621928    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pnv6x_fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa/util/0.log"
Jul 28 18:32:55.643774 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:55.643750    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pnv6x_fdcdf9cb-e5b6-4ab8-9867-d81ce71b67aa/pull/0.log"
Jul 28 18:32:55.673323 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:55.673301    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nvggl_cd6773b9-1881-431a-bd08-757ec5e03c00/extract/0.log"
Jul 28 18:32:55.696776 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:55.696752    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nvggl_cd6773b9-1881-431a-bd08-757ec5e03c00/util/0.log"
Jul 28 18:32:55.723503 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:55.723485    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nvggl_cd6773b9-1881-431a-bd08-757ec5e03c00/pull/0.log"
Jul 28 18:32:55.753747 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:55.753730    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlgc2_010eb983-47d5-4c20-a9d4-f8f27c32d136/extract/0.log"
Jul 28 18:32:55.777494 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:55.777472    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlgc2_010eb983-47d5-4c20-a9d4-f8f27c32d136/util/0.log"
Jul 28 18:32:55.802451 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:55.802415    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlgc2_010eb983-47d5-4c20-a9d4-f8f27c32d136/pull/0.log"
Jul 28 18:32:55.828803 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:55.828771    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1scs6r_3f67ec8b-7142-4b63-9df8-ed079ca8a9f7/extract/0.log"
Jul 28 18:32:55.850899 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:55.850831    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1scs6r_3f67ec8b-7142-4b63-9df8-ed079ca8a9f7/util/0.log"
Jul 28 18:32:55.874618 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:55.874597    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1scs6r_3f67ec8b-7142-4b63-9df8-ed079ca8a9f7/pull/0.log"
Jul 28 18:32:56.301916 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:56.301823    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-w487q_f6365e9a-e607-4e3a-b0f9-d7a904d120d6/manager/0.log"
Jul 28 18:32:56.590130 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:56.590096    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-6gbws_f61cb9c1-bef2-44df-b009-93d12dee3429/manager/0.log"
Jul 28 18:32:56.700282 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:56.700241    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-j56s6_0bb917a1-5d9d-4d4b-ae0e-3718aa32d030/manager/0.log"
Jul 28 18:32:58.681508 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:58.681475    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-jqft4_7393a17e-a626-450c-bcb9-04ea1774e002/cluster-monitoring-operator/0.log"
Jul 28 18:32:59.107498 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:59.107470    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jplhg_3573a7fb-6b9a-4b2b-99f0-99a7b4301719/node-exporter/0.log"
Jul 28 18:32:59.130716 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:59.130695    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jplhg_3573a7fb-6b9a-4b2b-99f0-99a7b4301719/kube-rbac-proxy/0.log"
Jul 28 18:32:59.152859 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:32:59.152843    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jplhg_3573a7fb-6b9a-4b2b-99f0-99a7b4301719/init-textfile/0.log"
Jul 28 18:33:00.520847 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:00.520816    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-sx94q/perf-node-gather-daemonset-xh7dh"]
Jul 28 18:33:00.521273 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:00.521256    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03c0eef1-73e0-4400-a6f7-1936634155a5" containerName="maas-api"
Jul 28 18:33:00.521273 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:00.521270    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="03c0eef1-73e0-4400-a6f7-1936634155a5" containerName="maas-api"
Jul 28 18:33:00.521359 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:00.521289    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41fc738b-ec7c-464b-8f0e-0678e1dc0fa3" containerName="revoke-keys"
Jul 28 18:33:00.521359 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:00.521295    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="41fc738b-ec7c-464b-8f0e-0678e1dc0fa3" containerName="revoke-keys"
Jul 28 18:33:00.521359 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:00.521307    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b10f1515-8d9d-4a52-ba48-50c787a5e9b5" containerName="payload-processing"
Jul 28 18:33:00.521359 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:00.521312    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b10f1515-8d9d-4a52-ba48-50c787a5e9b5" containerName="payload-processing"
Jul 28 18:33:00.521359 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:00.521321    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe167dee-9478-42ec-97a6-b291dbb3cf60" containerName="payload-pre-processing"
Jul 28 18:33:00.521359 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:00.521325    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe167dee-9478-42ec-97a6-b291dbb3cf60" containerName="payload-pre-processing"
Jul 28 18:33:00.521566 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:00.521385    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b10f1515-8d9d-4a52-ba48-50c787a5e9b5" containerName="payload-processing"
Jul 28 18:33:00.521566 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:00.521394    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="03c0eef1-73e0-4400-a6f7-1936634155a5" containerName="maas-api"
Jul 28 18:33:00.521566 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:00.521400    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="41fc738b-ec7c-464b-8f0e-0678e1dc0fa3" containerName="revoke-keys"
Jul 28 18:33:00.521566 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:00.521406    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe167dee-9478-42ec-97a6-b291dbb3cf60" containerName="payload-pre-processing"
Jul 28 18:33:00.524788 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:00.524772    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sx94q/perf-node-gather-daemonset-xh7dh"
Jul 28 18:33:00.527535 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:00.527511    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-sx94q\"/\"kube-root-ca.crt\""
Jul 28 18:33:00.527664 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:00.527563    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-sx94q\"/\"openshift-service-ca.crt\""
Jul 28 18:33:00.528597 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:00.528579    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-sx94q\"/\"default-dockercfg-t679m\""
Jul 28 18:33:00.533703 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:00.533680    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sx94q/perf-node-gather-daemonset-xh7dh"]
Jul 28 18:33:00.650235 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:00.650202    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9c40c746-cc7e-44de-b07a-2ff4f70c9078-sys\") pod \"perf-node-gather-daemonset-xh7dh\" (UID: \"9c40c746-cc7e-44de-b07a-2ff4f70c9078\") " pod="openshift-must-gather-sx94q/perf-node-gather-daemonset-xh7dh"
Jul 28 18:33:00.650235 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:00.650241    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9c40c746-cc7e-44de-b07a-2ff4f70c9078-podres\") pod \"perf-node-gather-daemonset-xh7dh\" (UID: \"9c40c746-cc7e-44de-b07a-2ff4f70c9078\") " pod="openshift-must-gather-sx94q/perf-node-gather-daemonset-xh7dh"
Jul 28 18:33:00.650478 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:00.650277    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9c40c746-cc7e-44de-b07a-2ff4f70c9078-lib-modules\") pod \"perf-node-gather-daemonset-xh7dh\" (UID: \"9c40c746-cc7e-44de-b07a-2ff4f70c9078\") " pod="openshift-must-gather-sx94q/perf-node-gather-daemonset-xh7dh"
Jul 28 18:33:00.650478 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:00.650298    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lsh5\" (UniqueName: \"kubernetes.io/projected/9c40c746-cc7e-44de-b07a-2ff4f70c9078-kube-api-access-5lsh5\") pod \"perf-node-gather-daemonset-xh7dh\" (UID: \"9c40c746-cc7e-44de-b07a-2ff4f70c9078\") " pod="openshift-must-gather-sx94q/perf-node-gather-daemonset-xh7dh"
Jul 28 18:33:00.650478 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:00.650324    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9c40c746-cc7e-44de-b07a-2ff4f70c9078-proc\") pod \"perf-node-gather-daemonset-xh7dh\" (UID: \"9c40c746-cc7e-44de-b07a-2ff4f70c9078\") " pod="openshift-must-gather-sx94q/perf-node-gather-daemonset-xh7dh"
Jul 28 18:33:00.751606 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:00.751554    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9c40c746-cc7e-44de-b07a-2ff4f70c9078-proc\") pod \"perf-node-gather-daemonset-xh7dh\" (UID: \"9c40c746-cc7e-44de-b07a-2ff4f70c9078\") " pod="openshift-must-gather-sx94q/perf-node-gather-daemonset-xh7dh"
Jul 28 18:33:00.751797 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:00.751661    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9c40c746-cc7e-44de-b07a-2ff4f70c9078-sys\") pod \"perf-node-gather-daemonset-xh7dh\" (UID: \"9c40c746-cc7e-44de-b07a-2ff4f70c9078\") " pod="openshift-must-gather-sx94q/perf-node-gather-daemonset-xh7dh"
Jul 28 18:33:00.751797 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:00.751687    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9c40c746-cc7e-44de-b07a-2ff4f70c9078-podres\") pod \"perf-node-gather-daemonset-xh7dh\" (UID: \"9c40c746-cc7e-44de-b07a-2ff4f70c9078\") " pod="openshift-must-gather-sx94q/perf-node-gather-daemonset-xh7dh"
Jul 28 18:33:00.751797 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:00.751690    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9c40c746-cc7e-44de-b07a-2ff4f70c9078-proc\") pod \"perf-node-gather-daemonset-xh7dh\" (UID: \"9c40c746-cc7e-44de-b07a-2ff4f70c9078\") " pod="openshift-must-gather-sx94q/perf-node-gather-daemonset-xh7dh"
Jul 28 18:33:00.751797 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:00.751730    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9c40c746-cc7e-44de-b07a-2ff4f70c9078-lib-modules\") pod \"perf-node-gather-daemonset-xh7dh\" (UID: \"9c40c746-cc7e-44de-b07a-2ff4f70c9078\") " pod="openshift-must-gather-sx94q/perf-node-gather-daemonset-xh7dh"
Jul 28 18:33:00.751797 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:00.751759    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lsh5\" (UniqueName: \"kubernetes.io/projected/9c40c746-cc7e-44de-b07a-2ff4f70c9078-kube-api-access-5lsh5\") pod \"perf-node-gather-daemonset-xh7dh\" (UID: \"9c40c746-cc7e-44de-b07a-2ff4f70c9078\") " pod="openshift-must-gather-sx94q/perf-node-gather-daemonset-xh7dh"
Jul 28 18:33:00.751797 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:00.751783    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9c40c746-cc7e-44de-b07a-2ff4f70c9078-sys\") pod \"perf-node-gather-daemonset-xh7dh\" (UID: \"9c40c746-cc7e-44de-b07a-2ff4f70c9078\") " pod="openshift-must-gather-sx94q/perf-node-gather-daemonset-xh7dh"
Jul 28 18:33:00.751797 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:00.751796    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9c40c746-cc7e-44de-b07a-2ff4f70c9078-podres\") pod \"perf-node-gather-daemonset-xh7dh\" (UID: \"9c40c746-cc7e-44de-b07a-2ff4f70c9078\") " pod="openshift-must-gather-sx94q/perf-node-gather-daemonset-xh7dh"
Jul 28 18:33:00.752097 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:00.751926    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9c40c746-cc7e-44de-b07a-2ff4f70c9078-lib-modules\") pod \"perf-node-gather-daemonset-xh7dh\" (UID: \"9c40c746-cc7e-44de-b07a-2ff4f70c9078\") " pod="openshift-must-gather-sx94q/perf-node-gather-daemonset-xh7dh"
Jul 28 18:33:00.762486 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:00.762464    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lsh5\" (UniqueName: \"kubernetes.io/projected/9c40c746-cc7e-44de-b07a-2ff4f70c9078-kube-api-access-5lsh5\") pod \"perf-node-gather-daemonset-xh7dh\" (UID: \"9c40c746-cc7e-44de-b07a-2ff4f70c9078\") " pod="openshift-must-gather-sx94q/perf-node-gather-daemonset-xh7dh"
Jul 28 18:33:00.835686 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:00.835658    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sx94q/perf-node-gather-daemonset-xh7dh"
Jul 28 18:33:00.966425 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:00.966401    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sx94q/perf-node-gather-daemonset-xh7dh"]
Jul 28 18:33:00.968250 ip-10-0-136-126 kubenswrapper[2563]: W0728 18:33:00.968219    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod9c40c746_cc7e_44de_b07a_2ff4f70c9078.slice/crio-417989c29fad93e1bfb84dedd5ec9f6ef36d5065230a4ec7803683961d2082dc WatchSource:0}: Error finding container 417989c29fad93e1bfb84dedd5ec9f6ef36d5065230a4ec7803683961d2082dc: Status 404 returned error can't find the container with id 417989c29fad93e1bfb84dedd5ec9f6ef36d5065230a4ec7803683961d2082dc
Jul 28 18:33:01.214827 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:01.214734    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sx94q/perf-node-gather-daemonset-xh7dh" event={"ID":"9c40c746-cc7e-44de-b07a-2ff4f70c9078","Type":"ContainerStarted","Data":"e7abf5d74121d1e825909d0dca46f96de73b0d2d969b87a237829b0a16892182"}
Jul 28 18:33:01.214827 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:01.214795    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sx94q/perf-node-gather-daemonset-xh7dh" event={"ID":"9c40c746-cc7e-44de-b07a-2ff4f70c9078","Type":"ContainerStarted","Data":"417989c29fad93e1bfb84dedd5ec9f6ef36d5065230a4ec7803683961d2082dc"}
Jul 28 18:33:01.215074 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:01.214907    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-sx94q/perf-node-gather-daemonset-xh7dh"
Jul 28 18:33:01.235270 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:01.235229    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-sx94q/perf-node-gather-daemonset-xh7dh" podStartSLOduration=1.235215609 podStartE2EDuration="1.235215609s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 18:33:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:33:01.232359722 +0000 UTC m=+3519.893516077" watchObservedRunningTime="2026-07-28 18:33:01.235215609 +0000 UTC m=+3519.896371964"
Jul 28 18:33:02.078847 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:02.078820    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-669956765d-fgv4m_ecf6fe5c-8ae5-458e-9964-e5974adb0c83/console/0.log"
Jul 28 18:33:03.864818 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:03.864787    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-psd2g_897c5c1b-7fb1-40eb-8b4a-46e737d8e991/dns/0.log"
Jul 28 18:33:03.891734 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:03.891708    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-psd2g_897c5c1b-7fb1-40eb-8b4a-46e737d8e991/kube-rbac-proxy/0.log"
Jul 28 18:33:04.004746 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:04.004719    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-j4wkp_654d1f40-f526-4325-b40c-fb544079bff4/dns-node-resolver/0.log"
Jul 28 18:33:04.601233 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:04.601202    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6b87dbb6b9-n25x2_91955790-de58-4f37-b897-77c442ec1a64/registry/0.log"
Jul 28 18:33:04.682526 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:04.682476    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-z8fhq_d6004102-aca2-4f56-8ddb-94374d3bc900/node-ca/0.log"
Jul 28 18:33:05.795735 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:05.795705    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-8pm6v_05df90ce-13cd-478f-864f-103c7cfa9689/discovery/0.log"
Jul 28 18:33:05.819068 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:05.819042    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6b8f9cc6c-4mggp_a977444b-6bdf-4448-9859-266756ff657f/kube-auth-proxy/0.log"
Jul 28 18:33:06.042309 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:06.042277    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7c67d86669-hftv7_0641104e-1805-4a32-97fd-e9ee7c550152/router/0.log"
Jul 28 18:33:06.905226 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:06.905188    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-lpwn9_093b4d76-2af2-4586-8421-68b7047ff74e/serve-healthcheck-canary/0.log"
Jul 28 18:33:07.229757 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:07.229688    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-sx94q/perf-node-gather-daemonset-xh7dh"
Jul 28 18:33:07.630813 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:07.630788    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mj678_dda2a096-8121-4e4d-a074-1565b2bf0872/kube-rbac-proxy/0.log"
Jul 28 18:33:07.654315 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:07.654292    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mj678_dda2a096-8121-4e4d-a074-1565b2bf0872/exporter/0.log"
Jul 28 18:33:07.678479 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:07.678450    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mj678_dda2a096-8121-4e4d-a074-1565b2bf0872/extractor/0.log"
Jul 28 18:33:10.015838 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:10.015813    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-5bfrd_d1058246-3d40-4ead-83f6-bed96bf7777c/manager/0.log"
Jul 28 18:33:10.230763 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:10.230730    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-zghk7_76cb19a8-73c6-4277-a484-608429e16222/manager/0.log"
Jul 28 18:33:10.569948 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:10.569925    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-ng7x4_0ae086ab-fa26-4ea1-94a2-b35920c32d3b/server/0.log"
Jul 28 18:33:10.653220 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:10.653180    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-fn4nb_7b824351-3974-48c2-a51d-279feba50bb2/manager/0.log"
Jul 28 18:33:10.742199 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:10.742160    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c7c5665b5-9g8wm_091c1bab-94c3-43f8-92c9-6c4937a172ce/manager/0.log"
Jul 28 18:33:10.760852 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:10.760820    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c7c5665b5-9g8wm_091c1bab-94c3-43f8-92c9-6c4937a172ce/copy-manifests/0.log"
Jul 28 18:33:11.034292 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:11.034254    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-8pm6v_05df90ce-13cd-478f-864f-103c7cfa9689/discovery/0.log"
Jul 28 18:33:11.047202 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:11.047177    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6b8f9cc6c-4mggp_a977444b-6bdf-4448-9859-266756ff657f/kube-auth-proxy/0.log"
Jul 28 18:33:11.231205 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:11.231178    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7c67d86669-hftv7_0641104e-1805-4a32-97fd-e9ee7c550152/router/0.log"
Jul 28 18:33:12.820231 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:12.820201    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-8pm6v_05df90ce-13cd-478f-864f-103c7cfa9689/discovery/0.log"
Jul 28 18:33:12.837000 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:12.836974    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6b8f9cc6c-4mggp_a977444b-6bdf-4448-9859-266756ff657f/kube-auth-proxy/0.log"
Jul 28 18:33:13.027705 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:13.027675    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7c67d86669-hftv7_0641104e-1805-4a32-97fd-e9ee7c550152/router/0.log"
Jul 28 18:33:17.639305 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:17.639270    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-25vvm_ea3e1303-f4dc-4511-abbc-9a7c0e789164/kube-storage-version-migrator-operator/1.log"
Jul 28 18:33:17.640184 ip-10-0-136-126 kubenswrapper[2563]: I0728 18:33:17.640166    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-25vvm_ea3e1303-f4dc-4511-abbc-9a7c0e789164/kube-storage-version-migrator-operator/0.log"